tltorch.CPTRL

class tltorch.CPTRL(input_shape, output_shape, rank, bias=False, verbose=0, **kwargs)[source]

Tensor Regression Layer with CP weights [1], [2]

Parameters:
input_shapeint iterable

shape of the input, excluding batch size

output_shapeint iterable

shape of the output, excluding batch size

verboseint, default is 0

level of verbosity

rankint

rank of the CP weights

verboseint, default is 1

level of verbosity, if 0, no information will be printed

References

[1]Tensor Regression Networks, Jean Kossaifi, Zachary C. Lipton, Arinbjorn Kolbeinsson, Aran Khanna, Tommaso Furlanello, Anima Anandkumar, JMLRs, 2020.
[2]Tensor Regression Networks with various Low-Rank Tensor Approximations Xingwei Cao, Guillaume Rabusseau, 2018

Methods

forward(x) Performs a forward pass
full_weight() Return the reconstructed weights from the low_rank
get_decomposition() Returns the decomposition parametrizing the layer
init_from_decomposition(weights, factors[, bias]) Initializes the factorization from the given decomposition
init_from_linear(weight[, bias]) Initialise the TRL from the weights of a fully connected layer
init_from_random([decompose_full_weight]) Initialize the module randomly
init_from_tensor(weight[, bias, …]) Initializes the layer by decomposing a full tensor
forward(x)[source]

Performs a forward pass

init_from_random(decompose_full_weight=True)[source]

Initialize the module randomly

Parameters:
decompose_full_weightbool, default is False

if True, constructs a full weight tensor and decomposes it to initialize the factors otherwise, the factors are directly initialized randomlys

init_from_decomposition(weights, factors, bias=None)[source]

Initializes the factorization from the given decomposition

Parameters:
decomposed_tensor

values to initialize the decomposition parametrizing the layer to

biastorch.Tensor or None, default is None
init_from_tensor(weight, bias=None, decomposition_kwargs={'init': 'random', 'n_iter_max': 10})[source]

Initializes the layer by decomposing a full tensor

Parameters:
tensortorch.Tensor

must be either a matrix or a tensor must verify np.prod(tensor.shape) == np.prod(self.tensorized_shape)

biastorch.Tensor or None, default is None
decomposition_kwargsdict

optional dictionary of parameters to pass to the decomposition

init_from_linear(weight, bias=None)[source]

Initialise the TRL from the weights of a fully connected layer

full_weight()[source]

Return the reconstructed weights from the low_rank

Returns:
tensor :

weights recoonstructed from the low-rank ones learnt

get_decomposition()[source]

Returns the decomposition parametrizing the layer