tltorch.TensorTrainTRL

class tltorch.TensorTrainTRL(input_shape, output_shape, rank, bias=False, verbose=0, **kwargs)[source]

Tensor Regression Layer with Tensor-Train weights [1], [2]

Parameters:	input_shapeint iterable shape of the input, excluding batch size output_shapeint iterable shape of the output, excluding batch size verboseint, default is 0 level of verbosity rankint rank of the Tensor-Train / tt weights verboseint, default is 1 level of verbosity, if 0, no information will be printed

See also

CPTRL

TuckerTRL

References

[1]	Tensor Regression Networks, Jean Kossaifi, Zachary C. Lipton, Arinbjorn Kolbeinsson, Aran Khanna, Tommaso Furlanello, Anima Anandkumar, JMLR 2020.

[2]	Tensor Regression Networks with various Low-Rank Tensor Approximations Xingwei Cao, Guillaume Rabusseau, 2018

Methods

forward(x)	Performs a forward pass
full_weight()	Return the reconstructed weights from the low_rank
get_decomposition()	Returns the decomposition parametrizing the layer
init_from_decomposition(factors[, bias])	Initializes the factorization from the given decomposition
init_from_linear(weight[, bias])	Initialise the TRL from the weights of a fully connected layer
init_from_random([decompose_full_weight])	Initialize the module randomly
init_from_tensor(weight[, bias, …])	Initializes the layer by decomposing a full tensor

forward(x)[source]

Performs a forward pass

init_from_random(decompose_full_weight=True)[source]

Initialize the module randomly

Parameters:	decompose_full_weightbool, default is False if True, constructs a full weight tensor and decomposes it to initialize the factors otherwise, the factors are directly initialized randomlys

init_from_decomposition(factors, bias=None)[source]

Initializes the factorization from the given decomposition

Parameters:	decomposed_tensor values to initialize the decomposition parametrizing the layer to biastorch.Tensor or None, default is None

init_from_tensor(weight, bias=None, decomposition_kwargs={})[source]

Initializes the layer by decomposing a full tensor

Parameters:	tensortorch.Tensor must be either a matrix or a tensor must verify np.prod(tensor.shape) == np.prod(self.tensorized_shape) biastorch.Tensor or None, default is None decomposition_kwargsdict optional dictionary of parameters to pass to the decomposition

init_from_linear(weight, bias=None)[source]

Initialise the TRL from the weights of a fully connected layer

full_weight()[source]

Return the reconstructed weights from the low_rank

Returns:	tensor : weights recoonstructed from the low-rank ones learnt

get_decomposition()[source]

Returns the decomposition parametrizing the layer