tltorch.TuckerTRL

class tltorch.TuckerTRL(input_shape, output_shape, rank, project_input=False, bias=False, verbose=0, **kwargs)[source]

Tensor Regression Layer with Tucker weights [1]

Parameters:

input_shapeint iterable shape of the input, excluding batch size output_shapeint iterable shape of the output, excluding batch size rankint or int list rank of the Tucker weights if int, the same rank will be used for all dimensions project_inputbool, default is False is True, the input activations are first projected using factors from the low-rank Tucker weights verboseint, default is 0 level of verbosity

See also

CPTRL

TensorTrainTRL

References

[1]	Tensor Regression Networks, Jean Kossaifi, Zachary C. Lipton, Arinbjorn Kolbeinsson, Aran Khanna, Tommaso Furlanello, Anima Anandkumar, JMLR, 2020.

Methods

forward(x)	Performs a forward pass
full_weight()	Return the reconstructed weights from the low_rank
get_decomposition()	Returns the decomposition parametrizing the layer
init_from_decomposition(tucker_tensor[, bias])	Initializes the factorization from the given decomposition
init_from_linear(weight, bias[, pooling_modes])	Initialise the TRL from the weights of a fully connected layer
init_from_random([decompose_full_weight])	Initialize the module randomly
init_from_tensor(tensor[, bias, …])	Initializes the layer by decomposing a full tensor

forward(x)[source]

Performs a forward pass

init_from_random(decompose_full_weight=False)[source]

Initialize the module randomly

Parameters:	decompose_full_weightbool, default is False if True, constructs a full weight tensor and decomposes it to initialize the factors otherwise, the factors are directly initialized randomlys

init_from_decomposition(tucker_tensor, bias=None)[source]

Initializes the factorization from the given decomposition

Parameters:	decomposed_tensor values to initialize the decomposition parametrizing the layer to biastorch.Tensor or None, default is None

init_from_tensor(tensor, bias=None, decomposition_kwargs={'init': 'random'})[source]

Initializes the layer by decomposing a full tensor

Parameters:	tensortorch.Tensor must be either a matrix or a tensor must verify np.prod(tensor.shape) == np.prod(self.tensorized_shape) biastorch.Tensor or None, default is None decomposition_kwargsdict optional dictionary of parameters to pass to the decomposition

init_from_linear(weight, bias, pooling_modes=None)[source]

Initialise the TRL from the weights of a fully connected layer

full_weight()[source]

Return the reconstructed weights from the low_rank

Returns:	tensor : weights recoonstructed from the low-rank ones learnt

get_decomposition()[source]

Returns the decomposition parametrizing the layer