tltorch.TTLinear

class tltorch.TTLinear(in_features, out_features, tensorized_shape, rank, bias=True)[source]

Tensorized Fully-Connected Layers

The weight matrice is tensorized to a tensor of size tensorized_shape. That tensor is expressed as a low-rank (TT) tensor. During inference, the full tensor is reconstructed, and unfolded back into a matrix, used for the forward pass in a regular linear layer.

Parameters:	in_featuresint out_featuresint tensorized_shapeint tuple rankint tuple or str biasbool, default is True

See also

TuckerLinear

CPLinear

TTMLinear

Notes

This is very similar to [1] except that the weight matrix is simply reshaped into a tensor, while in [1], the dimensions are then also permuted in order to jointly compress input and outputs. The original [1] is implemented in tltorch.TTMLinear() .

References

[1]	(1, 2, 3) Tensorizing Neural Networks, Alexander Novikov, Dmitry Podoprikhin, Anton Osokin, Dmitry Vetrov

Attributes:	full_weight Returns the reconstruced matrix weight of the linear layer

Methods

forward(input)	Inference using the tensorized and factorized weight matrix
get_decomposition()	Returns the decomposition parametrizing the layer
init_from_decomposition(tt_tensor[, bias])	Initializes the factorization from the given decomposition
init_from_random([decompose_full_weight])	Initializes the factorization randomly
init_from_tensor(tensor[, bias, …])	Initializes the layer by decomposing a full tensor

forward(input)[source]

Inference using the tensorized and factorized weight matrix

init_from_random(decompose_full_weight=True)[source]

Initializes the factorization randomly

Parameters:	decompose_full_weightbool, default is True if True, a full weight is created and decomposed to initialize the factors otherwise, the factors of the decomposition are directly initialized

init_from_decomposition(tt_tensor, bias=None)[source]

Initializes the factorization from the given decomposition

Parameters:	tucker_tensor(factors) values to initialize the decomposition parametrizing the layer to biastorch.Tensor or None, default is None

init_from_tensor(tensor, bias=None, decomposition_kwargs={})[source]

Initializes the layer by decomposing a full tensor

Parameters:	tensortorch.Tensor must be either a matrix or a tensor must verify np.prod(tensor.shape) == np.prod(self.tensorized_shape) biastorch.Tensor or None, default is None

get_decomposition()[source]

Returns the decomposition parametrizing the layer

property full_weight

Returns the reconstruced matrix weight of the linear layer