API reference

tltorch: Tensorized Deep Neural Networks

Factorized Tensors

TensorLy-Torch builds on top of TensorLy and provides out of the box PyTorch layers for tensor based operations. The core of this is the concept of factorized tensors, which factorize our layers, instead of regular, dense PyTorch tensors.

You can create any factorized tensor through the main class, or directly create a specific subclass:

FactorizedTensor(*args, **kwargs)

Tensor Factorization

CPTensor(*args, **kwargs)

CP Factorization

TuckerTensor(*args, **kwargs)

Tucker Factorization

TTTensor(*args, **kwargs)

Tensor-Train (Matrix-Product-State) Factorization

Tensorized Matrices

In TensorLy-Torch , you can also represent matrices in tensorized form, as low-rank tensors .

TensorizedMatrix(*args, **kwargs)

Tensor representing one or a batch of tensorized vectors/matrices/tensors

CPMatrix(*args, **kwargs)

Tensorized Matrix in CP Form

TuckerMatrix(*args, **kwargs)

Tensorized Matrix in Tucker Form

TTMatrix(*args, **kwargs)

Tensorized Matrix in the Tensor-Train Matrix (MPO) Form


Module for initializing tensor decompositions

tensor_init(tensor[, std])

Initializes directly the parameters of a factorized tensor so the reconstruction has the specified standard deviation and 0 mean

cp_init(cp_tensor[, std])

Initializes directly the weights and factors of a CP decomposition so the reconstruction has the specified std and 0 mean

tucker_init(tucker_tensor[, std])

Initializes directly the weights and factors of a Tucker decomposition so the reconstruction has the specified std and 0 mean

tt_init(tt_tensor[, std])

Initializes directly the weights and factors of a TT decomposition so the reconstruction has the specified std and 0 mean

Tensor Regression Layers

TRL(input_shape, output_shape[, bias, …])

Tensor Regression Layers

Tensor Contraction Layers

TCL(input_shape, rank[, verbose, bias])

Tensor Contraction Layer [R4c5b93526459-1]

Factorized Linear Layers

FactorizedLinear(in_tensorized_features, …)

Tensorized Fully-Connected Layers

Factorized Convolutions

General N-Dimensional convolutions in Factorized forms

FactorizedConv(in_channels, out_channels, …)

Create a factorized convolution of arbitrary order

Tensor Dropout

These functions allow you to easily add or remove tensor dropout from tensor layers.

tensor_dropout(factorized_tensor[, p])

Tensor Dropout


Removes the tensor dropout from a TensorModule

You can also use the class API below but unless you have a particular use for the classes, you should use the convenient functions provided instead.

TensorDropout(proba[, min_dim])

Decomposition Hook for Tensor Dropout on FactorizedTensor

L1 Regularization

L1 Regularization on tensor modules.

tensor_lasso([factorization, penalty, …])

Generalized Tensor Lasso from a factorized tensors


Removes the tensor lasso from a TensorModule