tltorch.tensor_hooks.tensor_lasso

tltorch.tensor_hooks.tensor_lasso(factorization='CP', penalty=0.01, clamp_weights=True, threshold=1e-06, normalize_loss=True)[source]

Generalized Tensor Lasso from a factorized tensors

Applies a generalized Lasso (l1 regularization) on a factorized tensor.

Parameters:

factorizationstr

penaltyfloat, default is 0.01

scaling factor for the loss

clamp_weightsbool, default is True

if True, the lasso weights are clamp between -1 and 1

thresholdfloat, default is 1e-6

if a lasso weight is lower than the set threshold, it is set to 0

normalize_lossbool, default is True

If True, the loss will be between 0 and 1. Otherwise, the raw sum of absolute weights will be returned.

Examples

Let’s say you have a set of factorized (here, CP) tensors:

>>> tensor = FactorizedTensor.new((3, 4, 2), rank='same', factorization='CP').normal_()
>>> tensor2 = FactorizedTensor.new((5, 6, 7), rank=0.5, factorization='CP').normal_()

First you need to create an instance of the regularizer:

>>> regularizer = TensorLasso(factorization='cp', penalty=penalty)

You can apply the regularizer to one or several layers:

>>> regularizer.apply(tensor)
>>> regularizer.apply(tensor2)

The lasso is automatically applied:

>>> sum = torch.sum(tensor() + tensor2())

You can access the Lasso loss from your instance:

>>> l1_loss = regularizer.loss

You can optimize and backpropagate through your loss as usual.

After you finish updating the weights, don’t forget to reset the regularizer, otherwise it will keep accumulating values!

>>> regularizer.reset()

You can also remove the regularizer with regularizer.remove(tensor), or remove_tensor_lasso(tensor).