tltorch.tensor_hooks
.tensor_lasso¶

tltorch.tensor_hooks.
tensor_lasso
(factorization='CP', penalty=0.01, clamp_weights=True, threshold=1e06, normalize_loss=True)[source]¶ Generalized Tensor Lasso from a factorized tensors
Applies a generalized Lasso (l1 regularization) on a factorized tensor.
 Parameters
 factorizationstr
 penaltyfloat, default is 0.01
scaling factor for the loss
 clamp_weightsbool, default is True
if True, the lasso weights are clamp between 1 and 1
 thresholdfloat, default is 1e6
if a lasso weight is lower than the set threshold, it is set to 0
 normalize_lossbool, default is True
If True, the loss will be between 0 and 1. Otherwise, the raw sum of absolute weights will be returned.
Examples
Let’s say you have a set of factorized (here, CP) tensors:
>>> tensor = FactorizedTensor.new((3, 4, 2), rank='same', factorization='CP').normal_() >>> tensor2 = FactorizedTensor.new((5, 6, 7), rank=0.5, factorization='CP').normal_()
First you need to create an instance of the regularizer:
>>> regularizer = TensorLasso(factorization='cp', penalty=penalty)
You can apply the regularizer to one or several layers:
>>> regularizer.apply(tensor) >>> regularizer.apply(tensor2)
The lasso is automatically applied:
>>> sum = torch.sum(tensor() + tensor2())
You can access the Lasso loss from your instance:
>>> l1_loss = regularizer.loss
You can optimize and backpropagate through your loss as usual.
After you finish updating the weights, don’t forget to reset the regularizer, otherwise it will keep accumulating values!
>>> regularizer.reset()
You can also remove the regularizer with regularizer.remove(tensor), or remove_tensor_lasso(tensor).