- tltorch.tensor_hooks.tensor_lasso(factorization='CP', penalty=0.01, clamp_weights=True, threshold=1e-06, normalize_loss=True)
Generalized Tensor Lasso from a factorized tensors
Applies a generalized Lasso (l1 regularization) on a factorized tensor.
- penaltyfloat, default is 0.01
scaling factor for the loss
- clamp_weightsbool, default is True
if True, the lasso weights are clamp between -1 and 1
- thresholdfloat, default is 1e-6
if a lasso weight is lower than the set threshold, it is set to 0
- normalize_lossbool, default is True
If True, the loss will be between 0 and 1. Otherwise, the raw sum of absolute weights will be returned.
Let’s say you have a set of factorized (here, CP) tensors:
>>> tensor = FactorizedTensor.new((3, 4, 2), rank='same', factorization='CP').normal_() >>> tensor2 = FactorizedTensor.new((5, 6, 7), rank=0.5, factorization='CP').normal_()
First you need to create an instance of the regularizer:
>>> regularizer = TensorLasso(factorization='cp', penalty=penalty)
You can apply the regularizer to one or several layers:
>>> regularizer.apply(tensor) >>> regularizer.apply(tensor2)
The lasso is automatically applied:
>>> sum = torch.sum(tensor() + tensor2())
You can access the Lasso loss from your instance:
>>> l1_loss = regularizer.loss
You can optimize and backpropagate through your loss as usual.
After you finish updating the weights, don’t forget to reset the regularizer, otherwise it will keep accumulating values!
You can also remove the regularizer with regularizer.remove(tensor), or remove_tensor_lasso(tensor).