Model evaluation¶
Functions:

Computes the core consistency [BK03] 

Efficient estimation of the Tucker core from a factor matrices and a data tensor. 

Compute the fit (1relative sum squared error) for a given cp_tensor. 

Use scikitlearn estimator to evaluate the predictive power of a factor matrix. 

Compute the relative sum of squared error for a given cp_tensor. 

Compute the sum of squared error for a given cp_tensor. 
 tlviz.model_evaluation.core_consistency(cp_tensor, dataset, normalised=False)[source]¶
Computes the core consistency [BK03]
A CP model can be interpreted as a restricted Tucker model, where the core tensor is constrained to be superdiagonal. For a third order tensor, this means that the core tensor, \(\mathcal{G}\), satisfy \(g_{ijk}\neq0\) only if \(i = j = k\). To compute the core consistency of a CP decomposition, we use this property, and calculate the optimal Tucker core tensor given the factor matrices of the CP model.
The key observation is that if the data tensor follows the assumptions of the CP model, then the optimal core tensor should be similar to that of the CP model, i. e. superdiagonal. However, if the data can be better described by allowing for interactions between the components across modes, then the core tensor will have nonzero offdiagonal. The core consistency quantifies this measure and is defined as:
\[\text{CC} = 100  100 \frac{\ \mathcal{G}  \mathcal{I} \_F^2}{N}\]where \(\mathcal{G}\) is the estimated core tensor, \(\mathcal{I}\) is a superdiagonal tensor only ones on the superdiagonal and \(N\) is a normalising factor, either equal to the number of components or the squared frobenius norm of the estimated core tensor. A core consistency score close to 100 indicates that the CP model is likely valid. If the core consistency is low, however, then the model either has components that describe noise or the data does not follow the model’s assumptions. So the core consistency can help determine if the chosen number of components is suitable.
 Parameters:
 cp_tensorCPTensor or tuple
TensorLystyle CPTensor object or tuple with weights as first argument and a tuple of components as second argument
 datasetnp.ndarray
Data tensor that the cp_tensor is fitted against
 normalisedBool (default=False)
If True, then the squared frobenius norm of the estimated core tensor is used to normalise the core consistency. Otherwise, the number of components is used.
If
normalised=False
, then the core consistency formula coincides with [BK03], and ifnormalised=True
, the core consistency formula coincides with that used in the NWay toolbox, and is unlikely to be less than 0. For core consistencies close to 100, the formulas approximately coincide.
 Returns:
 float
The core consistency
Examples
We can use the core consistency diagonstic to determine the correct number of components for a CP model. Here, we only fit one model, but in practice, you should fit multiple models and select the one with the lowest SSE (to account for local minima) before computing the core consistency.
>>> from tlviz.data import simulated_random_cp_tensor >>> from tensorly.decomposition import parafac >>> cp_tensor, dataset = simulated_random_cp_tensor((10,11,12), 3, seed=42) >>> # Fit many CP models with different number of components >>> for rank in range(1, 5): ... decomposition = parafac(dataset, rank=rank, random_state=42) ... cc = core_consistency(decomposition, dataset, normalised=True) ... print(f"No. components: {rank}  core consistency: {cc:.0f}") No. components: 1  core consistency: 100 No. components: 2  core consistency: 100 No. components: 3  core consistency: 81 No. components: 4  core consistency: 0
 tlviz.model_evaluation.estimate_core_tensor(factors, dataset)[source]¶
Efficient estimation of the Tucker core from a factor matrices and a data tensor.
 Parameters:
 factorstuple
Tuple of factor matrices used to estimate the core tensor from
 datasetnp.ndarray
The data tensor that the core tensor is estimated from
Notes
In the original paper, Papalexakis and Faloutsos [PF15] present an algorithm for 3way tensors. However, it is straightforward to generalise it to Nway tensors by using the inverse tensor product formula in [BD96].
 tlviz.model_evaluation.fit(cp_tensor, dataset, sum_squared_dataset=None)[source]¶
Compute the fit (1relative sum squared error) for a given cp_tensor.
 Parameters:
 cp_tensorCPTensor or tuple
TensorLystyle CPTensor object or tuple with weights as first argument and a tuple of components as second argument
 datasetndarray
Tensor approximated by
cp_tensor
 sum_squared_dataset: float (optional)
If
sum(dataset**2)
is already computed, you can optionally provide it using this argument to avoid unnecessary recalculation.
 Returns:
 float
The relative sum of squared error,
sum((X_hat  dataset)**2)/sum(dataset**2)
, whereX_hat
is the dense tensor represented bycp_tensor
Examples
Below, we create a random CP tensor and a random tensor and compute the sum of squared error for these two tensors.
>>> import tensorly as tl >>> from tensorly.random import random_cp >>> from tlviz.model_evaluation import fit >>> rng = tl.check_random_state(0) >>> cp = random_cp((4, 5, 6), 3, random_state=rng) >>> X = rng.random_sample((4, 5, 6)) >>> fit(cp, X) 0.5182592745038558
We can see that it is equal to 1  relative SSE
>>> from tlviz.model_evaluation import relative_sse >>> 1  relative_sse(cp, X) 0.5182592745038558
 tlviz.model_evaluation.predictive_power(cp_tensor, y, sklearn_estimator, mode=0, metric=None, axis=None)[source]¶
Use scikitlearn estimator to evaluate the predictive power of a factor matrix.
This is useful if you evaluate the components based on their predictive power with respect to some task.
 Parameters:
 factor_matrixndarray(ndim=2)
Factor matrix from a tensor decomposition model
 yndarray(ndim=1)
Prediction target for each row of the factor matrix in the given mode.
y
should have same length as the first dimension of this factor matrix (i.e. the length of the tensor along the given mode). sklearn_estimatorscikit learn estimator
Scikit learn estimator. Must have the
fit
andpredict
methods, and ifmetric
isNone
, then it should also have thescore
method. See https://scikitlearn.org/stable/developers/develop.html. modeint
Which mode to perform the scoring along
 metricCallable
Callable (typically function) with the signature
metric(y_true, y_pred)
, wherey_true=labels
andy_pred
is the predicted values obtained fromsklearn_estimator
. See https://scikitlearn.org/stable/developers/develop.html#specificmodels. axisint (optional)
Alias for mode, if set, then mode cannot be set.
 Returns:
 float
Score based on the estimator’s performance.
Examples
predictive_power
can be useful to evaluate the predictive power of a CP decomposition. To illustrate this, we start by creating a simulated CP tensor and a variable we want to predict that is linearly related to one of the factor matrices.>>> from tlviz.data import simulated_random_cp_tensor >>> import numpy as np >>> rng = np.random.default_rng(0) >>> cp_tensor, X = simulated_random_cp_tensor((30, 10, 10), 3, noise_level=0.1, seed=rng) >>> weights, (A, B, C) = cp_tensor >>> regression_coefficients = rng.standard_normal((3, 1)) >>> Y = A @ regression_coefficients
Next, we fit a PARAFAC model to this data
>>> from tensorly.decomposition import parafac >>> est_cp_tensor = parafac(X, 3)
Finally, we see how well the estimated decomposition can describe our target variable,
Y
. This will use the \(R^2\)coefficient for scoring, as that is the default scoring method for linear models.>>> from sklearn.linear_model import LinearRegression >>> from tlviz.model_evaluation import predictive_power >>> linear_regression = LinearRegression() >>> r_squared = predictive_power(cp_tensor, Y, linear_regression) >>> print(f"The R^2 coefficient is {r_squared:.2f}") The R^2 coefficient is 1.00
We can also specify our own scoring function
>>> from sklearn.metrics import max_error >>> highest_error = predictive_power(cp_tensor, Y, linear_regression, metric=max_error) >>> print(f"The maximum error is {highest_error:.2f}") The maximum error is 0.00
 tlviz.model_evaluation.relative_sse(cp_tensor, dataset, sum_squared_dataset=None)[source]¶
Compute the relative sum of squared error for a given cp_tensor.
 Parameters:
 cp_tensorCPTensor or tuple
TensorLystyle CPTensor object or tuple with weights as first argument and a tuple of components as second argument
 datasetndarray
Tensor approximated by
cp_tensor
 sum_squared_dataset: float (optional)
If
sum(dataset**2)
is already computed, you can optionally provide it using this argument to avoid unnecessary recalculation.
 Returns:
 float
The relative sum of squared error,
sum((X_hat  dataset)**2)/sum(dataset**2)
, whereX_hat
is the dense tensor represented bycp_tensor
Examples
Below, we create a random CP tensor and a random tensor and compute the sum of squared error for these two tensors.
>>> import tensorly as tl >>> from tensorly.random import random_cp >>> from tlviz.model_evaluation import relative_sse >>> rng = tl.check_random_state(0) >>> cp = random_cp((4, 5, 6), 3, random_state=rng) >>> X = rng.random_sample((4, 5, 6)) >>> relative_sse(cp, X) 0.4817407254961442
 tlviz.model_evaluation.sse(cp_tensor, dataset)[source]¶
Compute the sum of squared error for a given cp_tensor.
 Parameters:
 cp_tensorCPTensor or tuple
TensorLystyle CPTensor object or tuple with weights as first argument and a tuple of components as second argument
 datasetndarray
Tensor approximated by
cp_tensor
 Returns:
 float
The sum of squared error,
sum((X_hat  dataset)**2)
, whereX_hat
is the dense tensor represented bycp_tensor
Examples
Below, we create a random CP tensor and a random tensor and compute the sum of squared error for these two tensors.
>>> import tensorly as tl >>> from tensorly.random import random_cp >>> from tlviz.model_evaluation import sse >>> rng = tl.check_random_state(0) >>> cp = random_cp((4, 5, 6), 3, random_state=rng) >>> X = rng.random_sample((4, 5, 6)) >>> sse(cp, X) 18.948918157419186