Functions

The functions module contains all useful python functions and torch functions for the layers in the model.

anotherspdnet.functions.biMap(X: torch.Tensor, W: torch.Tensor, mode: str = 'einsum') → torch.Tensor

BiMap transform in a SPDnet layer according to the paper:

“A Riemannian Network for SPD Matrix Learning”, Huang et al AAAI Conference on Artificial Intelligence, 2017

The mapping is as follows:

\[\mathbf{Y} = \mathbf{W}^T \mathbf{X} \mathbf{W},\]

where \(\mathbf{X}\in\mathcal{S}_{n_{\mathrm{in}}}^{++}\) is the input SPD matrix and \(\mathbf{W}\in\mathcal{S}t(n_{\mathrm{in}}, n_{\mathrm{out}})\) is an orthogonal weight matrix. For convenience, the input W of this function is the transpose of the weight matrix.

Parameters:

• X (torch.Tensor of shape (..., n_in, n_in)) – Batches of input SPD matrices.

• W (torch.Tensor of shape (..., n_out, n_in)) – Batch of Stiefel weight matrices (or their transpose in case n_out > n_in). W.ndim must be 2 if X.ndim is 2. Then for higher number of dimensions, W.ndim must be X.ndim-1. (We will repeat the -3 dimension to match the number of matrices in X)

• mode (str) –

  Mode for the computation of the bilinear mapping. Choices are:

  ”einsum” or “bmm”. Default is “einsum”.

Returns:

Y

Return type:

torch.Tensor of shape (…, n_out, n_out)

anotherspdnet.functions.biMap_gradient(X: torch.Tensor, W: torch.Tensor, grad_output: torch.Tensor, mode: str = 'einsum') → Tuple[torch.Tensor, torch.Tensor]

Gradient of biMap towars input and weight matrix

Parameters:

• X (torch.Tensor of shape (..., n_in, n_in)) – Batches of input SPD matrices.

• W (torch.Tensor of shape (..., n_out, n_in)) – Batch of Stiefel weight matrices (or their transpose in case n_out > n_in). W.ndim must be 2 if X.ndim is 2. Then for higher number of dimesions, W.ndim must be X.ndim-1. (We will repeat the -3 dimension to match the number of matrices in X)

• grad_output (torch.Tensor of shape (..., n_out, n_out)) – Gradient of the loss with respect to the output of the layer.

• mode (str) –

  Mode for the computation of the bilinear mapping. Choices are:

  ”einsum” or “bmm”. Default is “einsum”.

Returns:

• grad_input (torch.Tensor of shape (…, n_in, n_in)) – Gradient of the loss with respect to the input of the layer.

• grad_weight (torch.Tensor of shape (…, n_out, n_in)) – Gradient of the loss with respect to the weight of the layer.

class anotherspdnet.functions.BiMapFunction(*args: Any, **kwargs: Any)

Bilinear mapping function.

static forward(ctx, X: torch.Tensor, W: torch.Tensor, mode: str = 'einsum') → torch.Tensor

Forward pass of the bilinear mapping function.

Parameters:

• ctx (torch.autograd.function._ContextMethodMixin) – Context object to save tensors for the backward pass.

• X (torch.Tensor of shape (n_bathces, n_matrices, n_in, n_in)) – Batch of several SPD matrices.

• W (torch.Tensor of shape (n_batches, n_in, n_out)) – Batch of Stiefel weight matrices.

• mode (str) –

  Mode for the computation of the bilinear mapping. Choices are:

  ”einsum” or “bmm”. Default is “einsum”.

Returns:

Y

Return type:

torch.Tensor of shape (n_batches, n_matrices, n_out, n_out)

static backward(ctx, grad_output: torch.Tensor) → Tuple[torch.Tensor, torch.Tensor, None]

Backward pass of the bilinear mapping function.

Parameters:

• ctx (torch.autograd.function._ContextMethodMixin) – Context object to retrieve tensors saved during the forward pass.

• grad_output (torch.Tensor of shape (n_batches, n_matrices, n_out, n_out)) – Gradient of the loss with respect to the output of the layer.

Returns:

• grad_input (torch.Tensor of shape (n_batches, n_matrices, n_in, n_in)) – Gradient of the loss with respect to the input of the layer.

• grad_weight (torch.Tensor of shape (n_batches, n_in, n_out)) – Gradient of the loss with respect to the weight of the layer.

anotherspdnet.functions.eig_operation(M: torch.Tensor, operation: Callable, eig_function: str = 'eigh', mm_mode: str = 'einsum', **kwargs) → Tuple[torch.Tensor, torch.Tensor, torch.Tensor]

Generic functions to compute an operation on the eigenvalues of a SPD matrix.

Parameters:

• M (torch.Tensor) – SPD matrix of shape (…, n_features, n_features)

• operation (Callable) – function to apply to the eigenvalues. Any unknown keyword args passed to this function are passed to the operation function.

• eig_function (str) – Name of the function to compute the eigenvalues and eigenvectors. Choices are: “eigh” or “eig”. Default is “eig” for torch.eig.

• mm_mode (str) –

  Mode for the computation of the matrix multiplication. Choices are:

  ”einsum” or “bmm”. Default is “einsum”.

• **kwargs – keyword arguments to pass to the operation function.

Returns:

• eigvals (torch.Tensor) – eigenvalues of shape (…, n_features) of the SPD matrices in M.

• eigvecs (torch.Tensor) – eigenvectors of shape (…, n_features, n_features) of the SPD matrices in M.

• result (torch.Tensor) – result of the operation on the eigenvalues of shape (…, n_matrices, n_features) and reconstructed from the eigenvectors.

anotherspdnet.functions.eig_operation_gradient_eigs(grad_output: torch.Tensor, eigvals: torch.Tensor, eigvecs: torch.Tensor, operation: Callable, grad_operation: Callable, mm_mode: str = 'einsum', **kwargs) → Tuple[torch.Tensor, torch.Tensor]

Gradient of an operation on the eigenvalues of a SPD matrix towards the eigenvalues and eigenvectors.

Parameters:

• grad_output (torch.Tensor) – gradient of the loss function wrt the output of the operation. of shape (…, n_features, n_features)

• eigvals (torch.Tensor) – eigenvalues of shape (…, n_features)

• eigvecs (torch.Tensor) – eigenvectors of shape (…, n_features, n_features)

• operation (Callable) – function to apply to the eigenvalues. Any unknown keyword args passed to this function are passed to the operation function.

• grad_operation (Callable) – function to apply to the gradient of the operation. Any unknown keyword args passed to this function are passed to the operation function.

• mm_mode (str) –

  Mode for the computation of the matrix multiplication. Choices are:

  ”einsum” or “bmm”. Default is “einsum”.

• **kwargs – keyword arguments to pass to the operation and gradient functions.

Returns:

• grad_eigvals (torch.Tensor) – gradient of the loss with respect to the eigenvalues of the layer.

• grad_eigvecs (torch.Tensor) – gradient of the loss with respect to the eigenvectors of the layer.

anotherspdnet.functions.construct_L_matrix(eigvals: torch.Tensor, eigvals_: torch.Tensor, deriveigvals: torch.Tensor) → torch.Tensor

Constructs the matrix L of brooks.

Parameters:

• eigvals (torch.Tensor of shape (..., n_features)) – eigenvalues of the SPD matrices

• eigvals – f(eigenvalues) of the SPD matrices

• deriveigvals (torch.Tensor of shape (..., n_features, n_features)) – f’(eigenvalues) of the SPD matrices

Returns:

L_matrix – matrix L of brooks

Return type:

torch.Tensor of shape (…, n_features, n_features)

anotherspdnet.functions.eig_operation_gradient_inputandbias(grad_output: torch.Tensor, eigvals: torch.Tensor, eigvecs: torch.Tensor, bias: torch.Tensor, operation: Callable, grad_operation: Callable, **kwargs) → Tuple[torch.Tensor, torch.Tensor]

Gradient of an operation on the eigenvalues of a SPD matrix towards the input and the bias for ReEigBias module.

Parameters:

• grad_output (torch.Tensor) – gradient of the loss function wrt the output of the operation. of shape (…, n_features, n_features)

• eigvals (torch.Tensor) – eigenvalues of shape (…, n_features)

• eigvecs (torch.Tensor) – eigenvectors of shape (…, n_features, n_features)

• bias (torch.Tensor) – bias of shape (…, n_features)

• operation (Callable) – function to apply to the eigenvalues. Any unknown keyword args passed to this function are passed to the operation function.

• grad_operation (Callable) – function to apply to the gradient of the operation. Any unknown keyword args passed to this function are passed to the operation function.

• **kwargs – keyword arguments to pass to the operation and gradient functions.

Returns:

• grad_input (torch.Tensor) – gradient of the loss with respect to the input of the layer.

• grad_bias (torch.Tensor) – gradient of the loss with respect to the bias of the layer.

anotherspdnet.functions.eig_operation_gradient(grad_output: torch.Tensor, eigvals: torch.Tensor, eigvecs: torch.Tensor, operation: Callable, grad_operation: Callable, mm_mode: str = 'einsum', formula: str = 'brooks', **kwargs) → torch.Tensor

Generic function to compute the gradient of an operation on the eigenvalues of a SPD matrix.

Parameters:

• grad_output (torch.Tensor) – gradient of the loss function wrt the output of the operation. of shape (…, n_features, n_features)

• eigvals (torch.Tensor) – eigenvalues of shape (…, n_features)

• eigvecs (torch.Tensor) – eigenvectors of shape (…, n_features, n_features)

• operation (Callable) – function to apply to the eigenvalues. Any unknown keyword args passed to this function are passed to the operation function.

• grad_operation (Callable) – function to apply to the gradient of the operation. Any unknown keyword args passed to this function are passed to the operation function.

• mm_mode (str) –

  Mode for the computation of the matrix multiplication. Choices are:

  ”einsum” or “bmm”. Default is “einsum”.

• formula (str) –

  Formula to compute the gradient of the operation. Choices are:

  ”brooks” or “ionescu”. Default is “brooks

• **kwargs – keyword arguments to pass to the operation and gradient functions.

Returns:

grad_input – gradient of the loss with respect to the input of the layer.

Return type:

torch.Tensor

anotherspdnet.functions.re_operation(eigvals: torch.Tensor, eps: float) → torch.Tensor

Rectification of the eigenvalues of a SPD matrix.

Parameters:

• eigvals (torch.Tensor of shape (..., n_features)) – eigenvalues of the SPD matrices

• eps (float) – Value for the rectification of the eigenvalues.

Returns:

eigvals_rect – eigenvalues of the SPD matrices with rectified eigenvalues.

Return type:

torch.Tensor of shape (…, n_features)

anotherspdnet.functions.re_operation_gradient(eigvals: torch.Tensor, eps: float, dtype: torch.dtype = torch.float64) → torch.Tensor

Gradient of the rectification of the eigenvalues of a SPD matrix.

Parameters:

• eigvals (torch.Tensor of shape (..., n_features)) – eigenvalues of the SPD matrices

• eps (float) – Value for the rectification of the eigenvalues.

• dtype (Callable) – Casting type of the gradient. Default is torch.float64.

Returns:

eigvals_rect – eigenvalues of the SPD matrices with rectified eigenvalues.

Return type:

torch.Tensor of shape (…, n_features)

class anotherspdnet.functions.ReEigBiasFunction(*args: Any, **kwargs: Any)

ReEigBias function.

static forward(ctx, M: torch.Tensor, bias: torch.Tensor, eps: float) → torch.Tensor

static backward(ctx, grad_output: torch.Tensor) → Tuple[torch.Tensor, torch.Tensor, None]

class anotherspdnet.functions.ReEigFunction(*args: Any, **kwargs: Any)

ReEig function.

static forward(ctx, M: torch.Tensor, eps: float, mm_mode: str = 'einsum', eig_function: str = 'eigh', formula: str = 'brooks') → torch.Tensor

Forward pass of the ReEig function.

Parameters:

• ctx (torch.autograd.function._ContextMethodMixin) – Context object to save tensors for the backward pass.

• M (torch.Tensor of shape (..., n_features, n_features)) – Batch of SPD matrices.

• eps (float) – Value for the rectification of the eigenvalues.

• mm_mode (str) –

  Mode for the computation of the matrix multiplication. Choices are:

  ”einsum” or “bmm”. Default is “einsum”.

• eig_function (str) – Name of the function to compute the eigenvalues and eigenvectors. Choices are: “eigh” or “eig”.

• formula (str) –

  Formula to compute the gradient of the operation. Choices are:

  ”brooks” or “ionescu”. Default is “brooks”

Returns:

M_rect – Batch of SPD matrices with rectified eigenvalues.

Return type:

torch.Tensor of shape (…, n_features, n_features)

static backward(ctx, grad_output: torch.Tensor) → Tuple[torch.Tensor, None, None, None, None]

Backward pass of the ReEig function.

Parameters:

• ctx (torch.autograd.function._ContextMethodMixin) – Context object to retrieve tensors saved during the forward pass.

• grad_output (torch.Tensor of shape (..., n_features, n_features)) – Gradient of the loss with respect to the output of the layer.

Returns:

grad_input – Gradient of the loss with respect to the input of the layer.

Return type:

torch.Tensor of shape (…, n_features, n_features)

class anotherspdnet.functions.LogEigFunction(*args: Any, **kwargs: Any)

LogEig function.

static forward(ctx, M: torch.Tensor, mm_mode: str = 'einsum', eig_function: str = 'eigh', formula: str = 'brooks') → torch.Tensor

Forward pass of the logEig function.

Parameters:

• ctx (torch.autograd.function._ContextMethodMixin) – Context object to save tensors for the backward pass.

• M (torch.Tensor of shape (..., n_features, n_features)) – Batch of SPD matrices.

• mm_mode (str) –

  Mode for the computation of the matrix multiplication. Choices are:

  ”einsum” or “bmm”. Default is “einsum”.

• eig_function (str) – Name of the function to compute the eigenvalues and eigenvectors. Choices are: “eigh” or “eig”.

• formula (str) –

  Formula to compute the gradient of the operation. Choices are:

  ”brooks” or “ionescu”. Default is “brooks”

Returns:

M_rect – Batch of SPD matrices with rectified eigenvalues.

Return type:

torch.Tensor of shape (…, n_features, n_features)

static backward(ctx, grad_output: torch.Tensor) → Tuple[torch.Tensor, None, None, None]

Backward pass of the logEig function.

Parameters:

• ctx (torch.autograd.function._ContextMethodMixin) – Context object to retrieve tensors saved during the forward pass.

• grad_output (torch.Tensor of shape (..., n_features, n_features)) – Gradient of the loss with respect to the output of the layer.

Returns:

grad_input – Gradient of the loss with respect to the input of the layer.

Return type:

torch.Tensor of shape (…, n_features, n_features)

class anotherspdnet.functions.ExpEigFunction(*args: Any, **kwargs: Any)

ExpEig function.

static forward(ctx, M: torch.Tensor, mm_mode: str = 'einsum', eig_function: str = 'eigh') → torch.Tensor

Forward pass of the ExpEig function.

Parameters:

• ctx (torch.autograd.function._ContextMethodMixin) – Context object to save tensors for the backward pass.

• M (torch.Tensor of shape (..., n_features, n_features)) – Batch of SPD matrices.

• mm_mode (str) –

  Mode for the computation of the matrix multiplication. Choices are:

  ”einsum” or “bmm”. Default is “einsum”.

• eig_function (str) – Name of the function to compute the eigenvalues and eigenvectors. Choices are: “eigh” or “eig”.

Returns:

M_exp – Batch of SPD matrices with exponentiated eigenvalues.

Return type:

torch.Tensor of shape (…, n_features, n_features)

static backward(ctx, grad_output: torch.Tensor) → Tuple[torch.Tensor, None, None]

Backward pass of the ExpEig function.

Parameters:

• ctx (torch.autograd.function._ContextMethodMixin) – Context object to retrieve tensors saved during the forward pass.

• grad_output (torch.Tensor of shape (..., n_features, n_features)) – Gradient of the loss with respect to the output of the layer.

Returns:

grad_input – Gradient of the loss with respect to the input of the layer.

Return type:

torch.Tensor of shape (…, n_features, n_features)

class anotherspdnet.functions.SqrtmEigFunction(*args: Any, **kwargs: Any)

SqrtmEig function.

static forward(ctx, M: torch.Tensor, mm_mode: str = 'einsum', eig_function: str = 'eigh') → torch.Tensor

Forward pass of the SqrtmEig function.

Parameters:

• ctx (torch.autograd.function._ContextMethodMixin) – Context object to save tensors for the backward pass.

• M (torch.Tensor of shape (..., n_features, n_features)) – Batch of SPD matrices.

• mm_mode (str) –

  Mode for the computation of the matrix multiplication. Choices are:

  ”einsum” or “bmm”. Default is “einsum”.

• eig_function (str) – Name of the function to compute the eigenvalues and eigenvectors. Choices are: “eigh” or “eig”.

Returns:

M_sqrtm – Batch of SPD matrices with square root of eigenvalues.

Return type:

torch.Tensor of shape (…, n_features, n_features)

static backward(ctx, grad_output: torch.Tensor) → Tuple[torch.Tensor, None, None]

Backward pass of the SqrtmEig function.

Parameters:

• ctx (torch.autograd.function._ContextMethodMixin) – Context object to retrieve tensors saved during the forward pass.

• grad_output (torch.Tensor of shape (..., n_features, n_features)) – Gradient of the loss with respect to the output of the layer.

Returns:

grad_input – Gradient of the loss with respect to the input of the layer.

Return type:

torch.Tensor of shape (…, n_features, n_features)

class anotherspdnet.functions.InvSqrtmEigFunction(*args: Any, **kwargs: Any)

static forward(ctx, M: torch.Tensor, mm_mode: str = 'einsum', eig_function: str = 'eigh') → torch.Tensor

Forward pass of the InvSqrtmEig function.

Parameters:

• ctx (torch.autograd.function._ContextMethodMixin) – Context object to save tensors for the backward pass.

• M (torch.Tensor of shape (..., n_features, n_features)) – Batch of SPD matrices.

• mm_mode (str) –

  Mode for the computation of the matrix multiplication. Choices are:

  ”einsum” or “bmm”. Default is “einsum”.

• eig_function (str) – Name of the function to compute the eigenvalues and eigenvectors. Choices are: “eigh” or “eig”.

Returns:

M_inv_sqrtm – Batch of SPD matrices with inverse square root of eigenvalues.

Return type:

torch.Tensor of shape (…, n_features, n_features)

static backward(ctx, grad_output: torch.Tensor) → Tuple[torch.Tensor, None, None]

Backward pass of the InvSqrtmEig function.

Parameters:

• ctx (torch.autograd.function._ContextMethodMixin) – Context object to retrieve tensors saved during the forward pass.

• grad_output (torch.Tensor of shape (..., n_features, n_features)) – Gradient of the loss with respect to the output of the layer.

Returns:

grad_input – Gradient of the loss with respect to the input of the layer.

Return type:

torch.Tensor of shape (…, n_features, n_features)

class anotherspdnet.functions.SqrtmAndInvSqrtmEigFunction(*args: Any, **kwargs: Any)

Function that computes the square root and inverse square root of the eigenvalues of a SPD matrix.

static forward(ctx, M: torch.Tensor, mm_mode: str = 'einsum', eig_function: str = 'eigh') → Tuple[torch.Tensor, torch.Tensor]

static backward(ctx, grad_output_sqrtm: torch.Tensor, grad_output_inv_sqrtm: torch.Tensor) → Tuple[torch.Tensor, None, None]

class anotherspdnet.functions.PowerEigFunction(*args: Any, **kwargs: Any)

PowerEig function.

static forward(ctx, M: torch.Tensor, p: float) → torch.Tensor

static backward(ctx, grad_output: torch.Tensor) → Tuple[torch.Tensor, None]

anotherspdnet.functions.vec_batch(X: torch.Tensor) → torch.Tensor

Vectorize a batch of tensors along last two dimensions.

Parameters:

X (torch.Tensor of shape (..., n, k)) – Batch of matrices.

Returns:

X_vec – Batch of vectorized matrices.

Return type:

torch.Tensor of shape (…, n*k)

anotherspdnet.functions.unvec_batch(X_vec: torch.Tensor, n: int) → torch.Tensor

Unvectorize a batch of tensors along last dimension. :param X_vec: Batch of vectorized matrices. :type X_vec: torch.Tensor of shape (…, n*k) :param n: Number of rows of the matrices. :type n: int

Returns:

X – Batch of matrices.

Return type:

torch.Tensor of shape (…, n, k)

anotherspdnet.functions.vech_batch(X: torch.Tensor) → torch.Tensor

Vectorize the lower triangular part of a batch of square matrices.

Parameters:

X (torch.Tensor of shape (..., n, n)) – Batch of matrices.

Returns:

X_vech – Batch of vectorized matrices.

Return type:

torch.Tensor of shape (…, n*(n+1)//2)

anotherspdnet.functions.unvech_batch(X_vech: torch.Tensor) → torch.Tensor

Unvectorize a batch of tensors along last dimension. :param X_vech: Batch of vectorized matrices. :type X_vech: torch.Tensor of shape (…, n*(n+1)//2)

Returns:

X – Batch of matrices.

Return type:

torch.Tensor of shape (…, n, n)

anotherspdnet.functions.spd_affine_invariant_geodesic(X: torch.Tensor, Y: torch.Tensor, t: float) → torch.Tensor

Affine invariant geodesic between two SPD matrices.

Parameters:

• X (torch.Tensor of shape (..., n, n)) – Batch of SPD matrices.

• Y (torch.Tensor of shape (..., n, n)) – Batch of SPD matrices.

• t (float) – Parameter of the geodesic. between 0 and 1.

Returns:

Z – Batch of SPD matrices.

Return type:

torch.Tensor of shape (…, n, n)

anotherspdnet.functions.spd_affine_invariant_distance(X: torch.Tensor, Y: torch.Tensor) → torch.Tensor

Affine invariant distance between two SPD matrices.

Parameters:

• X (torch.Tensor of shape (..., n, n)) – Batch of SPD matrices.

• Y (torch.Tensor of shape (..., n, n)) – Batch of SPD matrices.

Returns:

d – Batch of distances.

Return type:

torch.Tensor of shape (…,)