Skip to content

Templates

Here we provide a list of templates that can be used to create new structured matrices. This list is meant for internal purposes only. It exists because it is more convenient to read the rendered LaTeX code rather than the docstring source.

singd.structures.blockdiagonal.BlockDiagonalMatrixTemplate 

BlockDiagonalMatrixTemplate(blocks: Tensor, last: Tensor)

Bases: StructuredMatrix

Template class for symmetric block-diagonal dense matrix.

Note

This is a template class. To define an actual class, inherit from this class, then specify the BLOCK_DIM class attribute. See the example below.

Block-diagonal matrices have the following structure:

\( \begin{pmatrix} \mathbf{A}_1 & \mathbf{0} & \cdots & \cdots & \mathbf{0} \\ \mathbf{0} & \mathbf{A}_2 & \mathbf{0} & \cdots & \mathbf{0} \\ \vdots & \ddots & \ddots & \ddots & \vdots \\ \mathbf{0} & \cdots & \mathbf{0} & \mathbf{A}_N & \mathbf{0} & \\ \mathbf{0} & \cdots & \cdots & \mathbf{0} & \mathbf{B} \end{pmatrix} \in \mathbb{R}^{(N D + D') \times (N D + D')} \)

where

  • \(\mathbf{A}_n = \mathbf{A}_n^\top \in \mathbb{R}^{D \times D}\) are symmetric matrices containing the diagonal blocks of block dimension \(D\).
  • \(\mathbf{B} = \mathbf{B}^\top \in \mathbb{R}^{D' \times D'}\) is a symmetric matrix containing the last block of dimension \(D' < D\), which can be empty if \(D\) divides the matrix dimension.

Attributes:

  • BLOCK_DIM (int) –

    The dimension of a diagonal block (\(D\)).

Examples:

>>> from torch import ones
>>>
>>> class Block2DiagonalMatrix(BlockDiagonalMatrixTemplate):
...     '''Class to represent block-diagonal matrices with 2x2 blocks.'''
...     BLOCK_DIM = 2
>>>
>>> # A block-diagonal matrix of total dimension 7x7
>>> blocks, last = ones(3, 2, 2), 2 * ones(1, 1)
>>> mat = Block2DiagonalMatrix(blocks, last)
>>> mat.to_dense()
tensor([[1., 1., 0., 0., 0., 0., 0.],
        [1., 1., 0., 0., 0., 0., 0.],
        [0., 0., 1., 1., 0., 0., 0.],
        [0., 0., 1., 1., 0., 0., 0.],
        [0., 0., 0., 0., 1., 1., 0.],
        [0., 0., 0., 0., 1., 1., 0.],
        [0., 0., 0., 0., 0., 0., 2.]])

Store the matrix internally.

Parameters:

  • blocks (Tensor) –

    The diagonal blocks \(\{\mathbf{A}_n = \mathbf{A}_n^\top\}_{n = 1}^N\), supplied as a tensor of shape [N, BLOCK_DIM, BLOCK_DIM]. If there are no blocks, this argument has shape [0, BLOCK_DIM, BLOCK_DIM].

  • last (Tensor) –

    The last block \(\mathbf{B} = \mathbf{B}^\top\) which contains the remaining matrix if BLOCK_DIM does not divide the matrix dimension. Has shape [last_dim, last_dim] where last_dim may be zero.

Note

For performance reasons, symmetry is not checked internally and must be ensured by the caller.

Raises:

  • ValueError –

    If the passed tensors have incorrect shape.

Source code in singd/structures/blockdiagonal.py 
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
def __init__(self, blocks: Tensor, last: Tensor) -> None:
    r"""Store the matrix internally.

    Args:
        blocks: The diagonal blocks
            \(\{\mathbf{A}_n = \mathbf{A}_n^\top\}_{n = 1}^N\),
            supplied as a tensor of shape `[N, BLOCK_DIM, BLOCK_DIM]`. If there are
            no blocks, this argument has shape `[0, BLOCK_DIM, BLOCK_DIM]`.
        last: The last block \(\mathbf{B} = \mathbf{B}^\top\) which contains the
            remaining matrix if `BLOCK_DIM` does not divide the matrix dimension.
            Has shape `[last_dim, last_dim]` where `last_dim` may be zero.

    Note:
        For performance reasons, symmetry is not checked internally and must
        be ensured by the caller.

    Raises:
        ValueError: If the passed tensors have incorrect shape.
    """
    super().__init__()
    if blocks.ndim != 3:
        raise ValueError(
            f"Diagonal blocks must be 3-dimensional, got {blocks.ndim}."
        )
    if blocks.shape[1] != blocks.shape[2] != self.BLOCK_DIM:
        raise ValueError(
            f"Diagonal blocks must be square with dimension {self.BLOCK_DIM},"
            f" got {blocks.shape[1:]} instead."
        )
    if last.ndim != 2 or last.shape[0] != last.shape[1]:
        raise ValueError(f"Last block must be square, got {last.shape}.")
    if last.shape[0] >= self.BLOCK_DIM or last.shape[1] >= self.BLOCK_DIM:
        raise ValueError(
            f"Last block must have dimension at most {self.BLOCK_DIM},"
            f" got {last.shape} instead."
        )

    self._blocks: Tensor
    self.register_tensor(blocks, "_blocks")

    self._last: Tensor
    self.register_tensor(last, "_last")

singd.structures.hierarchical.HierarchicalMatrixTemplate 

HierarchicalMatrixTemplate(A: Tensor, B: Tensor, C: Tensor, D: Tensor, E: Tensor)

Bases: StructuredMatrix

Template class for creating hierarchical matrices.

Note

This is a template class. To define an actual class, inherit from this class, then specify the MAX_K1 and MAX_K2 class attributes. See the example below.

Hierarchical matrices have the following structure:

\( \begin{pmatrix} \mathbf{A} & \mathbf{B}_1 & \mathbf{B}_2 \\ \mathbf{0} & \mathbf{C} & \mathbf{0} \\ \mathbf{0} & \mathbf{D} & \mathbf{E} \\ \end{pmatrix} \in \mathbb{R}^{K \times K} \)

where (denoting \(\mathbf{B} := \begin{pmatrix}\mathbf{B}_1 & \mathbf{B}_2\end{pmatrix}\))

  • \(\mathbf{A} \in \mathbb{R}^{K_1 \times K_1}\) is dense symmetric
  • \(\mathbf{B} \in \mathbb{R}^{K_1 \times (K - K_1)}\) is dense rectangular
  • \(\mathbf{C} \in \mathbb{R}^{(K - K_2 - K_1) \times (K - K_2 - K_1)}\) is diagonal
  • \(\mathbf{D} \in \mathbb{R}^{K_2 \times (K - K_2 - K_1)}\) is dense rectangular
  • \(\mathbf{E} \in \mathbb{R}^{K_2 \times K_2}\) is dense symmetric

For fixed values of \(K_1, K_2\), if the matrix to be represented is not big enough to fit all structures, we use the following prioritization:

  1. If \(K \le K_1\), start by filling \(\mathbf{A}\).
  2. If \(K_1 < K \le K_1+K_2\), fill \(\mathbf{A}\) and start filling \(\mathbf{B}\) and \(\mathbf{E}\).
  3. If \(K_1+K_2 < K\), use all structures.

Attributes:

  • MAX_K1 (int) –

    Maximum dimension \(K_1\) of the top left block \(\mathbf{A}\).

  • MAX_K2 (int) –

    Maximum dimension \(K_2\) of the bottom right block \(\mathbf{E}\).

Examples:

>>> from torch import ones
>>>
>>> class Hierarchical2_3Matrix(HierarchicalMatrixTemplate):
...     '''Hierarchical matrix with 2x2 top left and 3x3 bottom right block.'''
...     MAX_K1 = 2
...     MAX_K2 = 3
>>>
>>> # A hierarchical matrix with total dimension K=7
>>> A, C, E = ones(2, 2), 3 * ones(2), 5 * ones(3, 3)
>>> B, D = 2 * ones(2, 5), 4 * ones(3, 2)
>>> mat = Hierarchical2_3Matrix(A, B, C, D, E)
>>> mat.to_dense()
tensor([[1., 1., 2., 2., 2., 2., 2.],
        [1., 1., 2., 2., 2., 2., 2.],
        [0., 0., 3., 0., 0., 0., 0.],
        [0., 0., 0., 3., 0., 0., 0.],
        [0., 0., 4., 4., 5., 5., 5.],
        [0., 0., 4., 4., 5., 5., 5.],
        [0., 0., 4., 4., 5., 5., 5.]])

Store the structural components internally.

Parameters:

  • A (Tensor) –

    Dense symmetric matrix of shape [K1, K1] or smaller representing \(\mathbf{A}\).

  • B (Tensor) –

    Dense rectangular matrix of shape [K1, K - K1] representing \(\mathbf{B}\).

  • C (Tensor) –

    Vector of shape [K - K1 - K2] representing the diagonal of \(\mathbf{C}\).

  • D (Tensor) –

    Dense rectangular matrix of shape [K2, K - K1 - K2] representing \(\mathbf{D}\).

  • E (Tensor) –

    Dense symmetric matrix of shape [K2, K2] or smaller representing \(\mathbf{E}\).

Note

For performance reasons, symmetry is not checked internally and must be ensured by the caller.

Raises:

  • ValueError –

    If the shapes of the arguments are invalid.

Source code in singd/structures/hierarchical.py 
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
def __init__(self, A: Tensor, B: Tensor, C: Tensor, D: Tensor, E: Tensor):
    r"""Store the structural components internally.

    Args:
        A: Dense symmetric matrix of shape `[K1, K1]` or smaller representing
            \(\mathbf{A}\).
        B: Dense rectangular matrix of shape `[K1, K - K1]` representing
            \(\mathbf{B}\).
        C: Vector of shape `[K - K1 - K2]` representing the diagonal of
            \(\mathbf{C}\).
        D: Dense rectangular matrix of shape `[K2, K - K1 - K2]` representing
            \(\mathbf{D}\).
        E: Dense symmetric matrix of shape `[K2, K2]` or smaller representing
            \(\mathbf{E}\).

    Note:
        For performance reasons, symmetry is not checked internally and must
        be ensured by the caller.

    Raises:
        ValueError: If the shapes of the arguments are invalid.
    """
    super().__init__()
    if A.ndim != 2 or B.ndim != 2 or C.ndim != 1 or D.ndim != 2 or E.ndim != 2:
        raise ValueError(
            "Invalid tensor dimensions. Expected 2, 2, 1, 2, 2."
            + f" Got {A.ndim}, {B.ndim}, {C.ndim}, {D.ndim}, {E.ndim}."
        )
    self._check_square(A, name="A")
    self._check_square(E, name="E")

    self.K1 = A.shape[0]
    self.K2 = E.shape[0]
    self.diag_dim = C.shape[0]
    self.dim = self.K1 + self.K2 + self.diag_dim

    if A.shape[0] > self.MAX_K1 or E.shape[0] > self.MAX_K2:
        raise ValueError(
            f"Expected A, E to be smaller than {self.MAX_K1}, {self.MAX_K2}."
            + f" Got {A.shape}, {E.shape}."
        )
    if D.shape != (self.K2, self.diag_dim):
        raise ValueError(
            f"Expected D to have shape {self.K2, self.diag_dim}. Got {D.shape}."
        )
    if B.shape != (self.K1, self.diag_dim + self.K2):
        raise ValueError(
            f"Expected B to have shape {self.K1, self.diag_dim + self.K2}."
            + " Got {B.shape}."
        )

    self.A: Tensor
    self.register_tensor(A, "A")

    self.B: Tensor
    self.register_tensor(B, "B")

    self.C: Tensor
    self.register_tensor(C, "C")

    self.D: Tensor
    self.register_tensor(D, "D")

    self.E: Tensor
    self.register_tensor(E, "E")

singd.structures.recursive.RecursiveBottomLeftMatrixTemplate 

RecursiveBottomLeftMatrixTemplate(A: StructuredMatrix, B: Tensor, C: StructuredMatrix)

Bases: RecursiveStructuredMatrix

Template to define recursive structured matrices with bottom left dense block.

Note

This is a template class. To define an actual class, inherit from this class, then specify the attributes MAX_DIMS, CLS_A, and CLS_C. See the example below.

This matrix is defined by

\(\begin{pmatrix} \mathbf{A} & \mathbf{0} \\ \mathbf{B} & \mathbf{C} \end{pmatrix}\)

where

  • \(\mathbf{A}, \mathbf{C}\) are structured matrices (which can be recursive).
  • \(\mathbf{B}\) is a dense rectangular matrix.

Attributes:

  • MAX_DIMS (Tuple[Union[int, float], Union[int, float]]) –

    A tuple that contains an integer and a float('inf') which indicate the maximum dimensions of \(\mathbf{A}\) and \(\mathbf{C}\). For example, (10, float('inf')) means that only \(\mathbf{A}\) will be used for dimensions up to 10, and \(\mathbf{C}\) will be used in addition for larger dimensions.

  • CLS_A (Type[StructuredMatrix]) –

    Structured matrix class used for the top left block \(\mathbf{A}\).

  • CLS_C (Type[StructuredMatrix]) –

    Structured matrix class used for the the bottom right block \(\mathbf{C}\).

Examples:

>>> from torch import ones
>>> from singd.structures.dense import DenseMatrix
>>> from singd.structures.diagonal import DiagonalMatrix
>>>
>>> class Dense3DiagonalBottomLeftMatrix(RecursiveBottomLeftMatrixTemplate):
...     '''Structured matrix with 3 left columns and right diagonal part.'''
...     MAX_DIMS = (3, float('inf'))
...     CLS_A = DenseMatrix
...     CLS_C = DiagonalMatrix
>>>
>>> # A 5x5 matrix with 3 left columns and right diagonal part
>>> A = DenseMatrix(ones(3, 3))
>>> B = 2 * ones(2, 3)
>>> C = DiagonalMatrix(3 * ones(2))
>>> mat = Dense3DiagonalBottomLeftMatrix(A, B, C)
>>> mat.to_dense()
tensor([[1., 1., 1., 0., 0.],
        [1., 1., 1., 0., 0.],
        [1., 1., 1., 0., 0.],
        [2., 2., 2., 3., 0.],
        [2., 2., 2., 0., 3.]])

Store the matrix internally.

Parameters:

  • A (StructuredMatrix) –

    Structured matrix representing the top left block \(\mathbf{A}\).

  • B (Tensor) –

    Rectangular tensor representing the bottom left block \(\mathbf{B}\).

  • C (StructuredMatrix) –

    Structured matrix representing the bottom right block \(\mathbf{C}\).

Note

For performance reasons, symmetry is not checked internally and must be ensured by the caller.

Raises:

  • ValueError –

    If the dimensions of the blocks do not match or the structured matrices are of wrong type.

Source code in singd/structures/recursive.py 
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
def __init__(self, A: StructuredMatrix, B: Tensor, C: StructuredMatrix):
    r"""Store the matrix internally.

    Args:
        A: Structured matrix representing the top left block \(\mathbf{A}\).
        B: Rectangular tensor representing the bottom left block \(\mathbf{B}\).
        C: Structured matrix representing the bottom right block \(\mathbf{C}\).

    Note:
        For performance reasons, symmetry is not checked internally and must
        be ensured by the caller.

    Raises:
        ValueError: If the dimensions of the blocks do not match or the structured
            matrices are of wrong type.
    """
    super().__init__()
    if not isinstance(A, self.CLS_A) or not isinstance(C, self.CLS_C):
        raise ValueError(
            f"Matrices A and C must be of type {self.CLS_A} and "
            f"{self.CLS_C}, respectively. Got {type(A)} and {type(C)}."
        )

    # TODO Add a `dim` property to make this cheaper
    dim_A, dim_C = A.to_dense().shape[0], C.to_dense().shape[0]
    if B.shape != (dim_C, dim_A):
        raise ValueError(f"Shape of `B` ({B.shape}) should be ({(dim_A, dim_C)}).")

    max_dim_A, max_dim_C = self.MAX_DIMS
    if dim_A > max_dim_A:
        raise ValueError(f"Dim. of A ({dim_A}) exceeds max dim. ({max_dim_A}).")
    if dim_C > max_dim_C:
        raise ValueError(f"Dim. of C ({dim_A}) exceeds max dim. ({max_dim_C}).")

    self.A: StructuredMatrix
    self.register_substructure(A, "A")

    self.B: Tensor
    self.register_tensor(B, "B")

    self.C: StructuredMatrix
    self.register_substructure(C, "C")

singd.structures.recursive.RecursiveTopRightMatrixTemplate 

RecursiveTopRightMatrixTemplate(A: StructuredMatrix, B: Tensor, C: StructuredMatrix)

Bases: RecursiveStructuredMatrix

Template to define recursive structured matrices with top right dense block.

Note

This is a template class. To define an actual class, inherit from this class, then specify the attributes MAX_DIMS, CLS_A, and CLS_C. See the example below.

This matrix is defined by

\(\begin{pmatrix} \mathbf{A} & \mathbf{B} \\ \mathbf{0} & \mathbf{C} \end{pmatrix}\)

where

  • \(\mathbf{A}, \mathbf{C}\) are structured matrices (which can be recursive).
  • \(\mathbf{B}\) is a dense rectangular matrix.

Attributes:

  • MAX_DIMS (Tuple[Union[int, float], Union[int, float]]) –

    A tuple that contains an integer and a float('inf') which indicate the maximum dimensions of \(\mathbf{A}\) and \(\mathbf{C}\). For example, (10, float('inf')) means that only \(\mathbf{A}\) will be used for dimensions up to 10, and \(\mathbf{C}\) will be used in addition for larger dimensions.

  • CLS_A (Type[StructuredMatrix]) –

    Structured matrix class used for the top left block \(\mathbf{A}\).

  • CLS_C (Type[StructuredMatrix]) –

    Structured matrix class used for the the bottom right block \(\mathbf{B}\).

Examples:

>>> from torch import ones
>>> from singd.structures.dense import DenseMatrix
>>> from singd.structures.diagonal import DiagonalMatrix
>>>
>>> class Dense3DiagonalTopRightMatrix(RecursiveTopRightMatrixTemplate):
...     '''Structured matrix with 3 dense rows upper and lower diagonal part.'''
...     MAX_DIMS = (3, float('inf'))
...     CLS_A = DenseMatrix
...     CLS_C = DiagonalMatrix
>>>
>>> # A 5x5 matrix with 3 dense rows in the upper and lower diagonal part
>>> A = DenseMatrix(ones(3, 3))
>>> B = 2 * ones(3, 2)
>>> C = DiagonalMatrix(3 * ones(2))
>>> mat = Dense3DiagonalTopRightMatrix(A, B, C)
>>> mat.to_dense()
tensor([[1., 1., 1., 2., 2.],
        [1., 1., 1., 2., 2.],
        [1., 1., 1., 2., 2.],
        [0., 0., 0., 3., 0.],
        [0., 0., 0., 0., 3.]])

Store the matrix internally.

Parameters:

  • A (StructuredMatrix) –

    Structured matrix representing the top left block \(\mathbf{A}\).

  • B (Tensor) –

    Rectangular tensor representing the top right block \(\mathbf{B}\).

  • C (StructuredMatrix) –

    Structured matrix representing the bottom right block \(\mathbf{C}\).

Note

For performance reasons, symmetry is not checked internally and must be ensured by the caller.

Raises:

  • ValueError –

    If the dimensions of the blocks do not match or the structured matrices are of wrong type.

Source code in singd/structures/recursive.py 
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
def __init__(self, A: StructuredMatrix, B: Tensor, C: StructuredMatrix):
    r"""Store the matrix internally.

    Args:
        A: Structured matrix representing the top left block \(\mathbf{A}\).
        B: Rectangular tensor representing the top right block \(\mathbf{B}\).
        C: Structured matrix representing the bottom right block \(\mathbf{C}\).

    Note:
        For performance reasons, symmetry is not checked internally and must
        be ensured by the caller.

    Raises:
        ValueError: If the dimensions of the blocks do not match or the
            structured matrices are of wrong type.
    """
    super().__init__()
    if not isinstance(A, self.CLS_A) or not isinstance(C, self.CLS_C):
        raise ValueError(
            f"Matrices A and C must be of type {self.CLS_A} and "
            f"{self.CLS_C}, respectively. Got {type(A)} and {type(C)}."
        )

    # TODO Add a `dim` property to make this cheaper
    dim_A, dim_C = A.to_dense().shape[0], C.to_dense().shape[0]
    if B.shape != (dim_A, dim_C):
        raise ValueError(f"Shape of `B` ({B.shape}) should be ({(dim_A, dim_C)}).")

    max_dim_A, max_dim_C = self.MAX_DIMS
    if dim_A > max_dim_A:
        raise ValueError(f"Dim. of A ({dim_A}) exceeds max dim. ({max_dim_A}).")
    if dim_C > max_dim_C:
        raise ValueError(f"Dim. of C ({dim_A}) exceeds max dim. ({max_dim_C}).")

    self.A: StructuredMatrix
    self.register_substructure(A, "A")

    self.B: Tensor
    self.register_tensor(B, "B")

    self.C: StructuredMatrix
    self.register_substructure(C, "C")