Tensor/matrix operations
Transpose
Transpose is an important operation defined for matrices or tensors. For a matrix, the transpose is defined as follows:

Here, AT denotes the transpose of A.
An example of the transpose operation can be illustrated as follows:

After the transpose operation:

For a tensor, transpose can be seen as permuting the dimensions order. For example, let's define a tensor S, as shown here:

Now a transpose operation (out of many) can be defined as follows:

Multiplication
Matrix multiplication is another important operation that appears quite frequently in linear algebra.
Given the matrices and
, the multiplication of A and B is defined as follows:

Here, .
Consider this example:


This gives

, and the value of C is as follows:

Element-wise multiplication
Element-wise matrix multiplication (or the Hadamard product) is computed for two matrices that have the same shape. Given the matrices and
, the element-wise multiplication of A and B is defined as follows...