It is important that we understand some of the terminology used within the Microsoft CNTK toolkit. Let’s look at some of that terminology now:
- Tensors: All CNTK inputs, outputs and parameters are organized as tensors. It should also be noted that minibatches are tensors as well.
- Rank: Each tensor has a rank. Scalars are tensors with a rank of 0, vectors are tensors and have a rank of 1, and matrices are tensors with a rank of 2.
- Static axis: The dimensions listed in 2 are referred to as axes. Every tensor has static and dynamic axes. A Static axis has the same length throughout its entire life.
- Dynamic axis: Dynamic axes, however, can vary their length from instance to instance. Their length is typically not known before each minibatch is presented. Additionally, they may be ordered.
- Minibatch: A minibatch is also a tensor. It has a dynamic axis, which...
