Core Tensor Operators

This page contains the list of core tensor operator primitives pre-defined in NNVM. The core tensor operator primitives(nnvm.top) covers typical workloads in deep learning. They can represent workloads in front-end frameworks, and provide basic building blocks for optimization. Since deep learning is a fast evolving field and it is that possible to have operators that are not in here. NNVM is designed for this problem and can easily new operators without changing the core library.

Note

Each operator node in the graph IR contains the following two kinds of parameters.

  • inputs: positional list of input tensors
  • attrs: attributes about operator(e.g. kernel_size in conv2d)

This document lists both inputs and attributes in the parameter field. You can distinguish them by the marked type. The inputs are of type Tensor, while the rest parameters are attributes. To construct the graph with NNVM python API, a user can pass in the input Tensors as positional arguments, and attributes as keyword arguments.

Overview of Operators

Level 1: Basic Operators

This level enables fully connected multi-layer perceptron.

nnvm.symbol.dense Applies a linear transformation: \(Y = XW^T + b\).
nnvm.symbol.relu Computes rectified linear.
nnvm.symbol.tanh Computes hyperbolic tangent.
nnvm.symbol.sigmoid Computes sigmoid.
nnvm.symbol.exp Returns the exp input array, computed element-wise.
nnvm.symbol.log Returns the log input array, computed element-wise.
nnvm.symbol.sqrt Returns the sqrt input array, computed element-wise.
nnvm.symbol.elemwise_add Element-wise add
nnvm.symbol.elemwise_sub Element-wise substraction
nnvm.symbol.elemwise_mul Element-wise multiplication
nnvm.symbol.elemwise_div Element-wise multiplication
nnvm.symbol.flatten Flattens the input into a 2-D array.
nnvm.symbol.concatenate Joins input arrays along a given axis.
nnvm.symbol.expand_dims Inserts a new axis of size 1 into the array shape
nnvm.symbol.squeeze Squeeze axises in the array.
nnvm.symbol.split Splits an array along a particular axis into multiple sub-arrays.
nnvm.symbol.dropout Applies dropout operation to input array.
nnvm.symbol.batch_norm Batch normalization layer (Ioffe and Szegedy, 2014).
nnvm.symbol.softmax Computes softmax.
nnvm.symbol.log_softmax Computes log softmax.

Level 2: Convolutions

This level enables typical convnet models.

nnvm.symbol.conv2d 2D convolution layer (e.g. spatial convolution over images).
nnvm.symbol.conv2d_transpose Transposed 2D convolution layer (sometimes called Deconvolution).
nnvm.symbol.max_pool2d Max pooling operation for one dimensional data.
nnvm.symbol.avg_pool2d Average pooling operation for one dimensional data.
nnvm.symbol.global_max_pool2d Global max pooling operation for 2D data.
nnvm.symbol.global_avg_pool2d Global average pooling operation for 2D data.

Level 3: Additional Tensor Ops

nnvm.symbol.reshape Reshapes the input array.
nnvm.symbol.copy Copy tensor to another one.
nnvm.symbol.negative Elemenwise numeric negative
nnvm.symbol.leaky_relu Leaky version of a Rectified Linear Unit.
nnvm.symbol.__add_scalar__ Tensor add scalar
nnvm.symbol.__sub_scalar__ Tensor substract scalar
nnvm.symbol.__rsub_scalar__ scalar substract Tensor
nnvm.symbol.__mul_scalar__ Tensor multiplies scalar
nnvm.symbol.__div_scalar__ Tensor divides scalar
nnvm.symbol.__rdiv_scalar__ scalar divides Tensor
nnvm.symbol.__pow_scalar__ Tensor power scalar
nnvm.symbol.__rpow_scalar__ scalar power Tensor

Level 4: Broadcast and Reductions

nnvm.symbol.transpose Permutes the dimensions of an array.
nnvm.symbol.broadcast_to Broadcasts the input array to a new shape.
nnvm.symbol.sum Computes the sum of array elements over given axes.
nnvm.symbol.min Computes the min of array elements over given axes.
nnvm.symbol.max Computes the max of array elements over given axes.
nnvm.symbol.broadcast_add Returns element-wise sum of the input arrays with broadcasting.
nnvm.symbol.broadcast_sub Returns element-wise difference of the input arrays with broadcasting.
nnvm.symbol.broadcast_mul Returns element-wise product of the input arrays with broadcasting.
nnvm.symbol.broadcast_div Returns element-wise division of the input arrays with broadcasting.

Detailed Definitions

nnvm.symbol.dense(*args, **kwargs)

Applies a linear transformation: \(Y = XW^T + b\).

  • data: (x1, x2, …, xn, input_dim)
  • weight: (units, input_dim)
  • bias: (units,)
  • out: (x1, x2, …, xn, num_hidden)

The learnable parameters include both weight and bias.

If use_bias is set to be false, then the bias term is ignored.

Defined in src/top/nn/nn.cc:L63

Parameters:
  • data (nD Tensor) – Input data.
  • weight (2D Tensor) – Weight matrix.
  • bias (1D Tensor) – Bias parameter.
  • units (int, required) – Number of hidden units of the dense transformation.
  • use_bias (boolean, optional, default=1) – Whether to use bias parameter
Returns:

result – The result Tensor.

Return type:

Tensor

nnvm.symbol.relu(*args, **kwargs)

Computes rectified linear.

\[max(input, 0)\]

Defined in src/top/nn/nn.cc:L84

Parameters:data (Tensor) – The input tensor.
Returns:result – The result Tensor.
Return type:Tensor
nnvm.symbol.tanh(*args, **kwargs)

Computes hyperbolic tangent.

\[Y = sinh(X) / cosh(X)\]

Defined in src/top/tensor/elemwise.cc:L42

Parameters:data (Tensor) – The input tensor.
Returns:result – The result Tensor.
Return type:Tensor
nnvm.symbol.sigmoid(*args, **kwargs)

Computes sigmoid.

\[Y = 1 / (1 + exp(-X))\]

Defined in src/top/tensor/elemwise.cc:L32

Parameters:data (Tensor) – The input tensor.
Returns:result – The result Tensor.
Return type:Tensor
nnvm.symbol.exp(*args, **kwargs)

Returns the exp input array, computed element-wise.

\[exp(x)\]

Defined in src/top/tensor/elemwise.cc:L52

Parameters:data (Tensor) – The input tensor.
Returns:result – The result Tensor.
Return type:Tensor
nnvm.symbol.log(*args, **kwargs)

Returns the log input array, computed element-wise.

\[log(x)\]

Defined in src/top/tensor/elemwise.cc:L62

Parameters:data (Tensor) – The input tensor.
Returns:result – The result Tensor.
Return type:Tensor
nnvm.symbol.sqrt(*args, **kwargs)

Returns the sqrt input array, computed element-wise.

\[\sqrt(x)\]

Defined in src/top/tensor/elemwise.cc:L72

Parameters:data (Tensor) – The input tensor.
Returns:result – The result Tensor.
Return type:Tensor
nnvm.symbol.elemwise_add(*args, **kwargs)

Element-wise add

Parameters:
  • lhs (Tensor) – first input
  • rhs (Tensor) – second input
Returns:

result – The result Tensor.

Return type:

Tensor

nnvm.symbol.elemwise_sub(*args, **kwargs)

Element-wise substraction

Defined in src/top/tensor/elemwise.cc:L86

Parameters:
  • lhs (Tensor) – first input
  • rhs (Tensor) – second input
Returns:

result – The result Tensor.

Return type:

Tensor

nnvm.symbol.elemwise_mul(*args, **kwargs)

Element-wise multiplication

Defined in src/top/tensor/elemwise.cc:L92

Parameters:
  • lhs (Tensor) – first input
  • rhs (Tensor) – second input
Returns:

result – The result Tensor.

Return type:

Tensor

nnvm.symbol.elemwise_div(*args, **kwargs)

Element-wise multiplication

Defined in src/top/tensor/elemwise.cc:L98

Parameters:
  • lhs (Tensor) – first input
  • rhs (Tensor) – second input
Returns:

result – The result Tensor.

Return type:

Tensor

nnvm.symbol.flatten(*args, **kwargs)

Flattens the input into a 2-D array.

For an input array with shape (d1, d2, ..., dk), flatten operation reshapes the input array into an output array of shape (d1, d2*...*dk).

Example:

x = [[
    [1,2,3],
    [4,5,6],
    [7,8,9]
],
[    [1,2,3],
    [4,5,6],
    [7,8,9]
]],

flatten(x) = [[ 1.,  2.,  3.,  4.,  5.,  6.,  7.,  8.,  9.],
   [ 1.,  2.,  3.,  4.,  5.,  6.,  7.,  8.,  9.]]

Defined in src/top/tensor/transform.cc:L55

Parameters:data (Tensor) – Input data.
Returns:result – The result Tensor.
Return type:Tensor
nnvm.symbol.concatenate(*args, **kwargs)

Joins input arrays along a given axis.

The dimensions of the input arrays should be the same except the axis along which they will be concatenated. The dimension of the output array along the concatenated axis will be equal to the sum of the corresponding dimensions of the input arrays.

Example:

x = [[1,1],[2,2]]
y = [[3,3],[4,4],[5,5]]
z = [[6,6], [7,7],[8,8]]

concatenate(x,y,z,dim=0) = [[ 1.,  1.],
                            [ 2.,  2.],
                            [ 3.,  3.],
                            [ 4.,  4.],
                            [ 5.,  5.],
                            [ 6.,  6.],
                            [ 7.,  7.],
                            [ 8.,  8.]]

Note that you cannot concat x,y,z along dimension 1 since dimension
0 is not the same for all the input arrays.

concatenate(y,z,dim=1) = [[ 3.,  3.,  6.,  6.],
                          [ 4.,  4.,  7.,  7.],
                          [ 5.,  5.,  8.,  8.]]

Defined in src/top/tensor/transform.cc:L134

Parameters:
  • data (Tensor-or-Tensor[]) – List of arrays to concatenate
  • axis (int, optional, default='1') – the axis to be concated.
Returns:

result – The result Tensor.

Return type:

Tensor

nnvm.symbol.expand_dims(*args, **kwargs)

Inserts a new axis of size 1 into the array shape

For example, given x with shape (2,3,4), then expand_dims(x, axis=1) will return a new array with shape (2,1,3,4).

Defined in src/top/tensor/transform.cc:L178

Parameters:
  • data (Tensor) – Input tensor
  • axis (int, required) – the axis to be expanded.
  • num_newaxis (int, optional, default='1') – Number of new axis to be inserted.
Returns:

result – The result Tensor.

Return type:

Tensor

nnvm.symbol.squeeze(*args, **kwargs)

Squeeze axises in the array.

Examples:

x = [[[0], [1], [2]]]

squeeze(x) = [0, 1, 2]

squeeze(x, 0) = [[0], [1], [2]]

squeeze(x, (0, 2)) = [0, 1, 2]

Defined in src/top/tensor/transform.cc:L511

Parameters:
  • data (Tensor) – Source input
  • axis (, optional, default=[]) – The axis to squeeze in the input tensor. If set to None, all size=1 axes will be squeezed
Returns:

result – The result Tensor.

Return type:

Tensor

nnvm.symbol.split(*args, **kwargs)

Splits an array along a particular axis into multiple sub-arrays.

Note that indices_or_sections should evenly divide the length of the axis along which to split the array.

Defined in src/top/tensor/transform.cc:L260

Parameters:
  • data (Tensor) – Array to be splitted
  • indices_or_sections (tuple of <int>, required) – Number of outputs to be splitted
  • axis (int, optional, default='1') – the axis to be splitted.
Returns:

result – The result Tensor.

Return type:

Tensor

nnvm.symbol.dropout(*args, **kwargs)

Applies dropout operation to input array.

  • During training, each element of the input is set to zero with probability p. The whole array is rescaled by \(1/(1-p)\) to keep the expected sum of the input unchanged.

Defined in src/top/nn/nn.cc:L97

Parameters:
  • data (Tensor) – Input to which dropout will be applied
  • rate (float, optional, default=0.5) – Fraction of the input that gets dropped out during training time.
Returns:

result – The result Tensor.

Return type:

Tensor

nnvm.symbol.batch_norm(*args, **kwargs)

Batch normalization layer (Ioffe and Szegedy, 2014). Normalizes the input at each batch, i.e. applies a transformation that maintains the mean activation close to 0 and the activation standard deviation close to 1.

\[\begin{split}data\_mean[i] = mean(data[:,i,:,...]) \\ data\_var[i] = var(data[:,i,:,...])\end{split}\]

Then compute the normalized output, which has the same shape as input, as following:

\[out[:,i,:,...] = \frac{data[:,i,:,...] - data\_mean[i]}{\sqrt{data\_var[i]+\epsilon}} * gamma[i] + beta[i]\]

Both mean and var returns a scalar by treating the input as a vector.

Assume the input has size k on axis 1, then both gamma and beta have shape (k,).

Besides the inputs and the outputs, this operator accepts two auxiliary states, moving_mean and moving_var, which are k-length vectors. They are global statistics for the whole dataset, which are updated by:

moving_mean = moving_mean * momentum + data_mean * (1 - momentum)
moving_var = moving_var * momentum + data_var * (1 - momentum)

The parameter axis specifies which axis of the input shape denotes the ‘channel’ (separately normalized groups). The default is 1. Specifying -1 sets the channel axis to be the last item in the input shape.

Note

This operator can be optimized away for inference.

Defined in src/top/nn/nn.cc:L171

Parameters:
  • data (Tensor) – Input to which dropout will be applied
  • gamma (Tensor) – The gamma scale factor
  • beta (Tensor) – The beta offset factor
  • moving_mean (Tensor) – running mean of input
  • moving_var (Tensor) – running variance of input
  • axis (int, optional, default='1') – Specify which shape axis the channel is specified.
  • epsilon (double, optional, default=1e-05) – Small float added to variance to avoid dividing by zero.
  • center (boolean, optional, default=1) – If True, add offset of beta to normalized tensor.If False, beta is ignored.
  • scale (boolean, optional, default=1) – If True, multiply by gamma. If False, gamma is not used.When the next layer is piecewise linear (also e.g. nn.relu),this can be disabled since the scalingwill be done by the next layer.
Returns:

result – The result Tensor.

Return type:

Tensor

nnvm.symbol.softmax(*args, **kwargs)

Computes softmax.

\[\text{softmax}(x)_i = \frac{exp(x_i)}{\sum_j exp(x_j)}\]

Note

This operator can be optimized away for inference.

Defined in src/top/nn/nn.cc:L208

Parameters:
  • data (Tensor) – Input data.
  • axis (int, optional, default='-1') – The axis to sum over when computing softmax.
Returns:

result – The result Tensor.

Return type:

Tensor

nnvm.symbol.log_softmax(*args, **kwargs)

Computes log softmax.

\[\text{log_softmax}(x)_i = \log \frac{exp(x_i)}{\sum_j exp(x_j)}\]

Note

This operator can be optimized away for inference.

Defined in src/top/nn/nn.cc:L227

Parameters:
  • data (Tensor) – Input data.
  • axis (int, optional, default='-1') – The axis to sum over when computing softmax.
Returns:

result – The result Tensor.

Return type:

Tensor

nnvm.symbol.conv2d(*args, **kwargs)

2D convolution layer (e.g. spatial convolution over images).

This layer creates a convolution kernel that is convolved with the layer input to produce a tensor of outputs. If use_bias is True, a bias vector is created and added to the outputs.

  • data: This depends on the layout parameter. Input is 4D array of shape
    (batch_size, in_channels, height, width) if layout is NCHW.
  • weight: (channels, in_channels, kernel_size[0], kernel_size[1])
  • bias: (channels,)
  • out: This depends on the layout parameter. Output is 4D array of shape
    (batch_size, channels, out_height, out_width) if layout is NCHW.

Defined in src/top/nn/convolution.cc:L111

Parameters:
  • data (4D Tensor) – Input data.
  • weight (4D Tensor) – Weight matrix.
  • bias (1D Tensor) – Bias parameter.
  • channels (int, required) – The dimensionality of the output spacei.e. the number of output channels in the convolution.
  • kernel_size (, required) – Specifies the dimensions of the convolution window.
  • strides (, optional, default=[1,1]) – Specifies the strides of the convolution.
  • padding (, optional, default=[0,0]) – If padding is non-zero, then the input is implicitly zero-paddedon both sides for padding number of points
  • dilation (, optional, default=[1,1]) – Specifies the dilation rate to use for dilated convolution.
  • groups (int, optional, default='1') – Controls the connections between inputs and outputs.At groups=1, all inputs are convolved to all outputs.At groups=2, the operation becomes equivalent to having two convolutionlayers side by side, each seeing half the input channels, and producinghalf the output channels, and both subsequently concatenated.
  • layout ({'NCHW', 'NHWC'},optional, default='NCHW') – Dimension ordering of data and weight. Can be ‘NCHW’, ‘NHWC’, etc.’N’, ‘C’, ‘H’, ‘W’ stands for batch, channel, height, and widthdimensions respectively. Convolution is applied on the ‘H’ and’W’ dimensions.
  • use_bias (boolean, optional, default=1) – Whether the layer uses a bias vector.
Returns:

result – The result Tensor.

Return type:

Tensor

nnvm.symbol.conv2d_transpose(*args, **kwargs)

Transposed 2D convolution layer (sometimes called Deconvolution).

The need for transposed convolutions generally arises from the desire to use a transformation going in the opposite direction of a normal convolution, i.e., from something that has the shape of the output of some convolution to something that has the shape of its input while maintaining a connectivity pattern that is compatible with said convolution.

  • data: This depends on the layout parameter. Input is 4D array of shape

    (batch_size, in_channels, height, width) if layout is NCHW.

  • weight: (channels, in_channels, kernel_size[0], kernel_size[1])

  • bias: (channels,)

  • out: This depends on the layout parameter. Output is 4D array of shape

    (batch_size, channels, out_height, out_width) if layout is NCHW.

    out_height and out_width are calculated as::

    out_height = (height-1)*strides[0]-2*padding[0]+kernel_size[0]+output_padding[0] out_width = (width-1)*strides[1]-2*padding[1]+kernel_size[1]+output_padding[1]

Defined in src/top/nn/convolution.cc:L201

Parameters:
  • data (4D Tensor) – Input data.
  • weight (4D Tensor) – Weight matrix.
  • bias (1D Tensor) – Bias parameter.
  • channels (int, required) – The dimensionality of the output spacei.e. the number of output channels in the convolution.
  • kernel_size (, required) – Specifies the dimensions of the convolution window.
  • strides (, optional, default=[1,1]) – Specifies the strides of the convolution.
  • output_padding (, optional, default=[0,0]) – Zero-padding added to one side of the output.
  • padding (, optional, default=[0,0]) – If padding is non-zero, then the input is implicitly zero-paddedon both sides for padding number of points
  • dilation (, optional, default=[1,1]) – Specifies the dilation rate to use for dilated convolution.
  • groups (int, optional, default='1') – Controls the connections between inputs and outputs.At groups=1, all inputs are convolved to all outputs.At groups=2, the operation becomes equivalent to having two convolutionlayers side by side, each seeing half the input channels, and producinghalf the output channels, and both subsequently concatenated.
  • layout ({'NCHW', 'NHWC'},optional, default='NCHW') – Dimension ordering of data and weight. Can be ‘NCHW’, ‘NHWC’, etc.’N’, ‘C’, ‘H’, ‘W’ stands for batch, channel, height, and widthdimensions respectively. Convolution is applied on the ‘H’ and’W’ dimensions.
  • use_bias (boolean, optional, default=1) – Whether the layer uses a bias vector.
Returns:

result – The result Tensor.

Return type:

Tensor

nnvm.symbol.max_pool2d(*args, **kwargs)

Max pooling operation for one dimensional data.

  • data: This depends on the layout parameter. Input is 4D array of shape

    (batch_size, channels, height, width) if layout is NCHW.

  • out: This depends on the layout parameter. Output is 4D array of shape

    (batch_size, channels, out_height, out_width) if layout is NCHW. out_height and out_width are calculated as:

    out_height = floor((height+2*padding[0]-pool_size[0])/strides[0])+1
    out_width = floor((width+2*padding[1]-pool_size[1])/strides[1])+1
    

    When ceil_mode is True, ceil will be used instead of floor in this equation.

Defined in src/top/nn/pooling.cc:L71

Parameters:
  • data (4D Tensor) – Input data.
  • pool_size (, required) – Size of the pooling windows..
  • strides (, optional, default=[1,1]) – Specifies the strides of the convolution.
  • padding (, optional, default=[0,0]) – If padding is non-zero, then the input is implicitly zero-paddedon both sides for padding number of points
  • layout ({'NCHW', 'NHWC'},optional, default='NCHW') – Dimension ordering of data and weight. Can be ‘NCHW’, ‘NHWC’, etc.’N’, ‘C’, ‘H’, ‘W’ stands for batch, channel, height, and widthdimensions respectively. Convolution is applied on the ‘H’ and’W’ dimensions.
  • ceil_mode (boolean, optional, default=0) – When true, will use ceil instead of floor to compute the output shape.
Returns:

result – The result Tensor.

Return type:

Tensor

nnvm.symbol.avg_pool2d(*args, **kwargs)

Average pooling operation for one dimensional data.

  • data: This depends on the layout parameter. Input is 4D array of shape

    (batch_size, channels, height, width) if layout is NCHW.

  • out: This depends on the layout parameter. Output is 4D array of shape

    (batch_size, channels, out_height, out_width) if layout is NCHW. out_height and out_width are calculated as:

    out_height = floor((height+2*padding[0]-pool_size[0])/strides[0])+1
    out_width = floor((width+2*padding[1]-pool_size[1])/strides[1])+1
    

    When ceil_mode is True, ceil will be used instead of floor in this equation.

Defined in src/top/nn/pooling.cc:L98

Parameters:
  • data (4D Tensor) – Input data.
  • pool_size (, required) – Size of the pooling windows..
  • strides (, optional, default=[1,1]) – Specifies the strides of the convolution.
  • padding (, optional, default=[0,0]) – If padding is non-zero, then the input is implicitly zero-paddedon both sides for padding number of points
  • layout ({'NCHW', 'NHWC'},optional, default='NCHW') – Dimension ordering of data and weight. Can be ‘NCHW’, ‘NHWC’, etc.’N’, ‘C’, ‘H’, ‘W’ stands for batch, channel, height, and widthdimensions respectively. Convolution is applied on the ‘H’ and’W’ dimensions.
  • ceil_mode (boolean, optional, default=0) – When true, will use ceil instead of floor to compute the output shape.
Returns:

result – The result Tensor.

Return type:

Tensor

nnvm.symbol.global_max_pool2d(*args, **kwargs)

Global max pooling operation for 2D data.

  • data: This depends on the layout parameter. Input is 4D array of shape
    (batch_size, channels, height, width) if layout is NCHW.
  • out: This depends on the layout parameter. Output is 4D array of shape
    (batch_size, channels, 1, 1) if layout is NCHW.

Defined in src/top/nn/pooling.cc:L136

Parameters:
  • data (4D Tensor) – Input data.
  • layout ({'NCHW', 'NHWC'},optional, default='NCHW') – Dimension ordering of data and weight. Can be ‘NCHW’, ‘NHWC’, etc.’N’, ‘C’, ‘H’, ‘W’ stands for batch, channel, height, and widthdimensions respectively. Convolution is applied on the ‘H’ and’W’ dimensions.
Returns:

result – The result Tensor.

Return type:

Tensor

nnvm.symbol.global_avg_pool2d(*args, **kwargs)

Global average pooling operation for 2D data.

  • data: This depends on the layout parameter. Input is 4D array of shape
    (batch_size, channels, height, width) if layout is NCHW.
  • out: This depends on the layout parameter. Output is 4D array of shape
    (batch_size, channels, 1, 1) if layout is NCHW.

Defined in src/top/nn/pooling.cc:L156

Parameters:
  • data (4D Tensor) – Input data.
  • layout ({'NCHW', 'NHWC'},optional, default='NCHW') – Dimension ordering of data and weight. Can be ‘NCHW’, ‘NHWC’, etc.’N’, ‘C’, ‘H’, ‘W’ stands for batch, channel, height, and widthdimensions respectively. Convolution is applied on the ‘H’ and’W’ dimensions.
Returns:

result – The result Tensor.

Return type:

Tensor

nnvm.symbol.reshape(*args, **kwargs)

Reshapes the input array.

Given an array and a shape, this function returns a copy of the array in the new shape. The shape is a tuple of integers such as (2,3,4).The size of the new shape should be same as the size of the input array.

Example:

reshape([1,2,3,4], shape=(2,2)) = [[1,2], [3,4]]

To give user more convenience in without doing manual shape inference, some dimensions of the shape can take special values from the set {0, -1, -2, -3, -4}. The significance of each is explained below:

  • 0 copy this dimension from the input to the output shape.

    Example:

    - input shape = (2,3,4), shape = (4,0,2), output shape = (4,3,2)
    - input shape = (2,3,4), shape = (2,0,0), output shape = (2,3,4)
    
  • -1 infers the dimension of the output shape by using the remainder of the input dimensions keeping the size of the new array same as that of the input array. At most one dimension of shape can be -1.

    Example:

    - input shape = (2,3,4), shape = (6,1,-1), output shape = (6,1,4)
    - input shape = (2,3,4), shape = (3,-1,8), output shape = (3,1,8)
    - input shape = (2,3,4), shape=(-1,), output shape = (24,)
    
  • -2 copy all/remainder of the input dimensions to the output shape.

    Example:

    - input shape = (2,3,4), shape = (-2,), output shape = (2,3,4)
    - input shape = (2,3,4), shape = (2,-2), output shape = (2,3,4)
    - input shape = (2,3,4), shape = (-2,1,1), output shape = (2,3,4,1,1)
    
  • -3 use the product of two consecutive dimensions of the input shape as the output dimension.

    Example:

    - input shape = (2,3,4), shape = (-3,4), output shape = (6,4)
    - input shape = (2,3,4,5), shape = (-3,-3), output shape = (6,20)
    - input shape = (2,3,4), shape = (0,-3), output shape = (2,12)
    - input shape = (2,3,4), shape = (-3,-2), output shape = (6,4)
    
  • -4 split one dimension of the input into two dimensions passed subsequent to -4 in shape (can contain -1).

    Example:

    - input shape = (2,3,4), shape = (-4,1,2,-2), output shape =(1,2,3,4)
    - input shape = (2,3,4), shape = (2,-4,-1,3,-2), output shape = (2,1,3,4)
    

Defined in src/top/tensor/transform.cc:L437

Parameters:
  • data (Tensor) – Input data.
  • shape (tuple of <>, required) –
Returns:

result – The result Tensor.

Return type:

Tensor

nnvm.symbol.copy(*args, **kwargs)

Copy tensor to another one.

Defined in src/top/tensor/elemwise.cc:L112

Parameters:data (Tensor) – The input tensor.
Returns:result – The result Tensor.
Return type:Tensor
nnvm.symbol.negative(*args, **kwargs)

Elemenwise numeric negative

Defined in src/top/tensor/elemwise.cc:L105

Parameters:data (Tensor) – The input tensor.
Returns:result – The result Tensor.
Return type:Tensor
nnvm.symbol.leaky_relu(*args, **kwargs)

Leaky version of a Rectified Linear Unit.

y = x > 0 ? x : alpha * x

Defined in src/top/nn/nn.cc:L246

Parameters:
  • data (Tensor) – Input data.
  • alpha (double, optional, default=0.25) – slope coefficient for the negative half axis.
Returns:

result – The result Tensor.

Return type:

Tensor

nnvm.symbol.__add_scalar__(*args, **kwargs)

Tensor add scalar

Defined in src/top/tensor/elemwise.cc:L128

Parameters:
  • data (Tensor) – The input tensor.
  • scalar (double, required) –
Returns:

result – The result Tensor.

Return type:

Tensor

nnvm.symbol.__sub_scalar__(*args, **kwargs)

Tensor substract scalar

Defined in src/top/tensor/elemwise.cc:L134

Parameters:
  • data (Tensor) – The input tensor.
  • scalar (double, required) –
Returns:

result – The result Tensor.

Return type:

Tensor

nnvm.symbol.__rsub_scalar__(*args, **kwargs)

scalar substract Tensor

Defined in src/top/tensor/elemwise.cc:L140

Parameters:
  • data (Tensor) – The input tensor.
  • scalar (double, required) –
Returns:

result – The result Tensor.

Return type:

Tensor

nnvm.symbol.__mul_scalar__(*args, **kwargs)

Tensor multiplies scalar

Defined in src/top/tensor/elemwise.cc:L146

Parameters:
  • data (Tensor) – The input tensor.
  • scalar (double, required) –
Returns:

result – The result Tensor.

Return type:

Tensor

nnvm.symbol.__div_scalar__(*args, **kwargs)

Tensor divides scalar

Defined in src/top/tensor/elemwise.cc:L152

Parameters:
  • data (Tensor) – The input tensor.
  • scalar (double, required) –
Returns:

result – The result Tensor.

Return type:

Tensor

nnvm.symbol.__rdiv_scalar__(*args, **kwargs)

scalar divides Tensor

Defined in src/top/tensor/elemwise.cc:L158

Parameters:
  • data (Tensor) – The input tensor.
  • scalar (double, required) –
Returns:

result – The result Tensor.

Return type:

Tensor

nnvm.symbol.__pow_scalar__(*args, **kwargs)

Tensor power scalar

Defined in src/top/tensor/elemwise.cc:L164

Parameters:
  • data (Tensor) – The input tensor.
  • scalar (double, required) –
Returns:

result – The result Tensor.

Return type:

Tensor

nnvm.symbol.__rpow_scalar__(*args, **kwargs)

scalar power Tensor

Defined in src/top/tensor/elemwise.cc:L170

Parameters:
  • data (Tensor) – The input tensor.
  • scalar (double, required) –
Returns:

result – The result Tensor.

Return type:

Tensor

nnvm.symbol.transpose(*args, **kwargs)

Permutes the dimensions of an array.

Examples:

x = [[ 1, 2],
     [ 3, 4]]

transpose(x) = [[ 1.,  3.],
                [ 2.,  4.]]

x = [[[ 1.,  2.],
      [ 3.,  4.]],

     [[ 5.,  6.],
      [ 7.,  8.]]]

transpose(x) = [[[ 1.,  5.],
                 [ 3.,  7.]],

                [[ 2.,  6.],
                 [ 4.,  8.]]]

transpose(x, axes=(1,0,2)) = [[[ 1.,  2.],
                               [ 5.,  6.]],

                              [[ 3.,  4.],
                               [ 7.,  8.]]]

Defined in src/top/tensor/transform.cc:L578

Parameters:
  • data (Tensor) – Source input
  • axes (, optional, default=[]) – Target axis order. By default the axes will be inverted.
Returns:

result – The result Tensor.

Return type:

Tensor

nnvm.symbol.broadcast_to(*args, **kwargs)

Broadcasts the input array to a new shape.

Broadcasting is a mechanism that allows NDArrays to perform arithmetic operations with arrays of different shapes efficiently without creating multiple copies of arrays. Also see, Broadcasting for more explanation.

Broadcasting is allowed on axes with size 1, such as from (2,1,3,1) to (2,8,3,9). Elements will be duplicated on the broadcasted axes.

For example:

broadcast_to([[1,2,3]], shape=(2,3)) = [[ 1.,  2.,  3.],
                                        [ 1.,  2.,  3.]])

The dimension which you do not want to change can also be kept as 0 which means copy the original value. So with shape=(2,0), we will obtain the same result as in the above example.

Defined in src/top/tensor/broadcast.cc:L63

Parameters:
  • data (Tensor) – Input data.
  • shape (, optional, default=[]) – The shape of the desired array. We can set the dim to zero if it’s same as the original. E.g A = broadcast_to(B, shape=(10, 0, 0))
Returns:

result – The result Tensor.

Return type:

Tensor

nnvm.symbol.sum(*args, **kwargs)

Computes the sum of array elements over given axes.

Example:

data = [[[1,2],[2,3],[1,3]],
        [[1,4],[4,3],[5,2]],
        [[7,1],[7,2],[7,3]]]

sum(data, axis=1)
[[  4.   8.]
 [ 10.   9.]
 [ 21.   6.]]

sum(data, axis=[1,2])
[ 12.  19.  27.]

Defined in src/top/tensor/reduce.cc:L123

Parameters:
  • data (Tensor) – The input
  • axis (, optional, default=[]) –

    The axis or axes along which to perform the reduction.

    The default, axis=(), will compute over all elements into a scalar array with shape (1,).

    If axis is int, a reduction is performed on a particular axis.

    If axis is a tuple of ints, a reduction is performed on all the axes specified in the tuple.

    If exclude is true, reduction will be performed on the axes that are NOT in axis instead.

  • keepdims (boolean, optional, default=0) – If this is set to True, the reduced axes are left in the result as dimension with size one.
  • exclude (boolean, optional, default=0) – Whether to perform reduction on axis that are NOT in axis instead.
Returns:

result – The result Tensor.

Return type:

Tensor

nnvm.symbol.min(*args, **kwargs)

Computes the min of array elements over given axes.

Defined in src/top/tensor/reduce.cc:L133

Parameters:
  • data (Tensor) – The input
  • axis (, optional, default=[]) –

    The axis or axes along which to perform the reduction.

    The default, axis=(), will compute over all elements into a scalar array with shape (1,).

    If axis is int, a reduction is performed on a particular axis.

    If axis is a tuple of ints, a reduction is performed on all the axes specified in the tuple.

    If exclude is true, reduction will be performed on the axes that are NOT in axis instead.

  • keepdims (boolean, optional, default=0) – If this is set to True, the reduced axes are left in the result as dimension with size one.
  • exclude (boolean, optional, default=0) – Whether to perform reduction on axis that are NOT in axis instead.
Returns:

result – The result Tensor.

Return type:

Tensor

nnvm.symbol.max(*args, **kwargs)

Computes the max of array elements over given axes.

Defined in src/top/tensor/reduce.cc:L128

Parameters:
  • data (Tensor) – The input
  • axis (, optional, default=[]) –

    The axis or axes along which to perform the reduction.

    The default, axis=(), will compute over all elements into a scalar array with shape (1,).

    If axis is int, a reduction is performed on a particular axis.

    If axis is a tuple of ints, a reduction is performed on all the axes specified in the tuple.

    If exclude is true, reduction will be performed on the axes that are NOT in axis instead.

  • keepdims (boolean, optional, default=0) – If this is set to True, the reduced axes are left in the result as dimension with size one.
  • exclude (boolean, optional, default=0) – Whether to perform reduction on axis that are NOT in axis instead.
Returns:

result – The result Tensor.

Return type:

Tensor

nnvm.symbol.broadcast_add(*args, **kwargs)

Returns element-wise sum of the input arrays with broadcasting.

Example:

x = [[ 1.,  1.,  1.],
     [ 1.,  1.,  1.]]

y = [[ 0.],
     [ 1.]]

broadcast_add(x, y) = [[ 1.,  1.,  1.],
                       [ 2.,  2.,  2.]]

Defined in src/top/tensor/broadcast.cc:L144

Parameters:
  • lhs (Tensor) – first input
  • rhs (Tensor) – second input
Returns:

result – The result Tensor.

Return type:

Tensor

nnvm.symbol.broadcast_sub(*args, **kwargs)

Returns element-wise difference of the input arrays with broadcasting.

Example:

x = [[ 1.,  1.,  1.],
     [ 1.,  1.,  1.]]

y = [[ 0.],
     [ 1.]]

broadcast_sub(x, y) = [[ 1.,  1.,  1.],
                       [ 0.,  0.,  0.]]

Defined in src/top/tensor/broadcast.cc:L162

Parameters:
  • lhs (Tensor) – first input
  • rhs (Tensor) – second input
Returns:

result – The result Tensor.

Return type:

Tensor

nnvm.symbol.broadcast_mul(*args, **kwargs)

Returns element-wise product of the input arrays with broadcasting.

Example:

x = [[ 1.,  1.,  1.],
     [ 1.,  1.,  1.]]

y = [[ 0.],
     [ 1.]]

broadcast_mul(x, y) = [[ 0.,  0.,  0.],
                       [ 1.,  1.,  1.]]

Defined in src/top/tensor/broadcast.cc:L179

Parameters:
  • lhs (Tensor) – first input
  • rhs (Tensor) – second input
Returns:

result – The result Tensor.

Return type:

Tensor

nnvm.symbol.broadcast_div(*args, **kwargs)

Returns element-wise division of the input arrays with broadcasting.

Example:

x = [[ 6.,  6.,  6.],
     [ 6.,  6.,  6.]]

y = [[ 2.],
     [ 3.]]

broadcast_div(x, y) = [[ 3.,  3.,  3.],
                       [ 2.,  2.,  2.]]

Defined in src/top/tensor/broadcast.cc:L197

Parameters:
  • lhs (Tensor) – first input
  • rhs (Tensor) – second input
Returns:

result – The result Tensor.

Return type:

Tensor