# Core Tensor Operators¶

This page contains the list of core tensor operator primitives pre-defined in NNVM. The core tensor operator primitives(nnvm.top) covers typical workloads in deep learning. They can represent workloads in front-end frameworks, and provide basic building blocks for optimization. Since deep learning is a fast evolving field and it is that possible to have operators that are not in here. NNVM is designed for this problem and can easily new operators without changing the core library.

Note

Each operator node in the graph IR contains the following two kinds of parameters.

• inputs: positional list of input tensors
• attrs: attributes about operator(e.g. kernel_size in conv2d)

This document lists both inputs and attributes in the parameter field. You can distinguish them by the marked type. The inputs are of type Tensor, while the rest parameters are attributes. To construct the graph with NNVM python API, a user can pass in the input Tensors as positional arguments, and attributes as keyword arguments.

## Overview of Operators¶

Level 1: Basic Operators

This level enables fully connected multi-layer perceptron.

 nnvm.symbol.dense Applies a linear transformation: $$Y = XW^T + b$$. nnvm.symbol.relu Computes rectified linear. nnvm.symbol.tanh Computes hyperbolic tangent. nnvm.symbol.sigmoid Computes sigmoid. nnvm.symbol.exp Returns the exp input array, computed element-wise. nnvm.symbol.log Returns the log input array, computed element-wise. nnvm.symbol.sqrt Returns the sqrt input array, computed element-wise. nnvm.symbol.elemwise_add Element-wise add nnvm.symbol.elemwise_sub Element-wise substraction nnvm.symbol.elemwise_mul Element-wise multiplication nnvm.symbol.elemwise_div Element-wise multiplication nnvm.symbol.flatten Flattens the input into a 2-D array. nnvm.symbol.concatenate Joins input arrays along a given axis. nnvm.symbol.expand_dims Inserts a new axis of size 1 into the array shape nnvm.symbol.squeeze Squeeze axises in the array. nnvm.symbol.split Splits an array along a particular axis into multiple sub-arrays. nnvm.symbol.dropout Applies dropout operation to input array. nnvm.symbol.batch_norm Batch normalization layer (Ioffe and Szegedy, 2014). nnvm.symbol.softmax Computes softmax. nnvm.symbol.log_softmax Computes log softmax.

Level 2: Convolutions

This level enables typical convnet models.

 nnvm.symbol.conv2d 2D convolution layer (e.g. spatial convolution over images). nnvm.symbol.conv2d_transpose Transposed 2D convolution layer (sometimes called Deconvolution). nnvm.symbol.max_pool2d Max pooling operation for one dimensional data. nnvm.symbol.avg_pool2d Average pooling operation for one dimensional data. nnvm.symbol.global_max_pool2d Global max pooling operation for 2D data. nnvm.symbol.global_avg_pool2d Global average pooling operation for 2D data.

Level 3: Additional Tensor Ops

 nnvm.symbol.reshape Reshapes the input array. nnvm.symbol.copy Copy tensor to another one. nnvm.symbol.negative Elemenwise numeric negative nnvm.symbol.leaky_relu Leaky version of a Rectified Linear Unit. nnvm.symbol.__add_scalar__ Tensor add scalar nnvm.symbol.__sub_scalar__ Tensor substract scalar nnvm.symbol.__rsub_scalar__ scalar substract Tensor nnvm.symbol.__mul_scalar__ Tensor multiplies scalar nnvm.symbol.__div_scalar__ Tensor divides scalar nnvm.symbol.__rdiv_scalar__ scalar divides Tensor nnvm.symbol.__pow_scalar__ Tensor power scalar nnvm.symbol.__rpow_scalar__ scalar power Tensor

Level 4: Broadcast and Reductions

 nnvm.symbol.transpose Permutes the dimensions of an array. nnvm.symbol.broadcast_to Broadcasts the input array to a new shape. nnvm.symbol.sum Computes the sum of array elements over given axes. nnvm.symbol.min Computes the min of array elements over given axes. nnvm.symbol.max Computes the max of array elements over given axes. nnvm.symbol.broadcast_add Returns element-wise sum of the input arrays with broadcasting. nnvm.symbol.broadcast_sub Returns element-wise difference of the input arrays with broadcasting. nnvm.symbol.broadcast_mul Returns element-wise product of the input arrays with broadcasting. nnvm.symbol.broadcast_div Returns element-wise division of the input arrays with broadcasting.

## Detailed Definitions¶

nnvm.symbol.dense(*args, **kwargs)

Applies a linear transformation: $$Y = XW^T + b$$.

• data: (x1, x2, …, xn, input_dim)
• weight: (units, input_dim)
• bias: (units,)
• out: (x1, x2, …, xn, num_hidden)

The learnable parameters include both weight and bias.

If use_bias is set to be false, then the bias term is ignored.

Defined in src/top/nn/nn.cc:L63

Parameters: data (nD Tensor) – Input data. weight (2D Tensor) – Weight matrix. bias (1D Tensor) – Bias parameter. units (int, required) – Number of hidden units of the dense transformation. use_bias (boolean, optional, default=1) – Whether to use bias parameter result – The result Tensor. Tensor
nnvm.symbol.relu(*args, **kwargs)

Computes rectified linear.

$max(input, 0)$

Defined in src/top/nn/nn.cc:L84

Parameters: data (Tensor) – The input tensor. result – The result Tensor. Tensor
nnvm.symbol.tanh(*args, **kwargs)

Computes hyperbolic tangent.

$Y = sinh(X) / cosh(X)$

Defined in src/top/tensor/elemwise.cc:L42

Parameters: data (Tensor) – The input tensor. result – The result Tensor. Tensor
nnvm.symbol.sigmoid(*args, **kwargs)

Computes sigmoid.

$Y = 1 / (1 + exp(-X))$

Defined in src/top/tensor/elemwise.cc:L32

Parameters: data (Tensor) – The input tensor. result – The result Tensor. Tensor
nnvm.symbol.exp(*args, **kwargs)

Returns the exp input array, computed element-wise.

$exp(x)$

Defined in src/top/tensor/elemwise.cc:L52

Parameters: data (Tensor) – The input tensor. result – The result Tensor. Tensor
nnvm.symbol.log(*args, **kwargs)

Returns the log input array, computed element-wise.

$log(x)$

Defined in src/top/tensor/elemwise.cc:L62

Parameters: data (Tensor) – The input tensor. result – The result Tensor. Tensor
nnvm.symbol.sqrt(*args, **kwargs)

Returns the sqrt input array, computed element-wise.

$\sqrt(x)$

Defined in src/top/tensor/elemwise.cc:L72

Parameters: data (Tensor) – The input tensor. result – The result Tensor. Tensor
nnvm.symbol.elemwise_add(*args, **kwargs)

Parameters: lhs (Tensor) – first input rhs (Tensor) – second input result – The result Tensor. Tensor
nnvm.symbol.elemwise_sub(*args, **kwargs)

Element-wise substraction

Defined in src/top/tensor/elemwise.cc:L86

Parameters: lhs (Tensor) – first input rhs (Tensor) – second input result – The result Tensor. Tensor
nnvm.symbol.elemwise_mul(*args, **kwargs)

Element-wise multiplication

Defined in src/top/tensor/elemwise.cc:L92

Parameters: lhs (Tensor) – first input rhs (Tensor) – second input result – The result Tensor. Tensor
nnvm.symbol.elemwise_div(*args, **kwargs)

Element-wise multiplication

Defined in src/top/tensor/elemwise.cc:L98

Parameters: lhs (Tensor) – first input rhs (Tensor) – second input result – The result Tensor. Tensor
nnvm.symbol.flatten(*args, **kwargs)

Flattens the input into a 2-D array.

For an input array with shape (d1, d2, ..., dk), flatten operation reshapes the input array into an output array of shape (d1, d2*...*dk).

Example:

x = [[
[1,2,3],
[4,5,6],
[7,8,9]
],
[    [1,2,3],
[4,5,6],
[7,8,9]
]],

flatten(x) = [[ 1.,  2.,  3.,  4.,  5.,  6.,  7.,  8.,  9.],
[ 1.,  2.,  3.,  4.,  5.,  6.,  7.,  8.,  9.]]


Defined in src/top/tensor/transform.cc:L55

Parameters: data (Tensor) – Input data. result – The result Tensor. Tensor
nnvm.symbol.concatenate(*args, **kwargs)

Joins input arrays along a given axis.

The dimensions of the input arrays should be the same except the axis along which they will be concatenated. The dimension of the output array along the concatenated axis will be equal to the sum of the corresponding dimensions of the input arrays.

Example:

x = [[1,1],[2,2]]
y = [[3,3],[4,4],[5,5]]
z = [[6,6], [7,7],[8,8]]

concatenate(x,y,z,dim=0) = [[ 1.,  1.],
[ 2.,  2.],
[ 3.,  3.],
[ 4.,  4.],
[ 5.,  5.],
[ 6.,  6.],
[ 7.,  7.],
[ 8.,  8.]]

Note that you cannot concat x,y,z along dimension 1 since dimension
0 is not the same for all the input arrays.

concatenate(y,z,dim=1) = [[ 3.,  3.,  6.,  6.],
[ 4.,  4.,  7.,  7.],
[ 5.,  5.,  8.,  8.]]


Defined in src/top/tensor/transform.cc:L134

Parameters: data (Tensor-or-Tensor[]) – List of arrays to concatenate axis (int, optional, default='1') – the axis to be concated. result – The result Tensor. Tensor
nnvm.symbol.expand_dims(*args, **kwargs)

Inserts a new axis of size 1 into the array shape

For example, given x with shape (2,3,4), then expand_dims(x, axis=1) will return a new array with shape (2,1,3,4).

Defined in src/top/tensor/transform.cc:L178

Parameters: data (Tensor) – Input tensor axis (int, required) – the axis to be expanded. num_newaxis (int, optional, default='1') – Number of new axis to be inserted. result – The result Tensor. Tensor
nnvm.symbol.squeeze(*args, **kwargs)

Squeeze axises in the array.

Examples:

x = [[[0], [1], [2]]]

squeeze(x) = [0, 1, 2]

squeeze(x, 0) = [[0], [1], [2]]

squeeze(x, (0, 2)) = [0, 1, 2]


Defined in src/top/tensor/transform.cc:L511

Parameters: data (Tensor) – Source input axis (, optional, default=[]) – The axis to squeeze in the input tensor. If set to None, all size=1 axes will be squeezed result – The result Tensor. Tensor
nnvm.symbol.split(*args, **kwargs)

Splits an array along a particular axis into multiple sub-arrays.

Note that indices_or_sections should evenly divide the length of the axis along which to split the array.

Defined in src/top/tensor/transform.cc:L260

Parameters: data (Tensor) – Array to be splitted indices_or_sections (tuple of , required) – Number of outputs to be splitted axis (int, optional, default='1') – the axis to be splitted. result – The result Tensor. Tensor
nnvm.symbol.dropout(*args, **kwargs)

Applies dropout operation to input array.

• During training, each element of the input is set to zero with probability p. The whole array is rescaled by $$1/(1-p)$$ to keep the expected sum of the input unchanged.

Defined in src/top/nn/nn.cc:L97

Parameters: data (Tensor) – Input to which dropout will be applied rate (float, optional, default=0.5) – Fraction of the input that gets dropped out during training time. result – The result Tensor. Tensor
nnvm.symbol.batch_norm(*args, **kwargs)

Batch normalization layer (Ioffe and Szegedy, 2014). Normalizes the input at each batch, i.e. applies a transformation that maintains the mean activation close to 0 and the activation standard deviation close to 1.

$\begin{split}data\_mean[i] = mean(data[:,i,:,...]) \\ data\_var[i] = var(data[:,i,:,...])\end{split}$

Then compute the normalized output, which has the same shape as input, as following:

$out[:,i,:,...] = \frac{data[:,i,:,...] - data\_mean[i]}{\sqrt{data\_var[i]+\epsilon}} * gamma[i] + beta[i]$

Both mean and var returns a scalar by treating the input as a vector.

Assume the input has size k on axis 1, then both gamma and beta have shape (k,).

Besides the inputs and the outputs, this operator accepts two auxiliary states, moving_mean and moving_var, which are k-length vectors. They are global statistics for the whole dataset, which are updated by:

moving_mean = moving_mean * momentum + data_mean * (1 - momentum)
moving_var = moving_var * momentum + data_var * (1 - momentum)


The parameter axis specifies which axis of the input shape denotes the ‘channel’ (separately normalized groups). The default is 1. Specifying -1 sets the channel axis to be the last item in the input shape.

Note

This operator can be optimized away for inference.

Defined in src/top/nn/nn.cc:L171

Parameters: data (Tensor) – Input to which dropout will be applied gamma (Tensor) – The gamma scale factor beta (Tensor) – The beta offset factor moving_mean (Tensor) – running mean of input moving_var (Tensor) – running variance of input axis (int, optional, default='1') – Specify which shape axis the channel is specified. epsilon (double, optional, default=1e-05) – Small float added to variance to avoid dividing by zero. center (boolean, optional, default=1) – If True, add offset of beta to normalized tensor.If False, beta is ignored. scale (boolean, optional, default=1) – If True, multiply by gamma. If False, gamma is not used.When the next layer is piecewise linear (also e.g. nn.relu),this can be disabled since the scalingwill be done by the next layer. result – The result Tensor. Tensor
nnvm.symbol.softmax(*args, **kwargs)

Computes softmax.

$\text{softmax}(x)_i = \frac{exp(x_i)}{\sum_j exp(x_j)}$

Note

This operator can be optimized away for inference.

Defined in src/top/nn/nn.cc:L208

Parameters: data (Tensor) – Input data. axis (int, optional, default='-1') – The axis to sum over when computing softmax. result – The result Tensor. Tensor
nnvm.symbol.log_softmax(*args, **kwargs)

Computes log softmax.

$\text{log_softmax}(x)_i = \log \frac{exp(x_i)}{\sum_j exp(x_j)}$

Note

This operator can be optimized away for inference.

Defined in src/top/nn/nn.cc:L227

Parameters: data (Tensor) – Input data. axis (int, optional, default='-1') – The axis to sum over when computing softmax. result – The result Tensor. Tensor
nnvm.symbol.conv2d(*args, **kwargs)

2D convolution layer (e.g. spatial convolution over images).

This layer creates a convolution kernel that is convolved with the layer input to produce a tensor of outputs. If use_bias is True, a bias vector is created and added to the outputs.

• data: This depends on the layout parameter. Input is 4D array of shape
(batch_size, in_channels, height, width) if layout is NCHW.
• weight: (channels, in_channels, kernel_size[0], kernel_size[1])
• bias: (channels,)
• out: This depends on the layout parameter. Output is 4D array of shape
(batch_size, channels, out_height, out_width) if layout is NCHW.

Defined in src/top/nn/convolution.cc:L111

Parameters: data (4D Tensor) – Input data. weight (4D Tensor) – Weight matrix. bias (1D Tensor) – Bias parameter. channels (int, required) – The dimensionality of the output spacei.e. the number of output channels in the convolution. kernel_size (, required) – Specifies the dimensions of the convolution window. strides (, optional, default=[1,1]) – Specifies the strides of the convolution. padding (, optional, default=[0,0]) – If padding is non-zero, then the input is implicitly zero-paddedon both sides for padding number of points dilation (, optional, default=[1,1]) – Specifies the dilation rate to use for dilated convolution. groups (int, optional, default='1') – Controls the connections between inputs and outputs.At groups=1, all inputs are convolved to all outputs.At groups=2, the operation becomes equivalent to having two convolutionlayers side by side, each seeing half the input channels, and producinghalf the output channels, and both subsequently concatenated. layout ({'NCHW', 'NHWC'},optional, default='NCHW') – Dimension ordering of data and weight. Can be ‘NCHW’, ‘NHWC’, etc.’N’, ‘C’, ‘H’, ‘W’ stands for batch, channel, height, and widthdimensions respectively. Convolution is applied on the ‘H’ and’W’ dimensions. use_bias (boolean, optional, default=1) – Whether the layer uses a bias vector. result – The result Tensor. Tensor
nnvm.symbol.conv2d_transpose(*args, **kwargs)

Transposed 2D convolution layer (sometimes called Deconvolution).

The need for transposed convolutions generally arises from the desire to use a transformation going in the opposite direction of a normal convolution, i.e., from something that has the shape of the output of some convolution to something that has the shape of its input while maintaining a connectivity pattern that is compatible with said convolution.

• data: This depends on the layout parameter. Input is 4D array of shape

(batch_size, in_channels, height, width) if layout is NCHW.

• weight: (channels, in_channels, kernel_size[0], kernel_size[1])

• bias: (channels,)

• out: This depends on the layout parameter. Output is 4D array of shape

(batch_size, channels, out_height, out_width) if layout is NCHW.

out_height and out_width are calculated as::

Defined in src/top/nn/convolution.cc:L201

Parameters: data (4D Tensor) – Input data. weight (4D Tensor) – Weight matrix. bias (1D Tensor) – Bias parameter. channels (int, required) – The dimensionality of the output spacei.e. the number of output channels in the convolution. kernel_size (, required) – Specifies the dimensions of the convolution window. strides (, optional, default=[1,1]) – Specifies the strides of the convolution. output_padding (, optional, default=[0,0]) – Zero-padding added to one side of the output. padding (, optional, default=[0,0]) – If padding is non-zero, then the input is implicitly zero-paddedon both sides for padding number of points dilation (, optional, default=[1,1]) – Specifies the dilation rate to use for dilated convolution. groups (int, optional, default='1') – Controls the connections between inputs and outputs.At groups=1, all inputs are convolved to all outputs.At groups=2, the operation becomes equivalent to having two convolutionlayers side by side, each seeing half the input channels, and producinghalf the output channels, and both subsequently concatenated. layout ({'NCHW', 'NHWC'},optional, default='NCHW') – Dimension ordering of data and weight. Can be ‘NCHW’, ‘NHWC’, etc.’N’, ‘C’, ‘H’, ‘W’ stands for batch, channel, height, and widthdimensions respectively. Convolution is applied on the ‘H’ and’W’ dimensions. use_bias (boolean, optional, default=1) – Whether the layer uses a bias vector. result – The result Tensor. Tensor
nnvm.symbol.max_pool2d(*args, **kwargs)

Max pooling operation for one dimensional data.

• data: This depends on the layout parameter. Input is 4D array of shape

(batch_size, channels, height, width) if layout is NCHW.

• out: This depends on the layout parameter. Output is 4D array of shape

(batch_size, channels, out_height, out_width) if layout is NCHW. out_height and out_width are calculated as:

out_height = floor((height+2*padding[0]-pool_size[0])/strides[0])+1


When ceil_mode is True, ceil will be used instead of floor in this equation.

Defined in src/top/nn/pooling.cc:L71

Parameters: data (4D Tensor) – Input data. pool_size (, required) – Size of the pooling windows.. strides (, optional, default=[1,1]) – Specifies the strides of the convolution. padding (, optional, default=[0,0]) – If padding is non-zero, then the input is implicitly zero-paddedon both sides for padding number of points layout ({'NCHW', 'NHWC'},optional, default='NCHW') – Dimension ordering of data and weight. Can be ‘NCHW’, ‘NHWC’, etc.’N’, ‘C’, ‘H’, ‘W’ stands for batch, channel, height, and widthdimensions respectively. Convolution is applied on the ‘H’ and’W’ dimensions. ceil_mode (boolean, optional, default=0) – When true, will use ceil instead of floor to compute the output shape. result – The result Tensor. Tensor
nnvm.symbol.avg_pool2d(*args, **kwargs)

Average pooling operation for one dimensional data.

• data: This depends on the layout parameter. Input is 4D array of shape

(batch_size, channels, height, width) if layout is NCHW.

• out: This depends on the layout parameter. Output is 4D array of shape

(batch_size, channels, out_height, out_width) if layout is NCHW. out_height and out_width are calculated as:

out_height = floor((height+2*padding[0]-pool_size[0])/strides[0])+1


When ceil_mode is True, ceil will be used instead of floor in this equation.

Defined in src/top/nn/pooling.cc:L98

Parameters: data (4D Tensor) – Input data. pool_size (, required) – Size of the pooling windows.. strides (, optional, default=[1,1]) – Specifies the strides of the convolution. padding (, optional, default=[0,0]) – If padding is non-zero, then the input is implicitly zero-paddedon both sides for padding number of points layout ({'NCHW', 'NHWC'},optional, default='NCHW') – Dimension ordering of data and weight. Can be ‘NCHW’, ‘NHWC’, etc.’N’, ‘C’, ‘H’, ‘W’ stands for batch, channel, height, and widthdimensions respectively. Convolution is applied on the ‘H’ and’W’ dimensions. ceil_mode (boolean, optional, default=0) – When true, will use ceil instead of floor to compute the output shape. result – The result Tensor. Tensor
nnvm.symbol.global_max_pool2d(*args, **kwargs)

Global max pooling operation for 2D data.

• data: This depends on the layout parameter. Input is 4D array of shape
(batch_size, channels, height, width) if layout is NCHW.
• out: This depends on the layout parameter. Output is 4D array of shape
(batch_size, channels, 1, 1) if layout is NCHW.

Defined in src/top/nn/pooling.cc:L136

Parameters: data (4D Tensor) – Input data. layout ({'NCHW', 'NHWC'},optional, default='NCHW') – Dimension ordering of data and weight. Can be ‘NCHW’, ‘NHWC’, etc.’N’, ‘C’, ‘H’, ‘W’ stands for batch, channel, height, and widthdimensions respectively. Convolution is applied on the ‘H’ and’W’ dimensions. result – The result Tensor. Tensor
nnvm.symbol.global_avg_pool2d(*args, **kwargs)

Global average pooling operation for 2D data.

• data: This depends on the layout parameter. Input is 4D array of shape
(batch_size, channels, height, width) if layout is NCHW.
• out: This depends on the layout parameter. Output is 4D array of shape
(batch_size, channels, 1, 1) if layout is NCHW.

Defined in src/top/nn/pooling.cc:L156

Parameters: data (4D Tensor) – Input data. layout ({'NCHW', 'NHWC'},optional, default='NCHW') – Dimension ordering of data and weight. Can be ‘NCHW’, ‘NHWC’, etc.’N’, ‘C’, ‘H’, ‘W’ stands for batch, channel, height, and widthdimensions respectively. Convolution is applied on the ‘H’ and’W’ dimensions. result – The result Tensor. Tensor
nnvm.symbol.reshape(*args, **kwargs)

Reshapes the input array.

Given an array and a shape, this function returns a copy of the array in the new shape. The shape is a tuple of integers such as (2,3,4).The size of the new shape should be same as the size of the input array.

Example:

reshape([1,2,3,4], shape=(2,2)) = [[1,2], [3,4]]


To give user more convenience in without doing manual shape inference, some dimensions of the shape can take special values from the set {0, -1, -2, -3, -4}. The significance of each is explained below:

• 0 copy this dimension from the input to the output shape.

Example:

- input shape = (2,3,4), shape = (4,0,2), output shape = (4,3,2)
- input shape = (2,3,4), shape = (2,0,0), output shape = (2,3,4)

• -1 infers the dimension of the output shape by using the remainder of the input dimensions keeping the size of the new array same as that of the input array. At most one dimension of shape can be -1.

Example:

- input shape = (2,3,4), shape = (6,1,-1), output shape = (6,1,4)
- input shape = (2,3,4), shape = (3,-1,8), output shape = (3,1,8)
- input shape = (2,3,4), shape=(-1,), output shape = (24,)

• -2 copy all/remainder of the input dimensions to the output shape.

Example:

- input shape = (2,3,4), shape = (-2,), output shape = (2,3,4)
- input shape = (2,3,4), shape = (2,-2), output shape = (2,3,4)
- input shape = (2,3,4), shape = (-2,1,1), output shape = (2,3,4,1,1)

• -3 use the product of two consecutive dimensions of the input shape as the output dimension.

Example:

- input shape = (2,3,4), shape = (-3,4), output shape = (6,4)
- input shape = (2,3,4,5), shape = (-3,-3), output shape = (6,20)
- input shape = (2,3,4), shape = (0,-3), output shape = (2,12)
- input shape = (2,3,4), shape = (-3,-2), output shape = (6,4)

• -4 split one dimension of the input into two dimensions passed subsequent to -4 in shape (can contain -1).

Example:

- input shape = (2,3,4), shape = (-4,1,2,-2), output shape =(1,2,3,4)
- input shape = (2,3,4), shape = (2,-4,-1,3,-2), output shape = (2,1,3,4)


Defined in src/top/tensor/transform.cc:L437

Parameters: data (Tensor) – Input data. shape (tuple of <>, required) – result – The result Tensor. Tensor
nnvm.symbol.copy(*args, **kwargs)

Copy tensor to another one.

Defined in src/top/tensor/elemwise.cc:L112

Parameters: data (Tensor) – The input tensor. result – The result Tensor. Tensor
nnvm.symbol.negative(*args, **kwargs)

Elemenwise numeric negative

Defined in src/top/tensor/elemwise.cc:L105

Parameters: data (Tensor) – The input tensor. result – The result Tensor. Tensor
nnvm.symbol.leaky_relu(*args, **kwargs)

Leaky version of a Rectified Linear Unit.

y = x > 0 ? x : alpha * x

Defined in src/top/nn/nn.cc:L246

Parameters: data (Tensor) – Input data. alpha (double, optional, default=0.25) – slope coefficient for the negative half axis. result – The result Tensor. Tensor
nnvm.symbol.__add_scalar__(*args, **kwargs)

Defined in src/top/tensor/elemwise.cc:L128

Parameters: data (Tensor) – The input tensor. scalar (double, required) – result – The result Tensor. Tensor
nnvm.symbol.__sub_scalar__(*args, **kwargs)

Tensor substract scalar

Defined in src/top/tensor/elemwise.cc:L134

Parameters: data (Tensor) – The input tensor. scalar (double, required) – result – The result Tensor. Tensor
nnvm.symbol.__rsub_scalar__(*args, **kwargs)

scalar substract Tensor

Defined in src/top/tensor/elemwise.cc:L140

Parameters: data (Tensor) – The input tensor. scalar (double, required) – result – The result Tensor. Tensor
nnvm.symbol.__mul_scalar__(*args, **kwargs)

Tensor multiplies scalar

Defined in src/top/tensor/elemwise.cc:L146

Parameters: data (Tensor) – The input tensor. scalar (double, required) – result – The result Tensor. Tensor
nnvm.symbol.__div_scalar__(*args, **kwargs)

Tensor divides scalar

Defined in src/top/tensor/elemwise.cc:L152

Parameters: data (Tensor) – The input tensor. scalar (double, required) – result – The result Tensor. Tensor
nnvm.symbol.__rdiv_scalar__(*args, **kwargs)

scalar divides Tensor

Defined in src/top/tensor/elemwise.cc:L158

Parameters: data (Tensor) – The input tensor. scalar (double, required) – result – The result Tensor. Tensor
nnvm.symbol.__pow_scalar__(*args, **kwargs)

Tensor power scalar

Defined in src/top/tensor/elemwise.cc:L164

Parameters: data (Tensor) – The input tensor. scalar (double, required) – result – The result Tensor. Tensor
nnvm.symbol.__rpow_scalar__(*args, **kwargs)

scalar power Tensor

Defined in src/top/tensor/elemwise.cc:L170

Parameters: data (Tensor) – The input tensor. scalar (double, required) – result – The result Tensor. Tensor
nnvm.symbol.transpose(*args, **kwargs)

Permutes the dimensions of an array.

Examples:

x = [[ 1, 2],
[ 3, 4]]

transpose(x) = [[ 1.,  3.],
[ 2.,  4.]]

x = [[[ 1.,  2.],
[ 3.,  4.]],

[[ 5.,  6.],
[ 7.,  8.]]]

transpose(x) = [[[ 1.,  5.],
[ 3.,  7.]],

[[ 2.,  6.],
[ 4.,  8.]]]

transpose(x, axes=(1,0,2)) = [[[ 1.,  2.],
[ 5.,  6.]],

[[ 3.,  4.],
[ 7.,  8.]]]


Defined in src/top/tensor/transform.cc:L578

Parameters: data (Tensor) – Source input axes (, optional, default=[]) – Target axis order. By default the axes will be inverted. result – The result Tensor. Tensor
nnvm.symbol.broadcast_to(*args, **kwargs)

Broadcasts the input array to a new shape.

Broadcasting is a mechanism that allows NDArrays to perform arithmetic operations with arrays of different shapes efficiently without creating multiple copies of arrays. Also see, Broadcasting for more explanation.

Broadcasting is allowed on axes with size 1, such as from (2,1,3,1) to (2,8,3,9). Elements will be duplicated on the broadcasted axes.

For example:

broadcast_to([[1,2,3]], shape=(2,3)) = [[ 1.,  2.,  3.],
[ 1.,  2.,  3.]])


The dimension which you do not want to change can also be kept as 0 which means copy the original value. So with shape=(2,0), we will obtain the same result as in the above example.

Parameters: data (Tensor) – Input data. shape (, optional, default=[]) – The shape of the desired array. We can set the dim to zero if it’s same as the original. E.g A = broadcast_to(B, shape=(10, 0, 0)) result – The result Tensor. Tensor
nnvm.symbol.sum(*args, **kwargs)

Computes the sum of array elements over given axes.

Example:

data = [[[1,2],[2,3],[1,3]],
[[1,4],[4,3],[5,2]],
[[7,1],[7,2],[7,3]]]

sum(data, axis=1)
[[  4.   8.]
[ 10.   9.]
[ 21.   6.]]

sum(data, axis=[1,2])
[ 12.  19.  27.]


Defined in src/top/tensor/reduce.cc:L123

Parameters: data (Tensor) – The input axis (, optional, default=[]) – The axis or axes along which to perform the reduction. The default, axis=(), will compute over all elements into a scalar array with shape (1,).If axis is int, a reduction is performed on a particular axis. If axis is a tuple of ints, a reduction is performed on all the axes specified in the tuple. If exclude is true, reduction will be performed on the axes that are NOT in axis instead. keepdims (boolean, optional, default=0) – If this is set to True, the reduced axes are left in the result as dimension with size one. exclude (boolean, optional, default=0) – Whether to perform reduction on axis that are NOT in axis instead. result – The result Tensor. Tensor
nnvm.symbol.min(*args, **kwargs)

Computes the min of array elements over given axes.

Defined in src/top/tensor/reduce.cc:L133

Parameters: data (Tensor) – The input axis (, optional, default=[]) – The axis or axes along which to perform the reduction. The default, axis=(), will compute over all elements into a scalar array with shape (1,).If axis is int, a reduction is performed on a particular axis. If axis is a tuple of ints, a reduction is performed on all the axes specified in the tuple. If exclude is true, reduction will be performed on the axes that are NOT in axis instead. keepdims (boolean, optional, default=0) – If this is set to True, the reduced axes are left in the result as dimension with size one. exclude (boolean, optional, default=0) – Whether to perform reduction on axis that are NOT in axis instead. result – The result Tensor. Tensor
nnvm.symbol.max(*args, **kwargs)

Computes the max of array elements over given axes.

Defined in src/top/tensor/reduce.cc:L128

Parameters: data (Tensor) – The input axis (, optional, default=[]) – The axis or axes along which to perform the reduction. The default, axis=(), will compute over all elements into a scalar array with shape (1,).If axis is int, a reduction is performed on a particular axis. If axis is a tuple of ints, a reduction is performed on all the axes specified in the tuple. If exclude is true, reduction will be performed on the axes that are NOT in axis instead. keepdims (boolean, optional, default=0) – If this is set to True, the reduced axes are left in the result as dimension with size one. exclude (boolean, optional, default=0) – Whether to perform reduction on axis that are NOT in axis instead. result – The result Tensor. Tensor
nnvm.symbol.broadcast_add(*args, **kwargs)

Returns element-wise sum of the input arrays with broadcasting.

Example:

x = [[ 1.,  1.,  1.],
[ 1.,  1.,  1.]]

y = [[ 0.],
[ 1.]]

broadcast_add(x, y) = [[ 1.,  1.,  1.],
[ 2.,  2.,  2.]]


Parameters: lhs (Tensor) – first input rhs (Tensor) – second input result – The result Tensor. Tensor
nnvm.symbol.broadcast_sub(*args, **kwargs)

Returns element-wise difference of the input arrays with broadcasting.

Example:

x = [[ 1.,  1.,  1.],
[ 1.,  1.,  1.]]

y = [[ 0.],
[ 1.]]

broadcast_sub(x, y) = [[ 1.,  1.,  1.],
[ 0.,  0.,  0.]]


Parameters: lhs (Tensor) – first input rhs (Tensor) – second input result – The result Tensor. Tensor
nnvm.symbol.broadcast_mul(*args, **kwargs)

Returns element-wise product of the input arrays with broadcasting.

Example:

x = [[ 1.,  1.,  1.],
[ 1.,  1.,  1.]]

y = [[ 0.],
[ 1.]]

broadcast_mul(x, y) = [[ 0.,  0.,  0.],
[ 1.,  1.,  1.]]


Parameters: lhs (Tensor) – first input rhs (Tensor) – second input result – The result Tensor. Tensor
nnvm.symbol.broadcast_div(*args, **kwargs)

Returns element-wise division of the input arrays with broadcasting.

Example:

x = [[ 6.,  6.,  6.],
[ 6.,  6.,  6.]]

y = [[ 2.],
[ 3.]]

broadcast_div(x, y) = [[ 3.,  3.,  3.],
[ 2.,  2.,  2.]]