3.1.4. clumsygrad.activation#
This module provides various activation functions for tensors.
- clumsygrad.activation.tanh(tensor: Tensor) Tensor[source]#
Element-wise hyperbolic tangent activation function.
\[\tanh(x) = \frac{e^x - e^{-x}}{e^x + e^{-x}}\]- Parameters:
tensor – Input tensor.
- Returns:
A new tensor containing the hyperbolic tangent of the input tensor.
- Return type:
- clumsygrad.activation.relu(tensor: Tensor) Tensor[source]#
Element-wise Rectified Linear Unit (ReLU) activation function.
\[\text{ReLU}(x) = \max(0, x)\]- Parameters:
tensor – Input tensor.
- Returns:
A new tensor containing the ReLU activation of the input tensor.
- Return type:
- clumsygrad.activation.sigmoid(tensor: Tensor) Tensor[source]#
Element-wise sigmoid activation function.
\[\sigma(x) = \frac{1}{1 + e^{-x}}\]- Parameters:
tensor – Input tensor.
- Returns:
A new tensor containing the sigmoid activation of the input tensor.
- Return type:
- clumsygrad.activation.softmax(tensor: Tensor, axis=-1) Tensor[source]#
Element-wise softmax activation function.
\[\text{softmax}(x_i) = \frac{e^{x_i}}{\sum_j e^{x_j}}\]- Parameters:
tensor – Input tensor.
axis – Axis along which to compute the softmax. Default is -1 (last axis).
- Returns:
A new tensor containing the softmax activation of the input tensor.
- Return type:
Note
The data in the input tensor is shifted by subtracting the maximum value along the specified axis to prevent overflow in the exponential computation.