5.8. clumsygrad.gradcheck#

Numerical gradient-checking utilities.

clumsygrad.gradcheck.check_gradient(fn: Callable[[Tensor], Tensor], tensor: Tensor, eps: float = 0.001, tol: float = 0.01) → tuple[bool, float][source]#

Numerically verify the analytical gradient of fn with respect to tensor.

fn is called with tensor and must return a Tensor built from it. If the output has more than one element, the analytical gradient is computed with an incoming gradient of ones (equivalent to backpropagating from sum(fn(tensor))), and each finite-difference probe uses the same element-sum so the two stay comparable:

\[\frac{\partial \sum \text{fn}(x)}{\partial x_i} \approx \frac{\sum \text{fn}(x_i + \epsilon) - \sum \text{fn}(x_i - \epsilon)}{2\epsilon}\]
Parameters:

  • fn – A function taking tensor and returning a Tensor derived from it.

  • tensor – The tensor (must have requires_grad=True) to check gradients for.

  • eps – Perturbation size for the central-difference approximation.

  • tol – Maximum allowed max-absolute-difference between the analytical and numerical gradients for the check to pass.

Returns:

A tuple (passed, max_abs_diff).

Raises:

ValueError – If tensor does not require gradients.