Tanhshrink Activation Function

From GM-RKB
Jump to: navigation, search

A Tanhshrink Activation Function is a Hyperbolic Tangent-based Activation Function that is defined as [math] f(x)=x-Tanh(x)[/math]



References

2018

  • (Pyttorch, 2018) ⇒ http://pytorch.org/docs/master/nn.html#tanhshrink
    • QUOTE: class torch.nn.Tanhshrink source

      Applies element-wise, [math]Tanhshrink(x)=x−Tanh(x)[/math]

      Shape:

      *** Input: [math](N,∗)[/math] where * means, any number of additional dimensions

      • Output: [math](N,∗)[/math], same shape as the input.
Examples:
>>> m = nn.Tanhshrink()
>>> input = autograd.Variable(torch.randn(2))
>>> print(input)
>>> print(m(input))