Neural Network Learnable Parameter

From GM-RKB
Jump to navigation Jump to search

A Neural Network Learnable Parameter is a Model Parameter that can be learned by training neural network.



References

2020

QUOTE: $ \operatorname{PReLU}(\mathrm{x})=\left\{\begin{array}{ll} \mathrm{x}, & \text { if } \mathrm{x} \geq 0 \\ \mathrm{ax}, & \text { otherwise } \end{array}\right. $

Here $a$ is a learnable parameter.

2017

[math]\displaystyle{ \begin{align} e^t_i &= \nu^T \mathrm{tanh}\left(W_hh_i +W_sS_t +b_{attn}\right) \\ a^t &= \mathrm{softmax}\left(e^t \right) \end{align} }[/math] (1)
(2)
where $\nu$, $W_h$, $W_s$ and $b_{attn}$ are learnable parameters.