LeakGAN Model

From GM-RKB
(Redirected from LeakGAN)
Jump to navigation Jump to search

A LeakGAN is a Generative Adversarial Network that leaks information from a feature extraction discriminator and a FuN-based generator to produce long coherent and semantically meaningful text.



References

2018

2018 LongTextGenerationviaAdversaria Fig1.png
Figure 1: An overview of our LeakGAN text generation framework. While the generator is responsible to generate the next word, the discriminator adversarially judges the generated sentence once it is complete. The chief novelty lies in that, unlike conventional adversarial training, during the process, the discriminator reveals its internal state (feature $f_t$) in order to guide the generator more informatively and frequently. (See Methodology Section for more details.)

{|style="border: 0px; text-align:center; border-spacing: 1px; margin: 1em auto; width: 80%"

|- |$f =\mathcal{F}\left(s ; \phi_{f}\right)$ |style="width:5%;text-align:right"|(1) |- |$D_{\phi}(s) =\operatorname{sigmoid}\left(\phi_{l} \cdot \mathcal{F}\left(s ; \phi_{f}\right)\right)=\operatorname{sigmoid}\left(\phi_{l}, f\right)$ |style="width:5%;text-align:right"|(2) |- |+ align="bottom" style="caption-side:top;text-align:center;font-weight:bold"|Discriminator

|}

{|style="border: 0px; text-align:center; border-spacing: 1px; margin: 1em auto; width: 80%" |- |$\hat{g}_{t}, h_{t}^{M} =\mathcal{M}\left(f_{t}, h_{t-1}^{M} ; \theta_{m}\right) $ |style="width:5%;text-align:right"|(3) |- |$g_{t} =\hat{g}_{t} /\left\|\hat{g}_{t}\right\|$ |style="width:5%;text-align:right"|(4) |- |$w_{t}=\psi\left(\sum_{i=1}^{c} g_{t-i}\right)=W_{\psi}\left(\sum_{i=1}^{c} g_{t-i}\right)$ |style="width:5%;text-align:right"|(5) |- |$O_{t}, h_{t}^{W}= \mathcal{W}\left(x_{t}, h_{t-1}^{W} ; \theta_{w}\right)$ |style="width:5%;text-align:right"|(6) |- |$G_{\theta}\left(\cdot \mid s_{t}\right)= \operatorname{sigmoid}\left(O_{t} \cdot w_{t} / \alpha\right)$ |style="width:5%;text-align:right"|(7) |- |$Q\left(f_{t}, g_{t}\right)=\mathbb{E}\left[r_{t}\right]$ |style="width:5%;text-align:right"|(8) |- |$\nabla_{\theta_{m}}^{\mathrm{adv}} g_{t}=-Q\left(f_{t}, g_{t}\right) \nabla_{\theta_{m}} d_{\cos }\left(\mathcal{F}\left(s_{t+c}\right)-\mathcal{F}\left(s_{t}\right), g_{t}\left(\theta_{m}\right)\right)$ |style="width:5%;text-align:right"|(9) |- |+ align="bottom" style="caption-side:top;text-align:center;font-weight:bold"|MANAGER of Generator |}

{|style="border: 0px; text-align:center; border-spacing: 1px; margin: 1em auto; width: 80%" |- |$\nabla_{\theta_{w}} \mathbb{E}_{s_{t-1} \sim G}\left[\sum_{x_{t}} r_{t}^{I} \mathcal{W}\left(x_{t} \mid s_{t-1} ; \theta_{w}\right)\right] =\mathbb{E}_{s_{t-1} \sim G, x_{t} \sim \mathcal{W}\left(x_{t} \mid s_{t-1}\right)}\left[r_{t}^{I} \nabla_{\theta_{w}} \log \mathcal{W}\left(x_{t} \mid s_{t-1} ; \theta_{w}\right)\right] $ |style="width:5%;text-align:right"|(10) |- |$r_{t}^{I}=\frac{1}{c} \sum_{i=1}^{c} d_{\cos }\left(\mathcal{F}\left(s_{t}\right)-\mathcal{F}\left(s_{t-i}\right), g_{t-i}\right)$ |style="width:5%;text-align:right"|(11) |- |+ align="bottom" style="caption-side:top;text-align:center;font-weight:bold"|WORKER of Generator |}

2017

$z_{t}=f^{\text {percept }}\left(x_{t}\right) ; s_{t}=f^{\text {Mspace}}\left(z_{t}\right)$ (1)
$h_{t}^{M}, \hat{g}_{t}=f^{M r n n}\left(s_{t}, h_{t-1}^{M}\right) ; g_{t}=\dfrac{\hat{g}_{t}}{\parallel\hat{g}_{t}\parallel}$ (2)
$w_{t}=\phi\left(\sum_{i=t-c}^{t} g_{i}\right) $ (3)
$h^{W}, U_{t}=f^{W r n n}\left(z_{t}, h_{t-1}^{W}\right) ; \pi_{t}=\operatorname{SoftMax}\left(U_{t} w_{t}\right)$ (4)

2017 FeUdalNetworksforHierarchicalRe Fig1.png
Figure 1. The schematic illustration of FuN.