Difference between revisions of "Trained Deep Neural Network"

From GM-RKB
Jump to: navigation, search
m (Text replacement - "[[::" to "[[")
 
Line 19: Line 19:
  
 
=== 2019 ===
 
=== 2019 ===
* ([[2019_TheLotteryTicketHypothesisFindi|Frankle & Carbin, 2019]]) ⇒ [[::Jonathan Frankle]], and [[::Michael Carbin]]. ([[::2019]]). “[https://arxiv.org/pdf/1803.03635.pdf The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networks].” In: International Conference on Learning Representations.  
+
* ([[2019_TheLotteryTicketHypothesisFindi|Frankle & Carbin, 2019]]) ⇒ [[Jonathan Frankle]], and [[Michael Carbin]]. ([[2019]]). “[https://arxiv.org/pdf/1803.03635.pdf The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networks].” In: International Conference on Learning Representations.  
 
** QUOTE: ...  In practice, [[trained deep neural network|neural network]]s tend to be dramatically [[overparameterized]]. </s> [[Distillation]] ([[Ba & Caruana, 2014]]; [[Hinton et al., 2015]]) and [[pruning]] ([[LeCun et al., 1990]]; [[Han et al., 2015]]) rely on the fact that [[parameter]]s can be reduced while preserving [[accuracy]]. </s> Even with sufficient [[capacity]] to [[memorize training data]], [[network]]s naturally [[learn]] [[simpler function]]s ([[Zhang et al., 2016]]; [[Neyshabur et al., 2014]]; [[Arpit et al., 2017]]). </s> [[Contemporary experience]] ([[Bengio et al., 2006]]; [[Hinton et al., 2015]]; [[Zhang et al., 2016]]) and Figure 1 suggest that [[overparameterized network]]s are easier to [[train]]. </s> [[Frankle & Carbin, 2019|We]] show that [[dense network]]s contain [[sparse subnetworks capable of learning]] on their own starting from their original [[initialization]]s. </s> Several other [[research direction]]s aim to [[NNet training|train]] [[small NNet|small]] or [[sparse network]]s. </s> ...
 
** QUOTE: ...  In practice, [[trained deep neural network|neural network]]s tend to be dramatically [[overparameterized]]. </s> [[Distillation]] ([[Ba & Caruana, 2014]]; [[Hinton et al., 2015]]) and [[pruning]] ([[LeCun et al., 1990]]; [[Han et al., 2015]]) rely on the fact that [[parameter]]s can be reduced while preserving [[accuracy]]. </s> Even with sufficient [[capacity]] to [[memorize training data]], [[network]]s naturally [[learn]] [[simpler function]]s ([[Zhang et al., 2016]]; [[Neyshabur et al., 2014]]; [[Arpit et al., 2017]]). </s> [[Contemporary experience]] ([[Bengio et al., 2006]]; [[Hinton et al., 2015]]; [[Zhang et al., 2016]]) and Figure 1 suggest that [[overparameterized network]]s are easier to [[train]]. </s> [[Frankle & Carbin, 2019|We]] show that [[dense network]]s contain [[sparse subnetworks capable of learning]] on their own starting from their original [[initialization]]s. </s> Several other [[research direction]]s aim to [[NNet training|train]] [[small NNet|small]] or [[sparse network]]s. </s> ...
  

Latest revision as of 22:37, 26 March 2020

A Trained Deep Neural Network is a trained neural network that is a deep neural network.



References

2020

2019