Sparse Coding Task

From GM-RKB
(Redirected from sparse coding)
Jump to navigation Jump to search

A Sparse Coding Task is a vector coding task that requires the creation of sparse vectors.



References

2016

  • (Wikipedia, 2016) ⇒ http://wikipedia.org/wiki/neural_coding#Sparse_coding Retrieved:2016-3-31.
    • The sparse code is when each item is encoded by the strong activation of a relatively small set of neurons. For each item to be encoded, this is a different subset of all available neurons.

      As a consequence, sparseness may be focused on temporal sparseness ("a relatively small number of time periods are active") or on the sparseness in an activated population of neurons. In this latter case, this may be defined in one time period as the number of activated neurons relative to the total number of neurons in the population. This seems to be a hallmark of neural computations since compared to traditional computers, information is massively distributed across neurons. A major result in neural coding from Olshausen and Field [1] is that sparse coding of natural images produces wavelet-like oriented filters that resemble the receptive fields of simple cells in the visual cortex. The capacity of sparse codes may be increased by simultaneous use of temporal coding, as found in the locust olfactory system.

      Given a potentially large set of input patterns, sparse coding algorithms (e.g. Sparse Autoencoder) attempt to automatically find a small number of representative patterns which, when combined in the right proportions, reproduce the original input patterns. The sparse coding for the input then consists of those representative patterns. For example, the very large set of English sentences can be encoded by a small number of symbols (i.e. letters, numbers, punctuation, and spaces) combined in a particular order for a particular sentence, and so a sparse coding for English would be those symbols.

  1. Olshausen, Bruno A; Field, David J. “Emergence of simple-cell receptive field properties by learning a sparse code for natural images." Nature 381.6583 (1996): 607-609. http://www.cs.ubc.ca/~little/cpsc425/olshausen_field_nature_1996.pdf

2012

2006

2002

1997