Synaptic weight

In neuroscience and computer science, synaptic weight refers to the strength or amplitude of a connection between two nodes, corresponding in biology to the amount of influence the firing of one neuron has on another. The term is typically used in artificial and biological neural network research.^[1]

Computation

In a computational neural network, a vector or set of inputs $\textbf{x}$ and outputs $\textbf{y}$, or pre- and post-synaptic neurons respectively, are interconnected with synaptic weights represented by the matrix $w$, where for a linear neuron

$y_j = \sum_i w_{ij} x_i ~~\textrm{or}~~ \textbf{y} = w\textbf{x}$.

The synaptic weight is changed by using a learning rule, the most basic of which is Hebb's rule, which is usually stated in biological terms as

Neurons that fire together, wire together.

Computationally, this means that if a large signal from one of the input neurons results in a large signal from one of the output neurons, then the synaptic weight between those two neurons will increase. The rule is unstable, however, and is typically modified using such variations as Oja's rule, radial basis functions or the backpropagation algorithm.

Biology

For biological networks, the effect of synaptic weights is not as simple as for linear neurons or Hebbian learning. However, biophysical models such as BCM theory have seen some success in mathematically describing these networks.

In the mammalian central nervous system, signal transmission is carried out by interconnected networks of nerve cells, or neurons. For the basic pyramidal neuron, the input signal is carried by the axon, which releases neurotransmitter chemicals into the synapse which is picked up by the dendrites of the next neuron, which can then generate an action potential which is analogous to the output signal in the computational case.

The synaptic weight in this process is determined by several variable factors:

• How well the input signal propagates through the axon (see myelination),
• The amount of neurotransmitter released into the synapse and the amount that can be absorbed in the following cell (determined by the number of AMPA and NMDA receptors on the cell membrane and the amount of intracellular calcium and other ions),
• The number of such connections made by the axon to the dendrites,
• How well the signal propagates and integrates in the postsynaptic cell.

The changes in synaptic weight that occur is known as synaptic plasticity, and the process behind long-term changes (long-term potentiation and depression) is still poorly understood. Hebb's original learning rule was originally applied to biological systems, but has had to undergo many modifications as a number of theoretical and experimental problems came to light.

References

1. Iyer, R; Menon, V; Buice, M; Koch, C; Mihalas, S (2013). "The influence of synaptic weight distribution on neuronal population dynamics". PLoS Computational Biology 9 (10): e1003248. doi:10.1371/journal.pcbi.1003248. PMC 3808453. PMID 24204219.

See also

• Neural network
• Synaptic plasticity
• Hebbian theory