Given that $\eta$ refers to learning rate and $x_i$ refers to the $i^{th}$ input to the neuron, Which of the followings most suitably describes the weight updation rule of a Kohonen SOM? (where 'j' refers to the $j^{th}$ neuron in the lattice)
Understanding Kohonen SOM Weight Updates
The Self-Organizing Map (SOM), developed by Teuvo Kohonen, is a powerful type of artificial neural network used for unsupervised learning. Its main purpose is to simplify complex, high-dimensional data by reducing its dimensionality while preserving the essential topological relationships present in the original data. This is achieved through a competitive learning process where neurons in the map compete to represent input data, and then adjust their internal weights accordingly.
The Weight Update Mechanism in Kohonen SOMs
When an input vector, denoted as $x$, is presented to the SOM, the network identifies the neuron whose weight vector $W_j$ is most similar (typically using Euclidean distance) to the input vector $x$. This most similar neuron is called the "best matching unit" (BMU) or the "winning neuron". Following this competition, the weights of the winning neuron, and potentially its neighbors within a defined radius on the map grid, are adjusted to become even more like the input vector $x$.
The core idea behind the weight update rule is to progressively refine the weights so that neurons specialize in representing specific regions of the input space. The adjustment process is influenced by:
- Learning Rate ($\eta$): This parameter controls the magnitude of the weight changes in each learning step. Typically, the learning rate starts high and decreases over time, allowing for larger adjustments initially and finer tuning later.
- Neighborhood Function: This function determines how strongly the weights of neurons other than the BMU are updated. Neurons closer to the BMU on the map grid are updated more significantly than those farther away.
- Input Vector ($x_i$): The specific input data point being processed influences the direction of the weight adjustment.
- Current Weights ($W_{ij}(\text{old})$): The existing weights of the neuron also play a role, as the update rule modifies them based on their current values.
Analyzing the Weight Updation Rule Options
Let's analyze the provided options for the weight update rule, where $W_{ij} (\text{new})$ represents the updated weight, $W_{ij} (\text{old})$ is the weight before the update, $\eta$ is the learning rate, and $x_i$ is the $i^{th}$ component of the input vector:
| Option | Formula | Analysis |
|---|---|---|
| 1 | $W_{ij} (\text{new}) = (1-\eta)*W_{ij} (\text{old}) + \eta*x_i$ |
This formula represents a weighted interpolation between the old weight and the input vector. The term $(1-\eta)*W_{ij} (\text{old})$ retains a portion of the previous weight, ensuring stability and gradual learning. The term $\eta*x_i$ pulls the weight towards the current input vector. This structure effectively balances preserving past learning with adapting to new information, making it a suitable representation for SOM weight updates, especially when focusing on the core mechanism of adjusting weights towards the input. It can be seen as moving the weight $\eta$ fraction of the way from $W_{ij}(\text{old})$ towards $x_i$. |
| 2 | $W_{ij} (\text{new}) = \eta*W_{ij} (\text{old}) + (1-\eta)*x_i$ |
In this option, the roles of $\eta$ and $(1-\eta)$ are swapped between the old weight and the input. When the learning rate $\eta$ is small (as it usually is during learning), this formula would give more importance to the old weight, hindering adaptation, which is contrary to the goal of learning. |
| 3 | $W_{ij} (\text{new}) = (1-\eta)*W_{ij} (\text{old}) + x_i$ |
This option lacks the learning rate ($\eta$) scaling factor for the input term ($x_i$). Without this scaling, the influence of the input vector might be too dominant, potentially leading to unstable weight adjustments or excessively large weight values, especially if $\eta$ is not extremely small. |
| 4 | $W_{ij} (\text{new}) = W_{ij}(\text{old}) + \eta*x_i$ |
This formula represents adding a scaled version of the input vector directly to the old weight. While it attempts to move the weight towards the input, it doesn't incorporate the damping effect provided by the $(1-\eta)$ factor. This can lead to more abrupt changes and potentially make the learning process less stable compared to rules that balance old and new information. |
Conclusion on Kohonen SOM Weight Rule
Based on the analysis, the weight update rule that most appropriately captures the principle of adjusting weights towards the input vector while maintaining learning stability is:
$W_{ij} (\text{new}) = (1-\eta)*W_{ij} (\text{old}) + \eta*x_i$
This formula correctly balances the influence of the neuron's previous weight state and the current input pattern, modulated effectively by the learning rate $\eta$. This balance is fundamental to the gradual convergence and topological mapping capabilities of the Kohonen SOM.
Important Questions from Artificial Neural Network
In a feed forward neural network with the following specifications:
Input layer has 4 neurons, hidden layer has 3 neurons and output layer has 2 neurons using the sigmoid activation function for given input values [0.5, 0.8, 0.2, 0.6] as well as the initial weights for the connections.
W1: [0.1, 0.3, 0.5, 0.2]
W2: [0.2, 0.4, 0.6, 0.2] Input layer to hidden layer weights
W3: [0.3, 0.5, 0.7, 0.2]
W4: [0.4, 0.1, 0.3]
W5: [0.5, 0.2, 0.4] Hidden layer to output layer weights
What is the output of the output layer when the given input values are passed through neural network? Round the answer to two decimal places :
Consider the following table defining the sample inputs and corresponding target values for a perceptron model.
Sample No x1 x2 target w1 w2 S1 0 0 0 0 0 S2 0 1 1 S3 1 0 1 S4 1 1 1
What shall be the value of updated weights after applying all the samples S1 to S4 (in the order S1, S2, S3, S4) to this model. Given that the initial weights w1=0, w2=0, learning rate =0.1 and no bias is involved in the perceptron. The activation function for this perceptron is given below
$y_{observed} = \{ \begin{matrix} 1 & \text{if } y_{in} > 0 \\ 0 & \text{if } y_{in} < 0 \end{matrix}$In a multilayer perceptron, an error signal originates at an output neuron of the network and propagates ___________ through the network.
- Which of the following is true about discrete hopfield network ?
(i) may be modified to a continuous model
(ii) can be used for associative memory problems
(iii) nodes of the network have a continuous graded output
(iv) energy of the network increases continuously with time Consider the two class classification task that consists of the following points :
Class \(C_{1}\) : [-1, -1], [-1, 1], [1, -1]
Class \(C_{2}\) : [1, 1]
The decision boundary between the two classes \(C_{1}\) and \(C_{2}\) using single perceptron is given by :
