Introduction to Oscillatory Networks based on Kuramoto Model

What is an oscillatory network? Oscillatory network is a special type of neural networks that is represented by dynamic system that consists of oscillatory elements. The general distinction from classical neural networks is dynamic output like in real neurons.

There is an assumption that synchronization processes are used to implement cognitive functions, for example, vision, motion or memory. Oscillatory networks use this assumption to encode patterns and features. How the networks does encodes features in synchronization? The answer is simple: oscillators that have the same phase coordinate forms special groups – synchronous ensembles. An oscillatory network might have more than one synchronous ensemble and it means that the network encodes more than one feature where each ensemble of synchronous oscillators encodes only one feature. For example, we want to extract colors from an image – color image segmentation; in this case, each synchronous ensemble of oscillators encodes only one color. Let us consider another example – data clustering, in this case synchronous ensemble encodes only one cluster of data.

How does synchronization look like in the network? Suppose we have network with four oscillators and the first, the second encodes a feature A and the third, and the fourth encodes a feature B. Thus there are two synchronous ensembles [0, 1] and [2, 3] – figure 1.

Lesson 01 - Synchronous ensembles example
Figure 1. Example of two synchronous ensembles – [neuron #1, neuron #2] and [neuron #3, neuron #4].

Spike neuron output is used for simplicity in the figure 1. Here neuron #1 and neuron #2 have the same phase coordinate and at the same time, their phases are different from phases of neuron #3 and #4 that from another synchronous ensemble. Output of neuron depends on type of transfer function, for example, it might have wave-form or even chaotic-form. Let us return to our example, suppose we have image 2×2 where two pixels have blue color and other two has green color. Each ensemble encodes only one feature as we mentioned it before, so for such image neuron #1 and #2 correspond to pixel #1 and #2 (blue color) and neuron #3 and #4 correspond to pixel #3 and #4 (green color).

Now we understood how oscillatory network encodes features, but there is next question: “How to synchronize oscillators?”. We need to synchronize oscillators that should correspond to the same feature. The easiest way to do it is to use Kuramoto model. Original Kuramoto model ensures global synchronization in a full-interconnected network that uses phase oscillator:

\dot{\theta}_{i}=\omega_{i} + \sum_{j=0}^{N}K_{ij}\sin(\theta_{j} - \theta_{i})

Here θi – phase that is considered as a current state of oscillator. ωi – oscillator intrinsic frequency that can be considered as a oscillator bias. Kij – is a connection between i and j oscillators. Oscillatory network based on Kuramoto model is able to ensure global synchronization when all oscillators have the same phase. Let us simulate the oscillatory network using 100 oscillators using pyclustering library 0.6.0:

from pyclustering.nnet import conn_type;
from pyclustering.nnet.sync import sync_network, sync_visualizer;

# amout of oscillators in the network
num_osc = 100;

# multiplier for coupling strength between oscillators
k = 1;

# type of network structure - all-to-all (full-interconnected)
conn = conn_type.ALL_TO_ALL;

# permission to use core of the library to reach maximum performance
ccore_flag = True;

# create the oscillatory network
network = sync_network(num_osc, k, type_conn = conn, ccore = ccore_flag);

# simulate it until global synchronization and collect out
sync_output_dynamic = network.simulate_dynamic(collect_dynamic = True);

# show output dynamic of oscillators in the network
sync_visualizer.show_output_dynamic(sync_output_dynamic);

# show animation of synchronization process in polar coordinates
sync_visualizer.animate_output_dynamic(sync_output_dynamic);

The model of oscillatory network is simple and it can be easily implemented. Moreover, simple Euler method can be used for solving the differential equation for each oscillator, no need to implement classical Runge-Kutta fourth order method. Nevertheless, it is enough for review to use pyclustering library, therefore the library is used in this technical note.

The presented code simulate oscillatory network where global synchronization is observed at the end of simulation. It means that oscillators have the phase coordinate. Amount of simulation steps that is required to reach global synchronization is different for various network structures as you can see it in figure 2.

network_structures_synchronizations
Figure 2. Synchronization process in oscillatory networks with various structures.

As you can see from example above synchronization is reached in all networks. Pay attention to forms of output dynamic for grid and bidirectional list structures. I have prepared several movies devoted to the global synchronization in Kuramoto model.

Kuramoto model is interesting system and many researches devoted to it. Using pyclustering library you can perform your own investigation and study the model. And I hope this article encourage you to check what this model can do, try to play with coupling strength, added new components of the differential equation such as noise or try to substitute sinus function to another.

Now we have next question that should be clarified to understand how to apply it for practical problem, for example, cluster analysis, graph coloring, image segmentation and so on. We have global synchronization, but how to get local synchronization when oscillatory network has more than one synchronous ensemble? This issue will be considered in the next technical note – Sync: Oscillatory based Clustering Algorithm.

Andrei

Advertisements