^{1, 2}

^{1}

^{2}

Being inspired by the Hopfield neural networks (Hopfield (1982) and Hopfield and Tank (1985)) and the nonlinear sigmoid power control algorithm for cellular radio systems in Uykan and Koivo (2004), in this paper, we present a novel discrete recurrent nonlinear system and extend the results in Uykan (2009), which are for autonomous linear systems, to nonlinear case. The proposed system can be viewed as a discrete-time realization of a recently proposed continuous-time network in Uykan (2013). In this paper, we focus on discrete-time analysis and provide various novel key results concerning the discrete-time dynamics of the proposed system, some of which are as follows: (i) the proposed system is shown to be stable in synchronous and asynchronous work mode in discrete time; (ii) a novel concept called Pseudo-SINR (pseudo-signal-to-interference-noise ratio) is introduced for discrete-time nonlinear systems; (iii) it is shown that when the system states approach an equilibrium point, the instantaneous Pseudo-SINRs are balanced; that is, they are equal to a target value. The simulation results confirm the novel results presented and show the effectiveness of the proposed discrete-time network as applied to various associative memory systems and clustering problems.

Artificial neural networks have been an important research area since 1970s. Since then, various biologically inspired neural network models have been developed. Hopfield Neural Networks [

In [

The paper is organized as follows. The proposed recurrent network and its stability features are analyzed in Section

Being inspired by the nonlinear sigmoid power control algorithm for cellular radio systems in [

In (

The proposed network includes both the sigmoid power control in [

Let us call the network in (

Then, the performance index is defined as

In what follows, we examine the evolution of the energy function in (

In asynchronous mode of the proposed network D-SP-SNN in (

if

In asynchronous mode, only one state is updated at an iteration time. Let

Using the error signal definition of (

So, the error signal for state

From (

Above, we examined only the state

The sigmoid function

We define the following system variable, which will be called pseudo-SINR, for the D-SP-SNN in (

Examining the

Prototype vectors are defined as those

In asynchronous mode, choosing the slope of

Since it is asynchronous mode, (

Note that it is straightforward to choose a sufficiently small

We observe from (

From (

In what follows, we examine the evolution of pseudo-SINR

In asynchronous mode, in the D-SP-SNN in (

Let

Without loss of generality, and for the sake of simplicity, let is take

In asynchronous mode, from (

From (

Provided that

Writing (

From (

As seen from (

The results in Propositions

In synchronous mode, all the states are updated at every step

Using (

From (

It is well known that the performance of Hopfield network may highly depend on the parameter setting of the weight matrix (e.g., [

In the simulation part, we examine the performance of the proposed D-SP-SNN in the area of associative memory systems and clustering problem. In Examples

In Examples

In this example of discrete-time networks, there are 8 neurons. The desired prototype vectors are as follows:

The weight matrices

Figure

As seen from Figure

The figure shows the percentage of correctly recovered desired patterns for all possible initial conditions in Example

The desired prototype vectors are

The weight matrices

Figure

The total number of different possible combinations for the initial conditions for this example is 64, 480, 2240 and 7280 for 1-, 2-, 3-, and 4-Hamming distance cases, respectively, which could be calculated by

As seen from Figure

Typical plots for evolution of states in Example

Evolutions of the Lyapunov function in (

The figure shows the percentage of correctly recovered desired patterns for all possible initial conditions in Example

Typical plot for evolutions of states (a) 1 to 8 and (b) 9 to 16 in Example

Evolutions of pseudo-SINRs for the states in Figure

Evolution of Lyapunov function in (

In Examples

Figure

The evolutions of the Lyapunov function in (

Figure

Evolutions of states in Example

Figure

The evolutions of the norm of the difference between the state vector and equilibrium point for pattern 2 in Figure

Figure

The evolutions of the Lyapunov function and the norm of the difference between the state vector and equilibrium point for pattern 3 in Figure

Figure

The evolutions of the Lyapunov function and the norm of the difference between the state vector and equilibrium point for pattern 4 in Figure

(a) Desired pattern 1, distorted pattern 1 (HD = 5), result of HNN-Euler, and result of D-SP-SNN in Example

Evolution of (a) Lyapunov function in (

(a) Desired pattern 2, (b) distorted pattern 2 (HD = 5), (c) result of HNN-Euler, and (d) Result of D-SP-SNN in Example

Evolutions of states (a) 1 to 8, (b) 9 to 16, and (c) 17 to 24 in Example

Evolutions of states (a) 1 to 8, (b) 9 to 16, and (c) 17 to 24 in Example

Evolutions of pseudo-SINRs of states (a) 1 to 8, (b) 9 to 16, and (c) 17 to 24 in Example

Evolutions of the norm of the difference between the state vector and equilibrium point in Example

(a) Desired pattern 3, (b) distorted pattern 3 (HD = 5), (c) result of HNN-Euler, and (d) result of D-SP-SNN in Example

Evolution of (a) Lyapunov function and (b) norm of the difference between the state vector and equilibrium point in Example

(a) Desired pattern 4, (b) distorted pattern 4 (HD = 5), (c) result of HNN-Euler, and (d) result of D-SP-SNN in Example

Evolutions of (a) Lyapunov function and (b) norm of the difference between the state vector and equilibrium point in Example

In this and in the following example, we examine the performance of the proposed D-SP-SNN in clustering problem. Clustering is used in a wide range of applications, such as engineering, biology, marketing, information retrieval, social network analysis, image processing, text mining, finding communities, influencers, and leaders in online or offline social networks. Data clustering is a technique that enables dividing large amounts of data into groups/clusters in an unsupervised manner such that the data points in the same group/cluster are similar and those in different clusters are dissimilar according to some defined similarity criteria. The clustering problem is an NP-complete, and its general solution even for 2-clustering case is not known. It is well known that the clustering problem can be formulated in the form of the Lyapunov function of the HNN. The weight matrix is chosen as the distance matrix of the dataset and is the same for both HNN-Euler and D-SP-SNN.

In what follows, we compare the performance of the proposed D-SP-SNN as compared to its HNN-Euler counterpart as applied to clustering problems for the very same parameter settings. Two-dimensional 16 data points to be bisected are shown in Figure

The evolutions of states in the clustering by HNN-Euler and by D-SP-SNN are shown in Figures

The evolutions of psuedo-SINRs of states in the clustering by D-SP-SNN in Example

The evolutions of Lyapunov function and the norm of the difference between the state vector and equilibrium point in Example

Result of clustering by D-SP-SNN,

Evolutions of states (a) 1 to 8 and (b) 9 to 16 in the clustering by HNN-Euler in Example

Evolutions of states (a) 1 to 8 and (b) 9 to 16 in the clustering by D-SP-SNN in Example

Evolutions of psuedo-SINRs of states (a) 1 to 8 and (b) 9 to 16 in the clustering by D-SP-SNN in Example

Evolution of (a) Lyapunov function and (b) norm of the difference between the state vector and equilibrium point in Example

In this example, there are 40 data points as shown in Figure

Figure

Evolutions of the Lyapunov function and the norm of the difference between the state vector and equilibrium point are shown in Figure

Bisecting clustering results by (a)

Evolution of pseudo-SINRs of states 1 to 8 in Example

Evolution of (a) Lyapunov function and (b) norm of the difference between the state vector and equilibrium point in Example

In this paper, we present and analyze a discrete recurrent nonlinear system which includes the Hopfield neural networks [

The simulation results confirm the novel results (e.g., Pseudo-SINR convergence, etc.) presented and show a superior performance of the proposed network as compared to its Hopfield network counterpart in various associative memory systems and clustering examples. Moreover, the results show that the proposed network minimizes the Lyapunov function of the Hopfield neural networks. The disadvantage of the D-SP-SNN is that it increases the computational burden.

In what follows, we will show the sigmoid function (

The derivative of

Let us assume that

Calculate the sum of outer products of the prototype vectors (Hebb Rule, [

Determine the diagonal matrix

Another choice of

From (

So, the prototype vectors