^{1}

^{1}

^{2}

^{1}

^{2}

A novel algorithm for decoding a general rate

With the rapid growth of digital communication systems in the recent years, lowering the power consumption and increasing the reliability of the system can both be achieved by using error-control coding (ECC). Convolutional code is a popular class of forward error-correcting codes, commonly used in space and satellite communication and digital video broadcasting [

Recently, researchers are finding different alternatives to VA, hoping to overcome this challenge. This has lead to many suboptimal decoding techniques. One of the techniques that have attracted considerable attention is one that is based on neural networks (NNs).

Neural networks (NNs) are characterised from the biological structure of the mammalian brain. There are several properties which make such structure suitable for digital communication applications. Some of these properties include highly parallelised structure, adaptive processing, self-organisation, and efficient hardware implementation [

Initially neural networks have only been applied for predicting errors in turbo decoders [

In 2003, there was a report published that proposed a new decoding algorithm based on recurrent neural network (RNN) for a generalised rate 1/

Unlike most existing convolutional decoders, which are based on trellis diagrams, the RNN decoder is a completely new approach to the decoding problem. Therefore it is important to note that the proposed method in this paper is not an implementation or optimisation of the existing trellis-based Viterbi or BCJR algorithms. Therefore all the conventional characteristics of the trellis decoding are not applicable in this paper. However, since the well-known Turbo decoder is also based on convolutional codes, the proposed RNN decoder can also be applied. The possibility of using the soft decision output of a 1/

Another unique advantage of the RNN decoder is on its computational speed. Most of the modern high-speed convolutional decoders are trellis based, in which high decoding speed/throughput is achieved through introducing parallelism in the existing algorithm and careful hardware design during implementation. Two good representations of such high-speed decoders for convolutional codes can be found in [

This paper contains the following sections. Section

A general rate

The structure diagram of the encoder, AWGN channel, and the received data.

The bits contained in the

Therefore the encoder can be represented by a large matrix that contains all submatrices from each subencoder, shown as

In order to reduce the mathematical complexities, polar mapping of additive group

Therefore the encoding becomes a process of mapping an

The main task for the decoder is to estimate a sequence of message bits that is the closest to the transmitted source message. Hence the decoding procedure can be redefined as a problem of finding the minimum difference between the message sequence to the one received. More strictly, it then becomes a problem of minimising the

Regarding the partial derivative, the previous investigation [

Therefore, if a nonlinear activation function

The GDA formula implicitly states that the decoding process has become an iterative procedure, similar to Turbo codes. Theoretically a larger number of iterations should always yield a more satisfactory result, because the estimation should tend closer to the actual value after successive iterations. The cost on the other hand is the excess decoding time required. Therefore at this point, some strategies are investigated in order to find the optimum trade-off between the performance and time. In this section three methods are presented to deal with this problem.

The most trivial method would be simply to fix on a predetermined number of iterations on the decoder, regardless of the result of estimation. All the received information is forced to pass through the required number of iterations before producing any result. The distinct advantages of this method are that it is very simple to implement, and also the overall decoding performance is totally under the designer’s control. This method is most suitable for situations where the optimum decoding result is required, regardless of the processing time.

However, without sufficient trials and the prior knowledge on the behaviour of the encoder/decoder, setting an “overly-estimated” large number would not yield the extra performance gain as expected, compared to a smaller iteration, and it also wastes much unnecessary decoding time too.

The second approach is to set a stopping criterion on the decoder prior to decoding. The decoding process would only terminate once this criterion has been met, regardless of the number of iterations it has gone through. This is the initial attempt to remove the constraint on the iteration number, and the decision to terminate is based on the result obtained from each iteration cycle. The main difference from the first method is that each code vector will not necessarily finish decoding at the same time; hence the time and the performance is no longer within the designer’s control. The aim of using this method is to reduce the processing time required by minimising the number of iteration.

One of the criterions is defined by the following condition: terminate the decoding if two successive iterations yield the same estimate of the source message. The underlying assumption is that since two successive iterations give the same estimate, then the estimate must be very close to the actual value; hence there is no point to continue searching further. In another word, this criterion is restricting the decoder to stop searching once it found the first minimum of the noise energy function. This minimum may be a local or global minimum point. However this criterion lacks the sophistication to continue searching for the global optimum solution. Therefore some performance loss is inevitable for its small decoding time. This finding is later reinforced from the simulation results. Therefore this solution is more suitable in places where the decoding time is at a higher priority compared to its performance.

From the first two approaches we realise that there is an inevitable trade-off between the desired decoding accuracy and the process time. The first method would always yield a much better result than the second but requires a much longer decoding time, especially if it is restricted to a high number of iterations, whereas the second approach can reduce the required processing time with a degraded decoding performance. In practice however, it is desirable for the decoder to hold both advantages. In this paper, a method is proposed that is not a separate approach from the previous two, but merely a combination of both, or can be regarded as an extension of the second one.

While still employing the same simple criterion on the decoder, now the decoded estimate is forced to pass through a minimum number of fixed iterations before the criterion can be triggered. The idea of this approach is raised from observing the simulation results, noticing that the errors reduce drastically after only initial few cycles of iterations. Hence this minimum threshold value can be a very small integer; thus not much time will be wasted. Furthermore, it is observed from the simulations that this slight modification can significantly increase the performance of the simple stopping criterion, as described previously. Therefore, at the time of investigation, this becomes a favourable approach that yields the best trade-off in terms of the decoding performance and the processing time.

In order to verify the theoretical decoding capability of the RNN decoder, two encoders with rate 1/2 and 2/3 are investigated.

Consider a simple rate 1/2 convolutional encoder with a constraint length of 3, which has the impulse generator matrix

The structure of this encoder is illustrated as in Figure

The structural diagram of the rate 1/2 convolutional encoder.

Using the general GDA formula with the specified value of

The neuron diagram of the RNN decoder in Example 1.

The place where multiplication occurs is marked with a circle in the neuron diagram. Each estimated bit needs to pass through a sigmoid activation function before feeding back into the network for the next iteration. This process carries on before it is terminated either by reaching to a fixed number of iterations or by triggering a stopping criterion as mentioned above.

In addition, the processing speed can be further increased if multiple neurons are used to estimate the entire message sequence concurrently. This parallel signal processing ability can thus accommodate a higher data transmission rate.

The second example is of an encoder with a higher rate of 2/3, which has the impulse generator matrix as

The structural diagram of this encoder can be referred in Appendix

Figure

The neuron diagram of the RNN decoder in example 2.

A conceptual diagram of the complete neural network decoder.

The main stages in decoding are outlined as follows. At first the receiver observes the incoming code words in parallel. In this case, it would be denoted as

In addition, a higher code rate can increase the speed of signal transmission, as

A simulated digital communication system using the two encoders described in the previous section was designed in order to verify the bit error rate (BER) performance of the RNN decoding scheme. For the sake of convenience, the rate 1/2 encoder and the rate 2/3 encoder shall be referred as encoders A and B, respectively, from this point on.

All the simulations were conducted by calculating the BER of decoding the encoded messages that are randomly generated and passed through an AWGN channel. The message sequences are transmitted in packets of different sizes. Each packet is encoded, transmitted, and decoded separately, and therefore the result from each packet can be regarded as independent. Numerous numbers of packets (in order of thousands) are required in order to truly reflect the BER in the specified signal to noise (SNR) range [

The main objective in this simulation is to investigate the effect of different packets sizes for the RNN decoder. The simulated

The simulations results are summarised in Figures

BER of the RNN decoder for a rate 1/2 encoder in different packet sizes.

BER of the RNN decoder for a rate 2/3 encoder in different packet sizes.

Nevertheless, the BER for encoder A is indeed comparable to the conventional Viterbi decoder. Similarly, this simulation also shows that the performance margin between encoder B and VA is close too, when the packet sizes are relatively small.

Therefore it can be concluded that when a multiple input encoder is going to be employed into a system with neural network decoders, then the size of the transmitted packet will have an impact on the decoding accuracy. As long as the packet size is kept as small as possible, the RNN decoder can provide an impressive performance.

A simulation was carried out to observe any changes in the BER for the RNN decoder in different numbers of iterations. This information is especially useful in implementing the first method of the stopping criterion as discussed earlier. In this investigation the SNR has to be fixed at 2 dB, and then the BER were recorded for the RNN decoder undergoing different numbers of iterations.

Figure

BER of the RNN decoder for a rate 1/2 encoder in different iterations at an SNR of 2 dB.

Conversely, such problem is not so obvious for encoder B, as shown in Figure

BER of the RNN decoder for a rate 2/3 encoder in different iterations at an SNR of 2 dB.

The objective of this simulation is to compare the decoding performance of employing the simple criterion and the extension of this criterion described in Sections

For both encoders, the maximum number of iterations is restricted to 50, in case that the stopping criterion (SC) is never fulfilled, and there is no obvious performance gain beyond that point (shown from Figures

When the extension of SC is applied, both decoders would need an extra few cycles before the decoding process terminates. This is reflected on about 10% increase in the decoding time. However, for the slight increase in time, the BER has decreased significantly after this modification. Figures

BER of the RNN decoder for a rate 1/2 encoder under two different stopping criterions.

BER of the RNN decoder for a rate 2/3 encoder under two different stopping criterions.

The structural diagram of the rate 2/3 convolutional encoder.

The simple criterion however, due to its simplicity in the design, would terminate decoding when the estimate is located on first local minimum of the noisy energy function. Although this method uses the least time in simulation, the results are not as satisfactory as expected. The difference between SC and the extension of SC is more pronounced for encoder B (multiple inputs) than for encoder A (single input). In short, the extension of the simple stopping criterion is an efficient way of yielding the optimal decoding accuracy in a minimum amount of time.

In conclusion, a mathematical model of a general rate

In order to verify the theoretical findings and performance of the RNN decoder from various aspects, simulations were carried out by using two encoders with rates 1/2 and 2/3. All results show that the packet size and the number of iterations have certain influence on the BER performance of the decoder. However, as long as the packet size is kept relatively small and employs a suitable stopping criterion, then the RNN decoder achieves a similar decoding accuracy with the traditional Viterbi decoders, with a reduction in complexity. Therefore this decoding algorithm, which has the same decoding performance and has more desirable characteristics that are superior to the conventional decoder, will possess great potentials for future communication systems.

Average number of iterations required to reach the stopping criterion.

Encoder A (rate 1/2) | Encoder B (rate 2/3) | |||

Stopping criterion | Extension of SC | Stopping criterion | Extension of SC | |

Average iterations | 3.04 | 6.02 | 4.6 | 6.3 |

In an earlier paper, Secker has analysed the computational complexity and speed of the RNN decoder [

The first assumption is that the RNN decoder has

Equation (

Thus, the number of addition operations required is

Thus, given a fixed number of encoder input

From inspecting equation (

The number of multiplication operations required to evaluate this expression depends on the elements of the generator matrix of that subencoder,

Bearing in mind that the above expression only accounts for one subencoder, therefore the total number of multiplication per neuron for

Thus, given a fixed number of encoder input