^{1}

^{2}

^{1}

^{2}

The error mechanisms of iterative message-passing decoders for low-density parity-check codes are studied. A tutorial review is given of the various graphical structures, including trapping sets, stopping sets, and absorbing sets that are frequently used to characterize the errors observed in simulations of iterative decoding of low-density parity-check codes. The connections between trapping sets and deviations on computation trees are explored in depth using the notion of

Prior to 1993, channel codes were typically designed with the goal of maximizing the minimum distance of the code [

With the discovery of turbo codes and the various subsequent iterative decoders, the phenomenon of the error floor has become prominent in practical code design. The term

Low-density parity-check codes with iterative decoding are also known to exhibit error floors [

LDPC codes are most commonly decoded using iterative message-passing decoders such as the min-sum decoder [

This paper attempts to make progress on this problem by integrating the precise, but computational intractable, work of Wiberg with the experimental studies of the error mechanisms observed when iteratively decoding LDPCs. The paper begins with a tutorial review of the existing methods for analyzing the performance of iterative message-passing decoders. Then, the notion of a

The following model for channel coding is used throughout this paper. First, a vector

Before a codeword

for all

where

for all

As mentioned earlier, each of the information sequences

From the generator matrix

A

Tanner graph of a

Note that in the Tanner graphs of irregular LDPC codes variable nodes and check nodes do not all have the same number of incident edges. The number of check nodes that a specific variable node

The min-sum (MS) and sum-product (SP) decoders are low-complexity, sub-optimal iterative decoders that can be used to decode low-density parity-check codes. Given a particular parity-check matrix, the MS decoder operates by passing messages between the check nodes and the variable nodes along the edges of the Tanner graph of the code.

Before introducing the decoders, some additional notation is necessary. The set of neighbors of check node

Set the number of iterations to

Set

For all messages

For all

One of the primary strengths of the min-sum and sum-product decoders is the relatively small number of operations performed during each iteration. During each iteration, the messages

In practice, the min-sum and sum-product decoders do not always output a codeword. It has been shown that when the MS decoder does not output a codeword after a large number (

The notion of stopping sets was first introduced by Forney et al. [

A stopping set

A small example of a stopping set is given in Figure

Example of a stopping set in the Tanner graph of an LDPC code.

The intuition behind stopping sets begins with an understanding of iterative message-passing decoders. Information given to a specific variable node from a neighboring check node is derived from all other variable nodes connected to that check node. Consider two variable nodes

Work relating linear programming (LP) pseudocodewords to stopping sets for the binary erasure channel [

Trapping sets, also referred to as near-codewords, were first introduced by MacKay and Postol [

Consider a length

Consider the trapping set shown in Figure

Example of a

It is important to note that any set of variable nodes can be considered a trapping set defined by some set of parameters, and the significance of trapping sets varies greatly depending on the parameters

In [

In an attempt to clarify the ambiguity of problematic trapping sets, Zhang et al. introduced the notion of absorbing sets [

Let

Note that an

Example of a

Simulations show that the majority of errors encountered in the error floor region during sum-product decoding of the IEEE 802.3 an low-density parity-check code could be attributed to absorbing sets [

In his 1996 dissertation, Wiberg [

In addition to introducing computation trees, Wiberg also introduced the concept of deviations. Wiberg proved that deviations on the computation tree with negative cost are required in order for errors to occur during MS and SP decoding. Because of the importance of computation trees and deviations in understanding finite tree-based decoding, they are examined in detail in this section.

Consider a low-density parity-check code represented by a Tanner graph

Figure

Computation tree of a simple repetition code after

Tanner graph

Computation tree

Computation trees are precise models for analyzing the performance and behavior of min-sum and sum-product decoding for a finite set of iterations. Each of these decoders can be precisely modeled after

Although the computation tree model is precise, after a small number of iterations it becomes impractical to analyze the performance of specific codes by considering all valid configurations on the computation tree. The number of valid configurations on the computation tree can be computed by treating the computation tree as a Tanner graph. In order to define a Tanner graph given the computation tree, treat all check nodes and variable nodes in the computation tree separately. For example, if multiple copies of variable node

To illustrate the growth rate in the number of valid configurations on the computation tree, consider an LDPC code where each variable node has degree

The number of nodes and valid configurations on the computation tree of a

Iterations | Variable Nodes | Check Nodes | Configurations |
---|---|---|---|

16 | 3 | 8192 | |

166 | 33 | ||

1666 | 333 |

the number of checks nodes given by

and the corresponding number of valid configurations on the computation tree after 1, 2, and 3 iterations. Note that the growth rate is not affected by the block length of the code.

Table

A

Each check node in the computation tree is adjacent to either two or zero variable nodes in the deviation set.

A deviation set contains the root node of the computation tree.

No proper and nonempty subset of variable nodes in the deviation from a valid configuration on the computation tree.

Figure

Example of a deviation on the computation tree.

Wiberg uses the set of deviations on the computation tree to derive an upper bound on the performance of the min-sum decoder. It is necessary, but not sufficient, for at least one deviation

where copies of

Using this condition, a bound can be derived on the probability that the minimum-cost configuration on the computation tree contains a binary 1 at the root node. This bound is

which can be further loosened to

Wiberg [

Let

By the definition of a deviation, we must assign the root node

Each of the previous

Following this pattern, the

After

The number of deviations on the computation tree of a

Number of deviations at iterations 1–5 for a

Iterations | # of deviations |
---|---|

1 | 125 |

2 | 1,953,125 |

3 | |

4 | |

5 |

Using computation trees, Wiberg provided a precise model of the behavior of the min-sum and sum-product decoders. Unfortunately, the size of the computation trees and the number of configurations on them grows too large for practical analysis. Deviations provide a simplified approach to the analysis of computation trees, but the number of deviations also grows exponentially with the number of iterations.

Deviations can be used to define a necessary condition for an error to occur during iterative decoding. The condition simply states that there must be at least one deviation with cost less than zero, assuming that the all-zeros codeword was sent. However, this condition says nothing about which deviations are more or less likely to cause errors. What is known is that at high SNRs low-weight, deviations are much more likely to cause errors than high-weight deviations. Thus it is reasonable to expect that low-weight stopping sets and low-weight deviations coincide over the BEC channel, since low-weight stopping sets are precisely the cause of errors for iterative decoding over the BEC [

Stopping sets consist of a subset

First, consider a computation tree where each of the variable nodes begins with an assignment of a binary 0. Then, assign all copies of variable nodes in

Using the method previously described, a deviation can be constructed using only the variable nodes contained in a stopping set. This is illustrated in Figure

A small portion of a deviation on a computation tree illustrating the impact of stopping set deviations on the BEC and BSC channels.

Deviation

Costs on the BEC

Costs on the BSC

The reason stopping sets will not cause errors as frequently over the BSC is illustrated in Figure

From Figure

Absorbing sets project to deviations on the computation tree in a different way than stopping sets. Since each check node connected to a stopping set

For an absorbing set

Consider an absorbing set

It is clear from (

For finite-length,

Consider the length

Using this same method on the

Any set of nodes can be interpreted as a trapping set, including stopping sets, absorbing sets, and fully absorbing sets. This is because trapping sets are only defined by the number of variable nodes in the set and the number of failed check nodes. In order to simplify analysis, the trapping sets studied in this paper are restricted to the study of problematic trapping sets.

A

Because trapping sets with small weight and a small number of failed check nodes are often the cause of errors at high SNR [

Set

Set

-Set

-Set

-Set

-Perform MS Decoding for

-Set

-Set

-Set

-Set

-Create a binary vector

-Compute the integar syndrome

-Compute the binary syndrome

Hamming weight

-Compute the integar vector

Once Algorithm

Stopping/trapping/absorbing sets of a length

Set | Size | Dev. % | Stop. | Abs. | Full Abs. | Observed |
---|---|---|---|---|---|---|

74% | X | 16 | ||||

73% | X | 7 | ||||

71% | X | 4 | ||||

72% | X | 7 | ||||

72% | X | 28 | ||||

76% | X | 0 | ||||

73% | X | 6 | ||||

75% | X | 21 | ||||

73% | X | 19 | ||||

73% | X | 4 | ||||

76% | X | 28 | ||||

76% | X | 0 | ||||

100% | X | X | X | 908 | ||

91% | X | X | 9 | |||

90% | X | X | 20 | |||

89% | X | X | 6 | |||

91% | X | X | 26 | |||

90% | X | X | 35 | |||

92% | X | X | 0 | |||

91% | X | X | 28 | |||

90% | X | X | 20 | |||

91% | X | X | 16 | |||

91% | X | X | 17 | |||

80% | X | 0 | ||||

64% | 0 | |||||

79% | X | 0 | ||||

79% | X | 0 | ||||

86% | 0 | |||||

86% | 0 | |||||

78% | X | 0 | ||||

85% | 0 | |||||

85% | 0 | |||||

57% | X | 0 | ||||

85% | 0 | |||||

Other | 275 |

In order to determine how effective Algorithm

Table

The problematic trapping sets with the highest proportion of nodes within their corresponding deviation were the

It is worth noting that the average value of received information within the absorbing sets of weight less than or equal to three was

Algorithm

As expected, the number of trapping sets grows very large when increasing the size and dimension of the code. Algorithm

In [

It may seem surprising that almost all problematic trapping sets listed in Table

Deviation designed to maximize the number of copies of

Subgraph of the Tanner graph.

To construct the deviation in Figure

Subgraph of the Tanner graph with repeated check nodes and directed edges.

Using the adjacency matrix of the directed bipartite graph, it is possible to compute the number of copies, or the multiplicity, of each node at each level in the deviation. The method for computing the multiplicity of nodes in the deviation is similar to the method given in [

The columns and rows of

Using the recursive formula given by (

The multiplicity of variable nodes contained in the deviation created using the directed graph given in Figure

0 | 0 | 0 | 0 | 0 | 1 |

2 | 1 | 2 | 0 | 0 | 0 |

4 | 2 | 1 | 1 | 0 | 2 |

6 | 2 | 2 | 2 | 1 | 5 |

8 | 4 | 11 | 2 | 2 | 5 |

20 | 326 | 468 | 158 | 78 | 506 |

60 | |||||

100 | |||||

Total |

where

where the mean

Practical methods for predicting and understanding the performance of low-density parity-check codes with iterative decoders are needed in order to avoid the use of codes with error floors. Trapping sets, which include absorbing sets and stopping sets, provide insight into the error mechanisms of iterative decoders but are too imprecise to be used to make design decisions with respect to error floors. Deviations on computation trees are precise and can be used to compute strict upper bounds on the performance of MS and SP decoding, but computing these bounds quickly becomes computationally intractable. The paper examined the connections between trapping sets and their corresponding deviations through the notion of problematic trapping sets in an attempt to find a practical and precise method for predicting the performance of LDPC codes with iterative decoding.

It was shown that the variable nodes in a stopping set can be used to define a deviation, while trapping sets and absorbing sets only define a deviation if a subset of their variable nodes forms a stopping set. When trapping sets and absorbing sets do not include a stopping set, it is necessary to include additional variable nodes in order to construct a corresponding deviation. The number and proportion of variable nodes outside the set that are needed to construct the deviation can be found experimentally using a modification of the MS decoder. This modified MS algorithm leads to an iterative method for identifying low-weight problematic trapping sets in an LDPC code. Simulation results demonstrate that this method is capable of finding many of the low-weight trapping sets that determine the performance of LDPC codes at moderate SNRs. The efficacy of this algorithm is limited by computational constraints.

Finally, an analytical approach for determining the weight of deviations induced by trapping sets on the computation tree was introduced. This approach involves determining the minimum-weight stopping set that contains a given trapping set, and then determining a directed Tanner graph from the stopping set that favors certain variable nodes within the trapping set. It was then shown that the effective weight of the deviation can be found using a recursive method for computing the multiplicity of variable nodes within the deviation, In one example, it was proven that a deviation exists on the computation tree with weight less than the Hamming weight of the code. This result shows that trapping sets probably result in a necessary condition for an error to occur during iterative decoding, and in certain cases, this condition is satisfied with probability higher than the probability of an ML codeword error.

This paper was funded in part by AFOSR Contract FA9550-06-1-0375 and Department of Education Grant no. P200A070344.