We analyse the asymptotic behaviour of random instances of the maximum set packing (MSP) optimization problem, also known as maximum matching or maximum strong independent set on hypergraphs. We give an analytic prediction of the MSPs size using the 1RSB cavity method from statistical mechanics of disordered systems. We also propose a heuristic algorithm, a generalization of the celebrated Karp-Sipser one, which allows us to rigorously prove that the replica symmetric cavity method prediction is exact for certain problem ensembles and breaks down when a core survives the leaf removal process. The e-phenomena threshold discovered by Karp and Sipser, marking the onset of core emergence and of replica symmetry breaking, is elegantly generalized to Cs=e/(d-1) for one of the ensembles considered, where d is the size of the sets.
1. Introduction
The maximum set packing is a very much studied problem in combinatorial optimization, one of Karp’s twenty-one NP-complete problems. Given a set F={1,…,M} and a collection of its subsets 𝒮={Si∣Si⊆F,i∈V} labeled by V={1,…,N}; a set packing (SP) is a collection of the subsets Si such that they are pairwise disjoint. The size of a SP 𝒮′⊆𝒮 is |𝒮′|. A maximum set packing (MSP) is an SP of maximum size. The integer programming formulation of the MSP problem reads
(1)maximize ∑i∈Vni,(2)subject to ∑i:r∈Sini≤1∀r∈F,(3)ni={0,1}∀i∈V.
The MSP problem, also known in the literature as the matching problem on hypergraphs or the strong independent set problem on hypergraphs, is an NP-Hard problem. This general formulation, however, can be specialized to obtain two other famous optimization problems: the restriction of the MSP problem to sets Si of size 2 corresponds to the problem of maximum matching on ordinary graphs and can be solved in polynomial time [1]; the restriction where each element of F appears exactly 2 times in 𝒮 is the maximum independent set and belongs to the NP-Hard class.
The formulation ((1)–(3)) of the MSP problem, therefore, encodes an ample class of packing problems and, as all packing problems, is related by duality to a covering problem, the minimum set covering problem. Another common specialization of the general MSP problem, known as k-set packing, is that in which all sets Si have size at most k. This is one of the most studied specializations in the computer science community, the efforts concentrating on minimal degree conditions to obtain a perfect matching [2], linear relaxations [3, 4], and approximability conditions [5–7]. Motivated by this interest, we choose a k-set packing problem ensemble as the principal application of the general analytical framework developed in the following sections. The asymptotic behaviour of random sparse instances of the MSP problem has not been investigated by mathematicians and computer scientists; only in the matching [8] and independent set [9] restrictions some work has been done. Extending some theorems of [8] (on which a part of this work is greatly inspired) to a greater class of problem ensembles is some of the main aims of the present work.
On the other hand also the statistical physics literature is lacking an accurate study of the random MSP problem. One of its specialization though, the matching problem, has been covered since the beginning of the physicists’ interest in optimization problems, with the work of Parisi and Mézard on the weighted and fully connected version of the problem [10, 11]. More recently the matching problem on sparse random graphs has also been accurately studied [12, 13] using the cavity method technique. Also the independent set problem on random graphs [14] and the dual problem to set packing, the set covering problem [15], received some attention by the disordered physics community. The SP problem was investigated with the cavity method formalism in a disguised form, as a glass model on a generalized Bethe lattice, in [16, 17]. This corresponds, as we will see in the next section, to a factor graph ensemble with fixed factor and variable degrees; thus, we will not cover this case in Section 7.
The paper is organized as follows.
In Section 2 we map the MSP problem ((1)–(3)) into a statistical physical model defined on a factor graph and relate the MSP size to the density ρ at infinite chemical potential.
We introduce the replica symmetric (RS) cavity method in Section 3 and give an estimate for the average MSPs size on sparse factor graph ensembles in the thermodynamic limit.
In Section 4 we establish a criterion for the validity of the RS ansatz and introduce the 1RSB formalism.
In Section 5 we propose a generalization of the Karp-Sipser heuristic algorithm [8] to the MSP problem and prove the validity of the RS ansatz for certain ensemble of problems. Moreover we find a relationship between a core emergence phenomena and the breaking of replica symmetry breaking.
Section 6 describes the numerical simulations performed.
In Section 7 we apply the analytical tools developed to some problem ensembles. We compare the numerical results obtained from an exact algorithm with the analytical predictions, focusing to greater extent to one ensemble modelling the k-set packing.
2. Statistical Physics Description
In order to turn the MSP combinatorial optimization problem into a useful statistical physical model let us recast ((1)–(3)) into a graphical model using the factor graph formalism [18, 19]. We define our variable nodes set to be V and to each i∈V we associate a variable ni taking values in {0,1} as in (3). F will be our factor nodes set, as its elements acts as hard constrains on the variables ni through (2). The edge set E is then naturally defined as E={(i,r)∣i∈V,r∈Si⊆F}. We call G=(V,F,E) the factor graph thus composed and can then rewrite (2) as
(4)∑i∈∂rni≤1∀r∈F.
A SP is a configuration {ni} satisfying (4) and its relative size is ρ({ni})=(1/N)∑i∈Vni that is simply the fraction of occupied sites.
It is now easy to define an appropriate Gibbs measure for the MSPs problem on G through the grand canonical partition function
(5)ΞG(μ)=∑{ni}∏i∈Veμni∏r∈F𝕀(∑i∈∂rni≤1).
Only SPs contribute to the partition function, and in the close packing limit, as we will call the limit μ↑+∞, the measure is dominated by MSPs. Equation (5) is also a model for a particle gas with hard core repulsion and chemical potential μ located on a hypergraph and as such has been studied mainly on lattice structures and more in general on ordinary graphs. Model (5) has been studied on a generalized Bethe lattice (i.e., the ensemble 𝔾RR(d,c) defined in Section 7, a d-uniform c-regular factor graph) in [16, 17] as a prototype of a system with finite connectivity showing a glassy behaviour. This has been the only approach, although disguised as a hard spheres model, from the statistical physics community to a general MSP problem.
The grand canonical potential is defined as
(6)ωG(μ)=-1μNlogΞG(μ),
and the particle density as
(7)ρG(μ)=1N〈∑i∈Vni〉G,μ=-ωG(μ)-μ∂μωG(μ).
Grand potential and density are related to entropy by the thermodynamic relation
(8)sG(μ)=-μ(ωG(μ)+ρG(μ))=μ2∂μωG(μ).
In the close packing limit (i.e., μ↑+∞) we recover the MSP problem, since in this limit the Gibbs measure is uniformly concentrated on MSPs and ρG gives the MSP relative size. Since entropy remains finite in this limit, from (8) we obtain the MSP relative size
(9)ρG≡limμ→+∞ρG(μ)=limμ→+∞-ωG(μ).
In this paper we focus on random instances of the MSP problem. As usual in statistical physics we will assume the number of variables N (the number of subsets Si) and the number of constrains M to diverge, keeping the ratio N/M finite. We will refer to this limit as the thermodynamic limit. Instances of the MSP problem will be encoded in factor graph ensembles which we assume to be locally tree-like in the thermodynamic limit.
The MSP relative size ρG is a self-averaging quantity in the thermodynamic limit and we want to compute its asymptotic value
(10)ρ=limN→+∞𝔼G[ρG],
where we denoted with 𝔼G[·] the expectation over the factor graph ensemble. In the last equation the N dependence is encoded in the graph ensemble considered. Computing (10) is not an easy task and some approximation have to be taken. We will employ the cavity method from the statistical physics of disordered systems [19, 20], using both the replica symmetric (RS) and the one-step replica symmetry breaking (1RSB) ansatz. We will prove in Section 5 that the RS ansatz is exact in a certain region of the phase space, while in Section 7 we will give numerical evidence that the 1RSB approximation gives very good results outside the RS region.
3. Replica Symmetry3.1. Bethe Approximation on a Single Instance
The RS cavity method has been known for many decades outside the statistical physics community as the Belief Propagation (BP) algorithm and only in recent years the two approaches have been bridged [18, 19]. We start with a variational approximation to the grand potential equation (6) of an instance of the problem, the Bethe free energy approximation:
(11)ωGRS[ν^]=1N[∑r∈∂Fωr[ν^]+∑i∈V(1-|∂i|)ωi[ν^]],
with the factor and variable contributions given by
(12)ωr[ν^]=-1μlog[∑ni∈∂r𝕀(∑i∈∂rni≤1)∏j∈∂r∏s∈∂j∖rν^s→j(nj)],ωi[ν^]=-1μlog[∑nieμni∏r∈∂iν^r→i(ni)].
The grand canonical potential is expressed as a function of the factor node to variable node messages ν^={ν^r→i}. Minimization of ωGRS[ν^] over the messages constrained to be normalized to one yields the fixed point BP equations for the set packing:
(13)ν^r→i(1)=1Zr→i∏j∈∂r∖i∏s∈∂j∖rν^s→j(0),ν^r→i(0)=1Zr→i[∏j∈∂r∖i∏s∈∂j∖rν^s→j(0)=1Zr→iiW+eμ∑j∈∂r∖i∏s∈∂j∖rν^s→j(1)=1Zr→iiW×∏j′∈∂r∖{i,j}∏s′∈∂j′∖rν^s′→j′(0)].
The coeeficients Zr→i are normalization factor. Equations (13) can be simplified introducing the fields {tr→i} defined as
(14)ν^r→i(1)ν^r→i(0)=e-μtr→i,
yielding
(15)tr→i=1μlog[1+∑j∈∂r∖ieμ(1-∑s∈∂j∖rts→j)].
Since we are interested in the close packing limit to solve the problem we will straightforwardly apply the zero temperature cavity method [21]. The related BP equations which can be found as the μ↑∞ limit of (15) read
(16)tr→i=max{0}∪{1-∑s∈∂j∖rts→j}j∈∂r∖i.
We note that the messages {tr→i} are bounded to take values in the interval [0,1] and that if we set the initial value of each tr→i in the discrete set {0,1}, at each BP iteration, all messages will take value either 0 or 1. These values can be directly interpreted as the occupational loss occurring in the subtree r→i if the subtree is connected to the occupied node i. This loss cannot be negative (thus a gain), since we put an additional constrain on the subtree demanding every j neighbour of r to be empty, and cannot be greater than 1 as well, in fact tr→i=1 corresponds to the worst case scenario where an otherwise occupied node j∈∂r∖i has to be emptied.
The Bethe free energy for model (5) on the factor graph G can be expressed as a function of the fixed point messages {tr→i} as
(17)ωGRS(μ)=-1μN[∑r∈∂Flog(1+∑j∈∂reμ(1-∑s∈∂j∖rts→j))=-1μN1+∑i∈V(1-|∂i|)log(1+eμ(1-∑r∈∂itr→i))(1+∑j∈∂reμ(1-∑s∈∂j∖rts→j))].
We finally arrive to the Bethe estimation of the MSPs relative size, taking the close packing limit of (17) and using ωGRS=-ρGRS, which is given by(18)ρGRS=1N[∑r∈Fmax{0}∪{1-∑s∈∂j∖rts→j}j∈r=1Nw+∑i∈V(1-|∂i|)max{0,1-∑r∈itr→i}[∑r∈Fmax{0}∪{1-∑s∈∂j∖rts→j}j∈r].
Let us examine the various contributions to (18) since we want to convince ourselves that it exactly counts the MSP size, at least on tree factor graphs. The term (1-|∂i|)max{0,1-∑r∈itr→i} contributes with 1-|∂i| to the sum only if all the incoming t messages are zero. In this case ni is frozen to 1, that is, the variable i takes part of all the MSPs in G. Obviously all the neighbours of a variable frozen to 1 have to be frozen to 0. To all its |∂i| neighbours r, the frozen to 1 variable i sends a message 1-∑s∈∂i∖rts→i=1, so that we have |∂i| contributions in the first sum of (18) max{0}∪{1-∑s∈∂j∖rts→j}j∈r=1 and the total contribution from i correctly sums up to 1. If for a certain i we have a total field τi≡1-∑r∈itr→i<0 (two or more incoming messages are equal to one) the variable is frozen to 0; that is, it does not take part of any MSPs. It correctly does not contribute to ωGRS since it sends a message 1-∑s∈i∖rts→i≤0 to each neighbour r; thus, it is not computed in max{0}∪{1-∑s∈∂j∖rts→j}j∈r.
The third case is the most interesting. It concerns variables i which take part to a fraction of the MSPs. We will call them unfrozen variables. The total field on an unfrozen variable i is τi=0 (thus we have no contribution from the second sum in (18)) and all incoming messages are 0 except for a single tr→i=1. To this sole function node r, the node i sends a message 1, so that the contribution of r to the first sum is 1. Actually BP equations impose that r has to have at least another unfrozen neighbour beside i. In other terms the function node r says that whatever MSP we consider, one of my neighbours has to be occupied. The corresponding term max{0}∪{1-∑s∈∂j∖rts→j}j∈r=1 in (18) accounts for that.
The presence of unfrozen variables is the reason why we cannot express the density ρG through the formula
(19)ρG=〈∑i∈Vni〉.
In fact, using the infinite chemical potential formalism, we cannot compute
(20)〈ni〉=limμ→+∞eμ(1-∑r∈itr→i)1+eμ(1-∑r∈itr→i)
when limμ→+∞1-∑r∈itr→i=0, and we would have to use the O(1/μ) corrections to the fields {tr→i}. We bypass the problem using the grand potential ωGRS to obtain ρGRS, also addressing a problem reported in [22] of extending an analysis suited for weighted matchings and independent sets to the unweighted case.
3.2. Ensemble Averages
To proceed further in the analysis and since one is often concerned with the average properties of a class of related factor graphs, let us consider the case where the factor graph G is sampled from a locally tree-like factor graph ensemble 𝔾(N). We will employ the following notation for the graph ensembles expectations: 𝔼G[·] for graphs averages; 𝔼C0[·] (𝔼D0[·]) for expectations over the factor (variable) degree distribution, which we will sometime call root degree distribution; and 𝔼C[·] (𝔼D[·]) for expectations over the excess degree distribution of factor (variable) nodes conditioned to have at least one adjacent edge, which we will sometime call residual degree distribution. The quantities c and d and the random variables C, C0, D, and D0, are related by
(21)ℙ[C=k]=(k+1)ℙ[C0=k+1]c,ℙ[D=k]=(k+1)ℙ[D0=k+1]d.
In Section 7 we discuss some specific factor graph ensembles, where C and D are fixed to a deterministic value or Poissonian distributed.
With these definitions the distributional equation corresponding to Belief Propagation formula (16) which reads
(22)T′=dmax{0}∪{1-∑s=1DjTsj}j∈{1,…,C},
where {Dj} are i.i.d. random residual variable degrees, C is a random residual factor degree, and {Tsj} are i.i.d. random incoming messages. Since on a given graph each message tr→i takes value only on {0,1} we can take the distribution of messages t to be of the form
(23)P(t)=pδ(t)+(1-p)δ(t-1).
For this to be a fixed point of (22), the parameter p has to satisfy the self-consistent equation
(24)p*=𝔼C(1-𝔼Dp*D)C.
Using (18) and (24), we obtain the replica symmetric approximation to the asymptotic MSP relative size
(25)ρRS=dc(1-𝔼C0(1-𝔼Dp*D)C0)+𝔼D0(1-D0)p*D0.
It turns out that the RS approximation is exact only when the ratio N/M is sufficiently low. In the SP language (25) holds true only when the number of the subsets among which we choose our MSP is not too big compared to the number of elements of which they are composed.
In the next section we will quantitatively establish the limits of validity of the RS ansatz and introduce the one-step replica symmetry breaking formalism which provides a better approximation to the exact results in the regime of large N/M ratio. In Section 5 we prove that (25) is exact for certain choices of the factor graph ensembles.
4. Replica Symmetry Breaking4.1. RS Consistency and Bugs Propagation
Here we propose two criterions in order to check the consistency of the RS cavity method, iIf any of those fails.
The first criterion is the assumption of unicity of the fixed point of (24) and its dynamical stability under iteration. We restrict ourselves to the subspace of distributions with support on {0,1}, although it is possible to extend the following analysis to the whole space of distributions over [0,1] with an argument based on stochastic dominance following [22]. Characterizing the distributions over {0,1} as in (23) with a real parameter p∈[0,1], from (22) we obtain the dynamical system
(26)p′=𝔼C(1-𝔼DpD)C≡f(p).
The stability criterion
(27)|f′(p*)|<1
suggests that the RS approximation to the MSP size (25) is exact as long as (27) is satisfied. This statement will be made rigorous in Section 5.
The second method we use to check the RS stability, called bugs proliferation, is the zero temperature analogous of spin glass susceptibility. We will compute the average number of changing t messages induced by a change in a single message tr→i (1→0 or 0→1). This is given by
(28)Nch=𝔼[∑(s,j)∑a0,a1b0,b1𝕀(ts→j=b0⟶b1∣tr→i=a0⟶a1)],
where a0, a1, b0, and b1 take value in {0,1}. Since a random factor graph is locally a tree, and assuming correlations decay fast enough, last equation can be expressed as
(29)Nch=∑s=0+∞(C¯D¯)s∑a0,a1b0,b1P(ts=b0⟶b1∣to=a0⟶a1),
where ts is a message at distance s from the tree root o, and we defined the average residual degrees C¯=𝔼C[C] and D¯=𝔼D[D]. The stability condition Nch<+∞ yields a constraint on the greatest eigenvalue λM of the transfer matrix P(b0→b1∣a0→a1):
(30)C¯D¯λM<1.
The two methods presented above give equivalent conditions for the RS ansatz to hold true and they simply express the independence for a finite subgraph from the tail boundary conditions.
4.2. The 1RSB Formalism
We are going to develop the 1RSB formalism for the MSP problem and then apply it in Section 7.1 to the ensemble 𝔾RP. We will not check the coherence of the 1RSB ansatz through the interstate and intrastate susceptibilities [23]; we are then not guaranteed against the need of further steps of replica symmetry breaking in order to recover the exact solution. Even in the worst case scenario though, when the 1RSB solution is trivially exact only in the RS region and a full RSB ansatz is needed otherwise, the 1RSB prediction for MSP relative size ρ should be everywhere more accurate than the RS one and possibly very close to the real value. We will refer to the textbook of Montanari and Mézard [19] for a detailed exposition of the 1RSB cavity method.
Let us fix a factor graph G from a locally tree-like ensemble 𝔾. We call Qr→i(tr→i) the distribution of messages on the directed edge r→i over the states of the system. We still expect the messages tr→i to take values 0 or 1, so that Qr→i can be parametrized as
(31)Qr→i(tr→i)=qr→iδ(tr→i)+(1-qr→i)δ(tr→i-1).
The 1RSB Parisi parameter x∈[0,1] has to be properly rescaled in order to correctly take the limit μ↑∞. Therefore we introduce the new 1RSB parameter y=μx which stays finite in the close packing limit and takes a value in [0,+∞). The reweighting factor e-yωiterr→i is defined as
(32)e-yωiterr→i=Zr→i∏j∈∂r∖i∏s∈∂j∖rZs→j.
Last equation combined with (13) and (14) gives ωiterr→i=-tr→i. Averaging over the whole ensemble we can then write the zero temperature 1RSB message passing rules (also called Survey Propagation equations):
(33)q′=d∏j=1C(1-∏s=1Djqsj)ey+(1-ey)∏j=1C(1-∏s=1Djqsj).
In preceding equation {qsj} are i.i.d.r.v. on [0,1] and, as usual, C is the random variable residual degree and {Dj} are random independent factors residual degrees. Fixed points of (33) take the form
(34)P(q)=p0δ(q)+p1δ(q-1)+p2P2(q),
where P2(q) is a continuous distribution on [0,1] and p2=1-p0-p1. Parameters p0 and p1 have to satisfy the closed equations
(35)p1=𝔼C(1-𝔼D(1-p0)D)C,p0=1-𝔼C(1-𝔼Dp1D)C.
Solutions of (35) with p2=0 correspond to replica symmetric solutions and their instability marks the onset of a spin glass phase. In this new phase the MSPs are clustered according to the general scenario displayed by constraint satisfaction problems [24].
From the stable fixed point of (33) we can calculate the 1RSB free energy functional ϕ(y) as
(36)-yϕ(y)=dc𝔼log[(1-ey)∏j=1C0(1-∏s=1Djqsj)+ey]+𝔼(1-D0)log[(1-ey)(1-∏r=1Dqr)+ey],
and 1RSB density, ρ1RSB(y)=-(∂yϕ(y)/∂y), as
(37)ρ1RSB(y)=dc𝔼ey(1-∏j=1C0(1-∏s=1Djqsj))(1-ey)∏j=1C0(1-∏s=1Djqsj)+ey+𝔼(1-D0)ey∏r=1D0qr(1-ey)(1-∏r=1D0qr)+ey,
with expectations intended over 𝔾 and over fixed point messages {qs} and {qsj}. Since the free energy functional ϕ(y) and the complexity Σ(ρ) are related by the Legendre transform
(38)Σ(ρ)=-yρ-yϕ(y),
with ∂Σ/∂ρ=-y, through (36) we can compute the complexity taking the inverse transform. Equilibrium states, that is, MSPs, are selected by
(39)ρ1RSB=arg maxρ{ρ:Σ(ρ)≥0}
or equivalently taking the 1RSB parameter y to be
(40)ys=arg maxy∈[0,+∞]ϕ(y).
In the static 1RSB phase we expect Σ(ρs)=0 so that from (38) we have ϕ(ys)=-ρs. We will see that this is generally true except for the ensemble 𝔾RP(2,c) of Section 7, corresponding to maximum matchings on ordinary graphs, where the equilibrium state have maximal complexity and ys=+∞. The relation
(41)ρ1RSB=-ϕ(ys)
is always valid though, since for ys↑∞ complexity stays finite.
5. A Heuristic Algorithm and Exact Results
In this section we propose a heuristic greedy algorithm to address the problem of MSP. It is a natural generalization of the algorithm that Karp and Sipser proposed to solve the maximum matching problem on Erdös-Rényi random graphs [8]; therefore, we will call it generalized Karp-Sipser (GKS). Extending their derivation concerning the leaf removal part of the algorithm we are able to prove that the RS prediction for MSP density is exact as long as the stability criterion (27) is satisfied. We will not give the proofs of the following theorems as they are lengthy but effortless extension of those given in [8]. In order to find the maximum matching on a graph Karp and Sipser noticed that as long as the graph contains a node of degree one (a leaf), its unique edge has to belong to one of the perfect matchings.
They considered the simplest randomized algorithm one can imagine: as long as there is any leaf remove it from the graph, otherwise remove a random edge; then iterate until the graph is depleted. They studied the behaviour of this leaf-removal algorithm on random graphs and were able to prove that it grants w.h.p a maximum matching (within an o(n) error).
To generalize some of their results we need to extend the definition of leaf to that of pendant. We call pendant a variable node whose factor neighbours all have degree one, except for one at most. Stating the same concept in different words, all of the neighbours of a pendant have the pendant itself as their sole neighbour, except for one of them at most. See Figure 1 for a pictorial representation of a pendant (in red). The GKS algorithm is articulated in two phases: a pendant removal and a random occupation phase. We give the pseudocode for the generalized Karp-Sipser algorithm (see Algorithm 1).
add to V′ all isolated variable nodes and remove them
from G
remove from G any isolated factor node
whileV is not empty do
ifG has any pendant then
choose a pendant i uniformly at random
add i to V′
remove i from G, then remove its factor neighbours
and their variable neighbours
else
pick uniformly at random a variable node and add
it to V′, remove it from G, then remove its factor
neighbours and their variable neighbours
end if
remove from G any isolated factor node
end while
ReturnV′
(left) A representation of a factor graph containing a pendant, depicted in red. (right) The factor graph after the removal of the pendant as occurs in the inner part of the while loop in Algorithm 1.
At each step the algorithm prioritizes the removal of pendants over that of random variable nodes. We notice that the removal of a pendant is always an optimal choice in order to achieve a MSP; we have no guarantees though on the effect of the occupation of a random node. We call phase 1 the execution of the algorithm up to the point where the first nonpendant variable is added to V′. It is trivial to show that phase 1 is enough to find an MSP on a tree factor graph. The interesting thing though is that phase 1 is also able to deplete nontree factor graphs and find an MSP as long as the factor graphs are sufficiently sparse and large enough.
We call core the subset of the variable nodes which has not been assigned to the MSP in phase 1. We will show how the emergence of a core is directly related to replica symmetry breaking. Let us define as usual
(42)f(x)≡𝔼C(1-𝔼DxD)C.
The function f is continuous, nonincreasing, and satisfies the relation 1≥f(0)≥f(1)=ℙ[C=0]. It turns out that, in the large graph limit, phase 1 of GKS is characterized by the solutions of the system of equations
(43)p=f(r),r=f(p).
We notice that system (43) is equivalent to 1RSB equation (35) once the substitutions p→p0 and r→1-p1 are made.
Lemma 1.
The system of (43) admits always a (unique) solution p*=r*.
Proof.
By monotony and continuity the function g(p)=f(p)-p has a single zero in the interval [0,1].
If other solutions are present the relevant one is the one with smallest p, as the following theorems certify.
Theorem 2.
Let (p,r) be the solution with smallest p of (43). Then the density of factor nodes surviving phase1 in the large graphs limit is given by
(44)ψ1=1+p~-r~+cp𝔼D[pD-rD]
with
(45)r~=𝔼C0(1-𝔼DpD)C0,p~=𝔼C0(1-𝔼DrD)C0.
In particular if the smallest solution is p*=r* the graph is depleted with high probability in phase1.
As already said we will not give the proof of this and of the following theorem, as they are lengthy and can be obtained from the derivation given in Karp and Sipser’s article [8] with little effort even if not in a completely trivial way. Theorem 2 affirms that as soon as (43) develops another solution a core survives phase 1. This phenomena coincides with the need for replica symmetry breaking in the cavity formalism. Let us now establish the exactness of the cavity prediction for ρ in the RS phase.
Theorem 3.
Let (p,r) be the solution with smallest p of (43). Then the density of variables assigned in phase1 in the large graphs limit is
(46)ρ1=dc(1-r~)+𝔼D0(1-D0)pD0,
where p~ has been defined in previous theorem. In particular if the smallest solution is p*=r* the replica symmetric cavity method prediction (25) is exact.
These two theorems imply that the RS prediction for ρ, (25), holds true as long as phase 1 manages to deplete w.h.p. the whole factor graph. A similar behaviour has been observed in other combinatorial optimization problem, for example, the random XORSAT [25]. Conversely it is easy to prove, given the equivalence between (43) and (35), that if a solution with p≠p* exists the RS fixed point is unstable in the 1RSB distributional space. Therefore the system is in the RS phase if and only if (35) admits a unique solution.
We notice that we could have also looked for the solutions of the single equation p=f2(p) instead of the two of (43), since the value of r is uniquely determined by the value of p. Then previous theorems state that the RS results hold as long as f2 has a single fixed point. In [22] it has been proven that the RS solution of the weighted maximum independent set and maximum weighted matching holds true as long as the corresponding squared cavity operator f2 has a unique fixed point. Since those are special cases of the weighted MSP problem, we conjecture that the f2 condition holds for the general case too, as it does for the unweighted case. Probably it would be possible to further generalize the Karp-Sipser algorithm to cover analytically the weighted case as well.
The scenario arising from the analysis of the algorithm and from 1RSB cavity method is the following: in the RS phase maximum set packings form a single cluster and it is possible to connect any two of them with a path involving MSPs separated by a rearrangement of a finite number of variables [26]. In the RSB phase instead MSPs can be grouped into many connected (in the sense we mentioned before) clusters, each one of them defined by the assigning of the variables in the core. Two MSPs who differ on the core are always separated by a global rearrangement of the variables (i.e., O(N)). In presence of a core the GKS algorithm makes some suboptimal random choices (after phase 1).
We did not make an analytical study of the random variable removal part of the GKS algorithm. This has been done by Wormald for the matching problem, associating to the graph process a set of differential equations amenable to analysis [9] (something similar has been done for random XORSAT as well [25]). In that case it turns out that the algorithm still achieves optimality and yields an almost perfect matching on the core. An appropriate analysis of the second phase of the GKS algorithm and the reason of its failure for most of the SP ensembles we covered deserve further studies.
6. Numerical Simulations
While the RS cavity (24) and (25) can be easily computed, the numerical solution of the 1RSB density evolution (33) is much more involved (although simplified by the zero temperature limit) and has been obtained with a standard population dynamics algorithm [27].
The implementation of the GKS algorithm is straightforward. While it is very fast during phase 1, we noticed a huge slowing down in the random removal part. We were able to find set packings for factor graphs with hundred of thousands of nodes.
In order to test the cavity and GKS predictions, we also computed the exact MSP size on factor graphs of small size. First we notice that a set packing problem, coded in a factor graph structure F, is equivalent to an independent set problem on an appropriate ordinary graph G. The node set of G will be the variable node set of F and we add an edge to G between each pair of nodes having a common factor node neighbour in F. Therefore each neighborhood of a factor node in F forms a clique in G. We then solve the maximum set problem for G using an exact algorithm [28] implemented in the igraph library [29]. Since the time complexity is exponential in the size of the graph we performed our simulations on graphs containing up to only one hundred nodes.
7. Applications to Problem Ensembles
We will now apply the methods developed in the previous sections to some factor graph ensembles, each modelling a class of MSP problem instances. We consider graph ensembles containing nodes with Poissonian random degree or regular degree: 𝔾PR(N,d,c), 𝔾RP(N,d,c), 𝔾PP(N,d,c), and 𝔾RR(N,d,c). Subscript R or P indicates whether the type of nodes to which they refer (variable nodes for the first subscript, factor nodes for the second) have regular or Poissonian random degree, respectively. We parametrize these ensemble by their average variable and factor degree; that is, d=𝔼D0D0 and c=𝔼C0C0. They are constituted by factor graphs having N variable nodes and M=⌊Nd/c⌋ factor nodes but they differ both for their elements and for their probability law. We define the ensembles giving the probability of sampling one of their elements.
𝔾RP(N,d,c): each element G has N variables and M=⌊Nd/c⌋ factors. Every variable node has fixed degree d. G is obtained linking each variable with d factors chosen uniformly and independently at random. The factor nodes degree distribution obtained is Poissonian of mean c with high probability. This ensemble is a model for the k-set packing and will be the main focus of our attention.
𝔾PR(N,d,c): each element G has N variables and M=⌊Nd/c⌋ factor. Every factor node has fixed degree c. G is built linking each factors with c variables chosen uniformly and independently at random. The variable nodes degree distribution obtained is Poissonian of mean d with high probability.
𝔾PP(N,d,c): each element G has N variables and M=⌊Nd/c⌋ factors. G is built adding an edge (i,r) with probability c/N independently for each choice of a variable i and a factor r. The factor graph obtained has w.h.p Poissonian variable nodes degree distribution of mean d and Poissonian factor nodes degree distribution of mean c.
𝔾RR(N,d,c): it is constituted of all factor graphs of N variable nodes of degree d and M=Nd/c factor nodes of degree c (Nd has to be multiple of c). Every factor graph of the ensemble is equiprobable and can be sampled using a generalization of the configuration model for random regular graph [30]. This ensemble is a model for the k-set packing.
We will omit the argument N when we refer to an ensemble in the N↑∞ limit.
7.1. 𝔾RP(d,c)
This is the ensemble with variable node degrees fixed to d and Poissonian factor node degree that is C~C0~Poisson(c). The case d=2 corresponds to the maximum matching problem on Erdös-Rényi random graph [8, 12].
Density evolution (22) for 𝔾RP reduces to
(47)T′=dmax{0}∪{1-∑s=1d-1Tsj}j∈{1,…,C}.
Considering distributions of the form P(t)=pδ(t)+(1-p)δ(1-t) fixed points of (47) reads
(48)p*=e-cp*d-1≡f(p*).
The last equation admits only one solution for each value of d and c, as it is easily seen through a monotony argument considering the left and right hand side of the equation. The values of c as a function of d satisfying |f′(p*)|=1 are the critical points cs(d) delimiting the RS phase (c<cs(d)) and are given by
(49)cs(d)=ed-1.
For c>e/(d-1) a core survives phase 1 of the GKS algorithm as stated by Theorem 2 and shown in Figure 2. We notice that the same threshold value e/(d-1) is found in the minimum set covering problem on the dual factor graph ensemble [31], where the application of a leaf removal algorithm, that is, phase 1 of GKS, unveils an analogous core emergence phenomena. In the RS phase, the MSP density is equal to the minimum set covering density, but in the RSB phase we cannot compare the cavity predictions for these two dual problems, since in [31] the 1RSB solution to the minimum set covering is not provided.
Results from GKS algorithm applied to 𝔾RP(3,c). The set packing size after phase 1 (blue line) and after the algorithm stops (red line) are reported. The vertical line is cs=e/2 and is drawn as a visual aid to determine the point where the RS assumptions stop to hold true. Above cs a nonempty core emerges continuously (green line).
In the matching case, that is, d=2 equation (49) expresses the notorious e-phenomena discovered by Karp and Sipser, while for higher values of d it provides an extension of the critical threshold.
We can recover the same critical condition equation (49) through the bug propagation method, as the transfer matrix P(b0→b1∣a0→a1) non-zero elements are only:
(50)P(0⟶1∣1⟶0)=pd-1,P(1⟶0∣0⟶1)=pd-1,
which give λM=pd-1. The average branching factor is C¯D¯=c(d-1) so that (30) and (48) yield (49). The analytical value for the relative size of MSPs, that is, the particle density ρ, is
(51)ρ(d,c)=dc(1-p*)+(1-d)p*dforc<cs(d).
In Figure 3 we compared ρ from (51) as a function of c for some values d with an exact algorithm applied to finite factor graphs (as explained in Section 6), both above and below cs. Clearly for c>cs the RS approximation is increasingly more inaccurate.
RS cavity method analytical prediction equation (51) for MSP size in 𝔾RP(d,c) compared with the average size of MSPs obtained from 1000 samples of factor graphs with 100 variable nodes (dots). Dashed parts of the lines are the RS estimations in the RSB phase, that is, for c>e/(d-1), where it is no longer exact.
We continue our analysis of 𝔾RP above the critical value cs through the 1RSB cavity method as outlined in Section 4.2. Fixed point messages of (33) are distributed as
(52)P(q)=p0δ(q)+p1δ(q-1)+p2P2(q),
where
(53)p1=e-c(1-p0)d-1,p0=1-e-cp1d,p2=1-p0-p1,
and P2(q) has to be determined through (33). Equation (53) admits always an RS solution p1=1-p0=p* which is stable up to cs, as already noticed. Above cs a new stable fixed point, with p2>0, continuously arises and we study it numerically with a population dynamics algorithm.
The 1RSB free energy, as a function of the Parisi parameter y, takes the form
(54)ϕ(y)=-dyc𝔼log[(1-ey)∏j=1C(1-∏s=1d-1qsj)+ey]+d-1y𝔼log[(1-ey)(1-∏r=1dqr)+ey].
As prescribed by the cavity method, the value ys which maximizes ϕ(y) over [0,+∞] yields the correct free energy; therefore, we have ϕ(ys)=-ρ1RSB.
Unsurprisingly, as they belong to different computational classes, the cases d=2 and d≥3 show qualitatively different pictures. In the case of maximum matching on the Poissonian graph ensemble, numerical estimates suggest that complexity is an increasing function of y on the whole real positive axis. Correct choice for parameter y is then ys=+∞, as already conjectured in [12], and we find that maximum matching size prediction from 1RSB cavity method fully agrees with rigorous results from Karp and Sipser [8] and with the size of the matchings given by their algorithm (see Figure 4). The 1RSB ansatz is therefore exact for d=2.
Maximum set packing size for 𝔾RP(d,c) as a function of c for d=2. Blue line corresponds to the RS analytical value (51), red line is from the Karp-Sipser algorithm and dotted black line is the 1RSB solution (54) with y≫1. Karp-Sipser and 1RSB cavity method yield the same and exact result.
The d≥3 case analysis does not yield such a definite result. The complexity of states Σ is no more a strictly increasing function of y. It reaches its maximum in yd, the choice of y that selects the most numerous states, which could be those where local search greedy algorithms are more likely to be trapped. Then it decreases up to the finite value ys where complexity changes sign and takes negative values. Therefore ys is the correct choice for the Parisi parameter which maximizes ϕ(y). Plotted as a function of ρ, complexity Σ has a convex nonphysical part, with extrema the RS solution (on the right) and the point corresponding to the dynamic 1RSB solution (on the left) and a concave physically relevant for y∈[yd,ys] (see Figure 5). The 1RSB seems to be in very good agreement with the exact algorithm and we are inclined to believe that no further steps of replica symmetry breaking are needed in this ensemble. The GKS algorithm instead falls short of the exact value (see Figure 6); therefore, it constitutes a lower bound which is not strict but at least it could probably be made rigorous carrying on the analysis of the GKS algorithm beyond phase 1.
Complexity in 𝔾RP(d,c), with d=3 and c=4.0, as a function of the relative MSP size ρ.
MSP size in 𝔾RP(3,c) as a function of c. RS and 1RSB cavity method is confronted with the GKS algorithm and with an exact algorithm on factor graphs of 100 variable nodes.
7.2. 𝔾PR(d,c)
The ensemble 𝔾PR(d,c) is constitute factor graphs containing factor nodes of degree fixed to c and variable nodes of Poissonian random degree of mean d. It has statistical properties with respect to the MSP problem quite different from those encountered in 𝔾RP, as we will readily show. The MSP problem on 𝔾PR(d,c) with d=2 is equivalent to the well known problem of maximum independent sets on Poissonian graphs [14, 32]. The real parameter p characterizing discrete support distributions of messages has to satisfy the fixed point (24) that reads
(55)p*=(1-ed(p*-1))c-1.
At fixed value of c the RS ansatz holds up to the critical value ds(c), which is implicitly given by first derivative condition (27):
(56)(c-1)p*(c-2)/(c-1)e(p*-1)dsds=1.
Although critical condition equation (56) is not as elegant as the one we obtained for the ensemble 𝔾RP, it can be easily solved numerically for ds as a function of c. For c=2 the threshold value is exactly ds(2)=e. For c>2 instead ds is an increasing function of the factor degree c. Thanks to (25) we readily compute the MSP size in the RS phase:
(57)ρ=dc(1-p*c/(c-1))+(1-p*d)ed(p*-1)forc<cs(d).
We can see in Figure 7 that the MSP size ρ(d,c) is a decreasing function in both arguments as expected. Equation (57) can be taken as the RS estimate for MSP size for d>ds(c). The RS estimate is strictly greater than the average size of SPs given by the GKS algorithm at all values of d>ds(c) (see Figure 7).
RS cavity method analytical prediction (57) for MSP size in 𝔾PR(d,c) (dot-dashed) compared with the set packing size from phase 1 of the GKS algorithm (dashed lines) and with a complete run of the algorithm (continuous lines).
7.3. 𝔾PP(d,c)
We will now briefly examine our MSP model on the ensemble 𝔾PP(d,c) where both factor nodes and variable nodes have Poissonian random degrees of mean c and d, respectively. From (23) and (24) we obtain the fixed point condition for the probability distribution of messages:
(58)p*=e-ced(p*-1).
As usual p is the parameter characterizing the distribution of messages, P(t)=pδ(t)+(1-p)δ(1-p). Equation (58) admits one and only one fixed point solution p* for each value of d and c. In fact f is continuous, strictly decreasing, and f(0)>0,f(1)<1. The first derivative condition |f′(p)|=1 defines the critical line ds(c) through
(59)ds(c)=-1p*log(p*).
The curve ds(c) separates the RS phase from the RSB phase in the c-d parametric space (see Figure 8). The unbounded RS region shares some resemblance with the corresponding (although d-discretized) region of 𝔾PR(d,c) and is at variance with the compact area of the RS phase in 𝔾RP.
MSP relative size for the ensemble 𝔾PP(d,c) as given by (60). Above the boundary separating RS and RSB phase values are only approximation to the exact value of ρ.
We can compute the MSP relative size ρ through (25) and obtain
(60)ρ=dc(1-p*)+(1-dp*)ed(p-1)ford<ds(c),
which holds only as an approximation in the RSB phase. We can see from Figure 8 that ρ is decreasing both in c and d as was observed in the other ensembles as well.
7.4. 𝔾RR(d,c)
The MSP problem on the ensemble 𝔾RR(d,c), the straightforward generalization to factor graphs of the random regular graphs ensemble, poses some simplification to the cavity formalism thanks to his homogeneity. It has already been the object of preliminary studies by Weigt and Hartmann [16] and then a much more deep work of Hansen-Goos and Weigt [17] who disguised it as a hard spheres model on a generalized Bethe lattice. The authors studied through the cavity method this hard spheres model on 𝔾RR(d,c) both at finite chemical potential μ and in the close packing limit and found out that the 1RSB solution is unstable in the close packing limit, therefore suggesting the need of a full RSB treatment of the problem.
8. Conclusions
We studied the average asymptotic behaviour of random instances of the maximum set packing problem, both from a mathematical and a physical viewpoint. We contributed to the known list of models where the replica symmetric cavity method can be proven to give exact results, thanks to the generalization of an algorithm (and of its analysis) first proposed by Karp and Sipser [8]. Moreover, our analysis address a problem reported in recent work on weighted maximum matchings and independent sets on random graphs [22], where the authors could not extend their results to the unweighted cases. We achieve here the desired result making use the grand canonical potential instead of the direct computation of single variable expectations. We also extend their condition for the system to be in what physicists call a replica symmetric phase, namely, the uniqueness of the fixed point of the square of a certain operator (which is the analogue of the one defined in (42)), to the more general setting of maximum set packing (although without weights). On some problem ensembles, where the assumptions of Theorems 2 and 3 no longer hold and the RS cavity method fails, we used the 1RSB cavity method machinery to obtain an analytical estimation of the MSP size. Numerical simulations show very good agreement of the 1RSB estimation with the exact values, although comparisons have been done only with small random problems due to the exact algorithm being of exponential time complexity. The GKS algorithm instead generally fails to find any MSPs except for some special instances of the problem.
Some questions remain open for further investigation. To validate the 1RSB approach the stability of the 1RSB solution has to be checked against more steps of replica symmetry breaking. Moreover a thorough analysis of the second phase of the GKS algorithm could shed some light on the mechanism of replica symmetry breaking and give a rigorous lower bound to the average maximum packing size.
Regarding the problem of looking for optimal solutions in single samples, we believe that an efficient heuristic algorithm, able to obtain near-to-optimal configurations also in the RSB phase, could be constructed following the ideas of [33].
Conflict of Interests
The authors declare that there is no conflict of interests regarding the publication of this paper.
Acknowledgments
This research has received financial support from the European Research Council (ERC) through Grant agreement no. 247328 and from the Italian Research Minister through the FIRB project no. RBFR086NN1.
MicaliS.VaziraniV. V.An O(vvcE) algoithm for finding maximum matching in general graphsProceedings of the 21st Annual IEEE Symposium on Foundations of Computer Science19801727AlonN.KimJ.-H.SpencerJ.Nearly perfect matchings in regular simple hypergraphs19971001711872-s2.0-0002373686FürediZ.KahnJ.SeymuP. D.On the fractional matching polytope of a hypergraph199313216718010.1007/BF01303202ChanY. H.LauL. C.On linear and semidefinite programming relaxations for hypergraph matching201113512314810.1007/s10107-011-0451-5RödlV.ThomaL.Asymptotic packing and the random greedy algorithm199683161177HalldórssonM. M.Approximations of weighted independent set and hereditary subset problems2000411610.7155/jgaa.00020HazanE.SafraS.SchwartzO.On the complexity of approximating K-set packing200615120392-s2.0-3364650899610.1007/s00037-006-0205-6KarpR. M.SipserM.Maximum matchings in sparse random graphs12Proceedings of the 22nd Annual Symposium on Foundations of Computer Science1981IEEE Computer Society36437510.1109/SFCS.1981.21WormaldN. C.The differential equation method for random graph processes and greedy algorithms199973155ParisiG.MézardM.Replicas and optimization198546177717782-s2.0-0347058873ParisiG.MézardM.A replica analysis of the travelling salesman problem1986471285129610.1051/jphys:019860047080128500ZhouH.Ou-YangZ.-C.Maximum matching on random graphsIn press, http://arxiv.org/abs/cond-mat/0309348ZdeborováL.MézardM.The number of matchings in random graphs20062006P0500310.1088/1742-5468/2006/05/P05003DallastaL.RamezanpourA.PinP.Statistical mechanics of maximal independent sets20098006113610.1103/PhysRevE.80.061136MézardM.TarziaM.Statistical mechanics of the hitting set problem20077641004112410.1103/PhysRevE.76.041124WeigtM.HartmannA. K.Glassy behavior induced by geometrical frustration in a hard-core lattice-gas model20036253302100510.1209/epl/i2003-00383-8Hansen-GoosH.WeigtM.A hard-sphere model on generalized bethe lattices: statics20052005P0400610.1088/1742-5468/2005/04/P04006KschischangF. R.FreyB. J.LoeligerH.-A.Factor graphs and the sum-product algorithm20014724985192-s2.0-003524656410.1109/18.910572MontanariA.MézardM.2009Oxford University PressParisiG.MézardM.VirasoroM. A.1987SingaporeWorld ScientificMézardM.ParisiG.The cavity method at zero temperature2003221-213410.1023/A:1022221005097GamarnikD.NowickiT.SwirszczG.Maximum weight independent sets and matchings in sparse random graphs. Exact results using the local weak convergence method2006281761062-s2.0-3364528854610.1002/rsa.20072MontanariA.ParisiG.Ricci-TersenghiF.Instability of one-step replica-symmetry-broken phase in satisfiability problems2004376207310.1088/0305-4470/37/6/008KrzakalaF.MontanariA.Ricci-TersenghiF.SemerjianG.ZdeborováL.Gibbs states and the set of solutions of random constraint satisfaction problems20071042510318103232-s2.0-3454727389110.1073/pnas.0703685104MézardM.Ricci-TersenghiF.ZecchinaR.Two solutions to diluted p-spin models and XORSAT problems20021113-450553310.1023/A:1022886412117SemerjianG.On the freezing of variables in random constraint satisfaction problems2008130225129310.1007/s10955-007-9417-7MézardM.ParisiG.The bethe lattice spin glass revisited200120221723310.1007/PL00011099TsukiyamaS.IdeM.AriyoshiH.ShirakawaI.A new algorithm for generating all the maximal independent sets19776350551710.1137/0206036CsárdiG.NepuszT.The igraph software package for complex network research20061695WormaldN. C.Models of random regular graphs1999239298London Mathematical Society Lecture NoteTakabeS.HukushimaK.Minimum vertex cover problems on random hypergraphs: replica symmetric solution and a leaf removal algorithmIn press, http://arxiv.org/abs/1301.5769SanghaviS.ShahD.WillskyA. S.Message passing for maximum weight independent set20095511482248342-s2.0-7035070602510.1109/TIT.2009.2030448WeigtM.ZhouH.Message passing for vertex covers2006741904611010.1103/PhysRevE.74.046110