The Reciprocal Influence Criterion: An Upgrade of the Information Quality Ratio

Understanding and quantifying the mutual influence between systems remain crucial but challenging tasks in any scientific enterprise. The Pearson correlation coefficient, the mutual information, and the information quality ratio are the most widely used indicators, only the last two being valid for nonlinear interactions. Given their limitations, a new criterion is proposed, the reciprocal influence criterion, which is very simple conceptually and does not make any assumption about the statistics of the stochastic variables involved. In addition to being normalised as the information quality ratio, it provides a much better resilience to noise and much higher stability to the issues related to the determination of the involved probability distribution functions. A conditional version, to counteract the effects of confounding variables, has also been developed, showing the same advantages compared to the more traditional indicators. A series of systematic tests with numerical examples is reported, to compare the properties of the new indicator with the more traditional ones, proving its clear superiority in practically all respects.


Determining Statistical
Dependence between Quantities e investigation of the reciprocal influence between systems is a major scientific objective in practically all disciplines. Both linear and nonlinear effects can be important, the latter becoming often dominant in complex phenomena [1]. e information about systems is typically obtained with measurements affected by various forms of uncertainties. e measurands can therefore be considered as random variables, and their mutual influence is partly deterministic and partly probabilistic in nature.
Quantifying the dependence between random variables is an essential statistical task for both bivariate and multivariate data. e most popular measure of dependence between two quantities is the Pearson product-moment correlation coefficient or "Pearson's correlation coefficient" (PCC) [2]. To alleviate its limitations (see Section 2), at the beginning of the last century, various alternative indicators based on ranked variables were developed, such as the Spearman ρ or Spearman rank correlation coefficient [3] and the rank correlation coefficient or Kendall τ [4]. After the Second World War information theoretic criteria, such as mutual information, became quite popular [5]. At the dawn of the new century, a lot of work was devoted to consolidating distance correlation, an indicator that is zero only when the two variables are not correlated [6]. In the last years, the advances in genetics have also motivated the development of various techniques for the analysis of the dependence between vectors in high dimensions. For this multivariate inference, very sophisticated approaches have been proposed, ranging from projection correlation [7] to Ball covariance [8] and Brownian distance [9]. All these techniques are based on specific assumptions about the statistical properties of the vectors of random variables to consider. ey are also quite involved, both conceptually and numerically. e indicators proposed in this paper are meant to deal only with bivariate cases, for which no prior information is known about the statistics of the data, so fully general techniques are required. A conditional version to counteract the effects of confounding variables has also been developed. e indicators are also very simple from the point of view of the conceptual background, interpretation of the results, and implementation requirements.
Regarding the structure of the paper, Section 2 introduces the most consolidated indicators, typically used to assess the dependence between two variables. Section 3 discusses the main rationale behind the proposed alternative indicator: the reciprocal influence criterion. e systematic tests, comparing the properties of RIC with PCC and IQR, are reported in Sections 4 and 5, for functional and nonfunctional dependencies, respectively. e robustness against the noise of statistics, different from the Gaussian and against outliers, is discussed in Section 6. e conditional version of RIC is introduced in Section 7, before conclusions are drawn in Section 8.

Traditional and Alternative Criteria to Quantify the Correlations between Quantities
In order to fully appreciate the potential of the new indicators proposed in the paper, it is worth starting the discussion with a review of the traditional and alternative tools most used by practitioners for the analysis of bivariate dependence. e linear correlations between two quantities are typically calculated with the Pearson correlation coefficient (PCC). For a couple of random variables X and Y, indicating with cov as the covariance and with σ as the standard deviation, the PCC is calculated as [10] PCC Even if it is very useful in practice, some limitations of the PCC, particularly, its vulnerability to noise, are very often overlooked. In any case, the detection of nonlinear correlations between two variables is a much more serious matter. To quantify nonlinear dependencies, historically, the first techniques developed were based on ranking variables (see Section 2.1). After the Second World War, indicators based on the probability distribution function (pdf ) of the data have been developed; particularly, information theoretic criteria are popular (Section 2.2). Distance correlation is a much more recent development, first introduced in 2005, to remedy the problem that the Pearson correlation coefficient can be zero for dependent variables (Section 2.3).
2.1. Rank-Based Criteria. In this paper, the symbol ρ S indicates the extension of the Pearson correlation coefficient introduced by Charles Spearman. Spearman's ρ S is a nonparametric measure of the statistical dependence between the rankings of two variables; as such, it quantifies how well the relationship between two variables can be modelled by a monotonic function [11]. Spearman's ρ S is calculated as the Pearson correlation between the rank values of the quantities involved and assesses how monotonic the relation between them is (whether linear or not). If there are no repeated data values, ρ S assumes the values +1 or − 1 when the two variables are perfect monotone functions of each other.
To calculate Spearman's ρ S, the raw data X i and Y i are converted into ranks first: rank X and rank Y ; the standard deviations of these two ranked variables are indicated with the symbols σ rank X and σ rank Y . Spearman's ρ S is then defined as e Kendall rank correlation coefficient, named after Maurice Kendall, was developed at the end of the 30 s, and it is usually indicated with the Greek letter τ. It is meant to measure the rank correlation, i.e., the similarity of the ordering of the data when ranked. Intuitively, the higher the Kendall correlation between two variables, the more similar their rank; the Kendall τ is also normalised in the sense that it ranges between 1 and − 1 [11].
Mathematically, the Kendall rank correlation coefficient is calculated as where the denominator is the binomial coefficient n 2 � n(n − 1)/2.

Information eoretic Criteria.
A widely used indicator to investigate nonlinear interactions is the mutual information, which quantifies the information shared by two systems. e Shannon or discrete version is defined as [12,13] MI Shan � p xy log p xy p x p y , where p x and p y are the discrete probabilities of two random variables X and Y and p xy is their joint probability.

Complexity
A differential version of the indicator can be formulated in terms of the probability densities f x and f y : where Δx and Δy are the dimensions of the bins. e differential mutual information, MI diff , is defined as It can be easily demonstrated that the two relations (4) and (6) are equivalent; the mutual information is, therefore, referred to as MI in the rest of the paper. MI has various positive properties but presents the main limitation of not being normalised. To obviate this drawback, the mutual information is typically divided by the joint entropy to obtain the so-called information quality ratio (IQR) [14]: is quantity is normalised in the sense that it assumes only values between zero and one. It is important to remember that, in equation (7), the discrete or Shannon version of the joint entropy is to be used: e Shannon entropy is to be compared with the differential one, which can also assume negative values: f xy log f xy dxdy . (9) Equation (7) is the commonly accepted version of the information quality ratio, normally adopted because the differential version of the joint entropy can be negative, with the obvious related problems and difficulties.
Unfortunately, mainly due to the denominator, the IQR S has some limitations, particularly, a lack of robustness to noise and a strong dependence on the choice of the bins.

Distance Correlation.
e objective of distance correlation (D corr ) consists of quantifying the dependence between two random vectors, which do not need to have necessarily equal dimension. D corr has the clear advantage, compared to the PCC, that the population distance correlation coefficient assumes a zero value only if the two random vectors are independent. erefore, distance correlation is meant to quantify both linear and nonlinear association between two random variables or random vectors [5]. e calculation of D corr requires the definition of some other preliminary quantities. Indicating with (X k , Y k ), k � 1, 2, . . ., n, a sample from a pair of real-valued or vector-valued random variables (X, Y), the elements of the n by n distance matrices (a j, k ) and (b j, k ) are all pairwise distances: where || || denotes the Euclidean norm. Defining a j and a k as the jth row mean and the kth column mean, respectively, and with a the grand mean of the first vector mutual distance X matrix, one can then calculate all doubly centred distances (with the same notation for the Y vector matrix): e desired sample distance covariance is then simply the arithmetic average of the product A j,k B j,k : Indicating with distance variance, finally, the distance correlation is . (14) e main properties of dCor are that it assumes values between 0 and 1, and it is zero only if the two vectors are independent. e distance correlation software used in this work is the one published by Shen Liu [15].

The Reciprocal Influence Criterion: Rationale
As will be shown in the rest of the paper, all criteria summarised in the previous section have several drawbacks. In this work, a new indicator, able to quantify linear and nonlinear correlations between variables, is introduced. e new indicator, based on information theoretic quantities and named reciprocal influence criterion (RIC), has been designed to have the following properties.

Property 1.
e indicator ranges from zero (no correlation) to one (perfectly correlated). e definition of correlation for this indicator is the following.
Two variables i and j are correlated when the knowledge of i helps to predict j and vice versa. e correlation indicator tends to one when the uncertainty of i (j) known j (i) goes to zero. Correlation must be equal to zero when the uncertainty of i (j) does not change when the j (i) is known. Note that, in this definition, symmetric equations cannot reach the value of one; as, for example, for the function y � x 2 , since y is known, there are two valid values of x (excluded x � 0 and y � 0).

Property 2.
e indicator does not vary as a function of the binning used for the probability density or distribution function calculations, i.e., where ΔRIC denotes the difference between the indicator values calculated with and without the outliers.

Property 3.
e indicator can be calculated indifferently for either discrete or continuous random variables.

Property 4. Small sensitivity to outliers is quantified as
e formulation of the RIC that ensures the former properties is e RIC indicator is based on the ratio between the amount of information (uncertainty) shared by the two variables with respect to the sum of their individual information (uncertainty). In some ways, its definition is very similar to the IQR indicator, with the main difference that the RIC is based on the exponential of the entropy A H . is approach allows obtaining a much more reliable indicator, which satisfies the aforementioned four desirable properties, as demonstrated in the following, and presents some additional positive qualities, namely, symmetry, asymptotic consistency, and unbiasedness, which are discussed in Appendix A.
With regard to the free parameter A, the choice of its numerical value can be optimised, depending on the nature of the data and the objectives of the investigation. e results, reported in the rest of the paper, have been obtained by setting A � 10, to maximise the coherence of RIC with PCC in case of linear correlations; the equations in the following are also particularised for this numerical value. A detailed discussion of the RIC behaviour with the parameter A is provided in Appendix B.
Some of the aforementioned four properties can be analytically demonstrated.
In the case of partially correlated quantities, RIC is always positive, and for high mutual information, it tends to one. More specifically, it is known that, for high correlation levels, the joint entropy tends to the lowest entropy of the two variables (H xy ⟶ min(H x , H y )) [16], which means Property 3 can also be proved. Indeed, writing the RIC using the differential information theoretic quantities and remembering that the sum of the entropies is equal to the sum of the mutual information and the joint entropy, it is possible to demonstrate that RIC D is the same as RIC S : Moreover, RIC is directly correlated to the mutual information value; also, property 2 is expected to be satisfied (and it is validated in Section 4).
In terms of interpretation, RIC remains an information theoretic indicator, since it relies on the mutual information. On the contrary, the monotonic transformation of (15) provides a practically normalised indicator without having to make recourse to the joint entropy. e advantages of such a reformulation will be illustrated in detail, with the help of numerical tests, in the next sections.

The Reciprocal Influence Criterion: Numerical Tests for Functional Dependencies
In this section, the properties of RIC are investigated with the help of a series of numerical tests with synthetic data. e reference indicators to benchmark the performance of RIC are the ones reviewed in Section 2, the most used by the practitioners. Only bivariate dependences due to functional relations are considered; nonfunctional dependencies are the subject of Section 4.1. A general overview of the results is  Complexity provided first and some specific aspects are discussed in dedicated sections.

Comparison Overview.
A first comparison between the RIC and the other criteria has been performed for the functional dependencies linear, quadratic, sinusoidal, and exponential. e results for a series of representative cases are reported in Table 1. A graphical overview is provided in Figure 1. Random noise of Gaussian distribution, with standard deviation equal to 10% of the quantity value, has been added to all variables. e values of Table 1 and inspection of Figure 1 reveal that RIC never performs significantly worse than the other criteria for linear dependencies. RIC starts outperforming all other indicators in the case of nonlinear functions. Moreover, as expected, RIC provides much more reliable and reasonable results whenever the functional dependence is nonmonotonic and when even the ranked methods fail miserably. Also, the indicators based on the pdf, IQR, and distance correlation show significant difficulties to provide acceptable results for the nonmonotonic dependencies. All these are general properties not only true for the examples reported but also confirmed in all cases tested.

Linear Correlations: Effects of Gaussian Noise and Binning.
is section, with the help of Figure 2, is simply aimed at supporting the statement that RIC can reproduce well the values of the PCC for linear correlations. e behaviour of the other indicators is also shown for completeness. e plots of Figure 2 refer to the case of perfect linear correlation between quantities: y � x. On the x-axis, the standard deviation of additive noise and zero mean and sampled randomly from a Gaussian distribution is reported. e results are fully general. e RIC reproduces quite well the values of the PCC, whereas the IQR is significantly more vulnerable to both noise and binning. A similar analysis indicates that RIC is also more robust against the presence of outliers; indeed, it can tolerate about even one order of magnitude more outliers than IQR, confirming that property 4 of Section 3 is satisfied (with 5% of outliers, the average Pearson coefficient variation is about 10% and the IQR variation is 12%, while the ΔRIC is 0.7%).

Nonlinear Correlations: Effects of Gaussian Noise and
Binning.
e competitive advantages of RIC become even more evident in the case of nonlinear correlations. ree exemplificative cases are reported in Figure 3, in which the proposed new criterion is compared with IQR S . e functional dependencies reported are y � x 2 , y � sin (x), and y � exp (x).
As expected, IQR is much more sensitive to the choice of the binning and the level of noise. RIC remains stable at a value very close to 1 for a much wider range of these factors. Moreover, IQR does not output a value of 1 even for perfect correlation between the two variables. is is a consequence of the denominator not being a normalised quantity. Again, also in the case of nonlinear correlations, RIC provides much more consistent results also in the presence of a significant number of outliers (the comparative resilience is similar to the case of linear correlations). e positive qualities of RIC, compared to PCC and IQR, are not a negligible matter in practice because, in real-life applications, the effects of the noise and the uncertainties It should be noted that, from the analysis performed as a function of the binning, it is clear that property 2 of Section 3 is satisfied, i.e., ΔN bin /N bin ≫ ΔRIC/RIC.

The Reciprocal Influence Criterion:
Numerical Tests for Nonfunctional Dependencies e cases treated in this section, to exemplify the properties of RIC for nonfunctional dependencies (with additive Gaussian noise of mean equal to zero and standard deviation equal to 0.1), are shown in Figure 4. ese types of dependencies are quite involved and difficult to resolve. ey are fully nonlinear and they cannot even be represented by functions. For all these cases, RIC performs significantly better than all other indicators. A synthetic overview of the results is reported in Table 2. Inspection of Table 2 and Figure 4 reveals that the RIC criterion is always higher than the others by a factor. A part of the case of the rhomboid dependence always provides a value of 0.8 or higher, whereas the other indicators are closer to zero. e RIC, therefore, provides a much more reliable indication that there is a strong correlation between the two variables involved. Even the two other most sophisticated criteria, the IQR and distance correlation, perform significantly worse for all the examples investigated.

Robustness to Noise of Different Statistics and Outliers
e signals and data acquired in many scientific disciplines are typically affected by noise. e assumption of Gaussian statistics is often justified, but there are also other important types of noise of great practical and theoretical importance. Two of the most relevant distributions are certainly the Poisson and gamma.
Poisson distribution: 8 Complexity Gamma distribution: Figure 5 reports some comparative examples of the performance of the various indicators for these two distributions. e cases reported are full representatives of a series of systematic tests performed to investigate this point. In general, as for the Gaussian distribution reported in Figure 5 as a reference, RIC is also much less affected by additive noise of different distributions.
Another potential source of data contamination, of great practical relevance, is the presence of outliers. If the user is aware of the problem and has some prior information about the statistics of the outliers, some measures to remedy the situation can be taken before applying the dependence indicators [17,18]. ese measures belong to the family of robust statistics and can be quite effective. On the contrary, it is not always the case that the practitioner is aware of the issue and therefore investigating the robustness of the various dependence criteria to outliers remains a significant subject. Some representative results of a series of systematic tests, performed to assess this aspect, are shown in Figure 6 for the main classes of functional dependencies. In these cases, the outliers are generated as random Gaussian points with a mean equal to zero and standard deviation  comparable to the range of the function. As expected, the most vulnerable indicator is the PCC. e rank-based criteria and the distance correlation are slightly more insensitive; the outliers must be relatively high both in number and amplitude to affect the ranking, before they have a detrimental effect on the values of these indicators. In any case, even in the presence of outliers, RIC remains the most robust criterion.

The Conditional Version of the Reciprocal Influence Criterion
In many applications of correlation analysis, one fundamental objective consists of determining the mutual influence between variables in the presence of confounding factors. It is therefore natural to investigate the potential of a conditional version of RIC: In the following plots, the performances of RIC cond are compared with the ones of a conditional version of IQR: e plots of Figure 7 report only the cases of nonlinear correlations (z � x 2 + y, z � x 3 + y, z � e x + y), but the same conclusions apply also to linear effects.
In addition, in this application, RIC provides much better resilience to noise, and it is less dependent on the choice of the binning to determine the pdfs of the quantities involved. Moreover, even for very low levels of uncertainty, in the limit of no noise, it manages to identify more clearly the mutual correlations actually at play.

Discussion and Conclusions
To quantify the mutual influence between quantities, a new indicator has been introduced, the reciprocal influence criterion. A conditional version to separate the effects of confounding factors has also been devised. RIC reproduces the results of PCC in the case of linear correlations but is more robust against the influence of additive noise and outliers. In the case of nonlinear influences, RIC outperforms not only the ranked criteria and the distance covariance but also the information theoretic indicators such as the IQR in many respects; it provides more interpretable results and is more robust against noise and less sensitive to the choice of the binning of the pdfs involved. All these competitive advantages can be quite important in practice.
Other aspects, not to be neglected in the perspective of a wide application of the proposed indicator, are the fact that RIC is conceptually very simple, easy to implement, and fully general, in the sense that it does not rely on specific assumptions about the properties of the stochastic variables involved. In terms of requirements on the data, of course, enough examples must be available to properly calculate the pdfs, but again RIC is more parsimonious than the other indicators, which require estimating the probability distribution functions of the quantities involved.
Regarding future developments, it is planned to investigate whether alternative versions of the entropy and, therefore, of the derived quantities, can help improve the performance of RIC [19][20][21][22]. Furthermore, additional formulations, more suited to the investigation of actual causal relations than simple correlations, are also under consideration [23][24][25][26]. In terms of practical applications, some of the most immediate range from the investigation of synchronization experiments and disruptions in thermonuclear fusion [27][28][29][30][31][32][33][34][35][36][37] to the refinement of measurement techniques   Asymptotical consistency: this property means basically that the indicator assumes appropriate values when the MI decreases toward zero or becomes very large. Also, in this respect, RIC behaves very satisfactorily. e indicator ranges from 0 to 1 and increases monotonically with MI, as can be derived directly from equations (19) and (20). Independence to offset (unbiased): this additional very important property assures that a constant offset or bias in the data does not affect the results and can be written as RIC (X + a, Y + b) � RIC (X, Y) ∀a, b ϵ P. Also, this property is a direct consequence of the unbiased nature of the mutual information.

B. The Choice of Parameter A
In general, the RIC indicator is defined as In this appendix, the behaviour of RIC with respect to A is discussed. e parameter A can indeed be optimised depending on the situation and the objectives of the analysis.
is degree of freedom can become handy in various applications. e reason why A has been set equal to 10, to obtain the results presented in his work, is that, for this choice, RIC produces quite well the trends of the PCC in case the correlation between the two quantities analysed is linear. Figure 8 shows how the RIC values vary as a function of the linear correlations between x and y and A.
For A � 10, RIC varies almost linearly with squared PCC, and the consistency with this very popular indicator can be considered a positive quality in most applications.

Data Availability
e data used to support the findings of the study are available from the corresponding author upon request.

Disclosure
Rossi and Gelfusa are the co-first authors. e funders had no role in the design of the study; in the collection, analyses, or interpretation of the data, in the writing of the manuscript, and in the decision to publish the results.

Conflicts of Interest
e authors declare no conflicts of interest.