Discrete Dynamics in Nature and Society
Volume 2019 (2019), Article ID 4989140, 12 pages
https://doi.org/10.1155/2019/4989140

Research Article

The Efficacy of Predictive Methods in Financial Statement Fraud

Mahdi Omidi,¹ Qingfei Min,¹ Vahab Moradinaftchali,² and Muhammad Piri³

¹Faculty of Management & Economics, Dalian University of Technology, Dalian, China
²School of Mathematical Sciences, Dalian University of Technology, Dalian, China
³Department of Business Administration, Malayer University, Iran

Correspondence should be addressed to Mahdi Omidi; momidi_pl@yahoo.com

Received 18 December 2018; Revised 8 March 2019; Accepted 2 April 2019; Published 2 May 2019

Academic Editor: Luca Pancioni

Copyright © 2019 Mahdi Omidi et al. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Abstract.

The existence and persistence of financial statement fraud (FSF) are detrimental to the financial health of global capital markets. A number of detective and predictive methods have been used to prevent, detect, and correct FSF, but their practicability has always been a big challenge for researchers and auditors, as they do not address real-world problems. In this paper, both supervised and unsupervised approaches are employed for analysing the financial data obtained from China’s stock market in detecting FSF. The variables used in this paper are 18 financial datasets, representing a fraud triangle. Additionally, this study examined the properties of five widely used supervised approaches, namely, multi-layer feed forward neural network (MFFNN), probabilistic neural network (PNN), support vector machine (SVM), multinomial log-linear model (MLM), and discriminant analysis (DA), applied in different real-life situations. The empirical results show that MFFNN yields the best classification results in detection of fraudulent data presented in financial statement. The outcomes of this study can be applied to different types of financial statement datasets, as they present a practical way for constructing predictive models using a combination of supervised and unsupervised approaches.

1. Introduction

The current business environment is experiencing an upsurge in financial accounting fraud. As a result, financial accounting fraud detection has become an emerging topic for business practitioners, industries, and academic research [1]. The high-profile financial frauds revealed in large companies such as Enron, Lucent, WorldCom, and Satyam over the last decade emphasize the importance of detecting and reporting financial accounting fraud [2].

The Association of Certified Fraud Examiners (ACFE) classifies occupational fraud into three primary categories, namely, asset misappropriation, corruption, and financial statement fraud [3]. In this study, the focus is on financial statement fraud (FSF), which is reportedly the costliest type of fraud, although, comparatively, it occurs at a lower frequency (Figure 1).

Figure 1: Occupational fraud and its frequency and median loss.

Fraud occurs when perpetrators-in-disguise “cook the books” by the intentional misstatement or manipulation of financial data. The users of financial statements may be investors, creditors, lenders, shareholders, pensioners, and other market participants [4]. Financial fraud is threatening the market increasingly for participants and the gravity of the problem is rising continuously. The effective detection of accounting fraud, however, remains a complex task for accounting professionals [5]. Traditional auditors fail to cope with emerging accounting frauds for many reasons, such as the lack of the required data mining knowledge, experience and expertise due to the infrequency of financial frauds, and the efforts made by other concerned people at finance departments, such as Chief Financial Officers (CFOs), financial managers, and accountants, for concealment and deception [6]. In addition, FST is usually committed by a smart team of knowledgeable perpetrators (e.g., top executives and auditors) with a well-planned scheme who are capable of masking their deceit [7]. The need for additional automatic data analysis procedures and tools for the effective detection of falsified financial statements is therefore more pronounced today than ever. Nowadays, a series of standards, such as SAS 82, are put forth by accounting and audit professionals in order to improve the auditors’ performance in detecting material misstatement in financial data.

Financial ratios can be mined for examining the key characteristics of financial frauds. These ratios are calculable based on the values taken from financial statements and can consolidate the detection of fraudulent materials presented in financial statements by quantifying many aspects of businesses. As an integral part of the financial statement analysis, these ratios can reveal the status of the receivables and bad debts, whether the business is carrying excess debt or inventory, whether the operation expenses are high, and whether the company assets are being properly used for generating income [8]. The liquidity, safety, profitability, and efficiency ratios are important indicators of financial ratios.

Due to the advancement of data mining techniques, sophisticated approaches to knowledge discovery are required for extracting previously unknown information from data. Supervised and unsupervised learning techniques are two widely used knowledge discovery techniques in data mining [9]. For example, clustering algorithms find group data points with natural similarities [10–15].

1.1. The Novelties of This Study Can Be Mentioned as the Following

(1)The novelty in application: the algorithm designed to conduct this study can be applied to any financial statement dataset with respect to the 18 variables used in this study. It is because the results of the study in the Research Methodology and Results section and also Conclusion part present the best approach to financial statement fraud detection. In other words, the result of this study enables any individual investor who wishes to buy stocks from the stock market and wants to know if the annual financial statement issued by companies reflects their real financial circumstance or it tends to defraud the investors by fraudulently manipulated data. Therefore, this study equips the stock market players with an applicable/accessible tool that provides them decision support. The model as well as the results of this study is investor friendly.(2)The novelty in theory: generally speaking, financial statement frauds detection approaches can be classified in two main families, which correspond to the two families of machine learning algorithms: supervised and unsupervised ones. In the former family (known as signature-based predictive models), past data are labeled as fraudulent or non-fraudulent; for instance, based on auditor’s judgment, the algorithms then learn over this data, to create a model that is applied to new instances appearing in the system. On the other hand, unsupervised techniques are based on the automatic detection of patterns and attributions that are considered “normal” for a given user (individual investor in our study), for then detecting data that are not consistent with such patterns, as fraudulent data are expected to depart from the normal data pattern. Both families have their own advantages and disadvantages. In this study both supervised and unsupervised techniques were employed, not simultaneously but sequently for different characteristics that they possess. By doing so the validity of the outcomes rises up.(3)Reducing the probability of data misplacement in clustering by adding the “suspicious” group: for the first time in the area of the financial statement fraud detection, this study suggests 3 segments for the outcomes of the algorithm. They are, namely, “fraudulent,” “non-fraudulent,” and “suspicious.” To refine the data that share some degree of attribution with both fraudulent and non-fraudulent cases and produce high quality clusters with higher intraclass similarity and low interclass similarity, we labeled an additional segment called “suspicious” to reduce the possibility of data misplacement. The data fallen into the “suspicious” group is subjected to further evaluation by machine and human intelligence. Each of these three clusters undergoes a different procedure in this study. They build the cornerstone of the hypothesis testing by contributing different fractions in the simulation tests presented in this research.(4)A major drawback with the existing works in the literature has been addressed effectively in this study: all the previous works have used balanced datasets that contain an equal size of fraudulent and non-fraudulent cases in machine learning. This draws an unrealistic pattern that cannot be inductive. Because it is always been observed that the proportion of the fraudulent cases is just a small fraction of the whole dataset. In our statistical analysis we conducted various possible scenarios that measure the performance of our model in different real-life situations.

In general, this study optimizes and refines the existing approaches to application of machine learning in financial statement fraud detection. This study finally benefits the academician and business practitioners in the strategic decision making regarding their investment in the stock market. Also the auditors of the financial statement can apply the suggested techniques in this research in their auditing procedures. The methodology and procedure used in this paper can easily be extended to other domains of financial crimes. It can be described as novelty in the mode of execution and application.

This research set and achieved the following main objectives:

(1)Putting financial data in proper categories using cluster analysis with the help of an auditor.(2)Evaluating the efficiency of 5 supervised predictive models with respect to precision and recall values.(3)Using the outcomes to answer several secondary objectives of the research, designed as statistical hypothesis testing.

In other words, a compatible cluster analysis is conducted to automatically partition the data, and the process of labeling the clusters is then performed with the help of an auditor. Later, the performance and learning rate of 5 widely used supervised predictive models are evaluated. For this purpose, certain fractions of fraudulent companies have been added to various sample sizes to study the behaviors of the models in different real-life situations. The primary beneficiaries of this research are interested investors in stock market, auditors involved in uncovering FSF, and emerging companies carefully observing their financial statements for profiting in the market [8].

It is worth mentioning that the imbalance data distribution used in this study does not hinder the learning task [16, 17]. Depending on the problem, design, and the objectives of the research, one or a combination of several methods (e.g., using the right evaluation metrics, resampling with different ratios, over-/undersampling, generating synthetic samples, using different algorithms, using penalized models, etc.) can be applied for a better result. However, each approach has its own pros and cons. It has been reported that the error rate caused by imbalanced class distribution decreases when the number of examples of the minority class is representative which means the positive class examples can be better learned [18].

2. Literature Review

Auditing procedures have proven to have many deficiencies for detection of fraudulent financial reporting. It is simply because they are not designed in that way. In an organization, the manager is the one who is morally responsible and accountable for detection of fraudulent financial data. But in reality majority of financial statement frauds are committed with the awareness or consent of management. Any failure in detection of misstated financial statement can severely harm the credibility of the audit profession [19]. Presently, auditing practices have to be conducted in a timely manner to cope with an increasing number and occurrence of financial statement fraud cases. The novel techniques such as data mining claim that it has advanced classification and prediction capabilities and can be employed to facilitate auditors’ role in terms of successfully accomplishing the task of fraud detection [20].

Data mining is a process in which different techniques are used to extract valid data patterns. This step involves choosing the most appropriate data mining method (such as classification, regression, clustering, or association) [21] and choosing the algorithm belonging to one of the previous families. Finally, the chosen algorithm is used to solve the problem by setting the main parameters and validation procedures. In general, data mining methods are divided into two types of predictive and descriptive (e.g., clustering) methods, and the former is divided into two parts: statistical methods and symbolic methods. Statistical methods are known by representing knowledge through mathematical models with their calculations. Symbolic methods display knowledge using symbols and associations, which ultimately produce more interpretable models.

Regression models, artificial neural networks, support vector machines, Bayesian models, etc. fall into the category of statistical methods. Regression models are among the classical models that require a class of equation modeling. Linear, quadratic, and logistic regressions are among the most well-known models in data mining. They impose basic requirements on data and try to use all its features whether useful or not. One of the most powerful mathematical models is artificial neural networks (ANNs), which are suitable for almost all data mining tasks. There are different formulations of neural networks; the most commonly used are learning vector quantization, and multi-layer feed forward neural networks, radial basis function networks, and multi-layer perception. Neural networks are based on the definition of neurons, which are atomic portions that collect their inputs for an output according to an activation function. These models are generally better than other models if the networks are configured properly, but they are not very popular for their complex structures. Support vector machines are based on learning theory and work very well when data is linearly separable. Unlike regression models, these models do not usually require producing interactions among variables, and similar to ANNs, they are robust to noise and outliers.

Predictive methods are mainly attributed to supervised learning. Supervised methods disclose the relationships between input traits and a target attribute in the structure that we call “model.”

Regression and classification problems are both in this category. A supervised scenario works so that a model is first fitted to a training data set and then used to predict unobserved instances. Here, the main goal is to map the inputs to an output whose correct values are determined by a supervisor. On the other hand, there is no such supervisor in unsupervised learning, and only input data is available. Therefore, the purpose is to find regulations, irregularities, similarities, relationships, and associations in the inputs. One of the advantages of the unsupervised learning is that, unlike supervised learning, more sophisticated models can be learned. Since in supervised learning the goal is to find the relation between two sets of observations, as the number of steps increases the difficulty of learning process grows exponentially due to high computational costs, and thus these models cannot be learned deeply. The two famous problems that belong to unsupervised learning are clustering and association rules.

Data mining is widely applied in many domains. Below is a summary of applications of data mining:

(i)Data mining applications in finance: money laundering and detection of other financial crimes, classification of customers and target marketing, loan payment prediction, and customer credit analysis.(ii)Data mining application in retail industry: analysis of effectiveness of sales campaign, customer retention and analysis of customer loyalty, product recommendation, and cross referencing of items.(iii)Data mining for the telecommunication industry: fraudulent pattern analysis and the identification of unusual patterns, and mobile telecommunication services.

Numerous studies have sought to detect fraud using innovative approaches. Nigrini and Mittermaier [22] described digital and number tests based on Benford’s law that could be employed by auditors for this purpose. Markou and Sameer [23, 24] reviewed anomaly detection using both statistical and neural network-based approaches. Rezaee [7] elaborated the causes, consequences, and deterrents of financial statement fraud (FSF) incidents and discussed the factors that might increase its likelihood and emphasized the role of corporate governance in financial fraud prevention. In his work, Rezaee introduced five interactive factors (Cooks, Recipes, Incentives, Monitoring, and End-Results [CRIME]) that can influence fraud occurrence, prevention, and detection in the financial domain. Phua et al. [25] categorized, compared, and summarized the algorithms and performance measurement of almost all the studies published before 2005. Patcha and Park [26] and Hodge and Austin [27] presented a comprehensive survey of anomaly detection systems and hybrid intrusion detection systems and discussed various anomaly-based fraud detection techniques.

Chandola et al. [28] discussed the application of various techniques for detecting anomaly and fraud using supervised approaches. Wu et al. [29] used clustering methods in a large imbalance class distribution to produce subclasses within each large cluster with relatively balanced class sizes. They employed signature-based techniques such as SVM for classification tasks and concluded that models can generally outperform auditors in detecting rates without assistance. Nonetheless, newly invented techniques used by perpetrators in the last decade have remained undetected. Sabau et al. [30] reviewed the techniques used for fraud detection in the last ten years and made a ranking to show the most common clustering techniques employed in fraud and anomaly detection.

Zhou and Kapoor [31] examine the effectiveness and limitations of data mining techniques such as regression, decision trees, and neural network and Bayesian networks. They explore a self-adaptive framework based on a response surface model with domain knowledge to detect financial statement fraud. Sharma and Panigrahi [32] propose a data mining framework for detection of financial fraud. Cecchini et al. [33] propose a methodology to aid in detecting fraudulent financial reporting by utilizing only basic and publicly available financial data. Chen and Roco [34] suggest that an artificial intelligence technique turns out to perform quite well in identifying a fraud lawsuit presence, and hence it can be a supportive tool for practitioners.

Ratio analysis is the most conventional approach used by auditors for fraud detection. The problem with this approach, however, is being subjective in the selection of ratios that are likely to indicate fraud [35]. Data mining techniques are for uncovering implicit, previously unknown, actionable knowledge [20]. To date, the use of these techniques has been limited in FSF. The majority of data mining techniques used in this area are supervised techniques such as logistic regression, neural networks, decision trees, and text mining. An even smaller number of studies have employed cluster analysis as a mining technique for categorizing data based on their natural tendencies and similarities. Han et al. [36] presented the critical conversion risks brought into business through IT from the auditors’ point of view. They suggested that firms with higher levels of IT investment are associated with higher audit risks for external auditors and emphasized the importance of enhancing the capabilities of auditors in highly IT-based systems.

Abtahi et al. [37] used Bayesian classifier model to suggest a system in which fraud in future market-trading coins can be detected. The primary labeling of data was done by K-means clustering. In the present study a similar approach is employed. The model could accurately classify 94.55% of the fraudulent cases.

Jan [38] takes 160 companies (including 40 fraudulent companies) to evaluate multiple data mining techniques including ANN and SVM. Also four types of decision trees (classification and regression tree (CART), chi-square automatic interaction detector (CHAID), C5.0, and quick unbiased efficient statistical tree (QUEST)) were used in this study. The results of this study show that the ANN + CART model yields the best classification results, with an accuracy of 90.83% in the detection of financial statements fraud.

Yao et al. [39] propose a hybrid fraudulent financial statement detection model combining the PCA, Xgboost, SVM, RF, DT, ANN, and LR to construct the fraud detection model, and the classification accuracy of each model was compared to determine the optimal model. The study indicated that random forest outperformed the other four methods.

Overall, in the past, expert analysis was used to judge the truthfulness of financial statements. A lot of doubts were raised about the efficacy of the traditional approaches to detect fraudulent financial data as auditors lack the required data mining techniques. That is the reason why traditional expert analysis often fails to identify fraud. Most previous studies use only 1-2 data mining techniques, without comparisons between each other. As for the input variables, existing studies obtain them from publicly available financial reports. The input variables need to be representative enough to reflect all aspects of the company’s financial status. For instance, quick ratio and liquidity ratio reflect the solvency; sales growth rate and EPS represent the profitability; and we can see operating capacity from inventory turnover and turnover of total capital. In addition, some studies investigate linguistic variables from MD&A and uncovered emotional characteristics of the language in financial reports [40]. Hajek and Henriques [40] found that fraudulent financial reports tend to include a higher rate of negative sentiment.

Some points have remained unaddressed in literature on the utilization of data mining for the detection of potential fraudulent financial statement data. Existing algorithms assign cases into two groups only, namely, “fraudulent” and “non-fraudulent,” and this limitation may lead to a significant rate of data misplacements. That is to say, some cases that are truly innocent can fall into the fraudulent group and vice versa. Adding another group as “suspicious” will resolve this error, as it allows the reexamination of cases that fall into somewhere between these two groups using other tools or human intelligence or a combination of both.

Another drawback is that most researchers have used balanced datasets for building their predictive models. These datasets contain an equal size of fraudulent and non-fraudulent cases. The problem with this method is that it does not reflect the real situation in real-world scenarios, since the ratio of fraudulent cases is very small compared to non-fraudulent cases.

In this study, the performance of the algorithm is measured in various situations. That can be a significant help in understanding how algorithms behave and which ones work better than others. Also, various real situations are simulated based on real data, and hypothesis tests are subsequently run to obtain applicable results.

3. Research Methodology and Results

The variables used in this research include 18 financial items (Table 1), which represent the fraud triangle and have been obtained from the study by Ravisankar et al. [8] and reflect liquidity, safety, profitability, and efficiency as indicative of the financial status. The dataset used in this research was obtained from 2659 companies listed in China’s stock market by the end of December 2015. To calculate the C11 and C12 variables, relevant data were obtained from the companies for the year 2014.

Table 1: A brief description of the variables used in this study.

Variable Name
Description

C1
Net profit

C2
Gross profit

C3
Primary business income

C4
Primary business income/Total assets

C5
Gross profit/Total assets

C6
Net profit/Total assets

C7
Inventory/Total assets

C8
Inventory/Current liabilities

C9
Net profit/Primary business income

C10
Primary business income/Fixed assets

C11
Primary business profit/Primary business profit of last year

C12
Primary business income/Primary business income of last year

C13
Fixed assets/Total assets

C14
Current assets/Current liabilities

C15
Capital and reserves/Total debt

C16
Long-term debt/Total capital and reserves

C17
Cash and deposits

C18
Inventory/Primary business income

After normalizing the variables, a generalized k-means cluster analysis was utilized to automatically extract the six clusters. These clusters were then labeled with the help of an auditor. The outcome of the labeling process was three clusters labeled as “fraudulent,” “suspicious,” and “non-fraudulent” (Table 2). The reason for including the “suspicious” cluster is that the algorithm was unable to quickly and accurately place the companies fallen in this group into either fraudulent or non-fraudulent groups. Consequently, to enhance the total performance of the algorithm, all the suspicious cases will later undergo further investigation.

Table 2: Clusters after labeling. The values in parentheses indicate the frequency of clusters that are merged.

Count
Cumulative count
% of all cases
Cumulative % of all cases

Fraudulent
153 
(7+146)
153
5.75
5.75

Suspicious
1596 
(418+346+832)
1749
60.03
65.78

Non-fraudulent
910
2659
34.22
100.00

To design a simulation study matching the real situation, we should note that over 60% of our observations pertained to the “suspicious” cluster (Table 2), which requires reevaluation. As these companies will always undergo further investigations, a fixed proportion of them (60%) should always be contained in all situations in our design. This study has taken sample sizes of 300, 500, and 800 and in each of these samples, various ratios of fraudulent cases, i.e., 1%, 3%, 5%, 7%, and 10%, are included. The aim is to examine and find out whether or not the precision and recall of the predictive models used in this study alter in different situations. Five predictive models have been used in this research, namely, MFFNN [41], PNN [42], SVM [43], MLM [44], and DA [45]. The first three models are powerful mathematical models suitable for almost all data mining tasks, and the last two are the widely used classification methods, which fall in the realm of classical statistics. In the body of this research the mathematic formulas are not presented. Interested readers can follow the mathematic concepts of this research in the reference list.

In this work, clustering process was continued until the reduction in error function (average distances of observation from the canter of clusters in two consecutive iterations) is less than 5%. For instance, if the iteration k+1 has less than 5% improvement, then we determine k as the number of clusters. To measure the distances, Euclidean distance formula was used in this study as follows:

To simulate a real-world situation, different sets of parameters have been assigned to each noted model, and each model is then evaluated 1000 times for each set of parameters. In each iteration, a training sample of size (300, 500, and 800) and a test sample of size are randomly chosen from a total of n=2659 cases, such that each sample contains a given proportion of fraudulent companies (1%, 3%, 5%, 7%, and 10%) and 60% of suspicious companies. It is worth noting that, in each iteration, the training and test samples are independent, so as to avoid overfitting. As each model is evaluated for different sets of parameters in each iteration, the best-fitted model for each iteration is chosen based on the highest values of precision and then recall in the test. This process is depicted in Figure 2. Table 3 shows the confusion matrix used for calculating the performance values, i.e., (1) precision and (2) recall, which are formulated as follows [46]:

Table 3: A scheme of a confusion matrix.

Predicted

True
Fraudulent
Suspicious
Non-fraudulent
Sum

Fraudulent

Suspicious

Non-fraudulent

Sum

Figure 2: The process of selecting samples, building and evaluating models in each iteration.

Precision can be defined as the proportion of events produced by the models that are correct. Recall is defined as the proportion of events occurring in the domain that are predicted as such by the models. In our case, we have 3 classes and the numbers of events that are correctly situated in their respective classes are both n11 and n33 (Table 3). In other words, since these two classes are of importance, we separate them from the “suspicious” class to calculate their precision and recall values while our classifier is still able to distinguish them as two distinct classes. An application of the use of precision and recall can be found in [46]. As the nature of our dataset shows an imbalance (skewed) dataset, precision and recall are adequate parameters to be utilized for our study [47].

For different sample sizes and proportions of fraudulent companies, Tables 4–6 report the mean, standard deviation, and minimum and maximum values of the precision and recall for each model after 1000 iterations.

Table 4: Descriptive statistics of the performance measures obtained from 1000 iterations segmented by different rate of fraudulent companies, and for training sample size of 300 and test sample size of 100.

1% Fraud
3% Fraud
5% Fraud
7% Fraud
10% Fraud

Prec.
recall
Prec.
recall
Prec.
recall
Prec.
recall
Prec.
recall

MFFNN
Avg.
0.955
0.949
0.955
0.947
0.957
0.946
0.957
0.945
0.957
0.940

SD
0.033
0.035
0.034
0.036
0.034
0.036
0.033
0.036
0.033
0.038

Min
0.805
0.796
0.805
0.813
0.805
0.800
0.829
0.816
0.829
0.796

Max
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000

PNN
Avg.
0.968
0.728
0.966
0.728
0.964
0.730
0.968
0.728
0.965
0.725

SD
0.027
0.053
0.027
0.052
0.029
0.055
0.028
0.055
0.030
0.055

Min
0.878
0.574
0.854
0.563
0.846
0.571
0.829
0.582
0.829
0.586

Max
1.000
0.909
1.000
0.930
1.000
0.909
1.000
0.884
1.000
0.930

SVM
Avg.
0.892
0.898
0.888
0.898
0.884
0.897
0.885
0.900
0.885
0.897

SD
0.049
0.045
0.050
0.045
0.052
0.046
0.052
0.047
0.050
0.048

Min.
0.707
0.720
0.659
0.761
0.683
0.750
0.634
0.735
0.659
0.708

Max
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000

MLM
Avg.
0.874
0.875
0.877
0.872
0.876
0.874
0.875
0.877
0.878
0.873

SD
0.055
0.049
0.055
0.048
0.055
0.049
0.056
0.049
0.054
0.050

Min
0.683
0.720
0.610
0.694
0.683
0.667
0.634
0.708
0.659
0.673

Max
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000

DA
Avg.
0.897
0.854
0.895
0.861
0.891
0.869
0.890
0.878
0.891
0.886

SD
0.050
0.049
0.051
0.048
0.052
0.048
0.052
0.048
0.051
0.048

Min
0.683
0.685
0.683
0.698
0.659
0.727
0.707
0.694
0.659
0.732

Max
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000

Table 5: Descriptive statistics of the performance measures obtained from 1000 iterations segmented by different rate of fraudulent companies, and for training sample size of 500 and test sample size of 170.

1% Fraud
3% Fraud
5% Fraud
7% Fraud
10% Fraud

Prec.
recall
Prec.
recall
Prec.
recall
Prec.
recall
Prec.
recall

MFFNN
Avg.
0.967
0.963
0.966
0.961
0.967
0.955
0.966
0.954
0.965
0.953

SD
0.022
0.024
0.022
0.025
0.023
0.026
0.023
0.025
0.024
0.025

Min
0.882
0.870
0.882
0.867
0.866
0.846
0.882
0.868
0.882
0.865

Max
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000

PNN
Avg.
0.972
0.729
0.969
0.727
0.970
0.721
0.970
0.723
0.970
0.723

SD
0.020
0.043
0.022
0.043
0.022
0.043
0.022
0.043
0.022
0.042

Min
0.882
0.607
0.882
0.598
0.896
0.591
0.897
0.598
0.881
0.608

Max
1.000
0.918
1.000
0.861
1.000
0.835
1.000
0.853
1.000
0.868

SVM
Avg.
0.905
0.909
0.897
0.908
0.897
0.906
0.900
0.907
0.898
0.907

SD
0.037
0.036
0.037
0.034
0.039
0.035
0.037
0.036
0.039
0.033

Min.
0.779
0.781
0.765
0.787
0.761
0.766
0.735
0.770
0.750
0.782

Max
1.000
1.000
1.000
1.000
0.985
1.000
1.000
1.000
1.000
0.985

MLM
Avg.
0.885
0.882
0.884
0.881
0.884
0.881
0.888
0.886
0.888
0.886

SD
0.042
0.037
0.042
0.037
0.041
0.037
0.041
0.039
0.041
0.036

Min
0.735
0.754
0.765
0.775
0.731
0.734
0.750
0.747
0.735
0.763

Max
0.985
0.984
0.985
0.983
1.000
0.984
0.985
1.000
0.985
0.984

DA
Avg.
0.904
0.865
0.901
0.871
0.900
0.878
0.902
0.888
0.900
0.896

SD
0.038
0.038
0.039
0.036
0.039
0.036
0.037
0.036
0.038
0.034

Min
0.779
0.733
0.735
0.757
0.746
0.756
0.750
0.782
0.750
0.753

Max
1.000
0.984
1.000
0.984
1.000
0.983
1.000
1.000
1.000
0.985

Table 6: Descriptive statistics of the performance measures obtained from 1000 iterations segmented by different rate of fraudulent companies, and for training sample size of 800 and test sample size of 270.

1% Fraud
3% Fraud
5% Fraud
7% Fraud
10% Fraud

Prec.
recall
Prec.
recall
Prec.
recall
Prec.
recall
Prec.
recall

MFFNN
Avg.
0.975
0.969
0.974
0.966
0.972
0.965
0.972
0.964
0.973
0.962

SD
0.015
0.017
0.016
0.018
0.016
0.019
0.016
0.019
0.017
0.019

Min
0.907
0.897
0.917
0.881
0.916
0.878
0.898
0.893
0.907
0.886

Max
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000

PNN
Avg.
0.975
0.726
0.974
0.720
0.974
0.720
0.975
0.718
0.976
0.718

SD
0.015
0.035
0.016
0.034
0.016
0.034
0.016
0.034
0.015
0.035

Min
0.916
0.623
0.906
0.630
0.906
0.614
0.879
0.631
0.916
0.578

Max
1.000
0.843
1.000
0.846
1.000
0.822
1.000
0.835
1.000
0.855

SVM
Avg.
0.916
0.921
0.909
0.916
0.905
0.914
0.906
0.914
0.907
0.916

SD
0.028
0.027
0.028
0.026
0.029
0.028
0.030
0.026
0.028
0.026

Min.
0.796
0.828
0.796
0.811
0.813
0.800
0.806
0.833
0.796
0.840

Max
0.981
0.990
0.991
1.000
0.972
0.990
0.981
0.990
0.972
0.980

MLM
Avg.
0.894
0.897
0.892
0.894
0.890
0.893
0.892
0.895
0.894
0.898

SD
0.032
0.027
0.032
0.027
0.032
0.028
0.032
0.027
0.029
0.027

Min
0.759
0.804
0.787
0.809
0.785
0.787
0.787
0.805
0.806
0.786

Max
0.981
0.980
0.991
0.980
0.972
0.980
0.981
0.980
0.981
0.970

DA
Avg.
0.912
0.870
0.908
0.877
0.906
0.884
0.907
0.892
0.905
0.902

SD
0.029
0.029
0.029
0.028
0.030
0.028
0.029
0.027
0.029
0.026

Min
0.806
0.784
0.815
0.800
0.804
0.771
0.806
0.800
0.796
0.814

Max
0.991
0.971
0.991
0.961
0.981
0.979
0.991
0.980
0.991
0.980

According to the findings shown in Tables 4–6, for all combinations of sample sizes (300, 500, and 800) and proportions of fraudulent companies, PNN and MFFNN have the highest precision values, while PNN has the lowest recall value. As the sample size increases, the precision value grows accordingly, and further, their deviation becomes smaller. Unlike PNN, however, MFFNN has the best recall value. Meanwhile, the recall values for the PNN approach show a decreasing trend.

According to the finding in Tables 4–6, for all three sample sizes (300, 500, and 100) and for various inclusions of fraudulent cases in samples, PNN has the highest precision. But taking recall into consideration, PNN has the poorest results. Having considered the value of precision, MFFNN yields in the second position. MFFNN also outperforms all the models in terms of recall value. We observed that sample size is positively associated with the model performances in terms of precision and recall, except for PNN that displays a negative association between the sample size and recall value (it decreases for all proportions except for 1% which shows an unclear trend). So as MFFNN outperforms all the other 4 models in terms of both precision and recall, we select MFFNN as the reference model. To make our findings generalizable and conclusive, we design the hypothesis statistical testing as follows.

For each approach, first, a one-way analysis of variance (ANOVA) was carried out to compare the mean values of precision and recall of different combinations for a given sample size. In other words, for a given sample size, the aim is to see whether the mean values of the performance measures of each approach differ as the rate of fraudulent companies changes in the sample. In the case of significance, the nature of this relationship will be determined by looking at the mean values of the data to find out whether they follow a strictly monotonic trend. The reports are displayed in Table 7.

Table 7: The outcome of one-way ANOVA for mean comparison of each method for given sample sizes. The downward (upward) arrow shows a strictly decreasing (increasing) trend.

N=300
N=500
N=800

F Prec.
F Recall
F Prec.
F Recall
F Prec.
F Recall

MFFNN
0.652
9.659
1.418
24.676
4.394
20.885

PNN
2.739
0.952
2.1764
5.646
3.118
10.342

SVM
4.047
0.728
7.788
0.887
23.551
12.244

MLM
0.954
1.287
2.3851
4.6429
2.425
6.91

DA
3.141
70.491
2.356
123.47
9.366
204.69

The mean values of precision and recall of the five approaches are compared for any given sample size and proportion of fraudulent companies using the one-way repeated measures ANOVA. The real goal is to know whether or not the mean values of performance measures differ among all five approaches. Table 8 reports the results of this analysis.

Table 8: The result of one-way repeated measures ANOVA for the second design.

N=300
N=500
N=800

F Prec.
F Recall
F Prec.
F Recall
F Prec.
F Recall

1% Fraudulent
1478
4992
2494
9105
3911
17057

3% Fraudulent
1420
5090
2496
9658
4033
17908

5% Fraudulent
1406
4536
2538
9723
4287
16697

7% Fraudulent
1546
4691
2390
9713
4076
17585

10% Fraudulent
1513
4639
2362
9651
4337
17685

A one-way ANOVA is conducted for any given proportion of fraudulent companies to see whether or not the performance measures of each approach differ for different samples sizes. Table 9 demonstrates the results of this analysis.

Table 9: The result of one-way ANOVA for mean comparison of each method for given proportions of fraudulent companies. The downward (upward) arrow shows a strictly decreasing (increasing) trend.

1% fraudulent
3% fraudulent
5% fraudulent
7% fraudulent
10% fraudulent

F Prec.
F Recall
F Prec.
F Recall
F Prec.
F Recall
F Prec.
F Recall
F Prec.
F Recall

MFFNN
163.3
150.2
139.9
122.9
99.8
116.2
95.8
121.1
100.1
158.1

PNN
32.9
0.814
28.7
9.4
44.1
14.4
24.7
11.9
51.9
7.9

SVM
96.5
99.1
72.3
63.2
66.2
51.0
73.6
34.7
75.1
69.3

MLM
50.6
80.9
28.5
81.1
26.6
55.3
40.44
52.1
33.0
104.0

DA
36.4
42.7
25.9
49.4
33.1
38.5
44.14
36.2
28.8
49.1

According to Table 7, of those significant performance measures, only recall values for MFFNN and DA show a monotonic behavior, which is downward for MFFNN and upward for DA. That is to say, as the rate of fraudulent companies increases in the sample, a decrease is expected in the recall value for MFFNN but an increase for DA.

For all the other significant cases reported in Table 7, no monotonic trends were observed as the rate of fraudulent companies increased in the sample. Therefore, it can be concluded that the proportion of fraudulent companies in this sample has no effects on the precision of the five approaches and only has an impact on the recall values of MFFNN and DA, as described.

The results in Table 8 indicate that, for all combinations of sample size and proportion of fraudulent companies, the performance measures of all five approaches differ significantly. As noted earlier, according to Tables 4–6, PNN has the highest values of precision compared to the other methods. Nonetheless, this method has small recall values. MFFNN is also the second best in terms of precision values, which are slightly lower than those of PNN. Nevertheless, this gap decreases as sample size increases. The MFFNN approach gives the best recall values among all five methods. It can therefore be concluded that MFFNN performs better than the other approaches in all situations.

Table 9 presents the results of the mean comparisons in each method for different sample sizes. As shown, except for the recall values of the PNN approach in cases where the rate of fraudulent companies is 1%, all the other performance measures are significant. Based on this finding and by looking at their values in Tables 4–6, it can be inferred that, as sample size increases, the precision value for each approach also increases accordingly. The same interpretation applies to the recall values of all the other approaches, except for the PNN approach. As for PNN, increasing the sample size leads to a reduced recall value. Larger sample sizes are recommended to be examined in order to have more reliable results.

4. Conclusion and Future Research

Financial statement is the mirror of financial status of companies that reflects useful information about the financial health of company. Fraudulent financial reporting is a deliberate misstatement of numbers for the aim of deceiving users of the financial statement. If there is a force from board to achieve a certain amount of earning, and the earned income is linked with management’s bonus, enough motivational stimuli have been created for fraudulent financial reporting. Other types of motivations for conducting fraudulent financial reporting include promotion to a higher position, salary raises, or raise in the share value of the company. Financial statement fraud damages corporate reputation and negatively affects the financial market, nation, and the world economy. Therefore the need for intelligent and novel tools to enable auditors in detecting fraudulent data is so urgent. The present data mining technique could greatly provide a decision support system to mangers in detection of financial statement fraud.

This study used cluster analysis for partitioning the financial statement data into three groups. The results of the cluster analysis could then be used to conduct statistical experiments in line with the research objectives and to examine the precision and recall values of the five supervised methods known as MFFNN, PNN, SVM, MLM, and DA in different situations, as defined in Section 3. The results show that, although the ratio of fraudulent companies in the sample has a significant impact in most cases, this significance does not follow a strictly monotonic form. There are only two methods, i.e., MFFNN and DA, in which the recall values are correlated by changing the ratio of fraudulent companies. Moreover, sample size has a significant impact on both the precision and recall values of all the methods used in this study, such that a larger sample size increases the precision values of all the methods. The same is true for the recall values of all the methods, except for PNN, which shows a strictly decreasing trend. The comparison of the approaches in different situations revealed that the MFFNN approach performs better than all in terms of performance measures. Although this approach has smaller precision values than PNN, this difference is small and negligible if a larger sample size is selected.

The results of this research allow a variety of groups, especially auditors and investors, to carefully select and decide about the companies which they are auditing or in which they are investing. Using these techniques can help auditors save time and cost by utilizing automated tools, and these steps decrease their work load considerably. The present research has simulated real scenarios to show how these techniques can be properly used in auditing and fraud detection. Further studies can be carried out by adding more variables, such as enterprise size or the inclusion of IT systems in the enterprise, which seems to be a determining factor. Comparative empirical studies can also be conducted when a dataset is obtained from other stock markets. Furthermore, the methodology and procedure used in this paper can be extended to other domains for different purposes and not only for fraud detection.

Data Availability

The data (financial statement data) used in this manuscript is taken from China Securities Market and Accounting Research (CSMAR) dataset through http://us.gtadata.com which is also available via WRDS with a proper subscription. The websites to obtain data are as follows: http://us.gtadata.com  https://wrds-web.wharton.upenn.edu/wrds/index.cfm Information regarding the variables, sample population, and methodology is plenty given in Section 3 of the manuscript (Research Methodology and Results).

Conflicts of Interest

The authors declare that there are no conflicts of interest regarding the publication of this paper.

Acknowledgments

This work was supported by the National Natural Sciences Foundation of China under Grants and .

References

A. Sharma and P. Kumar Panigrahi, “A review of financial accounting fraud detection based on data mining techniques,” International Journal of Computer Applications, vol. 39, no. 1, pp. 37–47, 2012.

D. Yue, X. Wu, Y. Wang, Y. Li, and C. Chu, “A review of data mining-based financial fraud detection research,” International Conference on Wireless Communications, Networking and Mobile Computing, pp. 5519–5522, 2007.

D. James and C. Ratley, Report to the Nations on Occupational Fraud and Abuse, Association of Certified Fraud Examiners, Austin, TX, USA, 2016.

S. Wang, “A comprehensive survey of data mining-based accounting-fraud detection research,” in Proceedings of the International Conference on Intelligent Computation Technology and Automation, vol. 3, pp. 50–53, 2010.

E. W. T. Ngai, Y. Hu, Y. H. Wong, Y. Chen, and X. Sun, “The application of data mining techniques in financial fraud detection: a classification framework and an academic review of literature,” Decision Support Systems, vol. 50, no. 3, pp. 559–569, 2011.

K. M. Fanning and K. O. Cogger, “Neural network detection of management fraud using published financial data,” International Journal of Intelligent Systems in Accounting, Finance & Management, vol. 7, no. 1, pp. 21–41, 1998.

Z. Rezaee, “Causes, consequences, and deterence of financial statement fraud,” Critical Perspectives on Accounting, vol. 16, no. 3, pp. 277–298, 2005.

P. Ravisankar, V. Ravi, G. Raghava Rao, and I. Bose, “Detection of financial statement fraud and feature selection using data mining techniques,” Decision Support Systems, vol. 50, no. 2, pp. 491–500, 2011.

M. Ahmed, A. N. Mahmood, and M. R. Islam, “A survey of anomaly detection techniques in financial domain,” Future Generation Computer Systems, vol. 55, pp. 278–288, 2016.

J. Han, M. Kamber, and J. Pei, Data mining: concepts and techniques, Elsevier, Amsterdam, Netherlands, 2012.

M. R. Anderberg, Cluster Analysis for Applications, Burlington: Academic Press/Elsevier, 2014.

M. Ester, H. P. Kriegel, J. Sander, and X. W. Xu, “A density-based algorithm for discovering clusters in large spatial databases with noise,” in Proceedings of the AAAI KDD-96 Proceedings, pp. 226–231, 1996.

E. M. Knorr and R. T. Ng, “Algorithms for mining distance-based outliers in large datasets,” in Proceedings of the 24rd International Conference on Very Large Data Bases, pp. 392–403, 1998.

R. T. Ng and J. Han, “Clarans: a method for clustering objects for spatial data mining,” IEEE Transactions on Knowledge and Data Engineering, vol. 14, no. 5, pp. 1003–1016, 2002.

X. Wang and G. Dong, “Research on money laundering detection based on improved minimum spanning tree clustering and its application,” in Proceedings of the Second International Symposium on Knowledge Acquisition and Modeling, vol. 3, pp. 62–64, 2009.

H. He and E. A. Garcia, “Learning from Imbalanced Data IEEE Transactions on Knowledge and Data Engineering,” vol. 21, no. 9, 2009.

Y. Sun, A. K. Wong, and M. S. Kamel, “Classification of imbalanced data: A review,” International Journal of Pattern Recognition and Artificial Intelligence, vol. 23, no. 4, pp. 687–719, 2009.

N. Japkowicz and S. Stephen, “The class imbalance problem: a systematic study,” Intelligent Data Analysis, vol. 6, no. 5, pp. 429–449, 2002.

L. Chui and B. Pike, Auditors’ responsibility for fraud detection: New wine in old bottles? Accounting Faculty Publications, 2013.

C. Lin, A. Chiu, S. Y. Huang, and D. C. Yen, “Detecting the financial statement fraud: The analysis of the differences between data mining techniques and experts’ judgments,” Knowledge-Based Systems, vol. 89, pp. 459–470, 2015.

D. Zhang and L. Zhou, “Discovering golden nuggets: Data mining in financial application,” IEEE Transactions on Systems, Man, and Cybernetics, Part C: Applications and Reviews, vol. 34, no. 4, pp. 513–522, 2004.

M. J. Nigrini and L. J. Mittermaier, “The use of Benfords Law as an aid in analytical procedures,” Auditing: A Journal of Practice and & Theory, vol. 16, pp. 52–67, 1997.

M. Markou and S. Singh, “Novelty detection: a review—part 1: statistical approaches,” Signal Processing, vol. 83, no. 12, pp. 2481–2497, 2003.

M. Markou and S. Singh, “Novelty detection: a review—part 2: neural network based approaches,” Signal Processing, vol. 83, no. 12, pp. 2499–2521, 2003.

C. Phua, V. Lee, K. Smith, and R. Gayler, “A comprehensive survey of data mining-based fraud detection research,” Artificial Intelligence Review, pp. 1–14, 2005.

A. Patcha and J.-M. Park, “An overview of anomaly detection techniques: existing solutions and latest technological trends,” Computer Networks, vol. 51, no. 12, pp. 3448–3470, 2007.

V. J. Hodge and J. Austin, “A survey of outlier detection methodologies,” Artificial Intelligence Review, vol. 22, no. 2, pp. 85–126, 2004.

V. Chandola, A. Banerjee, and V. Kumar, “Anomaly detection: a survey,” ACM Computing Surveys, vol. 41, pp. 15.1–15.58, 2009.

J. Wu, H. Xiong, and J. Chen, “COG: local decomposition for rare class analysis,” Data Mining and Knowledge Discovery, vol. 20, no. 2, pp. 191–220, 2010.

A. S. Sabau, “Survey of clustering based financial fraud detection research,” Informatica Economica, vol. 16, no. 2, pp. 116–122, 2012.

W. Zhou and G. J. D. S. S. Kapoor, “Detecting evolutionary financial statement fraud,” Decision Support Systems, vol. 50, no. 3, pp. 570–575, 2011.

A. Sharma and P. K. Panigrahi, “A review of financial accounting fraud detection based on data mining techniques,” 2013.

M. Cecchini, H. Aytug, G. J. Koehler, and P. Pathak, “Detecting management fraud in public companies,” Management Science, vol. 56, no. 7, pp. 1146–1160, 2010.

H. Chen and M. C. Roco, “Mapping nanotechnology knowledge via literature database: a longitudinal study,” in Mapping Nanotechnology Innovations and Knowledge: Global and Longitudinal Patent and Literature Analysis, pp. 1976–2004, Springer, Boston, MA, USA, 2009.

C. E. Hogan, Z. Rezaee, R. A. Riley, and U. K. Velury, “Financial statement fraud: Insights from the academic literature,” Auditing A Journal of Practice & Theory , vol. 27, no. 2, pp. 231–252, 2008.

S. Han, Z. Rezaee, L. Xue, and J. H. Zhang, “The association between information technology investments and audit risk,” Journal of Information Systems, vol. 30, no. 1, pp. 93–116, 2016.

A.-R. Abtahi, F. Elahi, and R. Yousefi-Zenouz, “An intelligent system for fraud detection in coin futures market's transactions of Iran mercantile exchange based on Bayesian network,” Journal of Information Technology Management, vol. 9, no. 1, pp. 1–20, 2017.

C.-L. Jan, “An effective financial statements fraud detection model for the sustainable development of financial markets: Evidence from Taiwan,” Sustainability, vol. 10, p. 513, 2018.

J. Yao, J. Zhang, and L. Wang, “A financial statement fraud detection model based on hybrid data mining methods,” in Proceedings of the International Conference on Artificial Intelligence and Big Data (ICAIBD), pp. 57–61, China, 2018.

P. Hajek and R. Henriques, “Mining corporate annual reports for intelligent detection of financial statement fraud–A comparative study of machine learning methods,” vol. 128, pp. 139-152, 2017.

D. E. Rumelhart, G. E. Hinton, and R. J. Williams, Learning Internal Representations by Error Propagation, Cambridge: MIT Press, 1986.

D. F. Specht, “Probabilistic neural networks,” Neural Networks, vol. 3, no. 1, pp. 109–118, 1990.

V. N. Vapnik, Statistical Learning Theory, John Wiley and Sons., New York, NY, USA, 1998.

W. H. Greene, Econometric Analysis, Pearson Education, Boston, Massachusetts, 7th edition, 2012.

P. A. Lachenbruch, Discriminant analysis, Hafner, New York, NY, USA, 1975.

L. Torgo, Data Mining with R: Learning with Case Studies, 2nd edition, 2016.

T. Saito and M. Rehmsmeier, “The precision-recall plot is more informative than the ROC plot when evaluating binary classifiers on imbalanced datasets,” PLoS ONE, vol. 10, no. 3, Article ID e0118432, 2015.