Analysis of the Application of Feedback Filtering and Seq2Seq Model in English Grammar

Natural language processing (NLP) technology is widely used in grammatical error correction, but its error correction logic is complex and fault-tolerant, which leads to low accuracy. With the progress of deep learning and big data analysis technology, a new method is proposed in the technical means of English grammar error correction. This paper proposes a deep learning model-based feedback grammar error correction method, which can e ﬀ ectively improve the accuracy and tolerance of grammar error correction. Firstly, the Seq2Seq model with attention mechanism is proposed, and then the feedback ﬁ ltering model is integrated, so that the existing errors or ine ﬃ cient grammars can be corrected again, thus improving the e ﬃ ciency of the model. Through a large number of text detection, the model proposed in this paper has high execution e ﬃ ciency and application ability and can widely meet the needs of English translation and grammar correction.


Introduction
As an international language, English plays a vital role in international communication. With the economic globalization and the increasing frequency of international exchanges, the demand for talents who are proficient in English is gradually increasing, which makes many non-English-speaking countries pay more and more attention to English teaching and education. In China, English is a compulsory language, which runs through all stages of education in primary school, junior high school, senior high school, and university. However, due to the lack and uneven distribution of English teaching resources, the feedback period of English teaching effect is long, and the level of English learners cannot be improved rapidly. To solve this problem, relevant experts and scholars have applied computer technology to assist English learning and studied and designed many Englishassisted learning systems. At present, English-assisted learning systems are mainly English grammar error correction systems, such as Literature [1,2]. Based on English corpus and other resources and computer technology, through model training, English grammar can be quickly corrected, and the feedback cycle of English education and teaching can be shortened. However, although these systems provide great convenience for English education and teaching, there are some problems such as low error correction accuracy and limited error correction ability. The reason is that most models based on English corpus resources depend on the quality of corpus resources, and the model training is complicated. Therefore, through the above analysis and research, combined with the characteristics and advantages of indepth learning, an English grammar error correction and feedback method based on Seq2Seq model is proposed to improve the error correction ability and effect of the system and meet the actual needs of English-assisted learning.

Seq2Seq
Model and Its Improvement 2.1. Basic Introduction of Seq2Seq Model. Seq2Seq model is a variant cyclic neural network for natural language processing, including encoder and decoder. Among them, the encoder is a cyclic neural network, which is mainly responsible for compressing the input sequence into vectors of specified length; the decoder is also usually a cyclic neural network, which is mainly responsible for generating the specified sequence [3,4] according to the semantic vector.
The Seq2Seq model can learn the probability distribution and make predictions. The specific mathematical description is as follows.
If the input sequence is x = jx 1 , x 2 , ⋯ | and the conversion rule is f , the hidden state HT at the current time can be expressed as where h t−1 represents the hiding state at the previous time.
According to the hidden state of each moment in Equation (1), all hidden states are summarized to obtain semantic vector C.
The decoding stage can be understood as the inverse process of encoding the above codes. According to C and the generated output sequence y 0 , y 1 , ⋯y t−1 , the next output word y t is predicted.
There are also calculation rules in the decoding process. h t ′ is used to represent the hidden state in the process, including 2.2. Improvement of Seq2Seq Model. The research shows that although Seq2Seq model is similar to machine translation algorithm, its translation efficiency and error correction ability need to be further improved. Therefore, this study improves the model from three aspects: attention mechanism, addition layer standardization, and word embedding processing.

Attention Mechanism.
Attention mechanism can be divided into local attention mechanism and global attention mechanism according to the calculation range of input sequence [5]. In this study, the problem of English grammar error correction needs to be dealt with according to the full text, so the global attention mechanism combined with Seq2Seq model is selected to deal with the problem of English grammar error correction. The specific operation process can be illustrated in Figure 1.
As can be seen from the figure, the value of the weight a ij can be determined by the i − 1 hidden state s i−1 and each hidden state variable in the input, and its calculation formula is as follows:  (7) and (8). After layer normalization, the performance of cyclic neural network is better than that of original network.
2.2.3. Word Embedding Processing. Since the computer cannot directly process natural language, it is necessary to vectorize the input English text to convert it into data that can be directly processed by the computer [6]. In this study, Continuous Bag of Words Model (CBOW) is used to vectorize the English text of the input model. Its basic transformation idea is to calculate the probability of a word appearing according to the first N times or the last N words of a word, as shown in Figure 2.

Feedback Filtering Algorithm Model Based on N-Gram
Considering that the correction results of grammatical error correction model may not be correct or achieve ideal error correction effect, especially for grammatical errors of complex sentences, manual error correction ability is stronger. Therefore, in order to improve the error correction ability of the error correction model, we should consider accepting the modification suggestions submitted by users. However, due to the uneven English level of users, the model Figure 1: Global attention mechanism model.

2
Wireless Communications and Mobile Computing needs to screen the user's suggestions before accepting the user's modification suggestions. The simplest screening method is to compare which of the system and the user's modification suggestions is more credible, that is, to score the modified sentences. The sentences with higher scores are considered to be sentences with higher probability without language disorders, while the sentences with lower scores are not adopted. In this study, n-gram model, a basic language model used in the field of natural language processing, is selected to score sentences. The mathematical description of the model is as follows. Suppose S is a string representing a statement; PðsÞ = P ðω 1 , ω 2 , ⋯, ω m Þ denotes the occurrence probability of a sentence composed of m words. First, N-gram model is used to model S. Then, according to the chain rule, such as Equation (9), the probability of S is calculated.
According to the formula, the probability of the i-th word ω i is related to the ω 1 ω 2 ⋯ ω i−1 word before it, and there is a dependency relationship between words, such as ω i depends on ω i−1 before it. However, considering the large calculation of this formula, the calculation is difficult, and the parameters of n-gram model are difficult to obtain. Therefore, the study introduces Markov hypothesis, that is, the current word is only related to the previous N words, then, Equation (9) can be simplified to Usually, n = 1, 2, 3. When n = 2, the calculation process is When n = 3, the calculation process is Second, the maximum value of Equation (9) can be taken according to the maximum likelihood estimation method. In the process of corpus training, the estimation can be made through the following equation: According to the trained n-gram model and the interface provided by SRILM, the sentence probability can be calculated simply.
Finally, by calculating the confusion degree of the sentence, the sentence can be scored. The calculation formula of confusion is as follows: According to the chain rule, the above formula is rewritten to obtain: As can be seen from the above formula, the smaller the degree of confusion, the greater the probability of sentences.
By comparing confusion degree and probability, it can be seen that confusion degree is more accurate in evaluating the model. The reason is that probability calculation is closely related to sentence length and word number, which affects the accuracy of model evaluation. Confusion degree can effectively reduce the influence of sentence length and word number on model evaluation, so its evaluation result is more accurate. Therefore, this study chooses confusion degree to judge the model.   Table 1, including sentence document ID, female tag, and sentence paragraph index ID. In addition, in order to facilitate the evaluation of the error correction results of this model, the data set was simply processed before the experiment. First, according to the annotation format of data set, it is transformed into m2 format to obtain the original English sentence and its grammatical error type and specific location; then write a script to process the transformed data, and finally get a statement without syntax errors.

Evaluation
Index. In this study, F 0:5 [9] is used as an index to evaluate the effect of the model, which is used to measure the effect of English grammar error correction. Its calculation method is as follows: Equation (9).
where R represents the correction rate and P represents the correction rate; they can be calculated by the following equations, respectively.  Figure 3. As can be seen from Figure 3 is language model and CAMB model training results. Although the recall rate is slightly lower than that of the comparison model, the overall F 0:5 value is higher than that of the comparison model. Therefore, the English grammar error correction model proposed in this study has good error correction effect and obvious advantages. At the same time, in order to verify the above English grammar error correction ability, some English sentences are selected for example testing. First, the English sentences to be corrected are divided into experimental group and control group. Among them, the experimental group is a sentence containing English grammatical errors, which is used to check whether the system can correct the sentence correctly; the control group is English grammati-cally correct sentences, which are used to check whether the system can correct errors. Figure 4 is the grammar correction result of the experimental group system, and Figure 5 is the error correction result of the control group system. As can be seen from Figure 4, there are many English grammatical errors in case 1 and case 2. After systematic error correction, the grammatical errors in case 1 have been corrected, but the tense error of "realized" in case 2 has not been corrected. Therefore, the error correction effect of this system model is good, but there is still room for improvement. As can be seen from Figure 5, the system does not correct the correct English sentences, which shows that the system model has good recognition ability for English grammatically correct sentences. Generally speaking, the error correction model of this system has high error correction accuracy and can be used for actual English grammar error correction.

Experiment and Result Analysis of Feedback Filtering
Algorithm. In order to verify the effectiveness of the feedback filtering algorithm in this study, this algorithm is used to train grammatically correct texts in NUCLE-release2.2 corpus. First, configure srilm in the Linux system, and execute the command n-gram-count-text train_data. txt-order3-lm model.lm to get the trained model file model.lm. Then, the trained model is used to calculate the sentence probability value and confusion degree to realize feedback filtering. Usually, the probability value of syntactically incorrect statements is lower, while the probability value of syntactically correct statements is higher. Therefore, the probability value of the corrected grammatical error statement will be improved to a certain extent. Take the following statement as an example.
First, security systems have improved in many areas such as on the school campus or at the workplace.
User modifications: first, security systems have improved in many areas such as school campus or at the workplace.
Calculation result is in Figure 6. From the above calculation results, it can be seen that the sentence confusion degree of the user and the system modification result is 171.859 and 134.366, respectively, that is, if the confusion degree of the modification result is lower, the suggestion will not be adopted. In addition, according to the grammar knowledge, the system modification method is better, so the modification proposal of the model is to perform filtering, which achieves the expected purpose.

Framework of English Grammar Error Correction
System. In order to verify the feasibility of the proposed grammar error correction again, a simple and practical English error correction website is constructed to assist English learning. The specific framework is shown in Figure 7. In this system, users can log in through account number and password and use the grammar error correction function of the system. After the user logs in to the system, input the English text to be corrected, and the system 4 Wireless Communications and Mobile Computing corrects grammar errors according to the trained grammar error correction model, and returns the results to the user.

Error Correction Algorithm
Training. According to the system flow, the training flow is designed as shown in Figure 8. When the system receives the English grammar error correction request, it first obtains the training data from the original corpus and preprocesses the corpus after the corpus reaches the set threshold. Then, initialize the model and start training. Finally, the error correction effect of the model is evaluated according to the training results, and if the error correction effect of the model is improved, the grammar error correction model is updated; on the contrary, the algorithm is directly terminated. In the whole

Case 1
Original sentence: erefore, the equipment of biometric identification tend to be in-expensive.
System error correction: erefore, the equipment of biometric identification tends to be inexpensive.

Case 2
Original sentence: in addition, we realized that burning of fuels produce the large amount of greenhouse gases.
System error correction: In addition, we realized that the burning of fuels produces a large amount of greenhouse gases.

Wireless Communications and Mobile Computing
process, if any link is abnormal, the system will send a training exception notification to the administrator, waiting for the system to repair.
"Original corpus" refers to the database that has been trained and can only be used as an input variable to provide rule reference for other models.

Implementation Process of English Grammar Error
Correction. English grammar error correction module is the core functional module of this system. When the module interface receives a grammar error correction request, it must first verify whether the parameters are legal, and if the parameters are legal, it will continue to perform grammar error correction. Then judging whether there are multiple statements in the input statement, and if it is only one statement, directly using the error correction model to correct errors; if there are multiple sentences, the error correction model should be used to correct the errors after the sentences are broken. Finally, the syntax correction can be ended by outputting the error correction result. The whole process is shown in Figure 9.
"Is the parameter legal" represents whether the entered parameter is valid, if so proceed to the next step, otherwise, end the program. "Multiple Statements" determines whether Case 1 Original sentence : our current population is 6 billion people and it is still growing exponentally.
System error correction: our current populationis 6 billion people and it is still growing exponentially.

Case 2
Original sentence : such as, global surface temperature has risen by 0.74 degree celsius in past century.
System error correction: such as, global surface temperature has risen by 0.74 degree celsius in the past century.

System Development Environment.
The front-end and back-end of this research system are developed based on react and Python languages, respectively. The related technologies used in the front end mainly include WebPack, React-Router, and Babel, while the related technologies used in the back end mainly include Django. MySQL is selected as the core database for storing data in this system, and Redis is selected as the cache database in some occasions of this system.

Error Correction
Effect. The user logs in to the system to enter the grammar error correction interface, as shown in Figure 10. According to the prompt, enter the statement and submit it. Click the corresponding button to view the error correction result, modification suggestions, etc.

Conclusion
To sum up, the English grammar error correction based on Seq2Seq constructed in this study realizes the effective extraction of text information and improves the effect of English grammar error correction. Compared with the hierarchical language model, the training results of CAMB model show that this model has better error correction effect and higher error correction accuracy. At the same time, through feedback filtering, the system can modify the user's statement, thus improving the error correction ability. Therefore, this training model can be used to correct English grammar errors, and then assist users to learn English. However, the research work of this paper still has the problem that the error correction accuracy of complex sentences is reduced, and the complexity of the model is high. Further application of the model needs to be further improved, and more models need to be combined. The next research work needs to analyze the complexity of the model and adopt a better method to achieve it. More models are needed to verify the error correction ability of the method in this paper, and a better method is used to achieve efficient error correction efficiency.

Data Availability
The experimental data used to support the findings of this study are available from the corresponding author upon request.