Adaptive Control of Artificial Pancreas Systems-A Review

Artificial pancreas (AP) systems offer an important improvement in regulating blood glucose concentration for patients with type 1 diabetes, compared to current approaches. AP consists of sensors, control algorithms and an insulin pump. Different AP control algorithms such as proportional-integral-derivative, model-predictive control, adaptive control, and fuzzy logic control have been investigated in simulation and clinical studies in the past three decades. The variability over time and complexity of the dynamics of blood glucose concentration, unsteady disturbances such as meals, time-varying delays on measurements and insulin infusion, and noisy data from sensors create a challenging system to AP. Adaptive control is a powerful control technique that can deal with such challenges. In this paper, a review of adaptive control techniques for blood glucose regulation with an AP system is presented. The investigations and advances in technology produced impressive results, but there is still a need for a reliable AP system that is both commercially viable and appealing to patients with type 1 diabetes.


INTRODUCTION
The development of artificial pancreas (AP) systems to regulate the blood glucose concentration (BGC) in patients with type 1 diabetes (T1D) has been in progress since 1960s [1].Currently, patients with T1D have to administer 3-5 insulin injections daily or infuse from pumps basal insulin and boluses before meals in order to regulate their BGC.AP will automate and coordinate frequent glucose concentration measurements, estimation of BGC, computation of the appropriate dose of insulin to be infused and adjustment of the pump flow rate to dispense the insulin.An AP consists of a sensor that measures a patient's glucose concentration, a controller with algorithms to estimate BGC and compute the control command transmitted to an insulin pump, and an insulin pump that infuses the computed dose of insulin to the patient (Figure 1).
Many model-based controller design strategies use detailed models that describe the dynamic behavior of a system.However, a large number of systems such as biological systems are too complex to fully understand or to develop detailed models that describe their dynamic behavior.Control design for such systems can rely on input-output models developed by using system identification techniques.Adaptive control systems based on data-based models can function successfully when unpredictable systemic changes and external disturbances affect the system by quickly adjusting the controller parameters [23] without any need for the knowledge about the initial parameters or conditions of the system [24].Some adaptive control systems embed the model in the controller and adjust controller parameters as the behavior of the system changes; selftuning regulators belong to that group.Other adaptive control systems update the parameters of the model recursively as new data are collected from the system, and use the latest model in the controller.Generalized predictive control (GPC) and linear quadratic Gaussian (LQG) control systems are examples of these controllers.
The remainder of this paper is structured as follows.Section 2 introduces the minimum variance control, self-tuning control, linear quadratic control, generalized predictive control and other adaptive control techniques, and reviews their use in AP systems.The review covers AP applications that use insulin as the manipulating variable and also applications that use glucose or insulin + glucagon or dextrose as the second manipulating variable.The section ends with a table of various adaptive control techniques used in AP systems and a summary of their performance.The discussion section provides an assessment of various adaptive control applications for the AP system.Conclusions are provided in the last section of the paper.

ADAPTIVE CONTROL AND AP APPLICATIONS
An adaptive controller is a controller that can modify itself in response to changes in the characteristics and dynamics of the system being controlled (Figure 2).For timevarying and nonlinear systems, responses to the same disturbances will yield outputs of different magnitudes and characteristics depending on the current state of the system.Hence, adaptive control is a promising approach to regulating such systems.

Figure 1.
Artificial Pancreas Most adaptive control techniques are designed by using linear models that are usually represented in the form of time series models: (1) where y(k) is the system output, u i (k) is the i th input variable at k th sampling time, ∈(k) is the white noise at k th sampling time, d i is the delay term for the corresponding input.The polynomials A(q −1 ), B(q −1 ) and C(q −1 ) are defined below: (2) where q -1 is the backward shift operator, a i , b i , c i , are the unknown model parameters that are estimated from measurements, and n A , n B i , n c are model orders.The model in Eqn. ( 1) is a general multi-input linear model representation called multivariable autoregressive moving average model with exogenous inputs (ARMAX).Various adaptive controller designs will be presented in different forms of Eqn. 1.

Minimum Variance Control
The minimum variance (MV) controller (Appendix A) was proposed by Astrom [25] and is discussed in detail for various applications [26].The MV controller has several appealing properties: (a) it considerably decreases the deviations of the controlled signal around its desired value; (b) it takes into consideration disturbances that occur in practical situations; (c) it is very simple and does not need high computational resources; (d) it can follow the changes of the system parameters; (e) it requires very The first experimental trials of artificial pancreas with predictive MV controller was performed by Pagurek et al. [27].Glucose infusion rate was used as the only manipulated variable to regulate the fasting glucose concentration via intravenous infusion.Glucose solution was infused by a systolic pump into vein in one of the subject's arms while a small quantity of blood was withdrawn continuously from the other arm and monitored with an auto-analyzer for glucose concentration.The secondorder linear model investigated by Ackerman et al. [28] was converted to the linear model of Eqn. 1 by Pagurek et al. [27] and used in controller design.An extended Kalman filter with least squares approach was used for the estimation of model parameters.Five hours of simulation and 167 minutes of clinical experiment on a nondiabetic human were performed with a sampling time of 25 seconds.A successful regulation performance with very small variations from the set point was achieved for each experiment.
Fischer et al. [29] hypothesized that only an adaptive algorithm would guarantee the optimal feedback control of BGC in T1D patients.Fischer and colleagues [29] designed an MV controller with the model of Eqn. 1.They excluded the disturbance term and used insulin infusion as the controller output.The unknown parameters of the model were identified in real time with recursive least squares (RLS) parameter estimation method [30].Thus, the effect of the initial uncertainties on parameter values was eliminated.Three different control strategies: adaptive control, fixed command control using on-line parameter estimation, and fixed command control using off-line parameter estimation were performed on seven diabetic dogs for 56 (±12) days.Fischer et al. [29] concluded that the difference between the performance of adaptive control and that of fixed command control using on-line estimation is not very significant; however, the results are much better compared to algorithms that use an off-line estimation method.The parameters of the fixed commands must be determined individually to fit each patient's BGC dynamics.The actions of the MV controller are very aggressive since the controller has a very ambitious objective of eliminating the variance of future outputs as soon as possible (immediately after the system delay).The method is also referred to as bangbang control because the control signals would switch frequently between their highest and lowest values.

Self-Tuning Regulator
Self-tuning controller (regulator) developed by Astrom and Wittenmark [31] is the extension of the minimum variance control (Appendix B).Since self-tuning controller is derived from MV controller, it satisfies most of the properties of MV and further improves identifiers of unknown parameters and control algorithm.A combination of a self-tuning controller and an error feedback controller for regulating of BGC was proposed by Sano et al. [32].They used the multi-input (insulin infusion and glucose infusion) version of the model of Eqn. 1 without the disturbance term.The stability of the closed-loop system was guaranteed by designing the self-tuning controller based on pole placement approach [33].The error feedback controller was used to accelerate the convergence speed of the output error between desired glucose level and system output by shifting pole locations.Five virtual patients with different parameters were created based on the model proposed by Yipintsoi et al. [34] and used for the simulation studies.Clinical experiments performed on diabetic dogs showed almost perfect reference tracking for 5 hours of fasting glucose periods.
Sarti et al. [35] performed the self-tuning control algorithm on 300 simulations with different sampling times (1 to 5 minutes).The results were compared with proportionalderivative (PD) algorithm of Biostator [36], and more realistic insulin concentration profiles were obtained with self-tuning controller.Domingo et al. [37] developed a selftuning pole assignment controller with a modified RLS parameter estimation method where lower control cost was obtained for real time glycemic control.Brunetti et al. [38] investigated the effect of the weight factor parameter used in the objective function of self-tuning algorithm on computer simulations.This parameter provides a balance between the amount of insulin infusion and hyperglycemia; i. e. , the larger the parameter, the lower the insulin suggested by the controller.Goh et al. [39] pointed out that the adjustability of adaptive control strategies is stronger compared to PID and MPC algorithms, as demonstrated by simulations of the pole-placement-based self-tuning controller with Glucosim simulator [40].
In daily life, many conditions such as meal consumption, exercise, emotional fluctuation cause large changes in BGC.Identification methods usually are not able to track large rapid changes.To provide a quicker response for such changes, Eren-Oruklu et al. [41] developed a change detection method to trigger the estimation of unknown model parameters with RLS for a self-tuning controller.The forgetting factor in RLS method was decreased to a small value (0. 005) when change was detected.Thus, earlier observations were discarded and the model was updated by using more recent measurements.When there were no large changes in the metabolism, the forgetting factor was reset to a large value (0. 4).Turksoy et al. [42] proposed an adaptive method to change the forgetting factor based on the modeling error where the forgetting factor varies in a range defined by a minimum and a maximum values adaptively without any predefinition of thresholds to trigger the adjustment of the forgetting factor.
Astrom and Wittenmark [31] emphasize two basic conditions for the design of selftuning regulators: polynomial B in Eqn.(3) has all it poles inside unit circle, and the first coefficient of the polynomial B is known.Most studies [32,35,[37][38][39] discussed have not checked these two conditions.Eren-Oruklu et al. [41] checked if these two conditions are satisfied at every sampling time and if not, the parameters computed at the previous sampling time were used in the controller.Turksoy et al. [42] converted the regular RLS method to a constrained optimization problem that guarantees the two conditions of Astrom and Wittenmark [31].

Linear Quadratic Control
Linear quadratic control (Appendix C) has been employed mostly for uncertain linear systems with additive white noise that may have incomplete state information.Linear quadratic Gaussian (LQG) control method is the combination of linear state estimation and optimal control based on linear state estimate feedback [43,44].The LQG algorithm can be implemented by selecting some design matrices that are used for desired closed-loop performance, solving matrix design equations, and obtaining a guaranteed solution that stabilizes the system.It provides little insight into the robustness of the control system.An LQG based closed-loop control for T1D [45] adopted a reduced version of the oral glucose "meal" model of Dalla Man et al. [46] for controller design.Patek and co-workers [45] focused on an ideal case where all parameters are assumed to be known, compared LQG-based simulation results for 100 virtual patients with PID control, and concluded that LQG achieved much better regulation.An adaptive version of LQG was proposed by Eren-Oruklu et al. [47] where the unknown model parameters were recursively estimated with RLS method.The model in Eqn. 1 was updated and converted to its state space version at every sampling time to modify the controller.A time-varying reference trajectory depending on current glucose measurements was included to prevent aggressive insulin infusion.Closed-loop simulation experiments were performed with Glucosim [40] and a simulator was developed using Hovorka's model [48].The effects of different values of insulin infusion time delays were studied.

Generalized Predictive Control
Generalized predictive control (GPC) is a receding-horizon method that predicts the output of a system over several steps into the future based on assumed future control actions, and selects these future control moves by minimizing an objective function (Appendix D).The objective function includes terms that represent the deviation of future output trajectories from reference values and penalties to excessive control actions [49,50].The system's output in the future is predicted by the model of Eqn. 1.
El-Khatib et al. [51] used GPC to calculate insulin infusion and PD to calculate glucagon infusion amounts for a dual subcutaneous closed-loop AP based on venous blood glucose measurements.Two customized infusion pumps were employed for subcutaneously delivery of both insulin and glucagon.Two ear-vein catheters were established in each experiment for blood glucose sampling and administering an IV glucose drip.Overall, 11 experiments with 4 diabetic pigs were performed without any additional information such as meal counting or other feed-forward information required by other systems [27,[52][53][54].El-Khatib et al. [17] performed the dual subcutaneous closed-loop approach on 11 human subjects with T1D for 27 hours of experiments.Insulin infusion was calculated with GPC while a PD algorithm was used for the glucagon infusion calculations.To prevent hyperinsulinemia, a model of accumulation of insulin in subcutaneous tissue was proposed [17].This model was used in GPC objective function as a second output for simultaneous minimization of glucose regulation and insulin accumulation.An improvement in hypoglycemia prevention was achieved in clinical environment.The same algorithm was employed in another clinical experiment with six T1D patients [18] provided with pre-meal bolus infusions, using a CGM device for glucose sampling.
Eren-Oruklu et al. [47] used GPC algorithm with only subcutaneous glucose sensing and subcutaneous insulin infusion.Simulations were conducted to compare results with LQG controller performance under the same conditions and found GPC to have better regulation performance.Smith Predictor was used as a pre-filter to accommodate the time delay between insulin delivery and its first effect on BGC.Turksoy et al. [15] used the multi-input version of the model of Eqn. 1 to develop the GPC algorithm.In addition to insulin and glucose information, some physiological signals such as energy expenditure and galvanic skin response were included as external inputs to the model for post-exercise hypoglycemia prevention.Experiments were performed on seven patients with T1D.Insulin on board models [55] were implemented to the GPC algorithm to prevent hyperinsulinemia.
In AP applications, insulin infusion and meal consumption may occur concurrently, but regular identification methods may not be able to distinguish between the effect of meal consumption and insulin infusion.Finan et al. [56] and Lee and Bequette [57] showed that even the sign of the effect of insulin on glucose concentration can be wrong with regular identification methods.The constrained optimization algorithm that is equivalent to RLS method, proposed by Turksoy et al. [42], guarantees the prevention of such abnormalities in the identification algorithm.Another important concern in recursive modeling is the stability of the models.Even if the modeled system is stable, information from signal noise can cause the development of unstable models.Turksoy et al. [42] developed stability constraints which guaranteed that the recursive identification method always produced stable models, and tested them with successful results in three clinical experiments and 30 virtual simulations [58].Turksoy et al. [42] also proposed a model for calculating the maximum amount of insulin allowed for the controller where the controller can update the maximum insulin infusion amount adaptively by the estimated insulin on board (IOB), the reference trajectory, and the patient's body weight.

Other Adaptive Control Techniques
The model of Eqn. 1 with no exogenous input was adopted by Candas and Radziuk [59] for controller design.The linear model was obtained from non-linear insulin/glucagon kinetics.Once the unknown parameters of the linear model were obtained by RLS method, the controller output was calculated based on the non-linear model kinetics.The controller was tested on 3 pigs to maintain the glycemia at fasting level.The main contribution of the model used was to consider insulin-dependent and insulin-independent glucose removal rates separately.The plasma glucose was maintained at the target levels almost all the time during the experiments.Hovorka et al. [60] developed a non-linear predictive controller where the unknowns are estimated on-line.Several clinical experiments [61,62] were conducted and promising results were reported.Li and Tao [63] developed a feedback controller with an adaptive method to update the controller parameters.Wang et al. [64] proposed a conditional technique that used different parameters from pre-defined gain mosaic based on blood glucose and its rate of change.
As reported in previous studies, acceptable BGC regulation was achieved with different adaptive control strategies.Figure 3 shows the results of a clinical experiment where an adaptive control system without any meal and activity announcement was able to keep glucose levels within a normal range (70-180 mg/dl) [15].A summary of representative clinical and simulation studies of various adaptive control implementations for AP systems is presented in Table 1.

DISCUSSION
The regulation problem of BGC in T1D patients with automated insulin pumps has been studied during the last three decades.Various clinical experiments have been performed to investigatethe feasibility of PID, MPC, logic control and GPC techniques that demonstrated promising results under various conditions and scenarios.Even though many studies reported satisfactory results, the current AP systems face many challenges.The first challenge is related to the measurement of BGC.Intravenous measurement of BGC is permitted only in hospital environments.Otherwise, sensors  Ins.
The proposed method showed improvement on the percentage of BGC that stayed in normal range.Optimal forgetting factor was identified.
A small number of hypoglycemia and hyperglycemia episodes for adult and adolescent virtual patients.Ins.
Less hypoglycemia in adaptive control simulations comparing to nurse-implemented insulin infusion protocols.The controller could deal with highly changing conditions such as large variations in time-varying insulin sensitivity for individual patients. [65] VP: virtual patient, IV: intravenous, SC: subcutaneous, MVC: minimum variance control, STC: self-tuning control, LQC: linear quadratic control, GPC: generalized predictive control,OAC: other adaptive control,Ins: insulin,Glu: glucagon, NP: not provided with useful lifetimes of 3 to 7 days were used to measure glucose concentration in subcutaneous tissue.The sampling times of these sensors range from 5 to 1 minutes.
There is a delay estimated to be in the range of 5 to 12 minutes between the BGC and the subcutaneous reading [66].The accuracy of readings from all sensors can be affected by various factors in the daily life of the patient and the probability of deviations between BGC and sensor readings increase when the BGC is in hypoglycemia and hyperglycemia regions or changing very rapidly.The suboptimal accuracy of current CGM devices [67][68][69][70] and the delay in the action of the insulin administration caused by diffusion from subcutaneous tissue to the bloodstream are other factors that make the system even harder to control.
Although some physiological glucose-insulin dynamics models are available in the literature [46,48,[71][72][73][74], the intra-and inter-subject variability of glucose-insulin dynamics prevents development of a physiological model that fits all patients.Meals, physical activity, emotional status, insulin type are some of the reasons that affect the BGC dynamics.Some of these disturbances such as the carbohydrate content of a meal to be consumed can be predicted, while others are unknown.
The body is a dynamic and nonlinear system rendering measurement and control action delays and subjected to various large disturbances.BGC measurement always has inaccuracies and missing information.An AP is expected to function reliably under a wide range of conditions to provide good glycemic control for this challenging system.An adaptive control system offers a promising solution to overcome such problems through linear and simple models adapted with every new measurement information at a small computation cost.Chronologically, LQG and MV control techniques provided the first generation of adaptive control systems.The use of recursive models with LQG represents a significant improvement in performance.MV control is too aggressive to be used in an AP system, but it germinated self-tuning regulators that provide a realistic option.GPC is currently the most powerful adaptive control approach for AP systems, and a number ofclinical studies have demonstrated its performance in single and dual-hormone systems.
The current sensor technologies pose challenges to AP systems that are beyond the capabilities of the control algorithm.The next generation of glucose sensors is expected to address many of the sensor accuracy and reliability issues.For example, the new Dexcom G4 has higher accuracy and longer service life [75].The orthogonal redundancy approach of Medtronic may provide significant improvement in reliability by utilizing both electrochemical and optical sensors in the system and reconciling the glucose concentration values sent by theses sensors before reporting the glucose concentration at that sampling time [76].CGM devices were investigated by Mauras et al. [77], Lane et al. [78], and Keenan et al. [79,80].The rate of adoption of these technologies and interest in APs will increase as the reliability and cost effectiveness of sensors improve.
The term of "Artificial Pancreas" should refer to a system that requires no input from the user.A manual pre-meal bolus injection and systems relying on meal announcement or exercise announcement may provide acceptable glucose regulation, but such systems cannot be considered as a fully automated closed-loop AP.While the safety of T1D patients is the most important criteria, the convenience and feasibility of the system must also be considered.Adaptive control of AP provides convenience and fully automated operation without announcements from patients.Steil et al. [2] proposed PID algorithm without announcement from patients as well.Weinzimer et al. [81] investigated the effect of pre-meal blousing, and less post-prandial hyperglycemia was achieved with announcement.After implementing an insulin feedback algorithm into PID algorithm, more experiments without meal announcements were performed [6,82].However, adaptive control (GPC) can respond to post-prandial glucose increase faster since the optimum amount of insulin is not calculated based on the last glucose measurement but based on a window of past measurement and future predictions.

CONCLUSIONS
Closed-loop BGC regulation with AP systems is a challenge that was addressed over the last 30 years.Different methodologies with various types of devices have been investigated to develop a fully automated AP system.The investigations and advances in technology produced impressive results, but there is still a need for a reliable AP system that is both commercially viable and appealing to patients with T1D.The subject-specific characteristics of blood glucose dynamics, unpredictability of disturbances, and variations in the BGC characteristics of patients with time make adaptive techniques a strong candidate for the control of AP systems.
where N 1 and N 2 are the first and last time instants of the modeling horizon, and N u is the control horizon.r(k) is a set-point and w(j) is a weight for penalizing the control input.yˆ(k+j) represents j steps ahead prediction of process output, and ∆ is the difference operator such that: (20) Predicted values are calculated using the following Diophantine equation: The future values of the system output can be calculated as: (24) The control signal in Eqn.24 is separated into past (u(k-1)) and future (u(k+j)) control signals and the predictions: (25) where H j (q −1 ) and Γ j (q −1 ) are obtained from a second Diophantine equation: (26) The vector of the future values of the system output can be calculated as: E q B q H q C q q q ( ) j j j j E q e e q e q j j j jj j 1 ,0 F q f f q f q j j j j n C q E q A q q F q j j j where the coefficients H j are the elements of matrix G and Γ j are rows of vector G′ ′ (boldface represents vector or matrix notation).The unconstrained optimal solution is the following: where I is the identity matrix and only the first element of the vector u is implemented inthe system.
Adaptive Control of Artificial Pancreas Systems -A Review

Figure 3 .
Figure 3. (a) Continuous glucose monitor readings (CGM), plasma or capillary glucose measurement using the Yellow Springs Instrument (YSI) or finger stick meter and (b) infused insulin (ins).BF, breakfast; L, lunch; D, dinner; S, snack; CHO, carbohydrate.The grey band indicates the normal range of BGC.The black band section indicates the closed-loop experiment period.Physiological activity information: (c) Energy expenditure (EE), and (d) Galvanic skin response (GSR).
Adaptive Control Block diagram little a priori knowledge such as time delay or the model order of the process.But its use is limited because it is very aggressive and can lead to large swings in manipulated variable values.

Table 1 . Summary of representative clinical and simulation experiments with various adaptive control techniques Clinical Number Sensing and/or Subject of and Control Sampling Closed-Loop Control Simulation Type Subjects Infusion Algorithm
.