Specific growth rate observer for the growing phase of a Polyhydroxybutyrate production process

This paper focuses on the specific growth rate estimation problem in a Polyhydroxybutyrate bioplastic production process by industrial fermentation. The kinetics of the process are unknown and there are uncertainties in the model parameters and inputs. During the first hours of the growth phase of the process, biomass concentration can be measured online by an optical density sensor, but as cell density increases this method becomes ineffective and biomass measurement is lost. An asymptotic observer is developed to estimate the growth rate for the case without biomass measurement based on corrections made by a pH control loop. Furthermore, an exponential observer based on the biomass measurement is developed to estimate the growth rate during the first hours, which gives the initial condition to the asymptotic observer. Error bounds and robustness to uncertainties in the models and in the inputs are found. The estimation is independent of the kinetic models of the microorganism. The characteristic features of the observer are illustrated by numerical simulations and validated by experimental results.


Introduction
Plastic materials have become an indispensable component of modern industry and society, being widely used in different application areas.As an alternative to synthetic plastics, research on new materials such as biopolymers and new fabrication methods is being done [20].One of these research lines aims to use microorganisms to produce easily degradable materials, requiring less energy for its production and generating less waste and pollution.Particularly, research is being done in the field of polyhydroxyalkanoates (also known as PHA), which are polyesters that can be produced by industrial fermentation.Polyhydroxybutyrate (PHB) is a PHA that can be produced by bacterias such as Cupriavidus necator (previously known as Alcaligenes eutrophus and Ralstonia eutropha) and is fully biodegradable and biocompatible.This kind of material offers attractive characteristics for thermoprocessing applications [4,26].
Production of PHB can be accomplished in a two-phase fed-batch process utilizing the microorganism Cupriavidus necator [22,24], which has specific growth and PHB production rates inhibited by the excess of nitrogen and carbon sources [11,17,24].The first phase of the process, or growth phase, involves producing a large amount of cells, while the second phase, or production phase, involves generating a large amount of PHB while keeping the total amount of cells in the bioreactor constant.Since PHB production is heavily inhibited by nitrogen, the PHB production phase is done under nitrogen starvation conditions, which at the same time prevent cell proliferation.The main goal of each phase is achieving the highest productivity, which means maximizing the growth and production rate, respectively.
Although at a laboratory scale, the optimal pure substrate concentrations can be identified from experimental data, uncertainties in the models and variability over time may lead to suboptimal operation and, moreover, to process instability.A solution to this problem is the implementation of growth and production rate closed-loop controllers.However, this requires online information of the process states, particularly about the growth and production rates.As these variables cannot be measured by any existing sensor, software sensors must be designed to estimate them from the measurement of other variables.
Previous studies on PHB production have proposed state observers to estimate some variables of the process and to adjust kinetic parameters online.Many works propose observers to estimate biomass, substrate and PHB concentration based on online measurement of some substrate concentration [7,13,25].Other works use biomass online measurements to obtain product and substrate concentrations and growth rate estimations [6].In [19] lactate and glucose consumption rates are estimated by measuring their concentrations.In [14] an extended Kalman filter is used to estimate many process concentrations based on the measurement of influent and outgoing oxygen and carbon dioxide, dissolved oxygen and cell concentration.A similar approach is taken in [2] with asymptotic observers.Despite the many attractive characteristics of these observers, such as tunable convergence speed or high noise rejection, its applicability requires instruments which are not usually available in a typical laboratory.Specific and expensive sensors are required and even if available at laboratory scale they are not economically viable at an industrial scale.
The objective of this work is to develop an unknown input observer to estimate the specific growth rate in the growth phase of the PHB production process using instruments available in a standard laboratory.A twoobserver switched scheme, specifically designed to be used in the growth phase of PHB processes with high cell densities is proposed.For the first hours of the process, when cell density is low and can be measured, an exponential observer is used to obtain an accurate growth rate estimation in a short time.Then, when cell density has grown to a level where its measure is no longer available, an asymptotic observer is used, initialized with the last estimations made by the exponential observer.The asymptotic observer is based on the nitrogen measurement obtained from a pH control loop.Introducing the asymptotic observer in the scheme provides great robustness against uncertainties in the feeding inputs, yield parameters and nitrogen concentration, as well as independence from specific rate models; all achieved without the use of biomass measurement, which makes it suitable for high cell density cultures.

Mathematical model
The dynamical model for Cupriavidus necator growth and PHB production can be obtained from mass balances as shown in Eqs. ( 1)-( 4) [21].The nomenclature for the model is depicted in Table 1.
Since PHB is an intracellular product, the total biomass is composed both of PHB and active biomass.In this work, the term active biomass refers to everything in the cells that is not PHB (organelles, membrane, cytoplasm), which is sometimes referred to as residual biomass.In the mathematical model ( 1)-( 4), PHB concentration and active biomass are considered as two separate and independent variables [17,18,21] The state variables of the process are active or residual biomass, carbon source, nitrogen source and PHB concentrations, X, S, N and P, respectively.The yields and the feeding carbon and nitrogen concentrations are assumed to be constant.The control inputs to the system are the carbon and nitrogen feeding dilutions defined as D s ¼ F s V and , where F s and F n are the carbon source and nitrogen source input flow rates, and V is the liquid medium volume.The specific growth rates l xs and l xp account for growth based on the consumption of the feeding carbon source (glucose or glycerol) and PHB, respectively; their sum l x is the total growth rate.The rate l ps is the PHB specific production rate based on feeding carbon source consumption (which is the only production rate since it is a nongrowth associated production).The values of these rates depend on the process concentrations.Several works have been carried out to model their relation: it has been proved that they follow Haldane-like kinetics [28] with an inhibiting effect due to both carbon and nitrogen excess [17,24].Other factors can be added to consider the maximal biomass concentration [23] and maximal PHB content per cell [11,17].

Available measurements and control
The fed-batch experiments displayed in this work were performed at the Flemish Institute for Technological Research (VITO), Belgium.The laboratory setup includes a 3-l bioreactor (Applikon Biotechnology, the Netherlands), an EZ-control system (Applikon Biotechnology, the Netherlands) for data acquisition and online monitoring and control of the measured variables.
The variables measured online are pH (AppliSens, The Netherlands, model Z001023551) and optical density OD (Optek-Danulat GmbH, Germany, model ASD19-N-EB-01).The biomass measurement given by the OD sensor is valid only until the 15th to 20th hour of the process, when the sensor output saturates due to the high biomass concentration (OD = 0.6 AU approximately).Temperature and dissolved oxygen concentration measurements are also available.The dissolved oxygen concentration level was regulated at 55 % of air saturation and temperature was kept at 30 C.
The measured bioreactor inputs are the carbon and nitrogen source flow rates which, after numerical integration to calculate the volume, are used to determine the dilution rates.Stirrer speed and influent air flow rate are also measured, while outgoing gaseous oxygen and carbon dioxide flow rates and composition are not.
A controller has been implemented in the bioreactor to regulate pH at 6.8 [22].Since hydrogen is released into the liquid medium (making it more acidic) as cells multiply, a base ammonium hydroxide (20 %NH 4 OH) solution is then dosed to increase pH and compensate for the change.The base solution is also used as nitrogen source for the process (as ammonia NH þ 4 after the ammonium hydroxide combines with the protons).It has been found [22] that there is a strong linear correlation between the nitrogen pumped into the bioreactor to regulate the pH and the one that is stoichiometrically needed to produce the amount of cells accumulated.As a result, the nitrogen concentration will remain steady as long as the pH is kept the same.Under these conditions nitrogen concentration is approximately constant, as the pH controller will compensate for any change in its concentration.Based on this, an exponential feeding law has been developed [22] to regulate the carbon source concentration, and therefore the specific growth rate value.The feeding law is described in Eq. ( 5).The amount of carbon source that should be fed to the bioreactor is calculated from the amount of nitrogen that has been fed to the bioreactor, which has been used to compensate for the consumption due to growth: To include the pumped nitrogen mass losses in gaseous form, a factor g has been introduced to the model as can be seen in the last term of Eq. ( 3).The nitrogen mass losses are also accounted in the carbon feeding calculation, as can be noticed by the addition of the g factor in Eq. ( 5).The magnitude of g has been empirically determined, but it shows a considerable degree of variation from one experiment to another.

Specific growth rate observer
In this section, the two-observer switched scheme is explained.First, an exponential observer is developed for the first hours of the growth phase, when there is a viable online biomass measurement.Second, an asymptotic observer is developed for the remaining part of the growth phase, when the biomass measurement is lost.The goal of the exponential observer is to provide a fast convergence of the estimate to the real value.Then, when the biomass concentration reaches a critical value, the asymptotic observer is started with the last estimate from the exponential.The exponential observer depicted in this work is an example of how fast convergence with biomass measurement can be executed, and there are extensive published examples of this case in the bibliography [6,14].
Other approaches to the same problem include first-and second-order sliding mode observers [9,10].The main contribution of this work is the development and design of the asymptotic observer for the growth phase of high cell density PHB processes.

Motivation and background
Standard control methods of the growth rate in fed-batch reactors aim to regulate the substrate concentrations at a fixed value.Such is the case for exponential feeding or substrate feedback control.These kinds of controllers are strongly dependent on the kinetic model, resulting in high sensitivity to model uncertainties and external disturbances.In processes where high substrate concentrations inhibit growth, any deviation from the optimal substrate concentration, caused by changes in the microorganism or errors in the model, will lead the process to operate at growth rates lower than optimal [3].A solution to this problem is to directly apply feedback control on the growth rates.For instance a proportional law can be used: where DðtÞ is the dilution, D 0 ðtÞ is a predefined dilution (for example an exponential feeding), k is a design gain, l is the specific growth rate and l ref is the set-point for the growth rate. 2More complex control laws include adaptive schemes such as [6,8,27] and extremum seeking control [7,15,16].
To apply this kind of feedback control, online information of the growth rate is needed.Since this variable cannot be measured by any existing sensor, state observers must be designed to estimate it from the measurement of other variables of the process.In addition, even if feedback control is not used, the capability of estimating variables such as the specific growth rates allows for a better monitoring of the process, which is useful to prevent undesired metabolic paths, overfed bioreactors or for fault detection.
As depicted in Fig. 1, a state observer is a system which estimates state variables or unmeasured outputs of a process based on the process inputs, measured outputs and a model of the process.The basic observer structure consists of a model of the process being fed by the process inputs, which ideally would have the same outputs and state variables as the process (open loop estimation).As models have uncertainties and errors which degrade open loop estimations, it is required to feedback the estimation error, defined as the difference between the outputs and their estimations [1,5,12].The main design requirement is convergence of the estimated variables to the real ones.

Asymptotic observer
As explained before, despite the availability of an optical density sensor, the biomass measurement is not always valid due to sensor saturation when cell concentration is too high.This shortcoming is inherent to the measurement method, which loses sensitivity as biomass increases.In this section, an asymptotic observer [2] is stated for the case without biomass measurement.Even though the observer convergence cannot be made as fast as in the case of an exponential or high-gain observer, the estimation error can be bounded to its initial condition.
In the field of bioprocesses, asymptotic observers are generally used to estimate substrates, biomass and product concentrations.Their main strength is that there is no need to know the model of the specific growth rates to perform the estimations.The price is that the convergence speed cannot be manually adjusted and depends on the dilution rate.For that reason, a persistent dilution is needed to stabilize the estimate.
To formulate the proposed growth rate observer, two steps are required.First, an auxiliary variable z is defined and estimated.Then, the growth rate estimation is obtained using the estimated auxiliary variable ẑ.The auxiliary variable z is defined by the following change of variables [2,12]: This change of coordinate is a tool used to hide the kinetics of the process.It should be noticed that by definition, z is always positive.The dynamics of the new variable can be obtained as: Equation ( 9) is obtained by replacing ( 1) and ( 3) in (8).
Then, from the previous change of variables the following observer equation can be stated to estimate z: It can be observed that both the steady state values of z and ẑ are: Defining the estimation error as z ¼ z À ẑ, it can be shown that the convergence error equation of this observer is: Equation ( 12) has a single eigenvalue k ¼ ÀD and its solution is an exponential with time constant equal to D À1 .From the fact that the dilution D is always positive, it follows that the exponential is stable, which means that the error converges to zero exponentially.From this result, it can be concluded that a large dilution rate is needed for a fast convergence.
In addition, from the estimation of ẑ a biomass estimation can be obtained: This estimation can be used for monitoring, but it is not the most reliable one because it is easily affected by errors in the parameter y xn or changes in the expected nitrogen concentration.
Up to this point, an asymptotic observer has been developed to estimate the auxiliary variable z.The next step is to obtain the growth rate estimation.From Eq. ( 3), it can be seen that: Then, the growth rate l x can be calculated from ( 14) and (7): The following growth rate observer equation is, therefore, proposed: Finally, the error for the growth rate estimation can be calculated as follows: Equation (11) states that both z and ẑ have the same limit, thus it is straightforward to conclude that the limit for the growth rate estimation error in Eq. ( 17) is: As was explained in Sect.2.2, the control loop that keeps the pH constant also regulates the nitrogen concentration, which means that _ N ¼ 0 and so the error converges to zero.However, even if N is not regulated perfectly, this error will be small because the derivative _ N cannot be large (this would imply an abrupt change of N).In addition, the dividing factor z À N or X=y xn represents the amount of nitrogen that was used to produce active biomass, which increases as the process develops.
To conclude the development of the asymptotic observer, uncertainties in the parameters y xn and g are analyzed.
When including the uncertain parameters, Eqs. ( 9), ( 10) and ( 12) change to: where ŷxn and ĝ are the estimated parameters, and g is the error in the nitrogen coefficient.From this, the error of the auxiliary variable z would not converge to zero: Then, Eq. ( 16) is rewritten as: where N is an estimation of the unmeasurable nitrogen concentration; we take its value directly as the optimal nitrogen concentration, thus the estimation error is the deviation of the real concentration from the optimal value.The error for the growth rate estimation can be calculated as follows: From Eqs. ( 19) and ( 20), the limits for g z and ĝ ẑ can be obtained: Then, the limit for the error in Eq. ( 25) is: From Eq. ( 28), it can be seen that although the uncertain parameters g and y xn affect the magnitude of the error, they are not its cause.In fact, if the derivatives _ y xn and _ N are zero the error will also be zero, no matter the value of ĝ and ŷxn , demonstrating that the observer is robust against the uncertain parameters.For instance, having _ y xn 6 ¼ 0 is not a probable situation in a controlled process because it means that the yield is constantly changing, that could be the case of a gradual metabolic change, for example.In a more realistic case, the yield is assumed constant, and the only error term present would be the one that depends on _ N. It should be noticed again that as nitrogen is being regulated, its variations occur slowly ( _ N is low) and that as the process develops and biomass is produced, the variable z will increase making the error smaller.Finally, it should be noticed that the production rate l ps does not appear in any of the terms of the observers, thus if there is some PHB production in the growth phase (as indeed occurs in practice) it will not affect the estimations.

Exponential observer for the first hours
Since there is a valid biomass measurement for the first hours of the process, a classical exponential observer can be proposed [1,2,5] to estimate l x during that period and give a better initial condition to the asymptotic observer.The main advantage of this kind of observer is its fast convergence, since the convergence speed can be adjusted by changing its gains.Also, the error converges exponentially to a neighborhood of zero in finite time, and to zero if the estimated variable remains constant.
The observer equations are the following: where X and lx are the estimated cell concentration and the estimated specific growth rate, respectively.Parameters k 1 and k 2 are the observer gains, which must be chosen to ensure stability and fast convergence.The stability of this observer can be analyzed on the estimation errors, being defined as X ¼ X À X and l ¼ l x À lx .Then, the dynamics of the error can be calculated: The dynamical system (31) has eigenvalues k 1 and k 2 such that: Since X and X 2 are always positive, the sign and magnitude of the eigenvalues can be set by properly adjusting the gains k 1 and k 2 .To ensure that the error is stable and converges to zero, it is sufficient to make both eigenvalues negative.The more negative the eigenvalues are, the faster the convergence will be.Nevertheless, the measurement noise should be taken into account when choosing the gains, since if they are too high it may introduce noise in the estimation.In this work The choice of an exponential observer is made out of simplicity, since this is not the main matter of this work.Other approaches can be followed to estimate the growth rate when there is biomass measurement, such as secondorder sliding mode observers [9,10], which exhibit stronger convergence and robustness properties.

Results
In this section, the simulation and experimental results for the designed observers are displayed.The simulations are aimed at depicting the general operation of the observer as described by the equations and to analyze the effect of different uncertainties in the l x estimation.The objective of the experimental results is to test and validate the growth rate observer in a real scenario.
The simulations were performed trying to imitate, as identically as possible, the experimental conditions in the real bioreactor; the same control laws and carbon and nitrogen source input concentrations (S in and N in ) were used.The models and parameters used for the growth rates and PHB production rate are described by Eqs.(33)-( 35), all of which have been validated experimentally [21].The kinetic expressions include Haldane, Monod, saturation and inhibition factors.
It is important to remark that these models are exclusively used to simulate the process and are not used in the observer equations.The estimation is independent of these models and will converge even if the expressions for the growth rates are different or their parameters change.The parameters used to simulate the process and the observer are listed in Table 2.Many of the same parameters are used later in the experimental validation of the observer.

Simulation results
The model of the process is depicted by Eqs. ( 1)-( 4).The models for the specific growth and production rates are the ones described by Eqs.(33)-(35).Both S and N were regulated at 12 and 0.7 g/l, respectively.The switching from the exponential observer to the asymptotic observer is done when biomass concentration reaches 7.67 g/l = 0.6 AU.
The top graph in Fig. 2 shows the simulation results for the specific growth rate estimation when an error of AE25 % is introduced in the input nitrogen concentration coefficient g, which has a nominal value of g N ¼ 0:75.The dashed black curve is the real value of l x (overlapped by the blue curve), the solid blue curve corresponds to the estimation when there is no error in the parameter, and the red dotdashed and green dashed lines correspond to the cases with AE25 % errors, respectively.The bottom graph shows the biomass concentration and its estimations obtained from Eq. ( 13), the same nomenclature for color and line pattern is used.The switching instant from one observer to the other is labeled as t switch .
As soon as the process is started, it can be seen that the exponential observer converges quickly to the real value despite the large initial overshoot that can be observed in the small box.When the asymptotic observer is started at t switch , two different situations can be observed:in the case without error in g, the estimation remains equal to the real value; in the cases with error in g, a difference appears in l x at the switching point, which is mainly dominated by the first term in Eq. ( 25).However, the estimation still approaches the real value asymptotically as expected.The slow convergence speed is caused by the low dilution rate that is used in the simulation, mainly due to the high concentrations of carbon source and nitrogen source (S in and N in ) used to feed the bioreactor.It should be also noticed that the steady state errors in the biomass concentration estimation are not reflected in the growth rate estimation, which is the variable of interest.
Figure 3 shows simulation results when there is a variation in the nitrogen to biomass yield.The yield remains constant until the 20th hour when it is intentionally decreased linearly up to the 30th hour, then it remains constant again.The top graph shows the specific growth rate in dashed black and the estimation in solid blue, while the middle graph shows the biomass evolution in dashed black and the estimated value from Eq. ( 13) in solid blue.The bottom graph shows the value of the yield normalized to its nominal value.The slope in the yield change was made steep to make more visible the errors introduced in the growth rate estimation, however, this should be considered as a pessimistic scenario.When the yield starts changing, the estimation starts to converge to a value different from the real one as described by Eq. ( 25); however, when the yield stops changing, the observer converges again to the real value despite the yield being different from the initial one or the one expected by the observer.This is in line with the fact that the estimation error for z depends on the derivative of the yield and not on its value (Eq.( 21)).
Figure 4 shows the simulation results when the nitrogen concentration is not regulated correctly.The top graph shows the growth rate in dashed black and its estimation in solid blue, while the middle graph shows the biomass concentration in dashed black and the value corresponding to the estimated rate in solid blue.The bottom graph shows the nitrogen concentration in dashed black and its expected value in solid blue.
It can be seen that when nitrogen concentration starts decreasing, the growth rate estimation shows a small perturbation and separates from the real value (10th hour).However, even though the nitrogen concentration keeps falling, the l x estimation quickly starts converging to a value very close to the real one.This steady state error is described by Eq. ( 18).The biomass estimation error is also very small, although it depends directly on the nitrogen concentration error (x ¼ ðz À ÑÞy xn ).The two reasons for this are that z is zero as it is independent of the estimated nitrogen concentration (Eq.( 21)), and that the magnitude of Ñ Á y xn is small compared to the cell concentration at the end of the growth phase.

Experimental results
In this section, experimental validation of the observer is performed.The experimental inputs used are the carbon source dilution rate D s , the nitrogen source dilution rate D n and the optical density measurement before it saturates (for the first 15-20 h).The switching from the exponential to the asymptotic observer is made when biomass concentration is around 7.67 g/l (0.6 AU).
As the equipment available at the laboratory setup is only capable of driving the pumps at a fixed flow rate, a duty cycle is used to obtain different flow rate values.In that way, the substrates are fed into the bioreactor in pulses, with several minutes of difference between doses, especially for the carbon source because of its high concentration in the reservoir.To smooth the estimates, a digital FIR filter was used to distribute each pulse over time.
Figures 5 and 7 show the growth rate observers response for two different experiments (experiment 1 and experiment 2, respectively); different values for the nitrogen concentration coefficient g were used to make the convergence of the estimation more explicit.As in the simulations, three values were used, the nominal value g N ¼ 0:75 and a AE25 % variation over the nominal value (g N AE 25 %).The black curve in each graph is the l x estimation made by the observer when g is at its nominal value, the red and blue curves are the estimations with variations in the coefficient of AE25 %, respectively.The graphs include in cyan color a noisy growth rate reference curve obtained from Eq. ( 1) by solving it for l x and differentiating the OD measurement.Besides being extremely noisy, this estimation loses accuracy as the OD sensor saturates.This explains the discrepancy between the reference and the estimation after the switching point.The first thing that can be noticed in Figs. 5 and 7 is that, as expected from the theoretical analysis, the exponential observer response is the same for all values of g.At the switching point (marked as t switch ), differences appear due to the variation of g.Although these variations have the same percent magnitude as the error in g, they tend to disappear asymptotically as described by Eq. ( 28), similarly to what happened in the simulation results.Note that, the observer shows a satisfactory performance in the presence of nitrogen variations as shown in Figs. 6 and 8.
Finally, to highlight the convergence of the exponential observer, Figs. 9 and 10 show the response of the observer when varying the growth rate estimation initial condition for the same experiments shown before.As it can be seen, in all the cases, the observer converges to the same curve approximately 7 h after starting the process.

Conclusions and future studies
In this work state, observers were developed to estimate the specific growth rate of a PHB production process in a reallife scenario, by the use of biomass measurement in the first hours of the growth phase and nitrogen variations in the later hours of the same.Simulation examples confirmed the convergence of the growth rate estimation despite uncertainties in the model parameters.Experimental results validated what was developed in the theory and predicted by simulations.
The proposed asymptotic observer was able to accurately estimate the growth rate without measuring the biomass, which cannot be performed in this particular case due to the high cell density.It was found that the observer is robust against the gaseous mass loses of the nitrogen input (correction factor g) and poor nitrogen regulation.Furthermore, drifts in the nitrogen to biomass yield y xn introduce just small errors that disappear after the yield establishes at a new value.The proposed switched observer is simple enough to be implemented with few code lines using typical laboratory control software.
The process conditions in the PHB production phase are different from the ones in the growth phase and the observer is not designed to operate under those conditions.However, as there is no growth during the PHB production phase, there is no need to estimate the growth rate.Nevertheless, a different kind of observer should be designed to estimate the specific production rate during the PHB production phase.Future work includes the design of such an observer.By the use of the observers, extremum seeking algorithms will be applied to optimize the growth rate and the production rate in each respective phase.

Fig. 2 lFig. 4 l
Fig. 2 l xs -Observer response for different values of g.Simulation results Fig. 3 l xs -Observer response for a time-varying y xn .Simulation results

Table 1
Nomenclature of the variables and parameters used in the model

Table 2
Values of the parameters used in the simulations