A Sub-network Approximate Dynamic Programming for Optimal Cylinder Balance Control of Idling Engine

Communications in Control Science and Engineering (CCSE) Volume 2, 2014

www.as-se.org/ccse

A Sub-network Approximate Dynamic Programming for Optimal Cylinder Balance Control of Idling Engine Zhijian Huang1, 2, 3, Jie Ma1, He Huang1 1

Institute of Power plant and Automation, Shanghai Jiao Tong University, Shanghai, 200030, China Merchant Marine College, Shanghai Maritime University, Shanghai, 200135, China 3 IMS Automotive Electronic Systems Co; Ltd., Shanghai, 200335, China 2

zjhuang@shmtu.edu.cn Abstract The fluctuations control of idling engine is focused on long-term average speed, whereas the short-term fluctuation of idling speed in one engine cycle has been neglected. This is the cylinder balance control problem. This paper first extends approximate dynamic programming method to multi-input multi-output form with sub-network scheme; then applied in cylinder balance control through simulation and experiment individually. Results show that this method can improve cylinder balance effect by intelligently manipulating fuel injection quality of each cylinder. The advantage of this method is the exclusion to detect which cylinder the fluctuation comes from. The presented method is directly applicable for nonlinear multi-input multi-output coupled system. In addition, this method provides an available scheme for nonlinear multi-input multi-output approximate dynamic programming.

Keywords Approximate Dynamic Programming; Multi-input Multi-output; Sub-network; Cylinder Balance; Idling Engine; Neural Network

Introduction The idling control of engine has focused on long-term average of engine speed, for example, an idling speed of 800 r/min. However, the short-term fluctuations of idling speed in one engine cycle caused by component difference, aging or disturbance etc. have been neglected. If there are different torque variations in engine cylinders, the periodic variation of engine speed influences idling stability, and causes more vibration of the vehicle, which is so-called cylinder balance control problem. So far, only several techniques have been explored in this area. In 1996, Shim et al. mainly adopted PI (Proportion Integration) control to balance cylinder fluctuation of an idling engine model by adjusting spark advances. In 1998, P. Bidan combined airflow rate control simultaneously with automobile alternator operating as a synchronous motor, to provide a fast supplementary torque. In 1999, Kim and Park reduced unbalanced combustion among cylinders of an idling engine model with neural network and genetic algorithm by changing spark timings. In 2007, Kim and Park used adaptive method: genetic algorithms and Alopex algorithm, to control the fluctuations of engine speed at idle. Also in 2007, Z. Ye gave an overview on modelling, identification, design, and implementation of idling control system for automobile, including cylinder balance control. In 2010, F. Payri used multiple injections to correct combustion stability with combustion centroid angle. Also in 2010, H. Ohn analyzed combustion stability and engine control characteristics using a fast responded flame ionization detector and a special tool for cycle by cycle and cylinder to cylinder investigation. In 2010 and 2011, P. Li et al. mainly adopted Model Predictive Control to improve speed performance of idling engine. However, the Shim’s PI controller is not highly effective for nonlinear multi-input multi-output (MIMO) coupled system. Both P. Bidan and H. Ohn’s methods need special devices. The Kim’s neural network method uses genetic algorithm as assistance, but it’s still not robust for practice industry. The P. Li’s method used a linear discrete time model of single input and single output, whereas the engine is a typical nonlinear MIMO system. P. Li also adopted average torque in ignition-event scale to evaluate cylinder-to-cylinder imbalance, and this is not accurate enough for cylinder unbalance measure. The above and other methods usually have to detect which cylinder the fluctuations come from, whereas this kind of detection is not convenient or costly for many occasions.

Therefore, this paper adopts action-dependent heuristic dynamic programming (ADHDP) for cylinder balance control of idling diesel engine. This is a kind of optimal method originating from Dynamic programming; and doesn’t need special devices to detect which cylinder the fluctuations come from, for the adoption of the existing revolving speed signals. However, the standard ADHDP has only one action output, and it’s usually applicable for single control variable system. Thus, this paper extends standard ADHDP to MIMO form with sub-network scheme. The presented

9

www.as-se.org/ccse

Communications in Control Science and Engineering (CCSE) Volume 2, 2014

method is directly suitable for nonlinear MIMO coupled system, and has fast iteration convergence. The method can promote cylinder balance by intelligently manipulating fuel injection quality of each cylinder; as well provids an available scheme for MIMO ADHDP. In addition, this paper adds another dimension to the existing literature on fluctuations control of idling engine.

The paper is organized as follows. In Section 2, it demonstrates the principle and method of sub- network ADHDP. In Section 3, the sub-network ADHDP is applied in the cylinder balance simulation control of idling engine. In Section 4, the sub-network ADHDP is applied in its experiment control. Section 5 concludes this paper with a few remarks.

Sub-network Approximate Dynamic Programming The Standard Approximate Dynamic Programming Fig. 1 is the schematic diagram of standard ADHDP, in which symbols are seen. The action module has only one action network, and it outputs one optimal control policy u ( t ) . This optimal output controls the nonlinear system, and the controlled system produces the state vector X ( t ) of this time. The critic module is used to approximate cost function Q ( t ) . The dashed lines are the paths for weights turning of critic and action module.

Environment

ADHDP Controller X(t) States X(t)

Nonlinear System

Critic Module

u(t)

Action Module

Q(t) α

―

Q(t-1)-r(t)

Action u(t)

FIG. 1 SCHEMATIC DIAGRAM DEMONSTRATING THE PRINCIPLE OF STANDARD ACTION-DEPENDENT HEURISTIC DYNAMIC PROGRAMMING. THE SOLID LINES REPRESENT SIGNAL FLOWS, WHILE THE DASHED LINES ARE THE PATHS FOR WEIGHTS TURNING.

The Principle of Sub-network ADHDP In this work, the MIMO ADHDP can be achieved through sub-network scheme: paralleling multiple single-output action module of standard ADHDP. The difference between sub-network scheme and standard ADHDP only lies in the structure of action and critic modules. The schematic diagram of action and critic modules is demonstrated in Fig. 2, in which symbols are also seen. The sub-network scheme has multiple identical action networks parallelling with each other. Moreover, each action network still outputs only one optimal control policy. All the action networks are fed with the same input state vector X ( t ) .The state vector X ( t ) and all action network outputs are connected to the input of the critic network, thus the structure of the critic network is also changed. So this paper promotes standard ADHDP to MIMO form with sub-network scheme, and gets the following action and critic networks design. x1(t)

wa(1) 1i , j (t )

h1

. . .

h1i

xn(t)

(1) wam i, j

hm

. . .

Action 1

( 2) wam i

S

hmi

S

vm(t) gmi(t)

S

x1(t)

u1(t)

. . . . . .

gm

S

. (t) . .

S

g1i(t)

N am

x2(t) xn(t)

v1(t)

. (t) . .

S

. . .

wa( 12)i (t )

S

x2(t)

x1(t)

g1

S

N a1

um(t)

      

(t ) wc(1) i, j

q

. . .

  x (t)  n

S S

p

wc(i2) (t )

Nc S

u1(t)

  u (t)  m . . .

qi(t)

/

. . .

pi(t)

S

Critic

Q(t)

Action m

FIG. 2 SCHEMATIC DIAGRAM DEMONSTRATING THE ACTION AND CRITIC NETWORKS OF MULTI-INPUT MULTI-OUTPUT ACTION-DEPENDENT HEURISTIC DYNAMIC PROGRAMMING OF SUB-NETWORK SCHEME.

The Action and Critic Networks Design As for action module, there are m paralleled action networks, which is different from standard ADHDP. The weights in the action network are updated to minimize the following error measure:

10

Communications in Control Science and Engineering (CCSE) Volume 2, 2014

www.as-se.org/ccse

1 2 1 2 = ea ( t ) Q (t ) .

2 2

= Ea ( t )

(1)

The weights of the m action networks should be trained and updated at the same time step to prevent learning algorithm from aliasing. Here, take the i th action network for example. Other action networks are the same principles except for different subscripts. The subscript indicates which action network it is in the action module.

According to the gradient descent algorithm,

wa ( t= + 1) wa ( t ) + ∆wa ( t ) , i = 1, 2,..., m

(2)

 ∂E ( t )   ∂Ea ( t ) ∂Q ( t ) ∂ui ( t )  ∆wa ( t ) = la ( t )  − a ⋅ ⋅  = la ( t )  −  , i = 1, 2,..., m  ∂wa ( t )   ∂Q ( t ) ∂ui ( t ) ∂wa ( t ) 

(3)

i

i

i

i

i

i

As for critic module, its weights update is the same principle as that of standard ADHDP, but the critic network has more than one action inputs, which is different from standard ADHDP. According to the gradient descent algorithm,

Ec= (t )

(

1 2 1 ec= α Q ( t ) − Q ( t − 1) − γ ( t )  (t ) 2 2

)

2

,

wc ( t= + 1) wc ( t ) + ∆wc ( t ) ,  ∂E ( t )   ∂E ( t ) ∂Q ( t )  ∆wc ( t ) = lc ( t )  − c  = lc ( t )  − c ⋅ .  ∂wc ( t )   ∂Q ( t ) ∂wc ( t ) 

(4) (5) (6)

Where lc ( t ) > 0 and la ( t ) > 0 are the learning rates of critic and action networks at time t , which decreases to a small value with time; and α is the discount factor.

Simulation and Results The intention of simulation is to get pre-trained weights of action and critic networks preparing for online control.

These weights should already have certain control effect. So the simulating ADHDP controller adopts the same controller as that for experiment in section 5. The detailed controller design and its training are also seen in section 5.

The numerical model of idling engine follows the model by Shim, Park and our simulation test. This model assumes a four-stroke diesel engine. The initial fluctuations condition of this model is shown in Fig. 3.

A typically successful training of sub-network ADHDP controller is shown in Fig. 4~6. The simulation shows that simulating ADHDP controller can suppress non-uniformity revolving among four cylinders of idling engine model within eight iterations (Fig. 4).

The cylinder 1~4 intelligently manipulates each fuel injection quality to suppress unbalanced fluctuations. This procedure also takes only several iterations to reach expected fuel injection qualities (Fig. 5).

FIG. 3 FLUCTUATIONS OF IDLING ENGINE MODEL IN INITIAL CONDITION

(a)

(b)

11

www.as-se.org/ccse

Communications in Control Science and Engineering (CCSE) Volume 2, 2014

(c) FIG. 4 FLUCTUATIONS CONTROL OF IDLING ENGINE MODEL USING SUB-NETWORK ADHDP: (a) 1st iteration; (b) 3rd iteration; (c) 8th iteration.

FIG. 5 ADJUSTMENT PROCEDURE OF FUEL INJECTION USING SUB-NETWORK ADHDP FOR CYLINDER 1~4

(a)

(b) FIG. 6 ERRORS CONVERGENCE PROCESS USING SUB-NETWORK ADHDP: (a) the errors convergence process of critic and action networks without initial value, and (b) the errors convergence process of critic network with initial value=0.2 and action network with initial value=0.5.

The critic and action network errors can also finally converge to an extremely small value, but the critic network error has a long oscillation procedure (Fig. 6(a)). If the errors of critic and action networks are given an initial value of 0.2 and 0.5 before training, the oscillation procedure of convergence to a minimum is not seen, because the error oscillates only within a very small value (Fig. 6(b)).

Experiment and Results The Experimental Objective Explore control effect and application method of sub-network ADHDP with experiment.

12

Communications in Control Science and Engineering (CCSE) Volume 2, 2014

www.as-se.org/ccse

The Experimental Principle The analysis and experiment show that engine revolving speed has a complex second-order nonlinear relationship with its fuel injection quantity, so cylinder balance is mainly controlled by adjusting fuel injection quality of each cylinder. This experiment firstly measures angular crankshaft speed over one engine cycle to estimate revolving difference among cylinders. The non-uniformity revolving is then feedback into ADHDP controller. Then the controller calculates a fine-turn variable of fuel injection quality to make up PID controller’s output. Finally, the manipulated fuel injection qualities suppress unbalanced combustions or device difference among cylinders. This principle is shown in Fig. 7.

Commanded speed Intake

Engine ECU

Combustion

Speed(t+1)

WAPS Pump Diesel Engine DFT

Action Controller

U(t)

minimize Q(t)

e(t+1)

Critic Network

E(t)=0.5e2(t)

Z-1

minimize E(t)

FIG. 7 APPLICATION SCHEMATIC OF SUB-NETWORK ADHDP CONTROLLER IN THIS EXPERIMENT

The Experimental Devices The main experiment devices are: a 2.8L, 4-cylinder high speed diesel engine of Jianglin 4JB1-T type and its ECU, a common rail fuel pump of WAP2 type, an Agilent power supply of 6032A type, an Agilent oscilloscope of DSO7054A type, two current clamps of Tektronix, a common rail pressure sensor of BOSCH, a PC with Vector CANape software, a MEAN WELL data collecting board of RSP-1000-27 type and a speed sensor of DG6 type. The debugging software is CANape of Germany Vector Company. This software is used to measure data and produce debugging graph. The test bench is shown in Fig.8.

The Sub-network ADHDP Controller Design (1) The optimal objective: The four periodic waves in one engine cycle are completely uniform, that is, both the magnitude of the first harmonic frequency and that of the second harmonic frequency of Discrete Fourier Transform (DFT) results over data samples in an engine cycle are zero.

FIG. 8 TEST BENCH OF ADHDP FLUCTUATION CONTROL OF DIESEL ENGINE

(2) The structure of critic and action networks: The critic network is chosen as a 6–12–1 structure with 6 input neurons, 12 hidden layer neurons and 1 output neuron. The hidden layer adopts the following sigmoidal function:

y=

1 − e− x .

1 + e− x

(7)

13

www.as-se.org/ccse

Communications in Control Science and Engineering (CCSE) Volume 2, 2014

The action module is designed such that it consists of four identical sub-networks. Each such sub-network is an action network. The four identical action networks are all chosen as 2–5–1 structure with 2 input neurons, 5 hidden layer neurons and 1 output neuron. Both the hidden layer and the output layer use the sigmoidal function in (7).

(3) The multi-input/output connections: The two inputs to each of the action network are: the magnitude of DFT fundamental harmonic frequency in one engine cycle, the magnitude of the second harmonic frequency in half the cycle. The outputs of the action network 1~4 correspond to the fuel injection qualities of cylinder 1~4. The six inputs to the critic network are: the magnitude of DFT fundamental harmonic frequency in one engine cycle, the magnitude of the second harmonic frequency in half the cycle, and the 4 fuel injection qualities of cylinder 1~4. The output of the critic network is Q ( t ) .

(4) The utility function: according to optimal objective, the utility function is defined as:

γ (t ) = n −1 n −1  c1 ∑ N k ωnk + c2 ∑ N k ωn2 k , if 0 ≤ γ ( t ) ≤ 0.3 , c1Y full + c= 2Yhalf = k 0= k 0  −1, if γ ( t ) > 0.3 

(8)

where N k = ( N 0 , N1 , N 2 ,..., N n −1 ) is the sampled speed in an engine cycle; Y full is the magnitude of DFT fundamental frequency in one engine cycle; Yhalf is the magnitude of the second harmonic frequency in half the cycle; ωnk = e 2π ik / n ;

. is complex number magnitude; c1 and c2 are the coefficients for γ ( t ) , they are both selected as 1; i is the imaginary unit; n is the sampling number in an engine cycle and k is the k th sampling of the engine cycle.

Then, the optimal ADHDP controller is designed by minimizing the following formula, N −1

J ( t ) = min ∑ α i −t γ ( i ), 0 < α < 1

(9)

u∈ℜ i = t

where α is the discount factor, 0 < α < 1 ; t is the time step; γ ( i ) is the utility function in (8); N is the time scale of a testing cycle.

The utility function defined in this way will make the ADHDP controller convergent and keep cylinder balanced effect.

(5) The controller parameters: TABLE 1 MAIN DESIGN SPECIFICATIONS OF THE SUB-NETWORK ADHDP CONTROLLER IN THIS WORK

Learning rate: Desired training error: Maximum training cycle: Discount factor: Network weight initialization:

The critic network

The action network

lc (t)=0.3, decreases 0.05 per iteration until 0.01

la (t)=0.3, decreases 0.05 per iteration until 0.01

Tc (t)= 0.05

Ta (t) = 0.005

50 times

500 times α= 0.95 (-1, +1) , random

(6) The discretization: both the idling mathematical model and the ADHDP method should be discretized before simulation/experiment to guarantee that any operation which updates utility function can reduce the error between current condition and the optimal objective.

(7) The training procedure: the weighs update of sub- network ADHDP should follow certain procedures to achieve a good training result, that is: 1). Initialize critic and action network weights: wc ( t ) , wa ( t ) with small pseudo random numbers.

2). Initialize controller parameters, for example, the learning rate of critic and action networks: lc ( t ) , la ( t ) ; the expected learning errors: Tc ( t ) , Ta ( t ) ; the discount factor α and the maximum cycle times etc.

3). Set initial running condition X ( t ) of controlled system.

14

Communications in Control Science and Engineering (CCSE) Volume 2, 2014

www.as-se.org/ccse

4). Calculate action network output U ( t ) and utility function γ ( t ) based on initial running condition. Then calculate

Q ( t ) of critic network based on current running condition and action output.

5). Calculate the next running condition X ( t + 1) based on action outputs U ( t ) and U ( t + 1) as well as utility function γ ( t + 1) . Then calculate Q ( t + 1) of critic network based on current running condition and action output.

6). Calculate error function Ec ( t ) according to the formula (4).

7). Train critic network according to the weights updating formulas (5), (6). If critic error can meet the expected error or reach the maximum training times, then go to the next step, or return to the 6th step.

8). Calculate error function Ea ( t ) according to the formula (1).

9). Train action network according to the weights updating formulas (2), (3). If action error can meet the expected error or reach the maximum training times, then go to the next step, or return to the 8th step.

10). Judge whether the condition of controlled object meets requirement, or whether the cycle reaches the maximum iteration times. If either of them reaches requirement, return to the 5th step, or go to the next step.

11). Judge whether the test reaches its maximum times, if it’s true, then go to the next step, or return to the 3rd step.

12). End. The Experiment and Results First of all, the Matlab/m file of simulating ADHDP controller is converted into Matlab-Function module. Then, this module is embedded into the existing engine managing system of Matlab/Simulink form. Thirdly, the TargetLink tool of dSPACE translates this engine managing system into C language code, and loads it into the ECU developed by Shanghai IMS automobile electronic Co., Ltd. This ECU is then connected to a PC with dSPACE software. So the control strategy in the ECU can be debugged on PC by revising Matlab-Function module to achieve the best control effect.

As for this type diesel engine of 800 r/min idling speed, it takes 0.075s for one revolution, i.e. 0.15 s for every engine cycle. This experiment adopts IBM Thinkpad T61i laptop of 1.46G main frequency to process algorithm. Adopting the DMA (Direct Memory Access) and double buffering function, this laptop takes less than 0.03ms to finish one iteration calculation of the algorithm. Thus, the sampling frequency of 16 kHz for revolving speed is enough.

The diesel engine is assumed to have the same intake and exhaust effect. This experiment takes the weights of simulating ADHDP controller as initial weights, which already has certain control effect. The experiment then carries on test bench training. The diesel engine first runs on idling. Then the ADHDP feedback is started. The optimal objective is to let the diesel engine have cylinder balance at idling speed of 800 r/min. The ADHDP optimizing effect is observed with Vector CANape as real-time idling fluctuation graph. If the fluctuation cannot be compensated, the controller network weights should be trained and revised again to improve optimal effect.

Here take the data of three successful experiments to give compared control effect, that is, the open loop control, the PID control and the ADHDP control. The control effect is given through the parameters of objective idling speed, real-time engine speed and main fuel injection quality.

Fig.9 is the control effect of fuel injection qualities and idling engine speed during open loop control without cylinder balance.

FIG. 9 OPEN LOOP CONTROL WITHOUT BALANCING EACH CYLINDER, RELATIONSHIP BETWEEN FUEL INDEX AND ENGINE SPEED

15

www.as-se.org/ccse

Communications in Control Science and Engineering (CCSE) Volume 2, 2014

FIG. 10 CLOSING LOOP PID CONTROL WITHOUT BALANCING EACH CYLINDER, RELATIONSHIP BETWEEN FUEL INDEX AND ENGINE SPEED

FIG. 11 ADHDP CYLINDER BALANCE CONTROL, RELATIONSHIP BETWEEN FUEL INDEX OF CYLINDER 1 AND ENGINE SPEED

Fig. 10 is the control effect of fuel injection qualities and idling engine speed during closing loop PID control without cylinder balance.

Fig.11 is the relationship of fuel injection quality with idling engine speed for the 1st cylinder under ADHDP cylinder balance control in a successful training.

Fig.12 is the relationship of fuel injection quality with idling engine speed for the 2nd cylinder under ADHDP cylinder balance control in a successful training.

FIG. 12 ADHDP CYLINDER BALANCE CONTROL, RELATIONSHIP BETWEEN FUEL INDEX OF CYLINDER 2 AND ENGINE SPEED

Fig.13 is the relationship of fuel injection quality with idling engine speed for the 3rd cylinder under ADHDP cylinder balance control in a successful training.

FIG. 13 ADHDP CYLINDER BALANCE CONTROL, RELATIONSHIP BETWEEN FUEL INDEX OF CYLINDER 3 AND ENGINE SPEED

16

Communications in Control Science and Engineering (CCSE) Volume 2, 2014

www.as-se.org/ccse

FIG. 14 ADHDP CYLINDER BALANCE CONTROL, RELATIONSHIP BETWEEN FUEL INDEX OF CYLINDER 4 AND ENGINE SPEED

Fig.14 is the relationship of fuel injection quality with idling engine speed for the 4th cylinder under ADHDP cylinder balance control in a successful training.

In these figures, the dotted wide lines are the ECU objective speeds, that is, the objective idling speeds; the solid thin lines are the real-time diesel engine speeds; and the solid wide lines are the fuel injection qualities of each cylinder.

The open loop control effect in Fig.9 shows that this engine has a relatively larger idling speed fluctuation, and it’s about 800±120 r/min. Fig.10 is the PID control effect, and its controller manipulates fuel injection qualities according to speed fluctuation. The speed range is about 800±40 r/min, and has a periodic fluctuation about ±40 r/min.

Fig.11~Fig.14 are the control effects of ADHDP optimal method based on PID controller, which reduces the difference among speed fluctuations of each cylinder and improves cylinder balance effect. After optimization, the speed range is about 800±20 r/min, and the periodic fluctuation has been reduced about ±20 r/min. The fuel injection qualities in these figures varying according to manipulation reduce gradually with time, because the cooling water temperature of diesel engine rises gradually with time.

Conclusions This work extended standard ADHDP for nonlinear MIMO system with sub-network scheme. The training algorithm of this method was also given. This novel method was then applied in the fluctuations control of idling engine with simulation and experiment individually. The sub-network ADHDP method intelligently made up differences among cylinders caused by device difference, aging or combustion etc. Therefore, here the following conclusions were drawn, according to the above fact and controlling performance: (1) This paper presents a nonlinear MIMO ADHDP by extending standard ADHDP with sub-network scheme.

(2) This is the first time that sub-network ADHDP has been applied in the cylinder balance experiment control of an idling engine.

(3) The sub-network ADHDP can suppress unbalanced fluctuations caused by device difference, aging or combustion among cylinders by intelligently manipulating fuel injection qualities of four cylinders. Specially, this method has the advantage of free from detection on which cylinder the fluctuations come from, and optimizes the cylinder balance performance.

(4) The applying example proved the correctness and effectiveness of sub-network ADHDP training algorithm.

(5) The sub-network ADHDP can nearly be applied in all nonlinear MIMO cases, especially the multiply outputs has different orders of magnitude. Since sub-network structure can eliminate interference among output vector or the weights by combining independent single-output action network. Thus, the sub-network method can be used to resist mis-learning of multiply action outputs.

(6) The problem on orders of magnitude or larger difference of multiply action outputs is one of the causes why some MIMO ADHDP suffers a failure design or has a poor effect. So the sub-network ADHDP can have a relatively higher learning success rate because of its structure.

(7) The presented method is a basic MIMO ADHDP scheme, and it provides an effective alternative for nonlinear MIMO ADHDP. Though illustrated for engine control, the ADHDP control system framework should also be applicable to general purpose nonlinear MIMO system.

Acknowledgements This work was supported by the NSFC Projects under Grant No. 50979058, No.31170952, by the Innovation Program of Shanghai Municipal Education Commission under Grant No.11ZZ143, and by Shanghai IMS Automotive Electronic

17

www.as-se.org/ccse

Communications in Control Science and Engineering (CCSE) Volume 2, 2014

Systems Company, Ltd. References [1]

D. E. Kim, and J. Park, “Application of adaptive control to the fluctuation of engine speed at idle,” Information Sciences; vol.

177, no. 16, pp.3341–3355, 2007.

[2]

D. Shim, J. Park, P. P. Khargonekar and W. B. Ribbens, “Reducing Automotive Engine Speed Fluctuation at Idle”, IEEE Transactions on Control Systems Technology, vol. 4, no. 4, pp. 404–410, 1996.

[3]

D. Shim, J. Park, P. P. Khargonekar, and W. B. Ribbens, “Reducing automotive engine speed fluctuation at idle,” IEEE Trans.

Contr. Syst. Technol., vol. 4, pp. 404–410, July 1996.

[4]

D.E. Kim and J. Park, “Neural network control for reducing engine speed fluctuation at idle,” IEEE Transactions on Control Systems Technology, vol. 4, no. 1999, pp. 629–634, 1999.

[5]

F. Payri, A. Broatch, J.M. Salavert, and J. Martín, “Investigation of Diesel combustion using multiple injection strategies for idling after cold start of passenger-car engines,” Experimental Thermal and Fluid Science, vol. 34, no. 2010, pp. 857–865, 2010.

[6]

G. Haraldsson, J. Hyvönen, P. Tunestål, B Johansson, “HCCI Combustion Phasing with Closed-Loop Combustion Control Using Variable Compression Ratio in a Multi Cylinder Engine”, JSAE 20030126 / SAE 2003-01-1830.

[7]

H. Husted, D. Kruger, G. Fattic, G. Ripley and E. Kelly, “Cylinder Pressure-Based Control of Pre-Mixed Diesel Combustion”, SAE 2007-01-0773

[8]

H. Ohn, S.Yu, and K. Min, “Spark timing and fuel injection strategy for combustion stability on HEV powertrain,” Control Engng Practice, vol. 18, no. 11, pp. 1272–1284, 2010.

[9]

J. J. Govindhasamy, S. F. McLoone, G. W. Irwin, J. J. French and R.P. Doyle, “Reinforcement learning for online control and optimisation,” IEE Control Engineering Book Series, vol. 70, no. 9, pp. 293–326, 2005.

[10] J. Si, and Y. T. Wang, “On-line learning control by association and reinforcement,” IEEE Transactions on Neural Networks, vol. 12, no. 2, pp. 264–276, 2001.

[11] L. Po, and S.Tielong, “Overlap model based unknown offset-free MPC scheme for torque balancing control in multi-cylinder SI engines,” China Control Conference (CCC), 30th Chinese, IEEE, pp. 6239–6244, 2011.

[12] Mark C. Sellnau, Frederic A. Matekunas, Paul A. Battiston, Chen-Fang Chang, and David R. Lancaster, "Cylinder-pressure-based engine control using pressure-ratio-management and low-cost non-Intrusive cylinder pressure sensor", SAE paper 2000-01-0932, 2000.

[13] P. Bidan and L. Kouadio, “Electrical assistance for S.I. engine idle-speed control,” Control Eng. Pract., vol. 6, no.

7, pp.829–836, 1998.

[14] P. Li, T. Shen, and D. Liu, “Idle speed performance improvement via torque balancing control in ignition-event scale for SI engines with multi-cylinders,” International Journal of Engine Research, vol. 13, no. 1, pp. 65–76, 2012.

[15] R. Bellman, “Dynamic Programming,” Princeton University Press: Princeton, NJ, 1957.

[16] R. Enns, and J. Si, “Helicopter trimming and tracking control using direct neural dynamic programming,” IEEE Transactions on Neural Networks, vol. 14, no. 4, pp. 929–939, 2003.

[17] X. Mao, “Research on the nonlinear speed control system of diesel engine with the variable structure control theory,” Wuhan: Wuhan university of technology, 2008.

[18] Z. Ye, “Modeling, Identification, Design and Implementation of Nonlinear Automotive Idle Speed Control Systems - An Overview,” IEEE Transactions on Systems, Man and Cybernetics, Vol. 37, no. 6, pp. 1137–1151 Nov. 2007.

18