Cmjv02i02p0198

J. Comp. & Math. Sci. Vol.2 (2), 198-207 (2011)

NNGD Algorithm for Active Noise Cancellation Using Non Linear Filters.

S. PANDA* and S. K. NAYAK Department of Electronic Science Berhampur University, Bhanja Bihar -760 007, Ganjam, Orissa, India. email: sudambu@yahoo.co.in ABSTRACT The problem of controlling the noise level in the environment has been the focus of a tremendous amount of research over the years.

In this work Active Noise Cancellation (ANC) is one such approach that has been proposed for reduction of steady state noise. ANC refers to an electromechanical or electro acoustic technique of canceling acoustic disturbance to yield a noise less environment. The basic principle of ANC is to introduce a canceling “anti noise� signal that has the same amplitude but the exact opposite phase, thus resulting in an attenuated residual noise signal. Wideband active noise control systems often involve adaptive filter lengths with hundreds of taps. We propose implementation of active noise-canceller based on nonlinear normalized gradient descent algorithm (NNGDA). The above algorithm converges much faster than the normalized least mean squares algorithm (NLMS) because of consideration of Taylor's series expansion of instant output error in the derivation of nonlinear normalized conjugate gradient descent (NNCGD) is made adaptive at every discrete time instant using gradient based approach.

Keywords: Band pass filter, Nonlinear normalized gradient descent algorithm.

1. A. INTRODUCTION Adaptive filtering is widely used in network active noise control1, data communications, echo cancellation, image processing and bio-medical signal processing. An adaptive filter can be characterized by its topology and adaptation algorithm. The conventional least mean

square (LMS) has some advantage of simplicity and robustness to its numerical error where as the rate of convergence is very slow2. But the normalized least mean square (NLMS) algorithm is some how complex than LMS, is widely used because of its stability and simplicity. However, the convergence performance degrades signifycantly with colored training signals such as

Journal of Computer and Mathematical Sciences Vol. 2, Issue 2, 30 April, 2011 Pages (170-398)

199

S. Panda, et al., J. Comp. & Math. Sci. Vol.2 (2), 198-207 (2011)

B. MOTIVATION

the primary signal which is corrupted by noise, called as reference signal. The reference signal is adaptively filtered and subtracted from the primary signal in case of noise and echo-cancellation problems. But in case of system identification problems, the usual method of estimating a signal from a noisy corrupted signal is by passing the signal through the filter that tends to suppress the noise while leaving the signal relatively unchanged. The design of such filters is in the domain of optimal filtering such as Weiner filter, Kalman and Extended Kalman Filters. Such filters are having the knowledge of both signal and noise.

Adaptive filters on the other hand have the ability to adjust their own parameters automatically and their design requires a little or no priori knowledge of both the signal and noise characteristics. This is based on the topology of various adaptation mechanisms. There are different types of adaptive filtering techniques like LMS, NLMS, RLS, FLS and etc. These are used in various applications like signal processing, image processing, communications and control systems. The conventional LMS method has some advantages of simplicity and robustness to its numerical error but rate of convergence is very slow for colored noises. But case of non stationary signals like speech the LMS algorithm requires 512 to 1024 taps for the purpose of noise as well as echo-cancellations problems. But the recursive least square algorithm converges very quickly but computational complexities are more due to involvement of matrix inversion problem.

Our proposed work deals with the noise cancellation problems from 1d-2d signals. The available methods normally use

The NLMS is an important variant of classical LMS algorithm for adaptive linear filtering techniques. It possess many

speech which happens in telecommunication networks3. In to-days applications, adaptive filters with 512 or 1024 taps of standard choices for network noise-cancellers. On the other hand, the recursive least square (RLS) algorithm4 converges more quickly but in some practical cases, its complexity prevents it to implement in a low cost commercial digital signal processors.

For nonlinear signals like speech signals, the noise cancel problem is a simple extension of band pass FIR filter, which is a dynamical perceptron, where the filter has a continuous nonlinear activation function. In this case, the real valued fully adaptive normalized nonlinear gradient descent algorithm5-8 has been implemented for noise cancellation in telephone network systems.

Majority of data exists in real values but some applications complex valued data also plays an important role, in such cases we normally use complex valued activation functions as well as complex learning algorithms9-12. Now a days echo cancellation problems are also available with the help of neural network based algorithms13 and sub band based approach14. Since the speech and noise are of real valued non-stationary data of nonlinear types, so nonlinear type of updation rules are more suitable for this case. In case of nonlinear filters, the choice of activation function plays an important role in the convergence of the algorithm. In our present case, the output error updates the coefficients of the filter according to the truncated Taylor’s series.

Journal of Computer and Mathematical Sciences Vol. 2, Issue 2, 30 April, 2011 Pages (170-398)

S. Panda, et al., J. Comp. & Math. Sci. Vol.2 (2), 198-207 (2011)

advantages over LMS algorithm including having faster convergence and automatic time-varying choice of step size parameters which effects the stability, steady state mean squared error and convergence speed of the algorithm. For non-linear and non-stationary cases its effects are of some limit. For this reason we propose to work out a nonlinear NLMS type of adaptive filter. In this, we are going to implement a real valued fully adaptive NNLMS method using non-linear gradient descent (NGD) based learning algorithm for training a nonlinear neural based finite impulse response (FIR) filter.

The proposed method will be helpful for the input signals with unknown, possible for large dynamic range and ill-conditioned tap input correlation matrix. In order to speed up the non-linear learning process, the minimization of output error is required. For such cases, here we consider non-linear techniques as mentioned. For optimal learning of nonlinear normalized gradient descent (NNGD) algorithm, the instaneous error is calculated and expanded by a Taylor series method for all the parameters to be adapted. For effective matching of the output signal with the primary signal the different order values from Taylor series expansion are considered and are used for updation of different parameters taken in NNLMS algorithm. The results are to be considered with LMS and NLMS. The organization of the paper is: section-2 presents real valued fully adaptive nonlinear normalized gradient descent algorithm, section-3 deals with simulation of this algorithm and section-4 presents the results, discussions and future extensions of this paper including bibliography.

200

2. REAL VALUED NON LINEAR NORMALISED LEAST SQUARE METHOD In case of noise-canceller (Fig.:2), the signal is the output of the filter which eliminates the noise signal. is the random Gaussian noise with SNR of 30dB generated at the noise path which mixes with the original signal. is the original speech signal. The signal excited at the noise path is .

The output of band pass filter at the noise-path is . The actual out put from the non-linear adaptive filter which can be expressed as

y (k ) = φ (u (k )) e( k ) = d ( k ) − y ( k )

(1)

where, is the instant output error of the filter at time instant k. is the nonlinear output of . is the desired output.

The output signal from the network can be expressed as: N

u ( k ) = ∑ xn ( k ) wn ( k ) = n =1

→T

→T

→

(2)

→

X (k ) W (k ) = W (k ) X (k ) →

Where,

X ( k ) = [ x1 ( k ), x 2 ( k ),.....x N ( k )]T

such that, xn (k ) = x(k − n + 1), n = 1,2, N , the weight vector is denoted as →

W ( k ) = [ w1 ( k ), w2 ( k ) 2 ,.....w N ( k )]T and N is the number of tap inputs. The error cost function of equation (1) is expressed as

Journal of Computer and Mathematical Sciences Vol. 2, Issue 2, 30 April, 2011 Pages (170-398)

201

E (k ) =

S. Panda, et al., J. Comp. & Math. Sci. Vol.2 (2), 198-207 (2011)

1 2 e( k ) 2

(3)

The weight updation of the adaptive filter structure is expressed as2

wn (k + 1) = wn (k ) + ∆wn (k )

∂e ( k ) ∆w n ( k ) n =1 ∂wn ( k ) N

e(k + 1) = e(k ) + ∑

(7)

Where, (4)

Where 1, 2 … .

∆wn (k ) = −η∇ w [ E (k )]w= wn ( k ) = ηe(k )φ ′(u (k )) xn (k )

neglected15 and the values of and are not known, the Taylor series can be truncated to:

(5)

Where, η is learning rate parameter. When the input signals of large dynamical range, an ill conditioned tap input vectors produce ill-conditioned input autocorrelation matrix and its coupling between different signal modes and improper choice of learning rate parameter slows down the learning process. In order to avoid the above process, the optimal choice of learning rate parameter and other coefficients of the algorithm are done with the help of normalized nonlinear gradient algorithms10, 11 by expanding the instant output error by Taylor's series expansion of error function,

; ; is expressed as:

∆wn (k ) = wn (k + 1) − wn (k ) , ∂e(k ) = −η , ∂wn (k ) =ηe(k )φ ′(u(k )) xn (k ).

Differentiating the value of

(8)

∂e(k ) and ∂wn ( k )

substituting the value of eqn. (8) in eqn. (7), we get

e(k + 1) = e(k ) − ηe(k ) φ ′(u (k )) x(k ) 2 , 2

= e(k )[1 − η φ ′(u (k ))

2

2

x(k ) 2 ] .

2

(9)

When, e(k + 1) → 0, η is solved by using10, which can be expressed as

η (k ) =

1

φ ′(u ( k )) x( k ) 2

When, x(k )

N

2 2

2

(10)

2

→ 0 , the value of

e(k + 1) = e(k ) + ∑

η (k ) becomes ∞ . So, this can be avoided

∂e(k ) ∂e(k ) ∆wn (k ) + ∆xn (k ) + ∂wn (k ) ∂xn (k )

by adding a constant at the denominator of above equation and the value of can be also varied at each iteration using Gradient Descent Method. So η can be represented as:

n =1

∂e(k ) ∆d (k ) ∂d (k )

(6)

When, ∂ w(k) is sufficiently small, the higher order terms of equation (6) are

η (k ) =

1

φ ′(u ( k )) x ( k ) 2 + C ( k − 1) 2

2

Journal of Computer and Mathematical Sciences Vol. 2, Issue 2, 30 April, 2011 Pages (170-398)

(11)

S. Panda, et al., J. Comp. & Math. Sci. Vol.2 (2), 198-207 (2011)

Similarly as above, the value of can be updated accordingly

C(k ) = C(k − 1) − ρ∇C [E(k )] C =C ( k −1)

(12)

The value of C (k ) can be evaluated as per15, which can be expressed as in Appendix,

C(k ) = C (k − 1) − 0.5 * ρξ (k )e(k )φ ′(u(k )) x(k )

(13)

Where ρ is a constant which is chosen to be small and ξ (k ) is expressed as

→T

e(k − 1)φ ′(u (k − 1)) X (k − 1) 1 ξ (k ) = 2 [ φ ′(u (k − 1)) 2 x(k − 1) 2 + C (k − 1)]2 2

(14) 3. A. SIMULATION The speech signal generated for second with the help of microphone interfaced with a personal computer by sampling a speech signal at a Nyquist rate of audio frequency range in a closed room. The noise signal generated inside the room is picked up by the microphone along with back ground random noise of 30dB. Such type of circumstances arises during the time of teleconferencing. Such a signal is passed through a band pass filter of 20Hz to 20 KHz (Fig: 4) with a tap length of 1024. Such a large tap length filter is not always possible. So in the present work, we have adapted normalized nonlinear least square filter for filtering out unwanted signal from the actual speech signal (Fig.2). The system identification type of noise canceller is

202

sketched in Fig.1, which is implemented as adaptive filter in Fig.2. Since the speech signals are highly non-stationary and magnitude changes over wide dynamic ranges which have high peaks and silent intervals. For nonlinear stochastic models, the initial condition of different parameters can affect the performance of the system drastically. So the choice of parameters like and at initial state are chosen in such a way that the learning curve converges very faster. Normally the values are chosen arbitrary between 0 and 1.

Here the noise-cancellation problem is dealt with a nonlinear normalized least mean square algorithm with full adaptation of different parameters presented in the eqns.(8 & 13).The comparison of this algorithm is not possible with fast converging algorithms like Recursive Least Squares (RLS) algorithm because of complication for practical application which may suffer numerical instability. In this case NNLMS algorithm is compared with NLMS with different step size. The error convergence curve is plotted in Fig.3. The noise mixed signal with original speech signal is trained with fully adaptive NNLMS algorithm with the adaptation of weight vector, step size and denominator coefficient. The results obtained from this algorithm are compared with standard NLMS which are plotted in the Fig.4.

3. B. SIMULATION ALGORITHM Step-1: Initialize the weights, Parameters like ρ .

Step-2: Submit an input vector to the adaptive filter network.

Journal of Computer and Mathematical Sciences Vol. 2, Issue 2, 30 April, 2011 Pages (170-398)

203

S. Panda, et al., J. Comp. & Math. Sci. Vol.2 (2), 198-207 (2011)

Step-3: Calculate the out put value and error signal from the input vector and weight vector.

Step-4: Evaluate →T

1 e(k − 1)φ ′(u (k − 1)) X (k − 1) ξ (k ) = 2 [ φ ′(u (k − 1)) 2 x (k − 1) 2 + C (k − 1)]2 2

Step-5: Evaluate C ( k ) = C ( k − 1) − 0 . 5 * ρ ξ ( k ) e ( k )φ ′ ( u ( k ) ) x ( k )

Step-6: Evaluate

η (k ) =

1

φ ′(u ( k )) x ( k ) 2 + C ( k − 1) 2

2

Step-7: Update the weights

η ′ ௡

Step-8:

weights until it reaches its optimal value.

The trained weights are utilized for validation of certain input-output mapping patterns which are plotted in the Figure.4.

Here the computational cost of the NNLMS algorithm is much higher because of the expansion of error function by Taylor’s series up to 1st order, which gives some extra terms in each step. But the updation of parameters like weight coefficients, learning rate becomes faster which we can see from the error convergence curve. By adjusting the parameters like gain etc. we can converge the error function much faster than the previous one.

ଵ ଶ

ଶ

Step-9: All iterations are over, if not go to step 2 If yes go to step 10 Step-10: If tolerance factor is reached yes than stop the training process if not change the weight vector to some other set and again repeat it.

4. A. RESULTS & DISCUSSIONS The noise speech signals are trained with the non linear type of sigmoid activation functions at the out put of single layer adaptive model with a single out put model. For different input-output mappings, the adaptive filter is used to update its

Here the noise speech signals are trained with the least mean square (LMS), NLMS and nonlinear normalized least mean squares (NNLMS) algorithms. We can observe from the convergence curve (Fig.3) the rate of convergence of NNLMS and LMS algorithm. It can not be compared with RLS type of algorithm because it may cause instability in convergence.

The present algorithm can be tried with affine projection algorithms, non-linear IIR filter structures for better convergence and better results than the present one irrespective of computational complexity.

4. B. FUTURE EXTENSION This present work can be extended to echo cancellation problems by using nonlinear system identification techniques with the help of IIR filter structures and taking higher order coefficients from Taylor’s series expansion of error signal.

Journal of Computer and Mathematical Sciences Vol. 2, Issue 2, 30 April, 2011 Pages (170-398)

204

S. Panda, et al., J. Comp. & Math. Sci. Vol.2 (2), 198-207 (2011) x(n)

Z-1

wo(k)

x(n-1) w1(k)

Z-1 u(k)

∑

∫

y(k)

x(n-2) Z-1

wN-1(k)

x(n-N+1)

Fig. - 1: Non linear model for fir filter

Near end speech signal ∑ Noise

FIR FILTER

Non linear NLMS Updation Rules

ADAPTIVE FILTER

y(n) d(n) e(n)

FIG-2: A model for fully adaptive nonlinear NLMS based active noise canceller.

Journal of Computer and Mathematical Sciences Vol. 2, Issue 2, 30 April, 2011 Pages (170-398)

205

S. Panda, et al., J. Comp. & Math. Sci. Vol.2 (2), 198-207 (2011)

Fig-3: Learning rate curve for FANNLMS and NLMS

Fig-4: Simulated Results of Speech Signal Journal of Computer and Mathematical Sciences Vol. 2, Issue 2, 30 April, 2011 Pages (170-398)

S. Panda, et al., J. Comp. & Math. Sci. Vol.2 (2), 198-207 (2011)

4. REFERENCES 1. "Acoustic echo and noise control. A practical approach", by E. Hänsler and G. Schmidt. John Wiley & Sons, (2004).

2. Honing.

M, Messerschmitt.

D, “Adaptive Filters: Structures, Algorithms and Applications”, ISBN-13: 9780898381634, Kluwer Academic Publishers, pp. 1061-1064 (1984).

3. Haykin, Simon, “Adaptive Filter Theory”, ISBN-10: 0130901261, Prentice-Hall, pp.231-38, 320-24 (2001).

4. B. Widrow and S.D. Stearns, Adaptive Signal Processing, Englewood Cliffs, New Jersey: Prentice-Hall, (1985).

5. D. P. Mandiac and J. A. Chambers, towards the optimal learning rate for back propagation, Neural Process. Lett.

Vol.11, no.1, pp.1-5, (2000).

6. V.J.Mathews and Z.Xie, Stochastic gradient adaptive filters with gradient adaptive step sizes, IEEE Trans. Signal Process. Vol. 41, no.6, pp. 2075-2086, June (1993).

7. W.Ang and B. Farhang - Boroujeny, A new class of gradient adaptive step size LMS algorithms, IEEE Trans. Signal Process., Vol.49, no.4, pp.805-810, Apr. (2001).

8. A. M. Kuzminiskly, Self-adjustment of an adaptation coefficient of noise compensator in a non-stationary process, Izvestiya VUZ. Radio electronika, Vol. 29, no.3, pp.103-105, (1986).

9. G. M. Georgiou and C. Koustougeras, Complex domain backpropogation, IEEE Trans. Circuits Syst. II Vol.39, pp.330-334, May (1992).

10. A. I. Hanna, D.P.Mandiac, and M.Razz, Normalized backpropogation learning

11.

12.

13.

14.

15.

206

algorithm for multilayer feed forward neural adaptive filters, in Proc. XI IEEE Workshop Neural Networks in Signal Process., pp.63-72 (2001).

D.P.Mandiac and J.A.Chambers, towards the optimal learning rate for backpropogation, Neural Process. Lett., Vol.11, no.1 pp.1-5, (2000).

A.Cichoki, R.Unbenhun, P.Signal , Neural networks for optimization and Signal Processing, New York:Wiley, (1996).

D.Mandiac and J.Chambers, recurrent Neural Networks for Prediction, Chicester, U.K.: Wiley, (2001).

Mari de courville and Pierre Duhamel, “Adaptive filtering in subbands using weighted criteria”, IEEE Trans. on Signal Process. Vol.46 (9) pp.23592371, (1998).

A.I.Hanna and Danilo P. Mandiac, A fully adaptive normalized nonlinear gradient descent algorithm for complex valued nonlinear adaptive filters, IEEE Trans. Signal Process., Vol.51, no.10, pp.2540-2549, (2003).

APPENDIX: The learning rate parameter as expressed in eqn. (11) as:

η (k ) =

1

φ ′(u (k )) x(k ) 2 + C (k − 1) 2

2

(15)

The above eqn. implies the learning rate parameter is also function of C (k − 1) .

Differentiating the learning rate parameter with respect to C (k − 1) we get

Journal of Computer and Mathematical Sciences Vol. 2, Issue 2, 30 April, 2011 Pages (170-398)

207

S. Panda, et al., J. Comp. & Math. Sci. Vol.2 (2), 198-207 (2011)

∂η (k ) = ∂C (k − 1)

wn (k ) = wn −1 (k ) + η (16)

−1 [ φ ′(u (k − 1))

2

x(k − 1) 2 + C (k − 1)] 2

∂E (k ) ∂w(k )

k = k −1

(19)

2

k = k −1

Differentiating the cost function with respect to C (k-1), we get

∂E (k ) ∂C (k − 1) ∂w(k ) = − φ (u ( k )) ′ x T ( k ). ∂C (k − 1)

(17)

Substituting the value in the eqn. (12), we get C ( k ) = C ( k − 1) + (0.5) * ρ e( k − 1).

∂w( k ) φ ′(u (k − 1)) x (k − 1). ∂C ( k − 1)

(18) k = k −1

As we know from equation (8)

∂w( k ) ∂C ( k − 1)

k = k −1

∂ (k ) k = k −1 ∂C ( k − 1)

∇ C [ E (k )] C =C ( k −1) =

T.

Then differentiating the w (k) with respect to C (k − 1) , we get =

∂w( k − 1) ∂η ( k ) − ∂C ( k − 1) ∂C ( k − 1)

∂ 2 E (k ) k = k −1 − η ( k ) ∂w( k )∂C ( k − 1)

(20)

The first term of R.H.S of above eqn. is zero, neglecting the higher order terms and substituting the values in the above eqn. (18) we get, C (k ) = C(k − 1) − 0.5* ρξ (k )e(k )φ ′(u(k )) x(k )

(21)

Where, →T

1 e(k − 1)φ ′(u (k − 1)) X (k − 1) ξ (k ) = 2 [ φ ′(u (k − 1)) 2 x(k − 1) 2 + C (k − 1)]2 2

Journal of Computer and Mathematical Sciences Vol. 2, Issue 2, 30 April, 2011 Pages (170-398)

(22)