Convergence and learning curve of 2nd order LMS predictor
** lab15_1.m * mw * 05/11/2007
Contents
2nd order model system, optimum prediction coefficients, MMSE
a = [1 -.71 .25]; % denominator coefficients b = .24192*[1 2 1]; % numerator coefficients Nh = 21; % lenght of impulse response h = impz(b,a,Nh); % impulse response Rhh = conv(h,flipud(h)); % acf R = [Rhh(Nh) Rhh(Nh+1); Rhh(Nh+1) Rhh(Nh)]/Rhh(Nh); % correlation matrix r = [Rhh(Nh+1); Rhh(Nh+2)]/Rhh(Nh); % crosscorrelation vector fprintf('lab15_1 mw 05/11/2007\n\n') fprintf('2nd order correlation matrix\n') fprintf('Rxx[1,1] = %6.4f ; Rxx[1,2] = %6.4f \n',R(1,1),R(1,2)) fprintf('Rxx[2,1] = %6.4f ; Rxx[2,2] = %6.4f \n',R(2,1),R(2,2)) fprintf('Crosscorrelation vector\n') fprintf('rxd[1] = %6.4f ; rxd[2] = %6.4f \n',r(1),r(2)) b_opt = R\r; % MMSE design MSEopt = R(1,1) + b_opt'*R*b_opt - 2*b_opt'*r; % minimum MSE fprintf('Prediction coefficients and error signal power\n') fprintf('b0 = %6.4f ; b1 = %6.4f ; MSEopt = %6.4f\n',b_opt(1),b_opt(2),MSEopt) lambda = eig(R); % eigenvalues of the correlation matrix fprintf('Eigenvalues\n') fprintf('lambda1 = %6.4f ; lambda2 = %6.4f \n',lambda(1),lambda(2))
lab15_1 mw 05/11/2007 2nd order correlation matrix Rxx[1,1] = 1.0000 ; Rxx[1,2] = 0.7885 Rxx[2,1] = 0.7885 ; Rxx[2,2] = 1.0000 Crosscorrelation vector rxd[1] = 0.7885 ; rxd[2] = 0.3684 Prediction coefficients and error signal power b0 = 1.3168 ; b1 = -0.6700 ; MSEopt = 0.2085 Eigenvalues lambda1 = 0.2115 ; lambda2 = 1.7885
Sample function of model process
M = 4e2; % number of iterations randn('state',0); w = randn(1,M); % innovation d = filter(b,a,w); % IIR filter for process modeling - desired signal
Simulation of adaptiv system
.alpha - convergence parameter, .beta - weighting parameter of variance estimator .sig2 - variance, .N - FIR filter order (N+1 coefficients) .Mode - mode for updating FIR filter coefficients, Mode = 'e*x' no sign function used Mode = 'x*sign(e)','e*sign(x)' and 'sign(e*x)' uses sign function
p = 2; % prediction order (FIR filter order + 1) lms = struct('alpha',1/16,'beta',1/32,'b',zeros(p,1),'e',0,'x',zeros(p,1),... 'sig2',p,'Mode','e*x'); x = [0 d(1:M-1)]; % input signal for adaptive FIR system [y,e,lms,SAVEb] = lms_algorithm(x,d,lms); % LMS algorithm fprintf('final prediction coefficients\n') fprintf('(%i) b0 = %6.4f ; b1 = %6.4f\n\n',M,lms.b(1),lms.b(2))
final prediction coefficients (400) b0 = 1.1628 ; b1 = -0.6785
Graphics
FIG1 = figure('Name',['lab15_1 : Desired signal, prediction error and variance estimate (',lms.Mode,')'],... 'NumberTitle','off'); subplot(3,1,1),plot(0:M-1,d,'LineWidth',2), grid, axis([0 M-1 -4 4]) xlabel('n \rightarrow'), ylabel('d[n] \rightarrow') title('Desired signal') subplot(3,1,2),plot(0:M-1,e,'LineWidth',2), grid, axis([0 M-1 -4 4]) xlabel('n \rightarrow'), ylabel('e[n] \rightarrow') title('Error') subplot(3,1,3),plot(0:M-1,SAVEb(1,:)/2,'LineWidth',2), grid, axis([0 M-1 0 2]) xlabel('n \rightarrow'), ylabel('\sigma^2_x[n] \rightarrow') title('Variance estimates') % MSE learning curve b = SAVEb(2:2+p-1,:); LC = zeros(1,M); for n = 1:M LC(n) = R(1,1) + b(:,n)'*R*b(:,n) - 2*b(:,n)'*r; % MSE, learning curve end FIG2 = figure('Name',['lab15_1 : Learning curve (',lms.Mode,')'],'NumberTitle','off'); plot(1:M-1,LC(1:M-1)/MSEopt,'LineWidth',2), grid axis([1 M 1 5]); xlabel('n \rightarrow'), ylabel('MSE[n] / MSEopt \rightarrow') title('Learning curve (MSE)') % 2nd order prediction error - MSE c0 = -2:.05:2; c1 = -2:.05:2; MSE = zeros(length(c0),length(c1)); for k=1:length(c0) for l=1:length(c1) cc = [c0(k); c1(l)] + b_opt; MSE(l,k) = R(1,1) + cc'*R*cc - 2*cc'*r; end end FIG3 = figure('Name',['lab15_1 : Realtiv MSE and convergence of coefficients c[n] (',lms.Mode,')'],... 'NumberTitle','off'); c = b - repmat(b_opt,1,M); V = [1 1.1 1.2 1.5 2 3 4 6]; [cc,h] = contour(c0,c1,MSE/MSEopt,V,'LineWidth',1,'Color','b'); grid clabel(cc); xlabel('c_0 \rightarrow'), ylabel('c_1 \rightarrow') title('Convergence of coefficients and MSE') hold on plot(c(1,:),c(2,:),'o','MarkerSize',4,'MarkerFaceColor','b') hold off
