ISBN: 3540652574
TITLE: Parameter Estimation and Hypothesis Testing in Linear Models
AUTHOR: Koch, Karl-Rudolf
TOC:

Introduction 1
1 Vector and Matrix Algebra 3
1.1 Sets and Fields 3
1.1.1 Notion of Sets 3
1.1.2 Composition of Sets 4
1.1.3 Relations 4
1.1.4 Field of Real Numbers 5
1.2 Vector Algebra 6
1.2.1 Definition of Vectors and Vector Space 6
1.2.2 Linear Dependence and Basis of a Vector Space 8
1.2.3 Inner Product and Euclidian Space 11
1.2.4 Orthogonal Subspaces 11
1.3 Matrices 13
1.3.1 Matrix Definition and Matrix Composition 13
1.3.2 Rank of a Matrix 19
1.3.3 Computation of Inverse Matrices 23
l.3.4 Matrix Identities 33
1.3.5 Column Space and Null Space of a Matrix 34
1.3.6 Determinants 36
1.3.7 Trace of a Matrix and Vector Representation 40
1.4 Quadratic Forms 41
1.4.1 Transformations 41
1.4.2 Eigenvalues and Eigenvectors 44
1.4.3 Definite Matrices 46
1.5 Generalized Inverses 48
1.5.1 Right Inverse and Left Inverse 48
1.5.2 Idempotent Matrices 48
1.5.3 Generalized Inverse, Reflexive Generalized Inverse and Pseudoinverse 
50
1.5.4 Systems of Linear Equations 54
1.5.5 Generalized Inverses of Symmetrical Matrices 57
1.5.6 Properties of the Pseudoinverse and of a Special Symmetrical 
Reflexive Generalized Inverse 62
1.6 Projections 64
1.6.1 General Projections 64
1.6.2 Orthogonal Projections 65
1.7 Differentiation and Integration of Vectors and Matrices 66
1.7.1 Extrema of Functions 66
1.7.2 Derivatives of Special Functions 68
1.7.3 Integration and Transformation of Variables 71
2 Probability Theory 75
2.1 Probability 75
2.1.1 Introduction 75
2.1.2 Random Events 77
2.1.3 Axioms of Probability 78
2.1.4 Conditional Probability and Bayes Formula 79
2.1.5 Independent Events 81
2.2 Random Variable 81
2.2.1 Definition 81
2.2.2 Distribution Function 82
2.2.3 Discrete and Continuous Random Variable 83
2.2.4 Binomial Distribution and Poisson Distribution 86
2.2.5 Multimensional Continuous Random Variable 88
2.2.6 Marginal Distribution 90
2.2.7 Conditional Distribution 90
2.2.8 Independent Random Variables 92
2.2.9 Transformation of Variables 93
2.3 Expected Values and Moments of Random Variables 93
2.3.1 Expectation 93
2.3.2 Multivariate Moments 96
2.3.3 Covariance Matrix and Covariances of Random Vectors 99
2.3.4 Moment Generating Function 106
2.4 Univariate Distributions 107
2.4.1 Normal Distribution 107
2.4.2 Derivation of the Normal Distribution as Distribution of 
Observational Errors 111
2.4.3 Gamma Distribution 112
2.4.4 Derivation of the Gamma Distribution as Waiting-Time Distribution 114
2.4.5 Beta Distribution 115
2.5 Multivariate Normal Distribution 117
2.5.1 Definition and Derivation 117
2.5.2 Moment Generating Function for the Normal Distribution 118
2.5.3 Marginal Distribution and Conditional Distribution 120
2.5.4 Independence of Normally Distributed Random Variables 122
2.5.5 Linear Functions of Normally Distributed Random Variables 122
2.5.6 Sum of Normally Distributed Random Variables 123
2.6 Test Distributions for Univariate Models 124
2.6.1 chi^2-Distribution 124
2.6.2 Non-Central chi^2-Distribution 126
2.6.3 F-Distribution 128
2.6.4 Non-Central F-Distribution 130
2.6.5 t-Distribution 132
2.7 Quadratic Forms 133
2.7.1 Expected Value and Covariance 133
2.7.2 Distribution of the Quadratic Form 135
2.7.3 Independence of Two Quadratic Forms 136
2.7.4 Independence of a Linear and a Quadratic Form 136
2.8 Test Distributions for Multivariate Models 137
2.8.1 Wishart Distribution 137
2.8.2 Derivation of the Wishart Distribution 137
2.8.3 Distribution of the Sum of Wishart Matrices 140
2.8.4 Distribution of the Transformed Wishart Matrix 140
2.8.5 Distribution of Matrices of Quadratic Forms and Independence of Wishart Matrices l41
2.8.6 Distribution of the Ratio of the Determinants of Two Wishart Matrices 143
2.8.7 Distribution of Special Functions of Wishart Matrices 146
3 Parameter Estimation in Linear Models 149
3.1 Methods of Estimating Parameters 150
3.1.1 Point Estimation 150
3.1.2 Best Unbiased Estimation 150
3.1.3 Method of Least Squares 152
3.1.4 Maximum-Likelihood Method 153
3.2 Gauss-Markoff Model 153
3.2.1 Definition and Linearization 153
3.2.2 Best Linear Unbiased Estimation 156
3.2.3 Method of Least Squares 158
3.2.4 Maximum-Likelihood Method 161
3.2.5 Unbiased Estimation of the Variance of Unit Weight 162
3.2.6 Numerical Computation of the Estimates and Their Covariances 165
3.2.7 Gauss-Markoff Model with Constraints 170
3.2.8 Recursive Parameter Estimation 177
3.2.9 Deviations From the Model 178
3.3 Gauss-Markoff Model not of Full Rank 181
3.3.1 Method of Least Squares and Maximum Likelihood Method 181
3.3.2 Estimable Functions 183
3.3.3 Projected Parameters as Estimable Functions 185
3.3.4 Gauss-Markoff Model not of Full Rank with Constraints 193
3.4 Special Gauss-Markoff Models 197
3.4.1 Polynomial Model 197
3.4.2 Analysis of Variance 200
3.4.3 Parameter Estimation for the Analysis of Variance by a Symmetrical 
Reflexive Generalised Inverse 204
3.4.4 Analysis of Covariance 207
3.4.5 Gauss-Markoff Model for Outliers in the Observations 208
3.5 Generalized Linear Models 210
3.5.1 Regression Model 210
3.5.2 Mixed Model 212
3.5.3 Best Linear Unbiased Estimation in the Mixed Model 214
3.5.4 Method of Least Squares and Maximum-Likelihood Method for the Mixed 
Model 216
3.5.5 Model of the Adjustment with Condition Equations 220
3.5.6 Prediction and Filtering 221
3.6 Estimation of Variance and Covariance Components 225
3.6.1 Best Invariant Quadratic Unbiased Estimation 225
3.6.2 Locally Best Estimation 229
3.6.3 Iterated Estimates 233
3.6.4 Best Unbiased Estimation of the Variance of Unit Weight 237
3.7 Multivariate Parameter Estimation 238
3.7.1 Multivariate Gauss-Markoff Model 238
3.7.2 Estimation of the Vectors of Parameters 240
3.7.3 Estimation of the Covariance Matrix 241
3.7.4 Numerical Computation of the Estimates and Incomplete Multivariate 
Models 246
3.7.5 Special Model for Estimating Covariance Matrices and Estimation of 
Covariances for Stochastic Processes 250
3.7.6 Multivariate Model with Constraints 253
3.8 Robust Parameter Estimation 255
3.8.1 Choice of the Score Function 255
3.8.2 Robust M-Estimation 255
3.8.3 M-Estimation of Huber 258
3.8.4 L_p-Norm Estimation 261
3.8.5 Leverage Points 263
3.8.6 Modified M-Estimation of Huber 265
3.8.7 Method of Rousseeuw 268
4 Hypothesis Testing, Interval Estimation and Test for Outliers 271
4.1 Distributions Based on Normally Distributed Observations 272
4.1.1 Distributions of Functions of the Residuals in the Univariate Model 
272
END
