Chapter 5

Question 1

What is the purpose of optimization?

Answer 1

To find the minimum of a scalar cost function

Question 2

What are the requirements for parameter estimation?

Answer 2

• Observations
• Mathematical Model for measurement
• Assumptions about the stochastic characteristics of the observations

Question 3

What are properties of estimators and what do they mean?

Answer 3

Bias: Deviation from real value

Covariance: Dispersion of estimator around expected value

Mean Square Error: Captures the effects of both Bias and covariance

Consistency

Asymp. Normality

Asymp. Efficiency

Question 4

What is consistency in estimators?

Answer 4

If an estimator converges in probability as the number of samples increases, it is called consistent. A consistent estimator is automatically unbiased

Question 5

What is Asymptotic normality?

Answer 5

“The more samples we use, the smaller the confidence intervals on the parameters will be, i.e. the ‘surer’ one can be about them

Question 6

What is asymptotic efficiency?

Answer 6

“An asymptotically efficient estimator makes the most of the available data, i.e. achieves the lowest possible parameter variances of all estimators”

Question 7

What is the basic idea of the maximum likelihood methods?

Answer 7

Adjust the parameters so the probability of obtaining a set of measurements is maximized

Question 8

What are the asymptotic properties of the maximum likelihood estimates?

Answer 8

1. Asymptotically consistent
2. Asymptotically Normal
3. Asymptotically Efficient

Question 9

What are the Three Estimation Models?

Answer 9

• Bayesisan
• Fisher
• Least Square

Question 10

For a multivariate cost function, what are the requirements for a vector to be a minimum?

Answer 10

1. First Derivative is 0
2. The Hessian is positive definite

Question 11

What is an estimator?

Answer 11

An estimator is a rule or method for calculating an estimate of a given quantity based on observed data.

Question 12

What are the parameters and residuals of the Fisher Model?

Answer 12

• \theta is a vector of unknown but constant parameters
• r is a random vector with probability density p(r) and covariance matrix Cov[r] = B

Question 13

What are the parameters and residuals of the Bayesian Model?

Answer 13

• \theta is a vector of random variables with probability density p(\theta)
• r is a random vector with probability density p(r) and covariance matrix Cov[r] = B

Question 14

What are the parameters and residuals of the Least-Square Model?

Answer 14

• \theta is a vector of unknown but constant
• r is a random vector of noise. No assumption is made about p(r)

Question 15

What is a good way of comparing two estimators?

Answer 15

Comparing the mean square error (MSE)

Question 16

How does the Bayesian Estimator work?

Answer 16

Prob. densities of _ and
residuals p(r) assumed to be known a priori, conditional prob. density is maximized p(_|z)

Question 17

How does the Fisher Estimator work?

Answer 17

Estimator based on concept of likelihood function; Prob. to obtain measurements Z, given a set of parameters _ is maximized,

Question 18

How does the Least Squares Estimator work?

Answer 18

(Best) estimate for parameters
is obtained from (weighted) sum of squares, no assumptions about prob. density of _ and residuals p(r) are made

Question 19

What is the Bayesian Rule?

Answer 19

[p(z|\theta)p(\theta)]/ p(z) = p(\theta|z)