1. Suppose that a list contains the values 20 44 48 55 62 66 74 88 93 99 at index positions 0 through 9. Trace the values of the variables….
Prove that the maximum likelihood estimator of the variance of a Gaussian variable is biased.
1. Consider a general optimization problem of the form max
2. Prove that the maximum likelihood estimator of the variance of a Gaussian variable is biased.
3. Regularization for Maximum Likelihood: Consider the following regularized loss minimization: 1 m _m i=1 log(1/ θ [xi ])+ 1 m _ log(1/θ)+log(1/(1−θ)) _
. _ Show that the preceding objective is equivalent to the usual empirical error had we added two pseudoexamples to the training set. Conclude that the regularized maximum likelihood estimator would be
. _ Derive a high probability bound on |ˆθ −θ_|. Hint: Rewrite this as |ˆθ −E[ˆθ ]+ E[ˆθ ]−θ_| and then use the triangle inequality and Hoeffding inequality. _ Use this to bound the true risk. Hint: Use the fact that now ˆθ ≥ 1 m+2 to relate