Revision #1398 → #1774 · back to history
addedOLS maximizes the likelihood under Gaussian errors24b95fc0f294
modifiedLikelihood functionbe3fea203cf0
| Field | From #1398 | To #1774 |
|---|
| anchors | [{"section":"Principles","snippet":"which is called the likelihood function"},{"type":"math_alttext","value":"{\\displaystyle {\\mathcal {L}}_{n}(\\theta )={\\mathcal {L}}_{n}(\\theta ;\\mathbf {y} )=f_{n}(\\mathbf {y} ;\\theta )\\;,}"},{"type":"math_alttext","value":"{\\displaystyle f_{n}(\\mathbf {y} ;\\theta )=\\prod _{k=1}^{n}\\,f_{k}^{\\mathsf {univar}}(y_{k};\\theta )~.}"}] | — |
modifiedLikelihood factorizes for independent variables6cfda0ee4e3b
| Field | From #1398 | To #1774 |
|---|
| anchors | [{"section":"Principles","snippet":"will be the product of univariate density functions"},{"type":"math_alttext","value":"{\\displaystyle {\\mathcal {L}}_{n}(\\theta )={\\mathcal {L}}_{n}(\\theta ;\\mathbf {y} )=f_{n}(\\mathbf {y} ;\\theta )\\;,}"},{"type":"math_alttext","value":"{\\displaystyle f_{n}(\\mathbf {y} ;\\theta )=\\prod _{k=1}^{n}\\,f_{k}^{\\mathsf {univar}}(y_{k};\\theta )~.}"}] | — |
modifiedMLE as maximizer of likelihoodbf24d9f3d3ea
| Field | From #1398 | To #1774 |
|---|
| anchors | [{"section":"Principles","snippet":"to find the values of the model parameters that maximize the likelihood function over the parameter space"},{"type":"math_alttext","value":"{\\displaystyle {\\hat {\\theta }}={\\underset {\\theta \\in \\Theta }{\\operatorname {arg\\;max} }}\\,{\\mathcal {L}}_{n}(\\theta \\,;\\mathbf {y} )~.}"}] | — |
modifiedLog-likelihood2119eb3c9fe1
| Field | From #1398 | To #1774 |
|---|
| anchors | [{"section":"Principles","snippet":"called the log-likelihood"},{"type":"math_alttext","value":"{\\displaystyle \\ell (\\theta \\,;\\mathbf {y} )=\\ln {\\mathcal {L}}_{n}(\\theta \\,;\\mathbf {y} )~.}"},{"type":"math_alttext","value":"{\\displaystyle {\\frac {\\partial \\ell }{\\partial \\theta _{1}}}=0,\\quad {\\frac {\\partial \\ell }{\\partial \\theta _{2}}}=0,\\quad \\ldots ,\\quad {\\frac {\\partial \\ell }{\\partial \\theta _{k}}}=0~,}"}] | — |
addedLog preserves the argmax of the likelihoodf33416298a84
modifiedLikelihood equationse5bf66cce0ad
| Field | From #1398 | To #1774 |
|---|
| anchors | [{"section":"Principles","snippet":"known as the likelihood equations"},{"type":"math_alttext","value":"{\\displaystyle \\ell (\\theta \\,;\\mathbf {y} )=\\ln {\\mathcal {L}}_{n}(\\theta \\,;\\mathbf {y} )~.}"},{"type":"math_alttext","value":"{\\displaystyle {\\frac {\\partial \\ell }{\\partial \\theta _{1}}}=0,\\quad {\\frac {\\partial \\ell }{\\partial \\theta _{2}}}=0,\\quad \\ldots ,\\quad {\\frac {\\partial \\ell }{\\partial \\theta _{k}}}=0~,}"}] | — |
modifiedRestricted likelihood equations via Lagranged479ddb5ca6b
| Field | From #1398 | To #1774 |
|---|
| anchors | [{"section":"Restricted parameter space","snippet":"leads to the restricted likelihood equations"},{"type":"math_alttext","value":"{\\displaystyle {\\frac {\\partial \\ell }{\\partial \\theta }}-{\\frac {\\partial h(\\theta )^{\\mathsf {T}}}{\\partial \\theta }}\\lambda =0}"}] | — |
modifiedMLE as extremum estimatora5103e50ca36
| Field | From #1398 | To #1774 |
|---|
| anchors | [{"section":"Properties","snippet":"is an extremum estimator obtained by maximizing"},{"type":"math_alttext","value":"{\\displaystyle {\\widehat {\\ell \\,}}(\\theta \\,;x)=\\sum _{i=1}^{n}\\ln f(x_{i}\\mid \\theta ),}"}] | — |
modifiedIdentification condition45808718085a
| Field | From #1398 | To #1774 |
|---|
| anchors | [{"section":"Consistency","snippet":"different parameter values θ correspond to different distributions within the model"},{"type":"math_alttext","value":"{\\displaystyle \\theta \\neq \\theta _{0}\\quad \\Leftrightarrow \\quad f(\\cdot \\mid \\theta )\\neq f(\\cdot \\mid \\theta _{0}).}"}] | — |
modifiedContinuity condition9a6cd8c7ddc7
| Field | From #1398 | To #1774 |
|---|
| anchors | [{"section":"Consistency","snippet":"is continuous in θ for almost all values of x"},{"type":"math_alttext","value":"{\\displaystyle \\operatorname {\\mathbb {P} } {\\Bigl [}\\;\\ln f(x\\mid \\theta )\\;\\in \\;C^{0}(\\Theta )\\;{\\Bigr ]}=1.}"}] | — |
modifiedDominance condition2cf29dd57a9a
| Field | From #1398 | To #1774 |
|---|
| anchors | [{"section":"Consistency","snippet":"there exists D ( x ) integrable with respect to the distribution"},{"type":"math_alttext","value":"{\\displaystyle {\\Bigl |}\\ln f(x\\mid \\theta ){\\Bigr |}<D(x)\\quad {\\text{ for all }}\\theta \\in \\Theta .}"},{"type":"math_alttext","value":"{\\displaystyle \\sup _{\\theta \\in \\Theta }\\left|{\\widehat {\\ell \\,}}(\\theta \\mid x)-\\ell (\\theta )\\,\\right|\\ \\xrightarrow {\\text{p}} \\ 0.}"}] | — |
modifiedAsymptotic normality of the MLE411f0edbed1b
| Field | From #1398 | To #1774 |
|---|
| anchors | [{"section":"Consistency","snippet":"the maximum likelihood estimator converges in distribution to a normal distribution"},{"type":"math_alttext","value":"{\\displaystyle {\\sqrt {n}}\\left({\\widehat {\\theta \\,}}_{\\mathrm {mle} }-\\theta _{0}\\right)\\ \\xrightarrow {d} \\ {\\mathcal {N}}\\left(0,\\,I^{-1}\\right)}"}] | — |
modifiedBias-corrected maximum likelihood estimatoref627a2da7c7
| Field | From #1398 | To #1774 |
|---|
| anchors | [{"section":"Second-order efficiency after correction for bias","snippet":"is called the bias-corrected maximum likelihood estimator"},{"type":"math_alttext","value":"{\\displaystyle {\\widehat {\\theta \\,}}_{\\text{mle}}^{*}={\\widehat {\\theta \\,}}_{\\text{mle}}-{\\widehat {b\\,}}~.}"}] | — |
modifiedBayes rule under zero-one loss48a61fe7ef26
| Field | From #1398 | To #1774 |
|---|
| anchors | [{"section":"Application of maximum-likelihood estimation in Bayes decision theory","snippet":"the Bayes Decision rule can be reformulated as"},{"type":"math_alttext","value":"{\\displaystyle \\operatorname {\\mathbb {P} } (w_{i}\\mid x)={\\frac {\\operatorname {\\mathbb {P} } (x\\mid w_{i})\\operatorname {\\mathbb {P} } (w_{i})}{\\operatorname {\\mathbb {P} } (x)}},}"},{"type":"math_alttext","value":"{\\displaystyle h_{\\text{Bayes}}={\\underset {w}{\\operatorname {arg\\;max} }}\\,{\\bigl [}\\,\\operatorname {\\mathbb {P} } (x\\mid w)\\,\\operatorname {\\mathbb {P} } (w)\\,{\\bigr ]}\\;,}"}] | — |
modifiedLikelihood equations not solvable explicitly354db22f05ff
| Field | From #1398 | To #1774 |
|---|
| anchors | [{"section":"Iterative procedures","snippet":"cannot be solved explicitly for an estimator"},{"type":"math_alttext","value":"{\\displaystyle {\\widehat {\\theta }}_{r+1}={\\widehat {\\theta }}_{r}+\\eta _{r}\\mathbf {d} _{r}\\left({\\widehat {\\theta }}\\right)}"}] | — |
modifiedIterative updating formula89ff0c6230ab
| Field | From #1398 | To #1774 |
|---|
| anchors | [{"section":"Iterative procedures","snippet":"algorithms based on an updating formula of the form"},{"type":"math_alttext","value":"{\\displaystyle {\\widehat {\\theta }}_{r+1}={\\widehat {\\theta }}_{r}+\\eta _{r}\\mathbf {d} _{r}\\left({\\widehat {\\theta }}\\right)}"}] | — |
modifiedDavidon–Fletcher–Powell formulaaa8d46f60092
| Field | From #1398 | To #1774 |
|---|
| anchors | [{"section":"Davidon–Fletcher–Powell formula","snippet":"DFP formula finds a solution that is symmetric, positive-definite and closest to the current approximate value of second-order derivative"},{"type":"math_alttext","value":"{\\displaystyle \\mathbf {H} _{k+1}=\\left(I-\\gamma _{k}y_{k}s_{k}^{\\mathsf {T}}\\right)\\mathbf {H} _{k}\\left(I-\\gamma _{k}s_{k}y_{k}^{\\mathsf {T}}\\right)+\\gamma _{k}y_{k}y_{k}^{\\mathsf {T}},}"}] | — |