Machine learning/Statistics Fundamentals

Point estimation is the process of calculating a statistic from a sample dataset.


 * A point estimator is any function for a sample.
 * Therefore, any statistic is a point estimator

Methods of finding point estimations


 * Method of Moments
 * Maximum Likelihood Estimators
 * Bayes Estimators
 * Maximum a posteriori estimator

Bayes Estimators
Using the Bayesian rule$$\begin{align} P(\theta|D) & = \frac{P(D|\theta)P(\theta)}{P(D)} \\ & = \frac{P(D|\theta)P(\theta)}{\int P(D|\theta) P(\theta) d\theta} \end{align}$$Where $$D = (X_1, X_2, \cdots, X_n)$$ is the sample dataset.

$$P(D|\theta) = \prod_{i=1}^{n} P(X_i|\theta)$$