Mathematics for Machine Learning

=Mathematics for Machine Learning=

Matrixes
Matrix algebra for stats by Tom P Minka Matrix cookback wikipedia article on matrix

Important Equality/Inequality
Probabilities Inequalities

Jensen's Inequality
Jensen's Inequality in Probability

Expectation of Indicator is its Probability
$$ \begin{matrix} Assertion: & P(X \in A) = & \mathbb{E}[ \mathbb{I}(X \in A)] \\ Proof: & rhs = & \mathbb{E}[\mathbb{I}(X \in A)] \\ \quad & \quad = & \mathbb {I}(X \in A)*P(X \in A) +  \mathbb{I}(X \notin A)*P(X \notin A) \\ \quad & \quad = & 1*P(X \in A)+0*P(X \notin A) \\ \quad & \quad = & P(X \in A) \\ \quad & \quad = & lhs \end{matrix} $$

Markov's Inequality
markov's inequality intuition

Chebyshev's Inequality
Chebyshev's inequility explanation

intuition

Cauchy-Schwarz inequality
cauchy schwarz inequality

Intuition of Cauchy-Schwarz inequality

Hoeffding's Inequality
Hoeffding's Inequality formal treatment

Intuition of Hoeffding Inequality

Hoeffding's inequality is useful to analyse the number of required samples needed to obtain a confidence interval

Law of Large Numbers
Law of large numbers

Law of Total (Iterater|Tower) Expectation and Variance
Law of iterated expectation and Variance

Convergence of Polynomial

 * 1) Note that we will need this for learning the statistical theory of ML. Watch section VII  Nice lecture from Herbert Gross on the polynomial and it's convergence

Polynomial Approximation
Here is the nice explanation of the polynomial approximations. conditions for successful polynomial approx (note not all polynomial converges to f(x) )

It covers
 * 1) Does $$lim_{n\to\infty} P_{n}(x) $$ exists?
 * 2) is $$f(x) = P_{n}(x)$$ true?
 * 3) In expression. $$P(x) = lim_{n\to\infty} P_{n}(x)$$ does lhs has the same polynomial property as the rhs?

the test for convergence
 * 1) ratio test
 * 2) Integral test for convergence
 * 3) Comparison test

Uniform Convergence
uniform convergence tutorial . The basic idea is that $$lim_{x\to x_{k}} [lim_{n\to\infty} f_{n}(x_{k})] = lim_{n\to\infty} f_{n}(x_{k}) = lim_{n\to\infty} [ lim_{x\to x_{k}} f_{n}(x_{k})] $$

Uniform Convergence for the Power Series
the application of the power series in evaluating the various functions

the above covers 1. Series is the limit of the partial sum. 2. a series is uniformly convergent means the sequence of partial sum converges uniformly to the series 3. Weierstrauss M-test

Various Visualization throughout history
Visualization especially multi-dimensional data is very important for the intuition of the algorithms and general understanding. Tutorial and history of various visualization core ideas Slides for various visualizations

More common visualization
More common used multidimensional visualizaion

(Fourier) Andrew's Curve
Nice introduction and examples

an example in PCA analysis

another example with data

Parallel Coordinate Plot
Parallel coordinates uses and applications

Radar(Star) Plot
Radar plot uses and applications