QR factorization latex

%!TEX encoding = UTF-8 Unicode
% Author: Abhijith
\documentclass[a4paper,11pt]{article}
\usepackage[utf8]{inputenc}
\usepackage{tikz}
\usepackage{verbatim}
\usepackage[active,tightpage]{preview}
\PreviewEnvironment{tikzpicture}
\setlength\PreviewBorder{5pt}


\usetikzlibrary[positioning]
\usetikzlibrary{patterns}
\usepackage[french]{babel}

\begin{document}
\begin{tikzpicture}

\pgfmathsetmacro\x{3}    
\pgfmathsetmacro\y{11}
\filldraw[fill=blue!20] (1,1) rectangle +(\x,\y);
\draw[thin] (1,1) rectangle +(\x,\y);
\node[above] at (.5*\x + 1,\y + 1){$A$};
\draw [thick,|-] (0,12) -- (0,7);
\node[below] at (0,7){$m$};
\draw [thick,-|] (0,6.5) -- (0,1);
\draw [thick,|-] (1,0) -- (2.25,0);
\node[right] at (2.25,0){$n$};
\draw [thick,-|] (2.75,0) -- (4,0);

% X=5
\node[above] at (\x + 2,.5*\y +1){$\boldmath{=}$};

%X=6
\draw [thick,|-] (6,12) -- (6,7);
\node[below] at (6,7){$m$};
\draw [thick,-|] (6,6.5) -- (6,1);
%X=7
\filldraw[fill=blue!20] (7,1) rectangle +(\x,\y);
\draw[thin] (7,1) rectangle +(\x,\y);
\node[above] at (.5*\x + 7,\y + 1){$Q$};
\draw [thick,|-] (7,0) -- (7.9,0);
\node[right] at (7.9,0){$k\leq n$};
\draw [thick,->] (9.1,0) -- (10,0);

%X=11
\draw [thick,|-] (11,12) -- (11,10.8);
\node[below] at (11,10.8){$k\leq n$};
\draw [thick,->] (11,10.2) -- (11,9);

%X=12
\draw[thin] (12,9) rectangle +(\x,\x);
\node[above] at (.5*\x + 12,\y + 1 ){$R$};
\draw [thick,|-] (12,8) -- (13.25,8);
\node[right] at (13.25,8){$n$};
\draw [thick,-|] (13.75,8) -- (15,8);
\draw[fill=blue!20] (15,9) -- (15,12) -- (12,12) -- (15,9);


\end{tikzpicture}

\end{document}

Basic Matrix methods for Data Sciences

This is the plan for a small course I am taking, Basic Linear Algebra and Matrix techniques for data science. 
  
The following techniques will give you an ability to understand complex machine learning techniques better, it is the first step towards becoming a complete ML expert.We will be working hands on the below mentioned applications and many more while learning the math. You will be playing with the data, testing your creative imagination towards finding new dimensions and patterns within it.

1. Vectors and matrices in Data Mining and pattern recognition.

2. Linear regression techniques. Spaces, span, basis, dimensionality.

3. Orthogonality, Least Squares, QR decompostion, SVD

4. Data mining applications:

 4.1 Classification of Handwritten digits.
    4.1.1 How to classify Handwritten Digits
    4.1.2 Classification Using SVD Bases
    4.1.3 Tangent Distance

 4.2 Text Mining
    4.2.2 Preprocessing
    4.2.3 Vector Space Model
    4.2.4 Latent Semantic Indexing
    4.2.5 Clustering

 4.3 Building a recommender system for NETFLIX data. (from the scratch)

Here we have an opportunity to work with a great new open source language called JULIA.

   
 Will be adding few more applications based on machine learning techniques like
 PCA, CCA, Boosting techniques, Reinforcement learning, SVM etc
 depending on the groups interests.  Many more goodies on the cards.

We are supported and guided by experts, researchers and JULIA open source community.

Interested people kindly add your name and mailID and contact number in the below doc:

https://docs.google.com/spreadsheet/ccc?key=0AmTPAEM_gTnadEl1SFFDdlcwQnNvOTVnWG52NDR4T1E&usp=sharing

ALS for low rank matrix approximation

ALS is a way of solving an matrix equivalent of an regular linear least squares problem. In a regular least sqaures problem we solve for Ax=b, i.e., ||b - Ax|| In the case of ALS we try to solve ||R - UM || in frobenius norm. Now we try to reduce ||R - UM||, to the kind ||b - Ax||.

Example:
Let us consider the first row of R, which will have all the ratings given by the
fist user. Let us assume the user has seen movies 1, 213, 315, 1744, 2133, 7044 and
12344, so in a total of 7 movies. Forming a vector of the known ratings for the first
user we have [1 213 315 1744 2133 7044 12344] , let this be denoted by b.
We initialize the M matrix in a particular way, and it has n_m(total number of movies) rows and n_f(reduced factors) columns. The matrix U has n_u(total number of users) rows and n_f columns. For the case of the first user we can form a sub-matrix of M , denote this by M_u . This matrix M_u will have all the rows from M corresponding to the movies seen by the user u, the first user in this case. Let u be the row vector from U corresponding to the considered user, .i.e, first user. Now we have three components b,M_u and u, so we have a regular least squares problem for ||b - Mu||. This is for the first user, we do this for all the users, hence forming the U matrix. since this cant be the final U we need to alternatively continue to solve for U and M.

You could try to just do an SVD on R, and initialize M=S*V', and use this M in ALS, in this case you might not need many iterations as we dont initialize with random values in M. Any approximation method involving random initial values needs a certain minimum number of iterations to tend towards the right values, as we in each iteration minimize the error to a certain extent only. We cant reduce the whole error in just one iteration.