Similarity transformation

  This post throws some light on the similarity tranformations and its applications.
 'An eigenvector defines a one-dimensional subspace that is invariant with respect to premultiplication by A.' (1)
Lets first see what is the formal definition of similarity transformation.

QR factorization latex

%!TEX encoding = UTF-8 Unicode
% Author: Abhijith
\documentclass[a4paper,11pt]{article}
\usepackage[utf8]{inputenc}
\usepackage{tikz}
\usepackage{verbatim}
\usepackage[active,tightpage]{preview}
\PreviewEnvironment{tikzpicture}
\setlength\PreviewBorder{5pt}


\usetikzlibrary[positioning]
\usetikzlibrary{patterns}
\usepackage[french]{babel}

\begin{document}
\begin{tikzpicture}

\pgfmathsetmacro\x{3}    
\pgfmathsetmacro\y{11}
\filldraw[fill=blue!20] (1,1) rectangle +(\x,\y);
\draw[thin] (1,1) rectangle +(\x,\y);
\node[above] at (.5*\x + 1,\y + 1){$A$};
\draw [thick,|-] (0,12) -- (0,7);
\node[below] at (0,7){$m$};
\draw [thick,-|] (0,6.5) -- (0,1);
\draw [thick,|-] (1,0) -- (2.25,0);
\node[right] at (2.25,0){$n$};
\draw [thick,-|] (2.75,0) -- (4,0);

% X=5
\node[above] at (\x + 2,.5*\y +1){$\boldmath{=}$};

%X=6
\draw [thick,|-] (6,12) -- (6,7);
\node[below] at (6,7){$m$};
\draw [thick,-|] (6,6.5) -- (6,1);
%X=7
\filldraw[fill=blue!20] (7,1) rectangle +(\x,\y);
\draw[thin] (7,1) rectangle +(\x,\y);
\node[above] at (.5*\x + 7,\y + 1){$Q$};
\draw [thick,|-] (7,0) -- (7.9,0);
\node[right] at (7.9,0){$k\leq n$};
\draw [thick,->] (9.1,0) -- (10,0);

%X=11
\draw [thick,|-] (11,12) -- (11,10.8);
\node[below] at (11,10.8){$k\leq n$};
\draw [thick,->] (11,10.2) -- (11,9);

%X=12
\draw[thin] (12,9) rectangle +(\x,\x);
\node[above] at (.5*\x + 12,\y + 1 ){$R$};
\draw [thick,|-] (12,8) -- (13.25,8);
\node[right] at (13.25,8){$n$};
\draw [thick,-|] (13.75,8) -- (15,8);
\draw[fill=blue!20] (15,9) -- (15,12) -- (12,12) -- (15,9);


\end{tikzpicture}

\end{document}

Basic Matrix methods for Data Sciences

This is the plan for a small course I am taking, Basic Linear Algebra and Matrix techniques for data science. 
  
The following techniques will give you an ability to understand complex machine learning techniques better, it is the first step towards becoming a complete ML expert.We will be working hands on the below mentioned applications and many more while learning the math. You will be playing with the data, testing your creative imagination towards finding new dimensions and patterns within it.

1. Vectors and matrices in Data Mining and pattern recognition.

2. Linear regression techniques. Spaces, span, basis, dimensionality.

3. Orthogonality, Least Squares, QR decompostion, SVD

4. Data mining applications:

 4.1 Classification of Handwritten digits.
    4.1.1 How to classify Handwritten Digits
    4.1.2 Classification Using SVD Bases
    4.1.3 Tangent Distance

 4.2 Text Mining
    4.2.2 Preprocessing
    4.2.3 Vector Space Model
    4.2.4 Latent Semantic Indexing
    4.2.5 Clustering

 4.3 Building a recommender system for NETFLIX data. (from the scratch)

Here we have an opportunity to work with a great new open source language called JULIA.

   
 Will be adding few more applications based on machine learning techniques like
 PCA, CCA, Boosting techniques, Reinforcement learning, SVM etc
 depending on the groups interests.  Many more goodies on the cards.

We are supported and guided by experts, researchers and JULIA open source community.

Interested people kindly add your name and mailID and contact number in the below doc:

https://docs.google.com/spreadsheet/ccc?key=0AmTPAEM_gTnadEl1SFFDdlcwQnNvOTVnWG52NDR4T1E&usp=sharing

ALS for low rank matrix approximation

ALS is a way of solving an matrix equivalent of an regular linear least squares problem. In a regular least sqaures problem we solve for Ax=b, i.e., ||b - Ax|| In the case of ALS we try to solve ||R - UM || in frobenius norm. Now we try to reduce ||R - UM||, to the kind ||b - Ax||.

Example:
Let us consider the first row of R, which will have all the ratings given by the
fist user. Let us assume the user has seen movies 1, 213, 315, 1744, 2133, 7044 and
12344, so in a total of 7 movies. Forming a vector of the known ratings for the first
user we have [1 213 315 1744 2133 7044 12344] , let this be denoted by b.
We initialize the M matrix in a particular way, and it has n_m(total number of movies) rows and n_f(reduced factors) columns. The matrix U has n_u(total number of users) rows and n_f columns. For the case of the first user we can form a sub-matrix of M , denote this by M_u . This matrix M_u will have all the rows from M corresponding to the movies seen by the user u, the first user in this case. Let u be the row vector from U corresponding to the considered user, .i.e, first user. Now we have three components b,M_u and u, so we have a regular least squares problem for ||b - Mu||. This is for the first user, we do this for all the users, hence forming the U matrix. since this cant be the final U we need to alternatively continue to solve for U and M.

You could try to just do an SVD on R, and initialize M=S*V', and use this M in ALS, in this case you might not need many iterations as we dont initialize with random values in M. Any approximation method involving random initial values needs a certain minimum number of iterations to tend towards the right values, as we in each iteration minimize the error to a certain extent only. We cant reduce the whole error in just one iteration.

Tutorial on text mining - Introduction Part 1

Well here we would be looking into 'How does textmining work'. As most peaple say textmining is analogous to finding a needle(your search keywords) in a haystack(world wide web). I have followed the book Matrix Methods in Data Mining and Pattern Recognition by Prof. Lars Eldén. The concepts in this tutorial is mainly based on the above book. The problems in datamining can be approached in either statistical or matrix factorization approach. I was fortunate to take the course under Lars, he is the most inspiring teacher I have had.
    Here we would be dealing with the matrix factorization technique. The task of retrieving a relevant tiny bit (relative to the entire domain) of information from large data which might be unstructured is a tricky one. These kind of tasks are also related to Information retrieval. Apart from the popular search engines like google, bing, yahoo.. IR also plays its role in the field of medicine. The medical community produces research articles pertaining to biomedical research at a prolific rate resulting in a very large knowledge bank. With the growth of such important data, the needs of the medical doctors and the like to extract relevant information had triggered research in test mining particularly for the biomedical knowledge domain. With the biomedical text mining there is a big difference with the normal World Wide Web search in that it includes semantics and natural language understanding. I would cover the NLP aspects in the Part 2 on text mining.
         Before going ahead I would mention the topics we would be learning here.
Preprocessing of the documents : Analyzing and Purification of documents, Indexing, Inverted Indexing, Item Normalization, Stop words, Stemming, Text parser.
Vector Space Model : TDM - Term Document Matrix, term weighting scheme, Sparse matrix storage, Query matching - cosine distance measure, Performance modeling - tolerance, Precision, Recall.
Latent Semantic Indexing : Matrix factorization, Low rank approximation, Singular Value Decomposition.
Clustering, NMF - Nonnegative Matrix Factorization, LGK Bidiagonalization.

Will be back!! 

A Cross Product Formula \[\mathbf{V}_1 \times \mathbf{V}_2 = \begin{vmatrix} \mathbf{i} & \mathbf{j} & \mathbf{k} \\ \frac{\partial X}{\partial u} & \frac{\partial Y}{\partial u} & 0 \\ \frac{\partial X}{\partial v} & \frac{\partial Y}{\partial v} & 0 \end{vmatrix} \]

Apache Mahout - Befitting of the name!!

Well, it was 7 days back that I posted my first elementary question to the Mahout user group, and today I was able to successfully test my first program. Being a first timer in (Java+Eclipse+Maven+Mahout) = JEMM, it me almost a month to get things running. I would like to thank kuba pawloch@interia.pl, Sean Owen and Julian Ortega for your support. 
  Well I dont think there would have been anyone as ignorant as myself with JEMM, as I could'nt find queries as basic as mine. But anyways now I have moved on, and hope to use Mahout on a more serious note. However if there are/will be anyone who has similar issues, I would like to simply share the issues I had with getting mahout running.

my experiments with NETFLIX problem.

Evolution has been the main defining quality in us humans right from the start. The doctrine of change being central to the universe given by Heraclitus, we can observe temporal dynamics in almost everything. The conception of 'Tool' during the stone age compared to that of today reflects the advancements we have suceeded in.

 My interest lies in the playing a small role towards modernizing The WEB aided with machine learning algorithms and of course computer/internet technologies. With the Web getting smarter and more proximate with us, I would love to be a part of Connected World.

 
This is the latest project I have been working on. The NETFLIX problem seems to have attracted a great of attention from data scientists world over. This is one problem which I have enjoyed solving the most. Berkant Savas has been supervising me who is great fun to work with. Initially my attempt was to model the temopral aspects involved in the NETFLIX data. The main article I was refering to is "Collaborative Filtering with Temporal Dynamics" by Yehuda Koren. Yehuda Koren was part of the team BellKor's Pragmatic Chaos which won the NETFLIX prize in September 2009.

 The concept of evolution applied to 'the way people concieve movies' is what I would like to emphasize and capitalize on.