How do I learn machine learning?

How do I learn machine learning? by Xavier Amatriain

Answer by Xavier Amatriain:

(Spanish version of this answer here)

I didn't do a PhD on machine learning (was mostly focused on Signal Processing and Software Engineering) so I get this question a lot. The typical person that asks me this question is a software engineer with a computer science background, so I will address it from that perspective. If you are a Math major, for example, my answer might be less useful.

Take an online course

The first thing I tell someone who wants to get into machine learning is to take Andrew Ng's online course. I think Ng's course is very much to-the-point and very well organized, so it is a great introduction for someone wanting to get into ML. I am surprised when people tell me the course is "too basic" or "too superficial". If they tell me that I ask them to explain the difference between Logistic Regression and Linear Kernel SVMs, PCA vs. Matrix Factorization, regularization, or gradient descent. I have interviewed candidates who claimed years of ML experience that did not know the answer to these questions. They are all clearly explained in Ng's course. There are many other other online courses you can take after this one (see My answer to What is the best MOOC to get started in Machine Learning?) but at this point you are mostly ready to go to the next step.

Implement an algorithm

My recommended next step is the following. Get a good ML book (my list below), read the first intro chapters, and then jump to whatever chapter includes an algorithm you are interested. Once you have found that algo, dive into it, understand all the details, and, especially, implement it. In the previous online course you would already have implemented some algorithms in Octave. But, here I am talking about implementing an algorithm from scratch in a "real" programming language. You can still start with an easy one such as L2-regularized Logistic Regression, or k-means, but you should also push yourself to implement more interesting ones such as LDA (Latent Dirichlet Allocation) or SVMs. You can use a reference implementation in one of the many existing libraries to make sure you are getting comparable results, but ideally you don't want to look at the code but actually force yourself to implement it directly from the mathematical formulation in the book.

Some book recommendations

So, what are some good books to do this? Many have been mentioned before. Some of my favorite (see my answer to What are the best books about machine learning? for more details):

You can also go directly to a research paper that introduces an algorithm or approach you are interested on and dive into it.

My main point is that machine learning is both about breadth as depth. You are expected to know the basics of the most important algorithms (see my answer to What are the top 10 data mining or machine learning algorithms?). On the other hand, you are also expected to understand low-level complicated details of algorithms and their implementation details. I think the approach I am describing addresses both these dimensions and I have seen it work.

Ready for a career in Machine Learning?

The next logic step some people ask about is whether they should now be ready to start a career in machine learning. That is, of course, a different question. Please refer to amy answer to How should one start a career in machine learning? for that.

Edit 08/26/2015

In response to some comments and questions, I feel that I should add another book recommendation. If you feel like you lack some background in Statistics, I would totally recommend:

This books covers all the topics in statistics that you need for understanding ML concepts. As a matter of fact, the book itself has a pretty good introduction to many of the typical ML approaches such as regression and classification.

How do I learn machine learning?

Android Volley Tutorial

I have just started using it…and making an awesome app.. Wait for it..volley is really awesome and takes alot of things in its hand and lets developer focus on other stuff..And plus is the caching.

Android Research Blog

Volley is an android library released by Google that can make your life easier when dealing with network operations. In this blog post I will mention the main features of the library and show a few example usages, in particular, how to make a request, how to download images, and how to use the cache.

Features of Volley library

a) Automatically schedules the network requests

b) Supports request prioritization. This means that you can load content depending of priorities, for example the main content could have a high priority, but the images a low priority.

c) Provides transparent disk and memory cache that allows for quick reloading of data. Transparent cache means that the caller doesn’t have to know about the existence of the cache. That is, the cache is implemented automatically. You do, however, have the possibility to disable the caching.

d) Provides a good API for canceling requests…

View original post 488 more words

Video Conference Part 1: These Things Suck

Ben Garney

What I cannot create, I do not understand. – Richard Feynman

I do a lot of video chat for work. If it’s not a one on one, it’s pair programming. If it’s not pair programming, it’s a client meeting. I use a lot of Skype and Hangouts.

Sometimes they don’t work for unclear reasons. Sometimes file transfers fail. Sometimes screenshare breaks, or when it’s active you don’t get webcam, too. Or the connection lags or drops even though everything is running fast.

Every time I experience such a failure, I get really angry and think, “I could do this better!” But I never quite got angry enough… until now. I guess the weight of years of frustration finally got to me.

I wrote my own (prototype) video conferencing app. It turned out pretty well. And that’s what these posts are about.

Conventions & Caveats

We will be referencing a 640×480…

View original post 933 more words

Tips for beginning systems and software engineers

Ilya's blog


From time to time I’m toying with the idea to give a lecture to newcomers in the  IT industry (systems or software engineers). Here are some of the points that I would include in it:

Human factor


Software bugs are inevitable. Just accept it. Thinking that you will get any significant piece of code right the first time is statistically wrong. The correct assumption is that bugs will be there, various kinds of bugs. The answer to this problem is the correct development process. The process should include:

  1. Automated tests – minimize chance of bugs getting into the production environment. There will be bugs not covered by tests. Production environment usually differs in some ways from the testing environment. This means some bugs will make their way to the production environment so monitoring is needed. Automated tests help combat the following problems:
    1. Slower development because one becomes hesitant…

View original post 1,208 more words

The Little Array That Could

To Each His Own Array


Hello everyone, and welcome to my new blog! It couldn’t be a more opportune time than the middle of the summer to start up a new programming site! I would like to start off by saying that I am very excited to share some of my great experiences I’ve had in the few years that I’ve been programming, and start some interesting discussions with other bloggers! Now, this being my first post, I would also like to advise you to sit tight for this, because you’re in for a wild ride!

The Making of PowArray

Let me backtrack a bit and pick up just before from where I ended off on my about me page. So at the peak of last year’s fateful summer, I was at my desktop computer, frantically coding away at my keyboard in the blistering heat, determined to solve Project Euler #16 without any Big Integer class…

View original post 644 more words

PE #20

To Each His Own Array

Intro and Problem Statement

Hello again everybody, this is the second follow up to my post about my Arbitrary Precision Calculator, PowArray. In this post I will be revealing and discussing my solution to Project Euler problem #20 (the basis for my Factorial calculator in PowArray).

Problem #20 states the following:

Factorial digit sum

n! means n (n − 1) … 3 2 1

For example, 10! = 10 × 9 × … 3 2 1 = 3628800,
and the sum of the digits in the number 10! is 3 + 6 + 2 + 8 + 8 + 0 + 0 = 27.

Find the sum of the digits in the number 100!

A Foreword

I’ll limit the number of words in this post in favor of code, as the ideas are pretty similar and not worth reiterating verbatim.

Before jumping right into the solution, it is important to…

View original post 354 more words