Machine Learning Blueprint Newsletter, Edition 3, 10/1/17

One of the most common criticisms of Deep Learning is what’s going on in the hidden layers can seem opaque when compared to more prosaic algorithms like decision trees. New work from Naftali Tishby has been touted as an answer to this criticism.

COMMENTS

This work has seen a lot of attention this week in several articles. To be clear, headlines notwithstanding, Tishby’s results do not solve the “black box” opacity criticism. At first read, the thesis that neural nets succeed by constructing an “information bottleneck” as the data passes through the network, seems almost intuitive. Deep learning, in a very direct way, is just representation learning. Instead of feature engineering by hand, the hidden layers let the data tell them how to transform and manipulate the raw inputs for optimal performance. Of course, the devil is in the details. Tishby’s theorem demands a solid information theoretical background, but the case he makes is as fascinating as it is complex, and deep learning research could benefit from further investigation.

ImageNet has been the benchmark for testing image recognition for the better part of a decade. The Matterport data set has the potential of playing this role for the decade to come.

COMMENTS

With attention to self driving continuing to accelerate, interest in deep learning models that can recognise depth is growing. Benchmark data sets are a useful asset when working on machine learning models because they help us know if our work represents an bonafide advance in the field. As with ImageNet or the Netflix prize dataset, creating a public challenge with a sizable bounty could solidify Matterport’s dataset as one of these benchmark sets.

A quick rundown of some of the essential skills and preparation that recent Data Science or Machine Learning grads will want to have as they enter the industry.

DataCamp has put together a comprehensive set of Machine Learning tools cheat sheets in a single place. This article is an essential bookmark for any practitioner to keep handy.

This paper reports new findings supporting the hypothesis that local minima in the error surface of a Deep NeuralNet tend to cluster together. The findings further present results that, in specialized cases, such minima also will tend to be equivalent in minimizing the error rate (meaning convergence at one local minima may be as good as the next).

A trend has been building in the research community towards finding ways to automate not just the feature representation (using primarily deep layers in neural nets), but also learning how to construct and apply machine learning algorithims themselves. This paper leverages evolutionary strategies specifically for convo-net architectures.

A review article of the various techniques used for music generation with deep learning (primarily recurrent neural network) algorithms. A good entry point to acquaint oneself with the field.