Global AI and Data Science

Wed June 19, 2019 03:48 PM

Michael Tamir

This newsletter is written and curated by Mike Tamir and Mike Mansour.

May 6, 2018

Hi all,

Hope you enjoy this week's ML Blueprint. This week is brought to you by fastdata.io.

Spotlight Articles

Watching ML-Algs Argue with One Another Might Teach Us About They Behave

Open.ai discusses a new “ML debate” training technique, in which two algorithms compete to reveal MNIST pixels one at a time in order to convince the “judge” (a MNIST trained classifier) that the image is a certain digit (or not).

Machine Learning Blueprint's Take

This article optimistically expands the scope of this three way algorithm competition architecture, suggesting a (stretched -- to put it mildly) comparison with AI’s arguing on points of fact with one another to convince a human judge. It is difficult to speculate exactly how the comparison of AI’s generating points of fact is similar to revealing pixels. (Perhaps through vector embedding the points of fact?) The analogy is a stretch, the use case presupposes a host of NLU and “point of fact” priors technology that is not yet developed. However, it is a very creative experiment and even the inching towards the targeted use case should be applauded. It is also possible that as with the inception of GANs generating a multitude of unanticipated applications over the years, there is a lot of potential for 3 way competing algorithm architectures to be used for new and creative present day use cases.

[Link]

Google Releases an Autonomous Agent, Duplex, that Completes Real Tasks with Human-Interaction

Duplex has been making waves after the I/O conference with their videos of an agent making reservations over the phone for restaurants and haircuts with an almost erie human-like pattern of speaking that includes conversational ticks and colloquialisms (like “hmm”s & “Umm”s). They’re able to handle difficult speech transactions by limiting the training data and applications to very limited scopes - this model would not be able to carry out other conversations. It’s using a combination of RNN’s, WaveNet/TacoTron, Speech-to-Text and TF-Extended to pull it all off.

Machine Learning Blueprint's Take

Google is walking down Facebook’s path of the discontinued assistant, M. Similarly to M, Duplex realizes when its failing and signals for human intervention. This drove costs up for M and ultimately ended it. To deploy a model in a new environment, they’re using a heavy human-in-the-loop process to actively coach an agent. The use cases for businesses at the end are certainly worth considering - they’re not glorious, but rather allow easier access to information and primitive tasks.

[Link]

A message from our sponsors...

Now witness the real-time power of this fully GPU-armed and operational stream processing engine.

FDIO Engine™ can process Spark Structured Streaming workloads 100 to 1000 times faster than Apache Spark, as it runs natively on NVIDIA GPUs and Apache Arrow.

To put 1000x performance in perspective, FDIO processes 1 Terabyte on a single AWS instance in 35 seconds, where Spark takes over 9 hours.

Sign up for a Test Flight today at fastdata.io

Learning Machine Learning

All About Einsum in TF, PyTorch and Numpy

Einsum allows users to execute Einstein Notation in code that perform arbitrary matrix & tensor operations, like multiplies and summations, without having to interface with a number of different package API endpoints for intermediary transformations & operations. This guide walks through how to write and use the DSL for using einsum across different packages.

[Link]

HyperTools: Package for Gaining Geometric Insights into High-Dimensional Data

A python package built on top of Matplotlib for projecting hyper-dimensional data into a 2D/3D representation. It comes with some built-in tools for doing T-SNE, clustering, normalization and even text-visualization tools.

Stochastic Weight Averaging Tutorial

A nice discussion about the error surface of hyperdimensional models, and the difference between shallow vs narrow solutions. They cover snapshot ensembling, cyclic learning rates, and fast geometric ensembling that crawl the error surface in search of “wide” solutions that will generalize well. The best approach maintains a running average the model weights through the learning-rate cycles while escaping local minima. This obviates the need to train multiple models: One wireframe holds the running average weights, and a model uses these weights to train.

[Link]

How TensorFlow Lite Works on iOS

A video guide for operating TF models on iOS devices. Not sure if this is harder than using the Apple x Turi Create platform for training deploying models to Apple products, which might be take less time and come with more compressed algorithms out of the box for accomplishing most required tasks. Also of note: TF integrates into Swift, where models written in Swift are compiled directly to a execution graph. It’s still very beta.

[Link]

Intuitive Approach to Understanding CNN’s

Some Advice on Hiring Data Scientists

Machine Learning News

EU Countries Team Up to Collaborate on AI to Keep Up with the Rest of the World

The International Conference on Learned Representations passed just over a week ago. All the papers are available, and here are some highlights by research groups.

DeepMind.

Google.

[Link]

ML Researchers are Mad at Nature for Paywalling Their New Machine Learning Journal, Boycott

Nature plans to release a new Nature Machine Learning Journal, but with paywalls. Several researchers from major institutions like CMU, Facebook, DeepMind, Microsoft and more vow not to publish there. They believe that publishing behind paywalls stymies the impact of their research in this fast moving field. Nature thinks they’re wrong and that readers will pay for the editorial value they provide.

Machine Learning Blueprint's Take

The ML field is already largely publishing in open-access like arxiv, and the trend will likely only increase further. While established journals are a great go-to for rigorous research that isn’t as flag-planty as what’s seen on arxiv, a number of reputable papers are available open source. The proposed cost of access is not mentioned (generally prohibitivly expensive without an employer purchasing access), but if it were something reasonable to independent researchers, then perhaps this conversation would change tone.

Paper Publish Counts on Arxiv by Category

[Link]

CMU Launches Undergrad AI Program

CMU announced the creation of a new undergraduate AI program, the first of its kind. Like the new minting of “Data Science” programs across academia over the past five years. With AI becoming the successor term of art for multidisciplinary machine learning (deep learning) data degrees, we can likely expect CMU’s program will not be the last of its kind.

[Link]

In-Memory Compute Chips May Be Coming Soon

Machine Learning for Evil: Keeping Gamblers Hooked