Welcome to the IBM TechXchange Community, a place to collaborate, share knowledge, & support one another in everyday challenges. Connect with your fellow members through forums, blogs, files, & face-to-face networking.
AI and Data Science Master the art of data science. Join now
When training Deep Learning models using GPUs, speeding up your data pipeline is paramount. The goal is to make sure that you are accessing and preparing data fast enough so that the data pipeline is not a bottleneck. In other terms: if your GPU is fast, and you should feed it data even faster!
In this article, I will show you how to set up a fast and elegant data pipeline leveraging Tensorflow’s tf.data API to train Computer Vision models using a GPU, when your data is stored in an Object Storage solution such as S3 or IBM Cloud Object Storage. The end to end code used for this article is available on my GitHub account in this repository.