List of data sources supported by Streams

 View Only

List of data sources supported by Streams 

Thu October 01, 2020 09:05 PM

A very common question from Streams users is whether Streams can connect to a particular data source to read or write data. In this article,  I’ll summarize the Streams toolkits that provide access to some of the more common data sources.

What is a toolkit?
 A toolkit is a package of artifacts that provide some function. For example, to connect to Apache Kafka, you would use the Streams Kafka toolkit. Streams provides dozens of toolkits for analyzing and ingesting data.

The following lists are evolving, as additional toolkits might be added to the Streams open source project on GitHub.

The first list below is of the toolkits that are included with Streams on Cloud Pak for connecting to data sources. 
The second is of toolkits on GitHub that are available in various stages of maturity. To use these toolkits you would have to download them from GitHub.

See below for next steps if your data source is not listed here.

Where listed, Python packages are on  PyPi and installable with pip.

Data sources supported by default in Streams 5.3+

Data Source Toolkit

Apache Kafka

Kafka Toolkit:
Python Package: streamsx.kafka

IBM Event Streams

­­Message Hub toolkit

Python package: streamsx.kafka

JDBC databases : IBM Db2, IBM Db2,
ORACLE, Teradata, PostgresSql, Microsoft SQL Server, MySql

JDBC Toolkit

Python package: streamsx.database

Db2 Event Store

Event Store toolkit­­­

Python package: streamsx.eventstore

Apache Hbase

HBase toolkit

Python package: streamsx.hbase


Rabbit MQ Toolkit


Python package: streamsx.mqtt

IBM MQ, ActiveMQ

JMS toolkit

Python package: streamsx.jms

Network data

HTTP endpoints/REST/Websockets

Inet toolkit

Python package: streamsx.inet


Messaging toolkit


Python package: streamsx.elasticsearch


Python package: streamsx.hdfs

Object Storage

Python package: streamsx.objectstorage

Additional toolkits available on GitHub

Data source




Watson Internet of Things Platform




Watson Explorer

InfoSphere Data Replication Change Data Capture

Streaming Analytics service in IBM Cloud

The Streaming Analytics service in the public IBM Cloud supports a subset of the aforementioned toolkits. View the full list of supported toolkits in the Streaming Analytics service.

Don't see your database or system?

  • If you do not see the data source you are looking for in this list, please search the Streams GitHub home page. If there is no project, you can either:

Useful links

Resources for Streams developers



0 Favorited
0 Files