IBM Event Streams 12.0.01 has just been released, bringing with it a collection of powerful new capabilities. Let's dive into the standout features that have been introduced over the recent releases and explore how they can benefit you-

1- Kafka Node Pools

Kafka Node Pools allow you to run different configuration of Kafka brokers within the same Kafka cluster, each optimized for specific workloads - like having dedicated high-memory nodes for heavy processing and cost-efficient nodes for lighter tasks. This reduces infrastructure costs while improving performance, as you only pay for the computing power each workload actually needs.

You can have designated node pools for specific need like high throughput where you assign dedicated hardware or for archival where you can use cost optimized storage

Key Features & Benefits

1.        Enhanced configuration flexibility: Different node configurations within same cluster

2.        Improved resource optimization: Customize memory, CPU per node group

3.        Better scheduling control: Configure affinity, tolerations per pool

4.        Simplified mixed workload management

5.        Streamlined operations: Inherit cluster settings, override per pool

2- Kraft  (Zookeeper-less Kafka Deployment)

Kafka Raft (KRaft) is Kafka's native metadata management system that eliminates ZooKeeper dependency, using Kafka's own consensus protocol for cluster coordination.KRaft provides a simplified architecture, improved scalability, and enhanced security.

If you are upgrading, your existing ZooKeeper-based cluster is migrated to KRaft during the upgrade process. Ensure your system meets the requirements for migration as described in the migration overview.

Note: KRaft migration is irreversible, and ZooKeeper is not supported 11.8.0 onwards

Check out my earlier blog on benefits of upgrading to KRaft.

3- Tiered storage support for Kafka topics

Tiered storage extends data retention without increasing local disk usage and keeps data available for compliance, auditing, and analytics. Event Streams supports tiered storage with Amazon S3-compatible storage by using the Aiven S3 plugin, which is bundled with and supported as part of Event Streams.

Key Features & Benefits

1.        Reduced storage costs: Offload old data to cheaper remote storage

2.        Extended data retention: Keep historical data without local disk growth

3.        Improved cluster performance: Less local storage reduces operational overhead

4.        Compliance and analytics ready: Historical data remains accessible when needed

5.        Integrated with Amazon S3

4- IBM Supported Connectors (newly added)

Our catalog now features over 70 connectors, including new additions driven by customer feedback. Through our innovative Connectivity Pack, we leverage existing IBM integration connectors from Automation Explorer to rapidly onboard connectors avoiding complete development cycle. We've also enhanced the catalog experience with improved search functionality and a streamlined process to request new connectors directly from the Connector Catalog documentation page. 

Newly added IBM-supported connectors:

•       Salesforce (source)

•       GitHub (source)

•       JIRA (source)

•       Google Calendar (source)

•       Google Big Query (sink)

•       MQTT (source)

•       ServiceNow (source)

We've simplified connector searches in the catalog, and you can now submit requests for new connectors directly from the Connector Catalog documentation page if you don’t find what you need .We have launched Connectivity Pack which helps onboard new connectors rapidly by leveraging existing IBM Automation Explorer connector. Let us know if you are looking for a new supported connector.

5-  Upgraded to Kafka 4.0

Here is list of key features you get with Kafka 4.0 , Here is blog from kafka about 4.1 release

Advanced Consumer Rebalance Protocol (KIP-848) introduces a new consumer rebalance mechanism that eliminates disruptive "stop-the-world" scenarios. This new protocol, now generally available, enhances consumer group stability and performance while reducing client implementation complexity

Share Groups (KIP-932) - Early Access Preview This enhancement introduces share groups as a novel approach to enable collaborative message consumption across Kafka topics. Rather than adding queuing concepts to Kafka(which is a mis-conception), it enables cooperative consumption patterns through Kafka topics. Share groups function similarly to "durable shared subscriptions" found in other messaging systems, opens up new possibilities for distributed message processing.

Want to learn more?

Check out whats new section of product documentation for detailed list of features with each release. Click here if you want to try out IBM Event Streams !