IBM Storage Ceph

IBM Storage Ceph

Connect, collaborate, and share expertise on IBM Storage Ceph

 View Only
Back to discussions
Expand all | Collapse all

IBM Storage Ceph Object Storage Deep Dive Series

  • 1.  IBM Storage Ceph Object Storage Deep Dive Series

    Posted 7 days ago
    Edited by Daniel Alexander Parkes 7 days ago

    Announcing: A Two-Part Deep Dive on the IBM Storage Ceph Object Gateway (RGW)

    This new two-part series explores the architecture and internal data structures of the Ceph Object Gateway (RGW). We move from high-level components to the underlying data representation, including multipart uploads and data inlining.

    Part 1: Architecture and Scalability Solutions.

    Part 1 details how the IBM Storage Ceph Object Gateway is architected to address the key scalability challenges faced by other storage systems.

    • The Concurrency Problem: Discover how the Beast frontend manages thousands of simultaneous connections without performance degradation.

    • The Slow Listing Problem: We analyze the OMAP-based index to reveal why your storage media choice for certain pools has a critical impact on listing speed.

    • The "Hot Bucket" Problem: A deep look at Dynamic Resharding as an automated solution to prevent performance bottlenecks in buckets with billions of objects.

    • RGW Data Placement: An overview of the specialized RGW RADOS pools, clarifying exactly where RGW metadata and data reside in your cluster.

    Read Part 1: https://community.ibm.com/community/user/blogs/daniel-alexander-parkes/2025/10/23/ibm-storage-ceph-object-storage-deep-dive-series-p

    Part 2: Data Internals and Advanced Management This article examines the complex data management operations that are crucial to efficiency and reliability.

    • Metadata Layout Explained: A clear breakdown of RGW's metadata, showing where user, bucket, and policy information is stored and how to inspect it safely.

    • The Head/Tail Data Model: Explore the internal data model that allows RGW to be highly efficient for both small files and massive, multi-terabyte objects.

    • Multipart Upload Internals: Understand how Multipart Upload is designed as a near-instantaneous, metadata-only commit that avoids performance penalties for large file writes.

    • Automated Background Operations: A deep dive into Garbage Collection and Lifecycle processes that ensure data is managed and space is reclaimed efficiently without manual intervention.

    Read Part 2: https://community.ibm.com/community/user/blogs/daniel-alexander-parkes/2025/10/24/ibm-storage-ceph-object-storage-deep-dive-series-p

    This series provides foundational knowledge for Storage Architects and SREs working with Ceph Object Storage.



Related Content