IBM Storage Ceph

IBM Storage Ceph

Connect, collaborate, and share expertise on IBM Storage Ceph

 View Only
Back to discussions
Expand all | Collapse all

New Tech Zone Lab: Build a Zero-Trust Lakehouse on IBM Storage Ceph

  • 1.  New Tech Zone Lab: Build a Zero-Trust Lakehouse on IBM Storage Ceph

    Posted 6 days ago
    Edited by Daniel Alexander Parkes 6 days ago

    Hi,

    We have added a new lab to our IBM Storage Ceph Collection available here: https://ibm.biz/techzone-ceph-collection

    The new lab: Build a Zero-Trust Lakehouse on IBM Storage Ceph is a guided exercise where you will construct a Zero‑Trust data‑lakehouse prototype on IBM Storage Ceph Object Storage. The lab is organised to familiarise you with both the architectural rationale and the practical implementation steps required for a modern, governed lakehouse.

    Book your lab and give it a spin! Happy to hear feedback on this thread about the experience.

    Why IBM Storage Ceph? This lab leans on Ceph Object Gateway (RGW) because it exposes a rich, AWS-compatible S3 + IAM + STS API surface, complete with multi-tenancy. That means we can drive on-prem object storage using the same Terraform + AWS CLI patterns teams use in public cloud, and immediately take advantage of scoped credentials, bucket policies, and identity isolation. Ceph's scale-out design also keeps the data path parallel and fast as analytics engines grow.

    Over an estimated 90 minutes, you will:

    1. Establish the Object Storage Ceph‑backed landing zone, creating a dedicated S3 IAM Account to isolate the analytic workloads.

    2. Apply fine‑grained access control by issuing time‑bound credentials through Polaris; table‑level policies are enforced directly by Ceph Object Gateway (RGW), Ceph's native support for S3 IAM and STS APIs enables us to issue scoped, short-lived tokens without custom proxies or third-party plugin

    3. Deploy the analytics Compute Stack-Spark for transformation, Trino for interactive SQL, and Superset for visual exploration-using Infrastructure‑as‑Code and container orchestration.

    4. Execute an end‑to‑end workflow: ingest raw objects, transform them into Iceberg tables, validate data quality, and present results in a dashboard.

    5. Inspect and verify the security posture at each stage to confirm that Zero‑Trust requirements are met.

    We also have other Labs available in the collection. If you haven't had a chance, please check them out:

    1. Install and Configure a Ceph Cluster: Learn to set up a robust IBM Storage Ceph environment from scratch and deploy the Object Storage service using the Dashboard. 
    2. Explore Advanced Object Storage Features: Dive deep into SSL configuration, bucket policies, audit logs, Storage Class Tiering, and lifecycle management to enhance the security and efficiency of your object storage. 
    3. Experience Unified Storage: Gain hands-on experience with Ceph's unified storage capabilities, including Block (RBD), File (NFS), and Object (S3), showcasing its versatility as an all-in-one storage solution. 
    4. Troubleshooting Workshop: This Hands-on Break and Fix Lab Workshop is an in-depth Ceph enablement designed to provide you with a solid understanding of the Core Ceph Concepts & Architecture. 

    Thanks!



    ------------------------------
    Daniel Alexander Parkes
    ------------------------------



Related Content