Netezza Native Cloud Object Storage (NCOS) Support 

Netezza has taken a major leap forward in its modernization journey by introducing Native Cloud Object Storage support directly within the engine. This powerful new capability allows users to create user tables in Netezza data format on S3-compatible object storage, unlocking a new tier of scalability, flexibility, and cost-efficiency.

Why This Matters

This enhancement is more than just a technical upgrade, it's a strategic shift that positions Netezza to fully leverage the benefits of cloud-native architectures. By integrating object storage natively, Netezza enables:

• Massive Scalability: Object storage can grow virtually without limits, making it ideal for big data workloads.
• Cost-Effectiveness: Lower storage costs compared to traditional block storage.
• Data Sharing & Deduplication: Easier collaboration and reduced redundancy.
• Enhanced Durability & Resilience: Built-in fault tolerance across cloud regions.

A Hybrid Storage Hierarchy

This new capability complements Netezza’s existing block storage support, forming a hybrid storage hierarchy. Users can now transition workloads from block to object storage seamlessly, enabling a smoother migration path to cloud-native deployments.

This hybrid model is a game-changer for customers looking to optimize performance and cost while modernizing their data infrastructure.

Architecture and modernization 

Netezza is currently available on leading cloud platforms such as AWS and Azure as SaaS and Bring Your Own Cloud (BYOC) offerings. These public cloud platforms provide object storage that is cheap, scalable, and highly capable. For example, AWS S3 is a popular object storage service that can be deployed through an AWS cloud account and scales extremely well while keeping the cost minimal. It is also possible to configure the storage buckets for versioning, redundancy and access control.

With the hybrid storage approach, customers can optionally specify the storage type for a table during creation. If no storage type is defined in the DDL, Netezza automatically selects one based on the database's default settings or the default system configuration, which are configurable.

Netezza continues to be built upon the concepts of distributed computing and MPP architecture. 

The storage type is opaque to the NPS front-end. Which means that the user should be able to query the tables and join them without worrying about the underlying storage type.

Netezza storage engine modernization 

Netezza offers object storage while continuing to support block storage to facilitate transitioning data and workloads from block storage to object storage. Once the object storage is configured, the customer can choose the storage type of the newly created tables. Data can be copied as required between object storage and block storage using simple SQL.

The users should create a separate bucket for each NPS instance. 

Table object naming and deletion

Each object stored in the object store follows a structured prefix format:

 “/nps/<instance name>/<dbuuid>/<dsid>”.

The object storage contains objects corresponding to the tables and metadata. Each data slice has its own metadata objects. The metadata is stored in a large buffer to improve the IO latency.

In Netezza's object storage architecture, obsolete objects, such as those resulting from dropped, truncated, or groomed tables, are tracked using a delete list object. This list is generated during those operations. Then, an asynchronous garbage cleaner is responsible for performing the actual deletion of those objects from the object store.

This means Netezza automatically handles cleanup in the background, ensuring efficient storage management without requiring manual intervention.

Local caching

To enhance performance and reduce operational costs, Netezza’s object storage leverages local caching. This caching layer delivers low latency and high throughput. A custom caching implementation was developed specifically for the Netezza storage engine, allowing rapid iteration and tight integration with Netezza-specific architecture.

Object storage key highlights for Netezza users 

Netezza’s native object storage integration brings powerful benefits to users—without disrupting existing workflows or applications. Here are the key advantages:

• No Changes to Customer Applications

• Applications built for block storage tables continue to work seamlessly with object storage tables.
• No code changes or reconfiguration required.

• Full SQL Compatibility

• All Netezza SQL syntax, tools, and operational features work identically with object storage tables.
• Users experience consistent behavior across both storage types.

• Improved Scalability

• Leverages cloud object storage’s inherent scalability for both capacity and IOPS.
• Ideal for growing workloads and large datasets.

• Transparent Cost Model

• Users are billed based on actual usage—storage volume and API calls.
• Enables better cost control and predictability.

• Optimized Price/Performance Over Time

• Netezza employs techniques like high performance local caching and batch writes to reduce API overhead.
• These optimizations help deliver better performance at lower operational cost, especially for read-heavy and analytical workloads.

Performance characteristic Block vs Object storage

To assess the efficiency and reliability of object storage, a diverse set of workloads was executed including read-only queries, mixed analytical operations, write-intensive tasks, and concurrent multi-user scenarios.

   Main Highlights

• Performance Parity with Block Storage

o   Object storage demonstrated comparable performance to traditional block storage across diverse workload types.

• Faster Batch Write Performance

o   Write-heavy operations, especially batch inserts, showed improved throughput thanks to optimized caching and write strategies.

• Lower Storage Costs Over Time

o   The total cost of running these workloads including API usage was consistently lower on object storage compared to block storage, making it a more cost-effective solution for long-term data management.

User Experience 

To support native object storage, Netezza introduces the storagetype keyword, allowing users to specify the desired storage type at various levels of configuration.

Storage Type Resolution Hierarchy

1.     Table-Level

• If specified in the CREATE TABLE statement, the table uses the defined storage type.

Example:

CREATE TABLE t1 (c1 INT) storagetype 'object';

2.     Session-Level

• •     If not specified at the table level, the storage type is inherited from the session.

               Example:

               SET default_storage_type = 'object';

               CREATE TABLE t1 (c1 INT)

3.     Database-Level

• If not specified at the session level, the storage type is inherited from the database.

Example:

CREATE DATABASE db1 storagetype 'object';

ALTER DATABASE db1 storagetype 'object';

4.     Global Setting

• If not specified at the database level either, the system falls back to the global default storage type set in system configuration.

Example:

SET default_storage_type = 'object';

This hierarchy ensures flexibility while maintaining consistency across deployments.

Availability Update

Native Cloud Object Storage for Netezza on AWS is now available in public preview, with General Availability planned for AWS and Azure in Q42025.

Summary

Netezza now supports native cloud object storage, enabling users to create and manage tables directly on S3-compatible cloud storage. This hybrid model combining block and object storage offers better scalability, lower costs, and seamless SQL compatibility. With caching, asynchronous cleanup, and flexible storage configuration via the storagetype keyword, Netezza delivers improved price-performance while maintaining a consistent user experience. Netezza NCOS public preview is a solid foundation to spur a wave of new capabilities that will excite our users. 

We recommend users to explore object storage by running their workloads and determine the best balance for their data and workload needs. 

We are super excited to receive feedback from this public preview.

--

Contributors: 

(Brajesh Pandey, Daniel Hancock, Harish Mara, Kiran Kumar Gahlot, Samuel Patterson, Shawn Rackley)