Primary Storage

As data storage requirements continue to grow, ensuring secure and efficient communication within storage networks has become paramount. The NVMe over TCP (NVMe-oF) protocol has revolutionised high-performance storage solutions, and adding mutual TLS (mTLS) enhances its security. This  blog will walk you through how to configure mTLS on both the NVMe-oF server and client side.

Introduction to NVMe-oF and mTLS

NVMe-oF allows data centers to connect NVMe storage devices over various network fabrics, enhancing performance and scalability. Mutual TLS (mTLS) is a security protocol that uses certificates for authenticating both the client and server, ensuring that data transfers are secure and trusted.

Create Self-Signed Server and Client  Certificates 

To configuring mTLS, ensure the server and client certificates should exists on each Ceph node that the NVMe-oF gateway service will run on:

1. Generate Server certificate and key files.

For example:

[ceph: root@adminnode1/]# openssl req -x509 -newkey rsa:4096 -nodes -keyout server.key -out server.crt -days 3650 -subj /CN=my.server -addext "subjectAltName=IP:10.243.64.15,IP:10.243.64.17,IP:10.243.64.13,IP:10.243.64.4"

2. Generate Client  certificate and key files.

[ceph: root@adminnode1/]# openssl req -x509 -newkey rsa:4096 -nodes -keyout client.key -out client.crt -days 3650 -subj /CN=my.client

Steps to Configuring mTLS on NVMe-oF Service.

1. Verify the  Ceph Cluster health status is OK.

[ceph: root@adminnode1/]# ceph -s

cluster:

    id:     cd3bfe3e-f11e-11ee-ac41-005056b4ea89

    health: HEALTH_OK

2. Login into Ceph Cluster Admin Node and export the NVMe-oF Gateway  configuration into the spec file.

[ceph: root@adminnode1/]# ceph orch ls nvmeof --export > gw-conf.yaml

3. Copy the NVMe-oF exported specification file to a new mTLS file.

[ceph: root@adminnode1/]# cp gw-conf.yaml mTLS_spec_file.yaml

4. Edit the NVMe-oF mTLS specification file.

[ceph: root@adminnode1/]# vi mTLS_spec_file.yaml

Add the below values into the specification file.

 enable_auth: true
root_ca_cert: root_ca_cert
client_cert: |
  -----BEGIN CERTIFICATE-----
  CLIENT_CERTIFICATE
  -----END CERTIFICATE-----
client_key: |
  -----BEGIN PRIVATE KEY-----
  CLIENT_PRIVATE_KEY
  -----END PRIVATE KEY-----
server_cert: |
  -----BEGIN CERTIFICATE-----
  SERVER_CERTIFICATE
  -----END CERTIFICATE-----
server_key: |
  -----BEGIN PRIVATE KEY-----
  SERVER_PRIVATE_KEY
  -----END PRIVATE KEY-----

For Eg.

5. Apply the changes to the specification file .

[ceph: root@adminnode1/]# ceph orch apply -i mTLS_spec_file.yaml

6. Redeploy the nvmeof service

[ceph: root@adminnode1/]# ceph orch redeploy nvmeof.rbd-pool

Configure the mTLS on the nvmeof client by pulling the nvmeof-cli images.

Note: Maintain all keys and certifications within the node where the Ceph NVMe-oF gateway command-line is run. In this example, everything is stored within the /tmp directory.

[ceph: root@gateway-node1/]# alias nvme= "podman run -v /tmp/server.crt:/root/server.crt:z -v /tmp/client.crt:/root/client.crt:z -v /tmp/client.key:/root/client.key:z -it --rm cp.stg.icr.io/cp/ibm-ceph/nvmeof-cli-rhel9:1.2.16-8 --server-address 10.x.x.x --client-key /root/client.key --client-cert /root/client.crt --server-cert /root/server.crt"

 [ceph: root@gateway-node1/]# nvme gw info

CLI's version: 1.2.16
Gateway's version: 1.2.16
Gateway's name: client.nvmeof.rbd.gateway-node1.fisfir
Gateway's group: ""
Gateway's host name: gateway-node1
Gateway's load balancing group: 1
Gateway's address: 10.x.x.x
Gateway's port: 5500
SPDK version: 24.01.1

Conclusion : 

Implementing mTLS in NVMe-oF setup significantly enhances the security of data transmissions, ensuring that only authorized clients and servers can communicate. This blog provides a step-by-step approach to configure mTLS, maintaining the integrity and confidentiality of ceph storage network communications.