IBM i

Hi,

We are building a test system for internal training purpose. Plan is to build 10 IBM i LPAR on S922 with 1TB disk size, no SAN, only vSCSI and dual VIOS. Any idea what should be the best option to size Power System. We have sized RAM/CPU, 2 10G NICs, no HBAs, as we donot need tape and backup.
I see two options.
1. Have four 3.2TB NVMEs installed in CEC, we get 12.8TB size. Assign those NVMEs to VIOS and create virtual disk from those NVME and assign to IBM i LPAR. Personally I have not done this method. I did option 2 below. Has anyone did that and can share more details ?
2. Have 4 SSD for VIOS in CEC and add disk drawer for 10TB size and attach via dual SAS adapter, one to VIOS1 and other to VIOS2. Create virtual disks and assign to IBM I LPAR.

Now a days standard is to use SAN via External Storage. We donot need SAN infrastructure.

Anyone can share best method for this.

Regards
Angela

------------------------------
Angela B
------------------------------

Dear Angela

Are the training workloads in those 10 IBM i LPARs CPU-bound or disk IO-bound or just moderate on both counts?   If CPU-bound or moderate, I would go for whichever config that costs less.  But if both configs cost in a comparable range of investment, going for all-NVMe is better as you can hope to enjoy decent disk IO performance. 

I notice you do not want to use disk mirroring for your first config.  Is this intentional?  If you actually want disk mirroring (RAID-5 not supported for NVMe), you need to buy 6.4TB units.   Please also configure NVMe namespace of 188GB rather than 393GB as IBM i likes more disk units (the configured namespace become disk unit size for IBM i).

BTW, Do the 10 LPARs carry workloads at the same time consistently? If not, please use "Uncapped Partitioning" feature so that LPARs that are carrying workloads can steal CPU power from neighboring LPARs that are idle or use little CPU power.  

Just my 2 cents.

------------------------------
Satid Singkorapoom
------------------------------

Original Message:
Sent: Fri May 06, 2022 05:41 AM
From: Angela B
Subject: Power System for IBM i using NVME

Hi,

We are building a test system for internal training purpose. Plan is to build 10 IBM i LPAR on S922 with 1TB disk size, no SAN, only vSCSI and dual VIOS. Any idea what should be the best option to size Power System. We have sized RAM/CPU, 2 10G NICs, no HBAs, as we donot need tape and backup.
I see two options.
1. Have four 3.2TB NVMEs installed in CEC, we get 12.8TB size. Assign those NVMEs to VIOS and create virtual disk from those NVME and assign to IBM i LPAR. Personally I have not done this method. I did option 2 below. Has anyone did that and can share more details ?
2. Have 4 SSD for VIOS in CEC and add disk drawer for 10TB size and attach via dual SAS adapter, one to VIOS1 and other to VIOS2. Create virtual disks and assign to IBM I LPAR.

Now a days standard is to use SAN via External Storage. We donot need SAN infrastructure.

Anyone can share best method for this.

Regards
Angela

------------------------------
Angela B
------------------------------

Hi Satid,

Thanks for your reply. Training workloads are moderate on both counts. So if I do disk mirroring for 2 (3.2TB) and another 2 (3.2TB)disks, I will get 6.4TB after mirroring. Can I install VIOS1 on same two disk(6.4TB) and create virtual disks from same VIOS1 for IBM i LPARs. Same for VIOS2, assigning 6.4TB (both 3.2 TB disks), install VIOS2 and create virtual disks for IBM i LPARs? Thanks for namespace guidance and CPU uncapping information.

With Regards
Angela


------------------------------
Angela B
------------------------------

Original Message:
Sent: Fri May 06, 2022 08:25 AM
From: Satid Singkorapoom
Subject: Power System for IBM i using NVME

Dear Angela

Are the training workloads in those 10 IBM i LPARs CPU-bound or disk IO-bound or just moderate on both counts?   If CPU-bound or moderate, I would go for whichever config that costs less.  But if both configs cost in a comparable range of investment, going for all-NVMe is better as you can hope to enjoy decent disk IO performance. 

I notice you do not want to use disk mirroring for your first config.  Is this intentional?  If you actually want disk mirroring (RAID-5 not supported for NVMe), you need to buy 6.4TB units.   Please also configure NVMe namespace of 188GB rather than 393GB as IBM i likes more disk units (the configured namespace become disk unit size for IBM i).

BTW, Do the 10 LPARs carry workloads at the same time consistently? If not, please use "Uncapped Partitioning" feature so that LPARs that are carrying workloads can steal CPU power from neighboring LPARs that are idle or use little CPU power.  

Just my 2 cents.

------------------------------
Satid Singkorapoom

Original Message:
Sent: Fri May 06, 2022 05:41 AM
From: Angela B
Subject: Power System for IBM i using NVME

Hi,

We are building a test system for internal training purpose. Plan is to build 10 IBM i LPAR on S922 with 1TB disk size, no SAN, only vSCSI and dual VIOS. Any idea what should be the best option to size Power System. We have sized RAM/CPU, 2 10G NICs, no HBAs, as we donot need tape and backup.
I see two options.
1. Have four 3.2TB NVMEs installed in CEC, we get 12.8TB size. Assign those NVMEs to VIOS and create virtual disk from those NVME and assign to IBM i LPAR. Personally I have not done this method. I did option 2 below. Has anyone did that and can share more details ?
2. Have 4 SSD for VIOS in CEC and add disk drawer for 10TB size and attach via dual SAS adapter, one to VIOS1 and other to VIOS2. Create virtual disks and assign to IBM I LPAR.

Now a days standard is to use SAN via External Storage. We donot need SAN infrastructure.

Anyone can share best method for this.

Regards
Angela

------------------------------
Angela B
------------------------------

Dear Angela

From this blog post (look at the Best Practices section), I see you can use only one VIOS if you buy only 2 adapter for NVMe : NVMe device support in Virtual IO Server (VIOS)

Since you stick with 3TB NVMe unit, I guess you want to use NVMe from backplane only but I have no idea on using NVMe from backplane. I suggest you ask your question in PowerVM group for a better chance to get an answer.

As for sharing LAN card in S922, you can use SR-IOV rather than VIOS but you need to order LAN cards that support SR-IOV.  This helps reduce workload in VIOS which means less memory needed there.  I used to configure SR-IOV in HMC and it was quite simple. The downsize was it did not support link aggregation (this may change by now, need to check more) but I see this should not be a big deal for your training system. If you still want LAN availability in an LPAR, you can use the old IBM i manual config of virtual IP address. 



------------------------------
Satid Singkorapoom
------------------------------

Original Message:
Sent: Fri May 06, 2022 12:59 PM
From: Angela B
Subject: Power System for IBM i using NVME

Hi Satid,

Thanks for your reply. Training workloads are moderate on both counts. So if I do disk mirroring for 2 (3.2TB) and another 2 (3.2TB)disks, I will get 6.4TB after mirroring. Can I install VIOS1 on same two disk(6.4TB) and create virtual disks from same VIOS1 for IBM i LPARs. Same for VIOS2, assigning 6.4TB (both 3.2 TB disks), install VIOS2 and create virtual disks for IBM i LPARs? Thanks for namespace guidance and CPU uncapping information.

With Regards
Angela


------------------------------
Angela B

Original Message:
Sent: Fri May 06, 2022 08:25 AM
From: Satid Singkorapoom
Subject: Power System for IBM i using NVME

Dear Angela

Are the training workloads in those 10 IBM i LPARs CPU-bound or disk IO-bound or just moderate on both counts?   If CPU-bound or moderate, I would go for whichever config that costs less.  But if both configs cost in a comparable range of investment, going for all-NVMe is better as you can hope to enjoy decent disk IO performance. 

I notice you do not want to use disk mirroring for your first config.  Is this intentional?  If you actually want disk mirroring (RAID-5 not supported for NVMe), you need to buy 6.4TB units.   Please also configure NVMe namespace of 188GB rather than 393GB as IBM i likes more disk units (the configured namespace become disk unit size for IBM i).

BTW, Do the 10 LPARs carry workloads at the same time consistently? If not, please use "Uncapped Partitioning" feature so that LPARs that are carrying workloads can steal CPU power from neighboring LPARs that are idle or use little CPU power.  

Just my 2 cents.

------------------------------
Satid Singkorapoom

Original Message:
Sent: Fri May 06, 2022 05:41 AM
From: Angela B
Subject: Power System for IBM i using NVME

Hi,

We are building a test system for internal training purpose. Plan is to build 10 IBM i LPAR on S922 with 1TB disk size, no SAN, only vSCSI and dual VIOS. Any idea what should be the best option to size Power System. We have sized RAM/CPU, 2 10G NICs, no HBAs, as we donot need tape and backup.
I see two options.
1. Have four 3.2TB NVMEs installed in CEC, we get 12.8TB size. Assign those NVMEs to VIOS and create virtual disk from those NVME and assign to IBM i LPAR. Personally I have not done this method. I did option 2 below. Has anyone did that and can share more details ?
2. Have 4 SSD for VIOS in CEC and add disk drawer for 10TB size and attach via dual SAS adapter, one to VIOS1 and other to VIOS2. Create virtual disks and assign to IBM I LPAR.

Now a days standard is to use SAN via External Storage. We donot need SAN infrastructure.

Anyone can share best method for this.

Regards
Angela

------------------------------
Angela B
------------------------------

I did some more research and found that when using NVMe from S922 storage backplane, this does not support split backplane. My high level idea is that you mount all 4 NVMe units onto the storage backplane and allocate 2 units to VIOS1 and the remaining 2 units to VIOS2.  Then within each VIOS, you create vSCSI disk units and I suggest you create each vDisk units at around 100GB size to ensure sufficient number to be allocated to all IBM i LPARs.  Then allocate vDisk units equally from VIOS1 and VIOS2 to each IBM i LPAR. Then within each IBM i LPAR, you start disk mirroring making sure you mirror vDisk units from VIOS1 against those from VIOS2.   

You also need to use VIOS vSCSI LVM client only (and I have no idea what this means but it is specified here : https://www.ibm.com/support/pages/ibm-i-io-support-details  )

Please keep in mind that, since split-backplane is not supported for NVMe, all NVMe units are under a single controller, If the controller fails, all disk units are gone.  The mirroring in IBM i LPAR is therefore at disk-unit-level only.   From my past experience, the base controller rarely fails.   

Please also check this idea with someone who has experience with setting up VIOS to confirm its feasibility.


------------------------------
Satid Singkorapoom
------------------------------

Original Message:
Sent: Fri May 06, 2022 12:59 PM
From: Angela B
Subject: Power System for IBM i using NVME

Hi Satid,

Thanks for your reply. Training workloads are moderate on both counts. So if I do disk mirroring for 2 (3.2TB) and another 2 (3.2TB)disks, I will get 6.4TB after mirroring. Can I install VIOS1 on same two disk(6.4TB) and create virtual disks from same VIOS1 for IBM i LPARs. Same for VIOS2, assigning 6.4TB (both 3.2 TB disks), install VIOS2 and create virtual disks for IBM i LPARs? Thanks for namespace guidance and CPU uncapping information.

With Regards
Angela


------------------------------
Angela B

Original Message:
Sent: Fri May 06, 2022 08:25 AM
From: Satid Singkorapoom
Subject: Power System for IBM i using NVME

Dear Angela

Are the training workloads in those 10 IBM i LPARs CPU-bound or disk IO-bound or just moderate on both counts?   If CPU-bound or moderate, I would go for whichever config that costs less.  But if both configs cost in a comparable range of investment, going for all-NVMe is better as you can hope to enjoy decent disk IO performance. 

I notice you do not want to use disk mirroring for your first config.  Is this intentional?  If you actually want disk mirroring (RAID-5 not supported for NVMe), you need to buy 6.4TB units.   Please also configure NVMe namespace of 188GB rather than 393GB as IBM i likes more disk units (the configured namespace become disk unit size for IBM i).

BTW, Do the 10 LPARs carry workloads at the same time consistently? If not, please use "Uncapped Partitioning" feature so that LPARs that are carrying workloads can steal CPU power from neighboring LPARs that are idle or use little CPU power.  

Just my 2 cents.

------------------------------
Satid Singkorapoom

Original Message:
Sent: Fri May 06, 2022 05:41 AM
From: Angela B
Subject: Power System for IBM i using NVME

Hi,

We are building a test system for internal training purpose. Plan is to build 10 IBM i LPAR on S922 with 1TB disk size, no SAN, only vSCSI and dual VIOS. Any idea what should be the best option to size Power System. We have sized RAM/CPU, 2 10G NICs, no HBAs, as we donot need tape and backup.
I see two options.
1. Have four 3.2TB NVMEs installed in CEC, we get 12.8TB size. Assign those NVMEs to VIOS and create virtual disk from those NVME and assign to IBM i LPAR. Personally I have not done this method. I did option 2 below. Has anyone did that and can share more details ?
2. Have 4 SSD for VIOS in CEC and add disk drawer for 10TB size and attach via dual SAS adapter, one to VIOS1 and other to VIOS2. Create virtual disks and assign to IBM I LPAR.

Now a days standard is to use SAN via External Storage. We donot need SAN infrastructure.

Anyone can share best method for this.

Regards
Angela

------------------------------
Angela B
------------------------------

Original Message:
Sent: 5/8/2022 11:37:00 PM
From: Satid Singkorapoom
Subject: RE: Power System for IBM i using NVME

I did some more research and found that when using NVMe from S922 storage backplane, this does not support split backplane. My high level idea is that you mount all 4 NVMe units onto the storage backplane and allocate 2 units to VIOS1 and the remaining 2 units to VIOS2.  Then within each VIOS, you create vSCSI disk units and I suggest you create each vDisk units at around 100GB size to ensure sufficient number to be allocated to all IBM i LPARs.  Then allocate vDisk units equally from VIOS1 and VIOS2 to each IBM i LPAR. Then within each IBM i LPAR, you start disk mirroring making sure you mirror vDisk units from VIOS1 against those from VIOS2.   

You also need to use VIOS vSCSI LVM client only (and I have no idea what this means but it is specified here : https://www.ibm.com/support/pages/ibm-i-io-support-details  )

Please keep in mind that, since split-backplane is not supported for NVMe, all NVMe units are under a single controller, If the controller fails, all disk units are gone.  The mirroring in IBM i LPAR is therefore at disk-unit-level only.   From my past experience, the base controller rarely fails.   

Please also check this idea with someone who has experience with setting up VIOS to confirm its feasibility.


------------------------------
Satid Singkorapoom
------------------------------

Original Message:
Sent: Fri May 06, 2022 12:59 PM
From: Angela B
Subject: Power System for IBM i using NVME

Hi Satid,

Thanks for your reply. Training workloads are moderate on both counts. So if I do disk mirroring for 2 (3.2TB) and another 2 (3.2TB)disks, I will get 6.4TB after mirroring. Can I install VIOS1 on same two disk(6.4TB) and create virtual disks from same VIOS1 for IBM i LPARs. Same for VIOS2, assigning 6.4TB (both 3.2 TB disks), install VIOS2 and create virtual disks for IBM i LPARs? Thanks for namespace guidance and CPU uncapping information.

With Regards
Angela


------------------------------
Angela B

Original Message:
Sent: Fri May 06, 2022 08:25 AM
From: Satid Singkorapoom
Subject: Power System for IBM i using NVME

Dear Angela

Are the training workloads in those 10 IBM i LPARs CPU-bound or disk IO-bound or just moderate on both counts?   If CPU-bound or moderate, I would go for whichever config that costs less.  But if both configs cost in a comparable range of investment, going for all-NVMe is better as you can hope to enjoy decent disk IO performance. 

I notice you do not want to use disk mirroring for your first config.  Is this intentional?  If you actually want disk mirroring (RAID-5 not supported for NVMe), you need to buy 6.4TB units.   Please also configure NVMe namespace of 188GB rather than 393GB as IBM i likes more disk units (the configured namespace become disk unit size for IBM i).

BTW, Do the 10 LPARs carry workloads at the same time consistently? If not, please use "Uncapped Partitioning" feature so that LPARs that are carrying workloads can steal CPU power from neighboring LPARs that are idle or use little CPU power.  

Just my 2 cents.

------------------------------
Satid Singkorapoom

Original Message:
Sent: Fri May 06, 2022 05:41 AM
From: Angela B
Subject: Power System for IBM i using NVME

Hi,

We are building a test system for internal training purpose. Plan is to build 10 IBM i LPAR on S922 with 1TB disk size, no SAN, only vSCSI and dual VIOS. Any idea what should be the best option to size Power System. We have sized RAM/CPU, 2 10G NICs, no HBAs, as we donot need tape and backup.
I see two options.
1. Have four 3.2TB NVMEs installed in CEC, we get 12.8TB size. Assign those NVMEs to VIOS and create virtual disk from those NVME and assign to IBM i LPAR. Personally I have not done this method. I did option 2 below. Has anyone did that and can share more details ?
2. Have 4 SSD for VIOS in CEC and add disk drawer for 10TB size and attach via dual SAS adapter, one to VIOS1 and other to VIOS2. Create virtual disks and assign to IBM I LPAR.

Now a days standard is to use SAN via External Storage. We donot need SAN infrastructure.

Anyone can share best method for this.

Regards
Angela

------------------------------
Angela B
------------------------------

Hi Steve,

Thank you for your inputs. I am checking i hosting on i solution.

Best Regards
Angela. B

------------------------------
Angela B
------------------------------

Original Message:
Sent: Mon May 09, 2022 07:46 AM
From: Steve Bradshaw
Subject: Power System for IBM i using NVME

Greetings one and all,

If you have not already ordered this system, it looks to me like you might benefit from using IBM i hosting IBM i rather than VIOS hosting IBM i. 

Either way when it comes to sizing your IBM i clients virtual disks, there are a number of factors to consider but the main one is to make sure you have a larger number of smaller virtual disks rather than a smaller number of larger disks.  I would suggest you start with the amount of storage you need for any one of these 10 LPARs then divide it through so that you have a minimum of 5 virtual disks, preferably more.  So, if for example one of your LPARs needed 500 GB of storage you could present 10 x 50GB virtual disks  but if they wanted 1TB of storage you might present 10 x 100GB disks.

Regards, Brad

Steve Bradshaw
Rowton IT Solutions Ltd

Unit 1, Amptronic Estate, Heath Mill Road, Wombourne, South Staffs, WV5 8AP.  
Web www.RowtonIT.com    Support Rowton.IT

Mobile +44 (0) 7970 978409

Office +44 (0) 1902 745409

Original Message:
Sent: 5/8/2022 11:37:00 PM
From: Satid Singkorapoom
Subject: RE: Power System for IBM i using NVME

I did some more research and found that when using NVMe from S922 storage backplane, this does not support split backplane. My high level idea is that you mount all 4 NVMe units onto the storage backplane and allocate 2 units to VIOS1 and the remaining 2 units to VIOS2.  Then within each VIOS, you create vSCSI disk units and I suggest you create each vDisk units at around 100GB size to ensure sufficient number to be allocated to all IBM i LPARs.  Then allocate vDisk units equally from VIOS1 and VIOS2 to each IBM i LPAR. Then within each IBM i LPAR, you start disk mirroring making sure you mirror vDisk units from VIOS1 against those from VIOS2.   

You also need to use VIOS vSCSI LVM client only (and I have no idea what this means but it is specified here : https://www.ibm.com/support/pages/ibm-i-io-support-details  )

Please keep in mind that, since split-backplane is not supported for NVMe, all NVMe units are under a single controller, If the controller fails, all disk units are gone.  The mirroring in IBM i LPAR is therefore at disk-unit-level only.   From my past experience, the base controller rarely fails.   

Please also check this idea with someone who has experience with setting up VIOS to confirm its feasibility.


------------------------------
Satid Singkorapoom

Original Message:
Sent: Fri May 06, 2022 12:59 PM
From: Angela B
Subject: Power System for IBM i using NVME

Hi Satid,

Thanks for your reply. Training workloads are moderate on both counts. So if I do disk mirroring for 2 (3.2TB) and another 2 (3.2TB)disks, I will get 6.4TB after mirroring. Can I install VIOS1 on same two disk(6.4TB) and create virtual disks from same VIOS1 for IBM i LPARs. Same for VIOS2, assigning 6.4TB (both 3.2 TB disks), install VIOS2 and create virtual disks for IBM i LPARs? Thanks for namespace guidance and CPU uncapping information.

With Regards
Angela


------------------------------
Angela B

Original Message:
Sent: Fri May 06, 2022 08:25 AM
From: Satid Singkorapoom
Subject: Power System for IBM i using NVME

Dear Angela

Are the training workloads in those 10 IBM i LPARs CPU-bound or disk IO-bound or just moderate on both counts?   If CPU-bound or moderate, I would go for whichever config that costs less.  But if both configs cost in a comparable range of investment, going for all-NVMe is better as you can hope to enjoy decent disk IO performance. 

I notice you do not want to use disk mirroring for your first config.  Is this intentional?  If you actually want disk mirroring (RAID-5 not supported for NVMe), you need to buy 6.4TB units.   Please also configure NVMe namespace of 188GB rather than 393GB as IBM i likes more disk units (the configured namespace become disk unit size for IBM i).

BTW, Do the 10 LPARs carry workloads at the same time consistently? If not, please use "Uncapped Partitioning" feature so that LPARs that are carrying workloads can steal CPU power from neighboring LPARs that are idle or use little CPU power.  

Just my 2 cents.

------------------------------
Satid Singkorapoom

Original Message:
Sent: Fri May 06, 2022 05:41 AM
From: Angela B
Subject: Power System for IBM i using NVME

Hi,

We are building a test system for internal training purpose. Plan is to build 10 IBM i LPAR on S922 with 1TB disk size, no SAN, only vSCSI and dual VIOS. Any idea what should be the best option to size Power System. We have sized RAM/CPU, 2 10G NICs, no HBAs, as we donot need tape and backup.
I see two options.
1. Have four 3.2TB NVMEs installed in CEC, we get 12.8TB size. Assign those NVMEs to VIOS and create virtual disk from those NVME and assign to IBM i LPAR. Personally I have not done this method. I did option 2 below. Has anyone did that and can share more details ?
2. Have 4 SSD for VIOS in CEC and add disk drawer for 10TB size and attach via dual SAS adapter, one to VIOS1 and other to VIOS2. Create virtual disks and assign to IBM I LPAR.

Now a days standard is to use SAN via External Storage. We donot need SAN infrastructure.

Anyone can share best method for this.

Regards
Angela

------------------------------
Angela B
------------------------------

Original Message:
Sent: 5/9/2022 9:53:00 AM
From: Angela B
Subject: RE: Power System for IBM i using NVME

Hi Steve,

Thank you for your inputs. I am checking i hosting on i solution.

Best Regards
Angela. B

------------------------------
Angela B
------------------------------

Original Message:
Sent: Mon May 09, 2022 07:46 AM
From: Steve Bradshaw
Subject: Power System for IBM i using NVME

Greetings one and all,

If you have not already ordered this system, it looks to me like you might benefit from using IBM i hosting IBM i rather than VIOS hosting IBM i. 

Either way when it comes to sizing your IBM i clients virtual disks, there are a number of factors to consider but the main one is to make sure you have a larger number of smaller virtual disks rather than a smaller number of larger disks.  I would suggest you start with the amount of storage you need for any one of these 10 LPARs then divide it through so that you have a minimum of 5 virtual disks, preferably more.  So, if for example one of your LPARs needed 500 GB of storage you could present 10 x 50GB virtual disks  but if they wanted 1TB of storage you might present 10 x 100GB disks.

Regards, Brad

Steve Bradshaw
Rowton IT Solutions Ltd

Unit 1, Amptronic Estate, Heath Mill Road, Wombourne, South Staffs, WV5 8AP.  
Web www.RowtonIT.com    Support Rowton.IT

Mobile +44 (0) 7970 978409

Office +44 (0) 1902 745409

Original Message:
Sent: 5/8/2022 11:37:00 PM
From: Satid Singkorapoom
Subject: RE: Power System for IBM i using NVME

I did some more research and found that when using NVMe from S922 storage backplane, this does not support split backplane. My high level idea is that you mount all 4 NVMe units onto the storage backplane and allocate 2 units to VIOS1 and the remaining 2 units to VIOS2.  Then within each VIOS, you create vSCSI disk units and I suggest you create each vDisk units at around 100GB size to ensure sufficient number to be allocated to all IBM i LPARs.  Then allocate vDisk units equally from VIOS1 and VIOS2 to each IBM i LPAR. Then within each IBM i LPAR, you start disk mirroring making sure you mirror vDisk units from VIOS1 against those from VIOS2.   

You also need to use VIOS vSCSI LVM client only (and I have no idea what this means but it is specified here : https://www.ibm.com/support/pages/ibm-i-io-support-details  )

Please keep in mind that, since split-backplane is not supported for NVMe, all NVMe units are under a single controller, If the controller fails, all disk units are gone.  The mirroring in IBM i LPAR is therefore at disk-unit-level only.   From my past experience, the base controller rarely fails.   

Please also check this idea with someone who has experience with setting up VIOS to confirm its feasibility.


------------------------------
Satid Singkorapoom

Original Message:
Sent: Fri May 06, 2022 12:59 PM
From: Angela B
Subject: Power System for IBM i using NVME

Hi Satid,

Thanks for your reply. Training workloads are moderate on both counts. So if I do disk mirroring for 2 (3.2TB) and another 2 (3.2TB)disks, I will get 6.4TB after mirroring. Can I install VIOS1 on same two disk(6.4TB) and create virtual disks from same VIOS1 for IBM i LPARs. Same for VIOS2, assigning 6.4TB (both 3.2 TB disks), install VIOS2 and create virtual disks for IBM i LPARs? Thanks for namespace guidance and CPU uncapping information.

With Regards
Angela


------------------------------
Angela B

Original Message:
Sent: Fri May 06, 2022 08:25 AM
From: Satid Singkorapoom
Subject: Power System for IBM i using NVME

Dear Angela

Are the training workloads in those 10 IBM i LPARs CPU-bound or disk IO-bound or just moderate on both counts?   If CPU-bound or moderate, I would go for whichever config that costs less.  But if both configs cost in a comparable range of investment, going for all-NVMe is better as you can hope to enjoy decent disk IO performance. 

I notice you do not want to use disk mirroring for your first config.  Is this intentional?  If you actually want disk mirroring (RAID-5 not supported for NVMe), you need to buy 6.4TB units.   Please also configure NVMe namespace of 188GB rather than 393GB as IBM i likes more disk units (the configured namespace become disk unit size for IBM i).

BTW, Do the 10 LPARs carry workloads at the same time consistently? If not, please use "Uncapped Partitioning" feature so that LPARs that are carrying workloads can steal CPU power from neighboring LPARs that are idle or use little CPU power.  

Just my 2 cents.

------------------------------
Satid Singkorapoom

Original Message:
Sent: Fri May 06, 2022 05:41 AM
From: Angela B
Subject: Power System for IBM i using NVME

Hi,

We are building a test system for internal training purpose. Plan is to build 10 IBM i LPAR on S922 with 1TB disk size, no SAN, only vSCSI and dual VIOS. Any idea what should be the best option to size Power System. We have sized RAM/CPU, 2 10G NICs, no HBAs, as we donot need tape and backup.
I see two options.
1. Have four 3.2TB NVMEs installed in CEC, we get 12.8TB size. Assign those NVMEs to VIOS and create virtual disk from those NVME and assign to IBM i LPAR. Personally I have not done this method. I did option 2 below. Has anyone did that and can share more details ?
2. Have 4 SSD for VIOS in CEC and add disk drawer for 10TB size and attach via dual SAS adapter, one to VIOS1 and other to VIOS2. Create virtual disks and assign to IBM I LPAR.

Now a days standard is to use SAN via External Storage. We donot need SAN infrastructure.

Anyone can share best method for this.

Regards
Angela

------------------------------
Angela B
------------------------------

Thank You Brad.

One Question - Do NVMe support hardware level encryption ? I don't see latest information, but some blogs say, it does not support hardware encryption for IBMi.

Kind Regards
Angela B.

------------------------------
Angela B
------------------------------

Original Message:
Sent: Mon May 09, 2022 12:21 PM
From: Steve Bradshaw
Subject: Power System for IBM i using NVME

Hi Angela

If you would like to know more about i hosting i, Larry and I recorded a webinar on IBM i hosting IBM i versus VIOS Hosting IBM i, we covered a lot of different topics, some technical some not.  I hope that this may be of use to you.  

https://youtu.be/2liJ5cDIn8M

Cheers Brad

Original Message:
Sent: 5/9/2022 9:53:00 AM
From: Angela B
Subject: RE: Power System for IBM i using NVME

Hi Steve,

Thank you for your inputs. I am checking i hosting on i solution.

Best Regards
Angela. B

------------------------------
Angela B

Original Message:
Sent: Mon May 09, 2022 07:46 AM
From: Steve Bradshaw
Subject: Power System for IBM i using NVME

Greetings one and all,

If you have not already ordered this system, it looks to me like you might benefit from using IBM i hosting IBM i rather than VIOS hosting IBM i. 

Either way when it comes to sizing your IBM i clients virtual disks, there are a number of factors to consider but the main one is to make sure you have a larger number of smaller virtual disks rather than a smaller number of larger disks.  I would suggest you start with the amount of storage you need for any one of these 10 LPARs then divide it through so that you have a minimum of 5 virtual disks, preferably more.  So, if for example one of your LPARs needed 500 GB of storage you could present 10 x 50GB virtual disks  but if they wanted 1TB of storage you might present 10 x 100GB disks.

Regards, Brad

Steve Bradshaw
Rowton IT Solutions Ltd

Unit 1, Amptronic Estate, Heath Mill Road, Wombourne, South Staffs, WV5 8AP.  
Web www.RowtonIT.com    Support Rowton.IT

Mobile +44 (0) 7970 978409

Office +44 (0) 1902 745409

Original Message:
Sent: 5/8/2022 11:37:00 PM
From: Satid Singkorapoom
Subject: RE: Power System for IBM i using NVME

I did some more research and found that when using NVMe from S922 storage backplane, this does not support split backplane. My high level idea is that you mount all 4 NVMe units onto the storage backplane and allocate 2 units to VIOS1 and the remaining 2 units to VIOS2.  Then within each VIOS, you create vSCSI disk units and I suggest you create each vDisk units at around 100GB size to ensure sufficient number to be allocated to all IBM i LPARs.  Then allocate vDisk units equally from VIOS1 and VIOS2 to each IBM i LPAR. Then within each IBM i LPAR, you start disk mirroring making sure you mirror vDisk units from VIOS1 against those from VIOS2.   

You also need to use VIOS vSCSI LVM client only (and I have no idea what this means but it is specified here : https://www.ibm.com/support/pages/ibm-i-io-support-details  )

Please keep in mind that, since split-backplane is not supported for NVMe, all NVMe units are under a single controller, If the controller fails, all disk units are gone.  The mirroring in IBM i LPAR is therefore at disk-unit-level only.   From my past experience, the base controller rarely fails.   

Please also check this idea with someone who has experience with setting up VIOS to confirm its feasibility.


------------------------------
Satid Singkorapoom

Original Message:
Sent: Fri May 06, 2022 12:59 PM
From: Angela B
Subject: Power System for IBM i using NVME

Hi Satid,

Thanks for your reply. Training workloads are moderate on both counts. So if I do disk mirroring for 2 (3.2TB) and another 2 (3.2TB)disks, I will get 6.4TB after mirroring. Can I install VIOS1 on same two disk(6.4TB) and create virtual disks from same VIOS1 for IBM i LPARs. Same for VIOS2, assigning 6.4TB (both 3.2 TB disks), install VIOS2 and create virtual disks for IBM i LPARs? Thanks for namespace guidance and CPU uncapping information.

With Regards
Angela


------------------------------
Angela B

Original Message:
Sent: Fri May 06, 2022 08:25 AM
From: Satid Singkorapoom
Subject: Power System for IBM i using NVME

Dear Angela

Are the training workloads in those 10 IBM i LPARs CPU-bound or disk IO-bound or just moderate on both counts?   If CPU-bound or moderate, I would go for whichever config that costs less.  But if both configs cost in a comparable range of investment, going for all-NVMe is better as you can hope to enjoy decent disk IO performance. 

I notice you do not want to use disk mirroring for your first config.  Is this intentional?  If you actually want disk mirroring (RAID-5 not supported for NVMe), you need to buy 6.4TB units.   Please also configure NVMe namespace of 188GB rather than 393GB as IBM i likes more disk units (the configured namespace become disk unit size for IBM i).

BTW, Do the 10 LPARs carry workloads at the same time consistently? If not, please use "Uncapped Partitioning" feature so that LPARs that are carrying workloads can steal CPU power from neighboring LPARs that are idle or use little CPU power.  

Just my 2 cents.

------------------------------
Satid Singkorapoom

Original Message:
Sent: Fri May 06, 2022 05:41 AM
From: Angela B
Subject: Power System for IBM i using NVME

Hi,

We are building a test system for internal training purpose. Plan is to build 10 IBM i LPAR on S922 with 1TB disk size, no SAN, only vSCSI and dual VIOS. Any idea what should be the best option to size Power System. We have sized RAM/CPU, 2 10G NICs, no HBAs, as we donot need tape and backup.
I see two options.
1. Have four 3.2TB NVMEs installed in CEC, we get 12.8TB size. Assign those NVMEs to VIOS and create virtual disk from those NVME and assign to IBM i LPAR. Personally I have not done this method. I did option 2 below. Has anyone did that and can share more details ?
2. Have 4 SSD for VIOS in CEC and add disk drawer for 10TB size and attach via dual SAS adapter, one to VIOS1 and other to VIOS2. Create virtual disks and assign to IBM I LPAR.

Now a days standard is to use SAN via External Storage. We donot need SAN infrastructure.

Anyone can share best method for this.

Regards
Angela

------------------------------
Angela B
------------------------------

Original Message:
Sent: 5/10/2022 5:17:00 AM
From: Angela B
Subject: RE: Power System for IBM i using NVME

Thank You Brad.

One Question - Do NVMe support hardware level encryption ? I don't see latest information, but some blogs say, it does not support hardware encryption for IBMi.

Kind Regards
Angela B.

------------------------------
Angela B
------------------------------

Original Message:
Sent: Mon May 09, 2022 12:21 PM
From: Steve Bradshaw
Subject: Power System for IBM i using NVME

Hi Angela

If you would like to know more about i hosting i, Larry and I recorded a webinar on IBM i hosting IBM i versus VIOS Hosting IBM i, we covered a lot of different topics, some technical some not.  I hope that this may be of use to you.  

https://youtu.be/2liJ5cDIn8M

Cheers Brad

Original Message:
Sent: 5/9/2022 9:53:00 AM
From: Angela B
Subject: RE: Power System for IBM i using NVME

Hi Steve,

Thank you for your inputs. I am checking i hosting on i solution.

Best Regards
Angela. B

------------------------------
Angela B

Original Message:
Sent: Mon May 09, 2022 07:46 AM
From: Steve Bradshaw
Subject: Power System for IBM i using NVME

Greetings one and all,

If you have not already ordered this system, it looks to me like you might benefit from using IBM i hosting IBM i rather than VIOS hosting IBM i. 

Either way when it comes to sizing your IBM i clients virtual disks, there are a number of factors to consider but the main one is to make sure you have a larger number of smaller virtual disks rather than a smaller number of larger disks.  I would suggest you start with the amount of storage you need for any one of these 10 LPARs then divide it through so that you have a minimum of 5 virtual disks, preferably more.  So, if for example one of your LPARs needed 500 GB of storage you could present 10 x 50GB virtual disks  but if they wanted 1TB of storage you might present 10 x 100GB disks.

Regards, Brad

Steve Bradshaw
Rowton IT Solutions Ltd

Unit 1, Amptronic Estate, Heath Mill Road, Wombourne, South Staffs, WV5 8AP.  
Web www.RowtonIT.com    Support Rowton.IT

Mobile +44 (0) 7970 978409

Office +44 (0) 1902 745409

Original Message:
Sent: 5/8/2022 11:37:00 PM
From: Satid Singkorapoom
Subject: RE: Power System for IBM i using NVME

I did some more research and found that when using NVMe from S922 storage backplane, this does not support split backplane. My high level idea is that you mount all 4 NVMe units onto the storage backplane and allocate 2 units to VIOS1 and the remaining 2 units to VIOS2.  Then within each VIOS, you create vSCSI disk units and I suggest you create each vDisk units at around 100GB size to ensure sufficient number to be allocated to all IBM i LPARs.  Then allocate vDisk units equally from VIOS1 and VIOS2 to each IBM i LPAR. Then within each IBM i LPAR, you start disk mirroring making sure you mirror vDisk units from VIOS1 against those from VIOS2.   

You also need to use VIOS vSCSI LVM client only (and I have no idea what this means but it is specified here : https://www.ibm.com/support/pages/ibm-i-io-support-details  )

Please keep in mind that, since split-backplane is not supported for NVMe, all NVMe units are under a single controller, If the controller fails, all disk units are gone.  The mirroring in IBM i LPAR is therefore at disk-unit-level only.   From my past experience, the base controller rarely fails.   

Please also check this idea with someone who has experience with setting up VIOS to confirm its feasibility.


------------------------------
Satid Singkorapoom

Original Message:
Sent: Fri May 06, 2022 12:59 PM
From: Angela B
Subject: Power System for IBM i using NVME

Hi Satid,

Thanks for your reply. Training workloads are moderate on both counts. So if I do disk mirroring for 2 (3.2TB) and another 2 (3.2TB)disks, I will get 6.4TB after mirroring. Can I install VIOS1 on same two disk(6.4TB) and create virtual disks from same VIOS1 for IBM i LPARs. Same for VIOS2, assigning 6.4TB (both 3.2 TB disks), install VIOS2 and create virtual disks for IBM i LPARs? Thanks for namespace guidance and CPU uncapping information.

With Regards
Angela


------------------------------
Angela B

Original Message:
Sent: Fri May 06, 2022 08:25 AM
From: Satid Singkorapoom
Subject: Power System for IBM i using NVME

Dear Angela

Are the training workloads in those 10 IBM i LPARs CPU-bound or disk IO-bound or just moderate on both counts?   If CPU-bound or moderate, I would go for whichever config that costs less.  But if both configs cost in a comparable range of investment, going for all-NVMe is better as you can hope to enjoy decent disk IO performance. 

I notice you do not want to use disk mirroring for your first config.  Is this intentional?  If you actually want disk mirroring (RAID-5 not supported for NVMe), you need to buy 6.4TB units.   Please also configure NVMe namespace of 188GB rather than 393GB as IBM i likes more disk units (the configured namespace become disk unit size for IBM i).

BTW, Do the 10 LPARs carry workloads at the same time consistently? If not, please use "Uncapped Partitioning" feature so that LPARs that are carrying workloads can steal CPU power from neighboring LPARs that are idle or use little CPU power.  

Just my 2 cents.

------------------------------
Satid Singkorapoom

Original Message:
Sent: Fri May 06, 2022 05:41 AM
From: Angela B
Subject: Power System for IBM i using NVME

Hi,

We are building a test system for internal training purpose. Plan is to build 10 IBM i LPAR on S922 with 1TB disk size, no SAN, only vSCSI and dual VIOS. Any idea what should be the best option to size Power System. We have sized RAM/CPU, 2 10G NICs, no HBAs, as we donot need tape and backup.
I see two options.
1. Have four 3.2TB NVMEs installed in CEC, we get 12.8TB size. Assign those NVMEs to VIOS and create virtual disk from those NVME and assign to IBM i LPAR. Personally I have not done this method. I did option 2 below. Has anyone did that and can share more details ?
2. Have 4 SSD for VIOS in CEC and add disk drawer for 10TB size and attach via dual SAS adapter, one to VIOS1 and other to VIOS2. Create virtual disks and assign to IBM I LPAR.

Now a days standard is to use SAN via External Storage. We donot need SAN infrastructure.

Anyone can share best method for this.

Regards
Angela

------------------------------
Angela B
------------------