IBM i Global

Does anyone know why I can download 50+megabytes of a video from YouTube or Netflix in a couple seconds, but reading 50MB from the IFS takes a very long time to load? 

I know, apples and oranges, but geeze, I remember how fast IFS *TYPE2 was when it came out compared to *TYPE1, but perhaps its time to advocate for *TYPE3? 

------------------------------
Robert Cozzi
------------------------------

This news piece may be of interest to you in this matter : Multi-Threaded IFS Streaming HA DR - Disaster Recovery for IBM i at  http://www.cn.maxava.guru/2015/03/03/product-release-multi-threaded-ifs/.  

IBM i users know that IFS replication speeds have long been an Achilles heel of some real-time replication solutions. Until now, IBM i replication software solutions have typically only been able to run IBM i IFS replication in a single-thread mode or with very limited fixed number multi-threading – drip-feeding data to the backup.  Now, Maxava HA offers the first fully dynamic multi-threaded IFS replication, for up to 255 parallel IFS dedicated replication processes, bringing IFS replication into line with the best of multi-streamed data and object replication.

------------------------------
Chance favors only the prepared mind.
-- Louis Pasteur
------------------------------
Satid S.
------------------------------

Original Message:
Sent: Thu December 14, 2023 06:51 PM
From: Robert Cozzi
Subject: IFS read/write performance

Does anyone know why I can download 50+megabytes of a video from YouTube or Netflix in a couple seconds, but reading 50MB from the IFS takes a very long time to load? 

I know, apples and oranges, but geeze, I remember how fast IFS *TYPE2 was when it came out compared to *TYPE1, but perhaps its time to advocate for *TYPE3? 

------------------------------
Robert Cozzi
------------------------------

Having all SSD units with no spinning disk at all positively contributes to IFS access performance as well. 

I think that the next IFS TYPE2 enhancement is for Rochester to deliver a way we can manually specify the degree of multithreading access to IFS files (QDLS is presumably out of the possibility for this support).  This will be good in the case of IBM i LPAR with many CPU cores at its disposal.

------------------------------
Chance favors only the prepared mind.
-- Louis Pasteur
------------------------------
Satid S.
------------------------------

Original Message:
Sent: Thu December 14, 2023 06:51 PM
From: Robert Cozzi
Subject: IFS read/write performance

Does anyone know why I can download 50+megabytes of a video from YouTube or Netflix in a couple seconds, but reading 50MB from the IFS takes a very long time to load? 

I know, apples and oranges, but geeze, I remember how fast IFS *TYPE2 was when it came out compared to *TYPE1, but perhaps its time to advocate for *TYPE3? 

------------------------------
Robert Cozzi
------------------------------

Have you tried it on a currently supported version of the operating system?  I was reading a little on this:

------------------------------
Robert Berendt IBMChampion
------------------------------

Original Message:
Sent: Thu December 14, 2023 06:51 PM
From: Robert Cozzi
Subject: IFS read/write performance

Does anyone know why I can download 50+megabytes of a video from YouTube or Netflix in a couple seconds, but reading 50MB from the IFS takes a very long time to load? 

I know, apples and oranges, but geeze, I remember how fast IFS *TYPE2 was when it came out compared to *TYPE1, but perhaps its time to advocate for *TYPE3? 

------------------------------
Robert Cozzi
------------------------------

Thanks Rob, I'll look at that link. I didn't even know there was a 512Kb buffer/size consideration to be aware of. 

Although I'm referring to core IFS TYPE2 file access, like those in /home/cozzi I don't use most legacy file systems like QDLS or QNTC.

------------------------------
Robert Cozzi
------------------------------

Original Message:
Sent: Mon December 18, 2023 08:24 AM
From: Robert Berendt
Subject: IFS read/write performance

Have you tried it on a currently supported version of the operating system?  I was reading a little on this:

7.4

https://www.ibm.com/support/pages/node/1164634

NetServer and QNTC updated to SMB3

Support larger read/write sizes (512 Kb) to improve performance in high latency networks 

End to end data encryption for entire client / server conversations 

 

Also, there are settings which may enhance performance

https://www.ibm.com/docs/en/i/7.5?topic=qntc-environment-variables

QIBM_ZLC_SMB_VERS: ... this can improve the performance of reading/writing operations.

Of course, the version mentioned is not supported in obsolete versions of IBM i, like 7.3.

------------------------------
Robert Berendt IBMChampion

Original Message:
Sent: Thu December 14, 2023 06:51 PM
From: Robert Cozzi
Subject: IFS read/write performance

Does anyone know why I can download 50+megabytes of a video from YouTube or Netflix in a couple seconds, but reading 50MB from the IFS takes a very long time to load? 

I know, apples and oranges, but geeze, I remember how fast IFS *TYPE2 was when it came out compared to *TYPE1, but perhaps its time to advocate for *TYPE3? 

------------------------------
Robert Cozzi
------------------------------

Regarding "I don't use most legacy file systems like QDLS or QNTC.".  The second link provided did pay sole homage to QNTC.  However the first link was also impactful for NetServer and access to stuff like /home/cozzi.

------------------------------
Robert Berendt IBMChampion
------------------------------

Original Message:
Sent: Tue January 30, 2024 10:08 AM
From: Robert Cozzi
Subject: IFS read/write performance

Thanks Rob, I'll look at that link. I didn't even know there was a 512Kb buffer/size consideration to be aware of. 

Although I'm referring to core IFS TYPE2 file access, like those in /home/cozzi I don't use most legacy file systems like QDLS or QNTC.

------------------------------
Robert Cozzi

Original Message:
Sent: Mon December 18, 2023 08:24 AM
From: Robert Berendt
Subject: IFS read/write performance

Have you tried it on a currently supported version of the operating system?  I was reading a little on this:

7.4

https://www.ibm.com/support/pages/node/1164634

NetServer and QNTC updated to SMB3

Support larger read/write sizes (512 Kb) to improve performance in high latency networks 

End to end data encryption for entire client / server conversations 

 

Also, there are settings which may enhance performance

https://www.ibm.com/docs/en/i/7.5?topic=qntc-environment-variables

QIBM_ZLC_SMB_VERS: ... this can improve the performance of reading/writing operations.

Of course, the version mentioned is not supported in obsolete versions of IBM i, like 7.3.

------------------------------
Robert Berendt IBMChampion

Original Message:
Sent: Thu December 14, 2023 06:51 PM
From: Robert Cozzi
Subject: IFS read/write performance

Does anyone know why I can download 50+megabytes of a video from YouTube or Netflix in a couple seconds, but reading 50MB from the IFS takes a very long time to load? 

I know, apples and oranges, but geeze, I remember how fast IFS *TYPE2 was when it came out compared to *TYPE1, but perhaps its time to advocate for *TYPE3? 

------------------------------
Robert Cozzi
------------------------------

Dear Robert

Netflix runs in cloud with practically hundred of CPU cores at its disposal for multithreaded download support.  IFS TYPE2 also utilizes multithreading for root, UDFS, and QOpenSys (and QFileSvr.400 in IBM i 7.2) ) file systems but the number of CPU core available to IBM i plays a role here. Using all SSD units and no spinning disk at all also contributes positively to the access speed.   I used to download IFS files from a customer's LPAR with 20 CPU cores (with SSD LUNs over 32 Gbps fibre) over 10Gbps LAN and it was impressively much faster than the cases with 1 or 2 cores with spinning disk. 

I see that, perhaps, a future IFS enhancement we need from Rochester is to be able to specify the mutithreading degree manually for IFS access. 

------------------------------
Chance favors only the prepared mind.
-- Louis Pasteur
------------------------------
Satid S.
------------------------------

Original Message:
Sent: Thu December 14, 2023 06:51 PM
From: Robert Cozzi
Subject: IFS read/write performance

Does anyone know why I can download 50+megabytes of a video from YouTube or Netflix in a couple seconds, but reading 50MB from the IFS takes a very long time to load? 

I know, apples and oranges, but geeze, I remember how fast IFS *TYPE2 was when it came out compared to *TYPE1, but perhaps its time to advocate for *TYPE3? 

------------------------------
Robert Cozzi
------------------------------

<geezer mode>

Many years ago we had a system where we had performance problems serving up file shares.  On that same system, using the same disk drives, we put in a FSIOP or whatever name that particular version of the card had that week.  These cards were basically Windows servers on a PC server card which plugged into your iSeries (or whatever).  But moving the share from IBM i to Windows on the same box, with the same disks, got much better performance.

</geezer mode>

Basically the version of IBM i we were running at the time was inferior for serving up file shares.  Since then IBM has put a lot of effort into improving file serving from IBM i.  So, throwing hardware at it isn't always the answer.  I've gotten burned by the "SSDs will make your performance issues go away" mentality.  We replaced all of our spinning disks on one upgrade with SSDs and did not get the performance enhancement we requested.  We got the performance boost we wanted by spending a lot less and going with 42-HA Journal Performance but that would have nothing to do with file serving.  Just another example of hardware wasn't the issue.

------------------------------
Robert Berendt IBMChampion
------------------------------

Original Message:
Sent: Fri December 15, 2023 09:04 PM
From: Satid Singkorapoom
Subject: IFS read/write performance

Dear Robert

Netflix runs in cloud with practically hundred of CPU cores at its disposal for multithreaded download support.  IFS TYPE2 also utilizes multithreading for root, UDFS, and QOpenSys (and QFileSvr.400 in IBM i 7.2) ) file systems but the number of CPU core available to IBM i plays a role here. Using all SSD units and no spinning disk at all also contributes positively to the access speed.   I used to download IFS files from a customer's LPAR with 20 CPU cores (with SSD LUNs over 32 Gbps fibre) over 10Gbps LAN and it was impressively much faster than the cases with 1 or 2 cores with spinning disk. 

I see that, perhaps, a future IFS enhancement we need from Rochester is to be able to specify the mutithreading degree manually for IFS access. 

------------------------------
Chance favors only the prepared mind.
-- Louis Pasteur
------------------------------
Satid S.

Original Message:
Sent: Thu December 14, 2023 06:51 PM
From: Robert Cozzi
Subject: IFS read/write performance

Does anyone know why I can download 50+megabytes of a video from YouTube or Netflix in a couple seconds, but reading 50MB from the IFS takes a very long time to load? 

I know, apples and oranges, but geeze, I remember how fast IFS *TYPE2 was when it came out compared to *TYPE1, but perhaps its time to advocate for *TYPE3? 

------------------------------
Robert Cozzi
------------------------------

------------------------------
Alen Wans
------------------------------

Original Message:
Sent: Thu December 14, 2023 06:51 PM
From: Robert Cozzi
Subject: IFS read/write performance

Does anyone know why I can download 50+megabytes of a video from YouTube or Netflix in a couple seconds, but reading 50MB from the IFS takes a very long time to load? 

I know, apples and oranges, but geeze, I remember how fast IFS *TYPE2 was when it came out compared to *TYPE1, but perhaps its time to advocate for *TYPE3? 

------------------------------
Robert Cozzi
------------------------------

IFS is not a network protocol. Or you mean a local read?

Anyway, if we are talking data transfer, 50MB should be downloaded to a PC in a about second with a modest nvme current machine using a a local very basic standard 1gbps LAN. Try using ACS or SSH/SFTP or Apache/HTTP if that changes.

------------------------------
--ft
------------------------------

Original Message:
Sent: Thu December 14, 2023 06:51 PM
From: Robert Cozzi
Subject: IFS read/write performance

Does anyone know why I can download 50+megabytes of a video from YouTube or Netflix in a couple seconds, but reading 50MB from the IFS takes a very long time to load? 

I know, apples and oranges, but geeze, I remember how fast IFS *TYPE2 was when it came out compared to *TYPE1, but perhaps its time to advocate for *TYPE3? 

------------------------------
Robert Cozzi
------------------------------