As customers look to move from IIOP based communications to WSI between applications and the CPE.

Are there functions, features, performance metrics, etc.. to watch out for?

Large File Upload / Download / View are some areas of particular interest. As well as configurations for Custom Apps as the client.

I had asked the question about Navigator previously here: QuestionDetail - Forums - IBM Support, but would like some feedback on  CPE and custom apps as well.

Thanks.

------------------------------
Chuck Hauble
Minneapolis MN
------------------------------

Hi Chuck,

the most important restriction I'm aware of is that your application cannot participate in client initiated JEE transactions based on JTA when using the WSI transport  (see section "Transport differences" in https://www.ibm.com/docs/en/filenet-p8-platform/5.5.x?topic=guide-getting-started). I rarely saw custom applications which relied on JTA for content manipulation but if this is the case, a re-write of the transaction logic including manual compensation will be required.

From a performance PoV, I do not remember that we saw significant differences between both transports. In fact, our CEBIT tool as well as the Expert Labs "BatchIt" importer always leveraged the WSI transport (which obviously only addresses the "content upload" use case).

@RUTH Hildebrand-Lund Maybe you can chime in here?

Best regards,
Michael

​​

------------------------------
Michael Kirchner
Leading Technical Specialist - Digital Business Automation
IBM Technology
Germany
------------------------------

Michael K is correct, that there is generally very little difference in performance between the WSI and EJB transports. There's a slight amount of additional overhead in the WSI transport for serializing/deserializing XML rather than the binary serialization used by EJB, but that is probably mostly lost in the noise.

However, the one area in which the WSI transport is definitely superior is in ingesting documents with content. A batch of N documents ingested over the EJB transport requires at least N+1 RPCs (network round trips) between client and server, more if the content is large, but over the WSI transport the whole batch is accomplished in one RPC. Depending on network latency and bandwidth that could be a substantial difference. So if you're looking to achieve high volume ingestion you should use the WSI transport.

------------------------------
Mike
------------------------------

Original Message:
Sent: Fri October 21, 2022 02:50 AM
From: Michael Kirchner
Subject: WSI vs IIOP Performance gotchas

Hi Chuck,

the most important restriction I'm aware of is that your application cannot participate in client initiated JEE transactions based on JTA when using the WSI transport  (see section "Transport differences" in https://www.ibm.com/docs/en/filenet-p8-platform/5.5.x?topic=guide-getting-started). I rarely saw custom applications which relied on JTA for content manipulation but if this is the case, a re-write of the transaction logic including manual compensation will be required.

From a performance PoV, I do not remember that we saw significant differences between both transports. In fact, our CEBIT tool as well as the Expert Labs "BatchIt" importer always leveraged the WSI transport (which obviously only addresses the "content upload" use case).

@RUTH Hildebrand-Lund Maybe you can chime in here?

Best regards,
Michael

​​

------------------------------
Michael Kirchner
Leading Technical Specialist - Digital Business Automation
IBM Technology
Germany
------------------------------

Do you really only see an improvement with a batch of documents from a single endpoint (client) ?

What about large single document/content item uploads?

------------------------------
Chuck Hauble
Minneapolis MN
------------------------------

Original Message:
Sent: Mon October 24, 2022 04:11 AM
From: Michael Seaman
Subject: WSI vs IIOP Performance gotchas

Michael K is correct, that there is generally very little difference in performance between the WSI and EJB transports. There's a slight amount of additional overhead in the WSI transport for serializing/deserializing XML rather than the binary serialization used by EJB, but that is probably mostly lost in the noise.

However, the one area in which the WSI transport is definitely superior is in ingesting documents with content. A batch of N documents ingested over the EJB transport requires at least N+1 RPCs (network round trips) between client and server, more if the content is large, but over the WSI transport the whole batch is accomplished in one RPC. Depending on network latency and bandwidth that could be a substantial difference. So if you're looking to achieve high volume ingestion you should use the WSI transport.

------------------------------
Mike
------------------------------


Original Message:
Sent: Fri October 21, 2022 02:50 AM
From: Michael Kirchner
Subject: WSI vs IIOP Performance gotchas

Hi Chuck,

the most important restriction I'm aware of is that your application cannot participate in client initiated JEE transactions based on JTA when using the WSI transport  (see section "Transport differences" in https://www.ibm.com/docs/en/filenet-p8-platform/5.5.x?topic=guide-getting-started). I rarely saw custom applications which relied on JTA for content manipulation but if this is the case, a re-write of the transaction logic including manual compensation will be required.

From a performance PoV, I do not remember that we saw significant differences between both transports. In fact, our CEBIT tool as well as the Expert Labs "BatchIt" importer always leveraged the WSI transport (which obviously only addresses the "content upload" use case).

@RUTH Hildebrand-Lund Maybe you can chime in here?

Best regards,
Michael

​​

------------------------------
Michael Kirchner
Leading Technical Specialist - Digital Business Automation
IBM Technology
Germany

The way this works over EJB is that the content is divided up into chunks of 1Mb and each chunk is sent to the server by a separate RPC. Then after all the content has been uploaded a further RPC is issued to execute the transaction which creates the document(s). Hence at least N+1 RPCs in the description I gave earlier, or more if any of the content elements exceeds 1Mb. For a single document with a single content element, just read N as 1.

In comparison, the WSI transport sends a single request in which the content is carried as binary attachments (streamed over the TCP connection) and the entire operation of storing the content and executing the document creation transaction is performed by the server from that one request.

The net amount of data sent by the two methods is pretty much the same, and for large content is pretty much dominated by that content, so whether or not you see a measurable difference between the two methods depends on the network latency for each request relative to the total time. If the latency is small then you may not notice the cost of the extra EJB round trips, but it is one of those "your mileage may vary" kind of things.

------------------------------
Mike
------------------------------

Original Message:
Sent: Mon October 24, 2022 04:35 PM
From: Chuck Hauble
Subject: WSI vs IIOP Performance gotchas

Do you really only see an improvement with a batch of documents from a single endpoint (client) ?

What about large single document/content item uploads?

------------------------------
Chuck Hauble
Minneapolis MN
------------------------------


Original Message:
Sent: Mon October 24, 2022 04:11 AM
From: Michael Seaman
Subject: WSI vs IIOP Performance gotchas

Michael K is correct, that there is generally very little difference in performance between the WSI and EJB transports. There's a slight amount of additional overhead in the WSI transport for serializing/deserializing XML rather than the binary serialization used by EJB, but that is probably mostly lost in the noise.

However, the one area in which the WSI transport is definitely superior is in ingesting documents with content. A batch of N documents ingested over the EJB transport requires at least N+1 RPCs (network round trips) between client and server, more if the content is large, but over the WSI transport the whole batch is accomplished in one RPC. Depending on network latency and bandwidth that could be a substantial difference. So if you're looking to achieve high volume ingestion you should use the WSI transport.

------------------------------
Mike


Original Message:
Sent: Fri October 21, 2022 02:50 AM
From: Michael Kirchner
Subject: WSI vs IIOP Performance gotchas

Hi Chuck,

the most important restriction I'm aware of is that your application cannot participate in client initiated JEE transactions based on JTA when using the WSI transport  (see section "Transport differences" in https://www.ibm.com/docs/en/filenet-p8-platform/5.5.x?topic=guide-getting-started). I rarely saw custom applications which relied on JTA for content manipulation but if this is the case, a re-write of the transaction logic including manual compensation will be required.

From a performance PoV, I do not remember that we saw significant differences between both transports. In fact, our CEBIT tool as well as the Expert Labs "BatchIt" importer always leveraged the WSI transport (which obviously only addresses the "content upload" use case).

@RUTH Hildebrand-Lund Maybe you can chime in here?

Best regards,
Michael

​​

------------------------------
Michael Kirchner
Leading Technical Specialist - Digital Business Automation
IBM Technology
Germany

Michael, Thanks for the input. Just the sort of thing I needed.

Is there more info available somewhere about CEBIT and BatchIt

------------------------------
Chuck Hauble
Minneapolis MN
------------------------------

Original Message:
Sent: Fri October 21, 2022 02:50 AM
From: Michael Kirchner
Subject: WSI vs IIOP Performance gotchas

Hi Chuck,

the most important restriction I'm aware of is that your application cannot participate in client initiated JEE transactions based on JTA when using the WSI transport  (see section "Transport differences" in https://www.ibm.com/docs/en/filenet-p8-platform/5.5.x?topic=guide-getting-started). I rarely saw custom applications which relied on JTA for content manipulation but if this is the case, a re-write of the transaction logic including manual compensation will be required.

From a performance PoV, I do not remember that we saw significant differences between both transports. In fact, our CEBIT tool as well as the Expert Labs "BatchIt" importer always leveraged the WSI transport (which obviously only addresses the "content upload" use case).

@RUTH Hildebrand-Lund Maybe you can chime in here?

Best regards,
Michael

​​

------------------------------
Michael Kirchner
Leading Technical Specialist - Digital Business Automation
IBM Technology
Germany
------------------------------

I concur with Michael that there are rather functional than performance differences.

One use case to use EJB is when you need transactional integrity between the PE and the CE (so updates need to be done for both or none). Ver few customers are that picky, but it is the serious way to do it.

I tested CEBI with EJB and WSI and the difference was never really significant or worth the additional efforts.

Hope this helps,

/Gerold

------------------------------
Gerold Krommer
------------------------------

Thanks, Yes very helpful

------------------------------
Chuck Hauble
Minneapolis MN
------------------------------

Original Message:
Sent: Mon October 24, 2022 04:21 AM
From: Gerold Krommer
Subject: WSI vs IIOP Performance gotchas

I concur with Michael that there are rather functional than performance differences.

One use case to use EJB is when you need transactional integrity between the PE and the CE (so updates need to be done for both or none). Ver few customers are that picky, but it is the serious way to do it.

I tested CEBI with EJB and WSI and the difference was never really significant or worth the additional efforts.

Hope this helps,

/Gerold

------------------------------
Gerold Krommer
------------------------------

I agree with everyone's notes here and I've certainly used both in high-throughput, high transaction volume situations very successfully.

There are some big differences, because of the functional difference in the underlying protocol, in the levers you have and can use to tune performance. WSI rides on an  HTTP connection, so your tuning is limited to the adjustable parameters of that protocol and optimizations to server-side processing tasks. EJB rides on an ORB connection which is, in my experience, a more complex to tune protocol--simply because it has more levers to pull. I have found myself spending more time and energy squeezing every drop of throughput from EJB and being more easily able to over tune or tune too heavily to a specific workload (i.e. Workflow vs Content requests). See, adjusting the 1Mb block size, @Michael Seaman noted above​, to reduce the number of RPC calls for a specific PE request type in the context of a specific workflow... Now in that extreme case, EJB outperformed WSI, but the effort required to do it was very large and the need to handle a billion plus workflow requests in 8 hours (3-5x peak production load) was a unique requirement.

The API performance concern on my mind is GraphQL... It runs in a separate context and is, effectively, translating requests from GraphQL JSON formatted REST to XML formatted SOAP and forwarding them to the WSI endpoint. I'm curious if anyone has done any testing there to determine if the overhead causes issues? I haven't had a need to so far...

------------------------------
Eric Walk
Director

O: 617-453-9983 | NASDAQ: PRFT | Perficient.com
------------------------------

Great discussion everyone.  This is exactly the sort of things I wanted to know about.

@Eric Walk thanks for bringing up GraphQL as well. It is something we are considering adding to our environment.

My original question really stems from the idea that we should look at moving workloads to containers and cloud architectures.

If we move the CPE to the Cloud Pak containerized environment, the custom applications and their developers are going to expect the same or better performance​ in the key areas a user will notice. (Uploads/downloads, etc)​

------------------------------
Chuck Hauble
Minneapolis MN
------------------------------

Original Message:
Sent: Wed October 26, 2022 08:31 AM
From: Eric Walk
Subject: WSI vs IIOP Performance gotchas

I agree with everyone's notes here and I've certainly used both in high-throughput, high transaction volume situations very successfully.

There are some big differences, because of the functional difference in the underlying protocol, in the levers you have and can use to tune performance. WSI rides on an  HTTP connection, so your tuning is limited to the adjustable parameters of that protocol and optimizations to server-side processing tasks. EJB rides on an ORB connection which is, in my experience, a more complex to tune protocol--simply because it has more levers to pull. I have found myself spending more time and energy squeezing every drop of throughput from EJB and being more easily able to over tune or tune too heavily to a specific workload (i.e. Workflow vs Content requests). See, adjusting the 1Mb block size, @Michael Seaman noted above​, to reduce the number of RPC calls for a specific PE request type in the context of a specific workflow... Now in that extreme case, EJB outperformed WSI, but the effort required to do it was very large and the need to handle a billion plus workflow requests in 8 hours (3-5x peak production load) was a unique requirement.

The API performance concern on my mind is GraphQL... It runs in a separate context and is, effectively, translating requests from GraphQL JSON formatted REST to XML formatted SOAP and forwarding them to the WSI endpoint. I'm curious if anyone has done any testing there to determine if the overhead causes issues? I haven't had a need to so far...

------------------------------
Eric Walk
Director

O: 617-453-9983 | NASDAQ: PRFT | Perficient.com
------------------------------

I've had incredibly good experiences with WSI and containers at super massive scale (30-50 CPE instances in a cluster).

Comparing traditional CPE to container CPE, there's no real noticeable performance difference. I just can't imagine running a 30-node cluster of traditional CPE without wanting to tear my hair out. That client has been using containers since before graphql came out, so they're all WSI all the time and the only performance issue anyone has is that request times for uploads increase, mostly, linearly with the size of the file being uploaded, but that's normal and not related to the nature of the containers.

The biggest benefit to containers is reduced maintenance overhead and significantly simplified patches and upgrades. You can get an interim fix deployed, especially if you use the operator, in under an hour with zero downtime, and the only difference with bigger upgrades is that you need 10-20mins outage for AutoUpgrade to do the schema changes. The work IBM has put into the automation has been a real coup.

------------------------------
Eric Walk
Director

O: 617-453-9983 | NASDAQ: PRFT | Perficient.com
------------------------------

Original Message:
Sent: Wed October 26, 2022 08:53 AM
From: Chuck Hauble
Subject: WSI vs IIOP Performance gotchas

Great discussion everyone.  This is exactly the sort of things I wanted to know about.

@Eric Walk thanks for bringing up GraphQL as well. It is something we are considering adding to our environment.

My original question really stems from the idea that we should look at moving workloads to containers and cloud architectures.

If we move the CPE to the Cloud Pak containerized environment, the custom applications and their developers are going to expect the same or better performance​ in the key areas a user will notice. (Uploads/downloads, etc)​

------------------------------
Chuck Hauble
Minneapolis MN
------------------------------


Original Message:
Sent: Wed October 26, 2022 08:31 AM
From: Eric Walk
Subject: WSI vs IIOP Performance gotchas

I agree with everyone's notes here and I've certainly used both in high-throughput, high transaction volume situations very successfully.

There are some big differences, because of the functional difference in the underlying protocol, in the levers you have and can use to tune performance. WSI rides on an  HTTP connection, so your tuning is limited to the adjustable parameters of that protocol and optimizations to server-side processing tasks. EJB rides on an ORB connection which is, in my experience, a more complex to tune protocol--simply because it has more levers to pull. I have found myself spending more time and energy squeezing every drop of throughput from EJB and being more easily able to over tune or tune too heavily to a specific workload (i.e. Workflow vs Content requests). See, adjusting the 1Mb block size, @Michael Seaman noted above​, to reduce the number of RPC calls for a specific PE request type in the context of a specific workflow... Now in that extreme case, EJB outperformed WSI, but the effort required to do it was very large and the need to handle a billion plus workflow requests in 8 hours (3-5x peak production load) was a unique requirement.

The API performance concern on my mind is GraphQL... It runs in a separate context and is, effectively, translating requests from GraphQL JSON formatted REST to XML formatted SOAP and forwarding them to the WSI endpoint. I'm curious if anyone has done any testing there to determine if the overhead causes issues? I haven't had a need to so far...

------------------------------
Eric Walk
Director

O: 617-453-9983 | NASDAQ: PRFT | Perficient.com

Super discussion - very valuable! Thanks for sharing!
This community is just the place to do it. Thanks again.

------------------------------
Christoph Sünderkamp
------------------------------

Original Message:
Sent: Wed October 26, 2022 08:53 AM
From: Chuck Hauble
Subject: WSI vs IIOP Performance gotchas

Great discussion everyone.  This is exactly the sort of things I wanted to know about.

@Eric Walk thanks for bringing up GraphQL as well. It is something we are considering adding to our environment.

My original question really stems from the idea that we should look at moving workloads to containers and cloud architectures.

If we move the CPE to the Cloud Pak containerized environment, the custom applications and their developers are going to expect the same or better performance​ in the key areas a user will notice. (Uploads/downloads, etc)​

------------------------------
Chuck Hauble
Minneapolis MN
------------------------------


Original Message:
Sent: Wed October 26, 2022 08:31 AM
From: Eric Walk
Subject: WSI vs IIOP Performance gotchas

I agree with everyone's notes here and I've certainly used both in high-throughput, high transaction volume situations very successfully.

There are some big differences, because of the functional difference in the underlying protocol, in the levers you have and can use to tune performance. WSI rides on an  HTTP connection, so your tuning is limited to the adjustable parameters of that protocol and optimizations to server-side processing tasks. EJB rides on an ORB connection which is, in my experience, a more complex to tune protocol--simply because it has more levers to pull. I have found myself spending more time and energy squeezing every drop of throughput from EJB and being more easily able to over tune or tune too heavily to a specific workload (i.e. Workflow vs Content requests). See, adjusting the 1Mb block size, @Michael Seaman noted above​, to reduce the number of RPC calls for a specific PE request type in the context of a specific workflow... Now in that extreme case, EJB outperformed WSI, but the effort required to do it was very large and the need to handle a billion plus workflow requests in 8 hours (3-5x peak production load) was a unique requirement.

The API performance concern on my mind is GraphQL... It runs in a separate context and is, effectively, translating requests from GraphQL JSON formatted REST to XML formatted SOAP and forwarding them to the WSI endpoint. I'm curious if anyone has done any testing there to determine if the overhead causes issues? I haven't had a need to so far...

------------------------------
Eric Walk
Director

O: 617-453-9983 | NASDAQ: PRFT | Perficient.com