MQ

What does the MFT agent property “maxInlineFileSize” do?

By Paul Titheridge posted Thu September 17, 2020 10:35 AM

  

I was talking to a colleague the other day about the agent property maxInlineFileSize. The description for this property in the The MFT agent.properties topic in IBM Knowledge Center says this:

For single file-to-file, or file-to-message transfers, the maximum file size (in bytes) that can be automatically included in the initial transfer request message.

You can use this property to improve the speed of your transfers, but if you set the file size to too large a value, this might degrade performance. A suggested initial size for this property is 100 KB but you are recommended to thoroughly test different values until you find the best file size for your system.

They had read this description, and asked “What does this actually mean?”

This is a very good question! Setting this property enables something called inline transfers. In order to understand what these are and how they work, we need to spend a few minutes looking at how normal managed transfers work at a high level.

Transfer Negotiation

The first stage in a managed transfer is the negotiation phase. In this stage, the source agent sends an “initial transfer request" message (sometimes known as a negotiation message) to the destination agent, to check that it is OK to participate in the managed transfer.

The destination agent picks up the request, and performs some checks to make sure it has the capacity to process the managed transfer. If it is already acting as the destination agent for its maximum number of managed transfers (as specified by the maxDestinationTransfers property), then it sends back a reply indicating that it is currently too busy. In this situation, the source agent will wait for a bit before asking again. However, if the agent does have the capacity to take part in the managed transfer, then it sends back a reply indicating that it is good to go.

Figure 2: The destination agent performs some checks to see if it can take part in the managed transfer, and sends back an

 

Data Transfer

Once the negotiation has completed, the source agent will read the data for the transfer item that it needs to transfer, put into "transfer data" messages and send it over to the destination agent. The destination agent reads theses messages, extracts the data and then writes it out to the destination file.

Figure 3: The source agent reads the file data, puts it into one or more


By default, each "transfer data" message contains 32kb of transfer item data.

The destination agent sends "transfer data acknowledgement" messages back to the source agent, indicating that it has received the "transfer data" messages,  extracted the data from them and successfully written it out to the destination item. These messages allow the source agent to know how the managed transfer is progressing.

Figure 4: After the destination agent has processed


Transfer Completion

After the destination agent has received the last "transfer data" message and written the destination file, it sends a "transfer complete" message to the source agent indicating that it has finished processing this managed transfer. The source agent receives that message, performs any post-processing of the managed transfer (such as deleting the source item if the source disposition was set to “delete”) and then puts the managed transfer into a “Completed” state.

Figure 5: After all of the file data has been processed, the destination agent sends back a

 

How inline transfers work

Now, as you can see, there is a lot of communication between the source and destination agents during a managed transfer. This means that if a managed transfer is transferring a small file, then a significant proportion of the total time taken to perform the transfer is taken up by the transfer negotiation and transfer completion stages.

This is where inline transfers can help.

These types of transfer are enabled by setting the maxInlineFileSize property on a source agent to a positive value. When it is turned on, then if the size of the source item is less than the value of the property, the source agent will read the data for that file and include it in the "Initial transfer request" message that it sends to the destination agent.

When the destination agent receives the transfer negotiation message, it will perform its usual check to see if it has the capacity to participate in the managed transfer. If it has, then it extracts the file data from the "Initial transfer request" message and writes it out to the destination file.

Figure 6: When inline transfers are being used, the source agent includes all of the file data in the


The destination agent then sends back a "Transfer complete" message. When the source agent receives this, it performs the post-processing of the managed transfer, and then moves it into a “Completed” state.

This can improve the time taken for a managed transfer to complete, as the source and destination agents only need to exchange two messages in order to process it.

One thing to note is inline transfers can only be used for file-to-file or file-to-message transfers. They cannot be used for message-to-file transfers.

 

As always, I hope this helps! If you have any questions on this, feel free to ask and I’ll be happy to answer them.


#MQ

Permalink