fteCleanAgent is a powerful utility that allows users to reset their MQ Managed File Transfer (MFT) agents to a clean, empty state. In this blog post, we’ll look at how to use it to remove managed transfers from an agent.
How agents store information about managed transfers.
Before we do that though, it’s worth spending a few minutes talking about how agents keep track of managed transfers.
Each agent maintains its own state store, which holds details of the managed transfers that it needs to process. Entries are added to the store when an agent receives a request to participate in a managed transfer, are updated as the transfer progresses and are removed when it completes.
Now, managed transfers take place between pairs of agents – one agent will act as the source agent, and the other will be the destination agent. Because of this, the state store is divided into three sections:
- The first section holds the state of the managed transfers that the agent is the source agent for.
- The second section contains state information for managed transfers that the agent is the destination agent for.
- The final section is the “queued” list (sometimes referred to as the agent’s backlog). This contains details of new managed transfer requests that are waiting to be processed, and managed transfers that are in recovery waiting to restart.
This means that, when a managed transfer is in progress, there will be an entry for it in both the source and destination agent’s state store, as shown in the diagram below:
Figure 1: Here, AGENT1 is the source agent for Transfer-A, and AGENT2 is the destination agent. Because of this, both agents have an entry for it in their state store.
In terms of the implementation, the agent uses the SYSTEM.FTE.STATE.<agent name> queue for the state store. There is one message on the queue for every entry in the store.
How to use fteCleanAgent to remove a managed transfer.
Now, one of the options that can be passed into the fteCleanAgent command is -transfers (or -trs). When the command is run with this option specified, it will remove all of the messages from the SYSTEM.FTE.STATE.<agent name> queue, which essentially cleans out the state store and gets rid of all of the managed transfers that the agent knows above.
As we discussed above, managed transfers take place between pairs of agents. This means that if one agent is cleaned and details of managed transfers are removed from its state store, then the other agent involved in those transfers also needs to be cleaned too. If you don’t do that, then you can end up in the situation where one agent knows about a managed transfer and the other agent doesn’t, which will result in an entry being taken up in the state store for a transfer that will never complete.
Figure 2: fteCleanAgent has been run with the -transfers option on AGENT1 and not on AGENT2. This means that AGENT2 still has an entry for Transfer-A in its state store, even though that transfer will never complete.
You are probably thinking “If I run fteCleanAgent -transfers on one agent, how do I know what other agent (or agents) to clean?”
The good news is that the output from the command gives you this information! When the command is run, it shows the transfer identifiers for those transfers that it has removed from the state store along with the name of the other agent in the transfer. You can then run fteCleanAgent with the -transfers option on that agent too.
Let’s look at example of how to do this.
Suppose there’s been an issue with AGENT1 which means that it needs to be reset to a clean transfer state. The first thing I need to do is stop the agent, to prevent it from trying to access its state store while the clean operation is taking place. I can do this using the following command:
fteStopAgent -i AGENT1
Notice that the agent needs to be stopped with the -i option specified to force an immediate shutdown. Without this, the agent will try to stop gracefully and wait for any inflight managed transfers to complete – however, we don’t want that as any managed transfers associated with this agent need to be discarded.
Now, I can run the fteCleanAgent command:
fteCleanAgent -transfers AGENT1
This generates the following output:
All messages will be deleted from the state and command queues
State Queue Entries:
Transfer Identifier: 414d51207061756c745639334c545320349bae6501c51140
Source Agent Name: AGENT1
Destination Agent Name: AGENT2
Command Queue New Transfer Entries:
BFGCL0149I: The agent 'AGENT1' has been cleaned.
There are a couple of things of interest here.
The first is that the command has also tried to remove new transfer request messages from the agent’s command queue (SYSTEM.FTE.COMMAND.<agent name>). These transfer requests haven’t been picked up by the agent yet, and so do not have a corresponding state store.
The second thing is that the output shows that the managed transfer with transfer identifier 414d51207061756c745639334c545320349bae6501c51140 has been removed from AGENT1’s state store. Based on this, we can see that the destination agent for this transfer is AGENT2, so we need to clean that one too.
Once again, we need to stop that agent first using the following command:
fteStopAgent -i AGENT2
The -i option is needed here, to ensure that AGENT2 doesn’t wait for the managed transfer 414d51207061756c745639334c545320349bae6501c51140 to finish before stopping (because AGENT1 has no longer has any knowledge of this transfer, it won’t try and progress it)
After the agent has stopped, we can run the fteCleanAgent command for it:
fteCleanAgent -transfers AGENT2
The output from this command looks like this:
All messages will be deleted from the state and command queues
State Queue Entries:
Transfer Identifier: 414d51207061756c745639334c545320349bae6501c51140
Source Agent Name: AGENT1
Destination Agent Name: AGENT2
Command Queue New Transfer Entries:
BFGCL0149I: The agent 'AGENT2' has been cleaned.
which shows that the managed transfer has also been removed from this agent’s state store too.
At this point, both agents have an empty state store and don’t have any transfers registered. They can then be restarted, ready to do more work.
fteCleanAgent and complex topologies.
The example above is nice and simple – there are just two agents (AGENT1 and AGENT2) involved in a single managed transfer.
However, its possible that you might have a complex topology, where a single agent is taking part in multiple managed transfers with lots of different agents at the same time. In this situation, if fteCleanAgent is used to remove transfers from an agent, then it will need to be run for all the other agents that the original agent was working with. When the other agents are cleaned, you will need to clean any agents that they were working with too, and so on.
For example, suppose we have this configuration:
Figure 3: Transfer-A, Transfer-B and Transfer-C are registered with the four agents in an MFT topology.
If we initially run fteCleanAgent -transfers on AGENT1, it will remove Transfer-A and Transfer-B from that agent’s state store. This would leave the state stores looking like this:
Figure 4: AGENT1 is cleaned, and Transfer-A and Transfer-B are removed from its state store. However, those transfers are still in the state store for AGENT2 and AGENT3.
So, now, we need to run fteCleanAgent -transfers on AGENT2 and AGENT3 to clean their state stores. Once this has been done, the state stores look like this:
Figure 5: AGENT2 and AGENT3 are cleaned, and details about Transfer-B and Transfer-C are removed from their state stores. However, AGENT4 still knows about Transfer-C.
Finally, we need to run fteCleanAgent -transfers against AGENT4 too. This will remove the entry for Transfer-C from its state store, and return the topology to a clean state.
Figure 6: fteCleanAgent is run for AGENT4, and the entry for Transfer-C is removed from its state store. The topology has now been restored to a clean state.
I hope this helps! If you have any questions on this, feel free to ask and I’ll be happy to answer them.