Originally posted by: SystemAdmin

I have one set of maps that are run on the event server with a system, that create an output that becomes the input for another system once per hour. The second system has a time trigger and reads in the file that was created by the first set of maps. Originally we had just the time trigger set and everything was working great but when there wasn't any data to process, a 'Source Not Available' error was written to the launcher.txt which we monitor using Sitescope. Since these errors are essentially false alarms, I need to alleviate the problem.

I tried to do this by turning on the Source Event for the file in the folder and adding "All of NONE" to the Event Coordination settings in the IFD. I thought that this would prevent the system from initiating if the file didn't exist, but this completely backfired. As soon as the file appeared in the folder, the second set of maps locked it and nothing would process in the first set of maps.

We are back to getting the false alarms and I am out of ideas on how to solve this. Does anyone have any suggestions?
#IBM-Websphere-Transformation-Extender
#IBMSterlingTransformationExtender
#DataExchange

Originally posted by: SystemAdmin

Hello Jennifer,

we went into the same problem during an upgrade from version 1.4.2 (where the second map running time and event triggered and didn't locked the file) to version 6.7.1. The only way out was running a map time triggered, which calls a batch file and the batch file renames the output from the first map into a file, which trigger the second map. This is a safe way, because if the first map ist just appending date to it's output file and the bacyh file want's to rename the file, then the renaming is not possible, because the file is locked by the event server. The file will be renamed with the next run of the batch file.

Hope this will solve your problem, we did it in the same way and we didn't had any problems until now.

Joerg
#IBMSterlingTransformationExtender
#DataExchange
#IBM-Websphere-Transformation-Extender

Originally posted by: SystemAdmin

Timing conflicts always warrant close examination. A long-used technique that can assist is the use of semaphore files or messages. Initial processes creates a signal upon completion. It might be an empty file, a file containing a list of files to process, or a triggering message on a queue. Your secondary process will now look for this signal and only execute if it is present.

In the case of DSTX, you can define an additional output card in your first map that creates the signal. You then specify the second map or system to execute upon BOTH a timed base and an event basis. The time signal is your scheduled execution and the event is the presence of the semaphore file or message.

There is more than one way way to implement what you want but basically you want to decouple sequential map executions and define a scenario that ensures your requirements are met before a dependent, subsequent map executes.
#IBMSterlingTransformationExtender
#DataExchange
#IBM-Websphere-Transformation-Extender

Originally posted by: SystemAdmin

Jim:

I tried to think of a way to implement your suggestion but each instance left me with the same scenario. The problem is that my initial process runs continuously but sometimes not at all for hours. The maps are executed repeatedly over the course of an hour. Then the output file is translated by the second set of maps and a new version of the original file is created. No matter which way I work it, if the original maps do not run for an hour, I end up with a process that is either locking a file or generating the error message because the source isn't available.

If the original process creates the dummy file and the second process has a source event on that, everything locks up. If I have a time trigger on it, I get another process generating a "source not available" error.

I can't create it independent of the original maps because then I may end up with a dummy file and no real file.

I also can't use the real data as a source event for the dummy map because then that process would lock it once created.

We tried moving the file to a different folder and ended up with dropped transactions.

Any suggestions are much appreciated!
#DataExchange
#IBMSterlingTransformationExtender
#IBM-Websphere-Transformation-Extender

Originally posted by: SystemAdmin

I was suggesting that perhaps you can have a map that only executes when a file exists and the time is right. You can set both conditions in the Integration Flow Designer by defining a Time & a Source Event. You then set Time Event-->Event Coordination to All or None. The map doesn't execute until both requirements are fulfilled.

It sounds like you need to decouple your initial process from the subsequent one. The question is "How?"

You usually never want a map to ever access a file that is created by a continuous or even frequently executed map. This might be affected by having the first process create multiple, uniquely named files and a secondary process creates a list of available files. The next process in the flow processes the list of files. Meanwhile the initial process can proceed independently.

One technique that may be useful is an intermediary map that has map rules of the type"
Rule 1: GET("SomeFile")
Rule 2: IF(LASTERRORCODE() = 0),
RUN("SomeMap.mmc") )

This means SomeMap.mmc won't execute unless SomeFile exists or any condition or conditions you test for in the previous map rule.

Also, look at the Retries map setting. It can help avoid conflicts. I have used it on both the initial and subsequent map to allow some slop time. Never continuously append to a file that you want another process to pick up. Clashes are are almost guaranteed.

I have some maps that solved a similar problem on my old computer. I'll try and take a look over the weekend.
#DataExchange
#IBM-Websphere-Transformation-Extender
#IBMSterlingTransformationExtender

Originally posted by: SystemAdmin

I think you have two different issues here.

1) Objective: Trigger once per hour if file exists. If you have a time event and a source event and have set Event Coordination to "All or NONE" this is what you will get. If you don't, call support. (If your map could run longer than the schedule, you may also want to set Skip if Busy to Yes on the time event.)

2) Objective: Allow Map 1 to run and write to file while Map 2 is reading it (?) Not sure if I am understanding you properly because I can't see why you would want to do this or why you think it should work. Maybe you can clarify? If Map 1 is to run continuously and Map 2 is to grab whatever has accumulated on an hourly basis but you want Map 1 to keep processing even while Map 2 does its thing you should (i) have a "blob" map (type tree has one element for the whole file) read the file in and write it out to a new directory or a new name and delete the input, or (ii) use a database.
#IBM-Websphere-Transformation-Extender
#DataExchange
#IBMSterlingTransformationExtender

Originally posted by: SystemAdmin

Why does the second set of maps have to wait an hour to process?

If you need to accumulate data, then process it in a batch mode, you might want to send the data to a database and trigger a map once an hour to get any records not marked as processed. The second set of maps would update the rows processed so they don't go through again, and you could also run a clean up map to delete process rows if you don't need the data.
#IBM-Websphere-Transformation-Extender
#DataExchange
#IBMSterlingTransformationExtender

Originally posted by: SystemAdmin

The easiest solution is to have map 1 with 2 output cards. It runs on the time trigger. Output card 1 does a GET on the file required. If it exists run your process map from card 2 otherwise terminate map 1.
#IBM-Websphere-Transformation-Extender
#DataExchange
#IBMSterlingTransformationExtender