Originally posted by: SystemAdmin

Hi,

I try to run a map on our HP-UX 11i server with the dstx command and I have this error message:

Ascential DataStage(TM) TX HP-UX(TM) Command Server - version 7.5(119)
Copyright (c) 2004 Ascential Software Corporation.

Map file: /dsdata/CIAR5P1uat/tx_bin/Volume_Partner_X12_Transmission.hp
In #29: 65226039 Out # 0: 0 Time: 6:46
Map could not complete (2): Not enough memory to execute map (488.951 seconds)
Error detected in running map, exiting.

When I check in the .mtr file, I have this message at the end of the file:

*** Memory Error #19: 77808.

I have already open a case with the Online Support but if you have an idea, don't hesitate!

Thanks
#DataExchange
#IBM-Websphere-Transformation-Extender
#IBMSterlingTransformationExtender

Originally posted by: SystemAdmin

Run ulimit -a.
How large is(are) the input file(s)?
What is the map's page size & count settings?
Is the work area file or memory?
Do you have output cards set to sink?
Are you passing large amounts of data to a Functional map, from card to card or a RUN map?
How many input cards?
Can you burst the input data?
#DataExchange
#IBM-Websphere-Transformation-Extender
#IBMSterlingTransformationExtender

Originally posted by: SystemAdmin

Hi,

Here is the answers to your good questions.

-Run ulimit -a.
time(seconds) unlimited
file(blocks) unlimited
data(kbytes) 4292870144
stack(kbytes) 8192
memory(kbytes) unlimited
coredump(blocks) 4194303

-How large is(are) the input file(s)?
The job fails on input card 29. Here is some stats about the major input cards.
input card 19 : 505 700 rows
input card 27 : 505 700 rows
input card 29 : 4 075 708 rows

And here is the data volume of the work files before the job fails.
25248696 Nov 14 16:27 Volume_Partner_X12_Transmission.FNL
1092367 Nov 14 16:19 Volume_Partner_X12_Transmission.I01
1092367 Nov 14 16:19 Volume_Partner_X12_Transmission.I02
1092367 Nov 14 16:19 Volume_Partner_X12_Transmission.I03
1092367 Nov 14 16:19 Volume_Partner_X12_Transmission.I04
1092367 Nov 14 16:19 Volume_Partner_X12_Transmission.I05
1092367 Nov 14 16:19 Volume_Partner_X12_Transmission.I06
1092367 Nov 14 16:19 Volume_Partner_X12_Transmission.I07
1092367 Nov 14 16:19 Volume_Partner_X12_Transmission.I08
1092367 Nov 14 16:19 Volume_Partner_X12_Transmission.I09
1092367 Nov 14 16:19 Volume_Partner_X12_Transmission.I10
1092367 Nov 14 16:19 Volume_Partner_X12_Transmission.I11
1092367 Nov 14 16:19 Volume_Partner_X12_Transmission.I12
1092367 Nov 14 16:19 Volume_Partner_X12_Transmission.I13
1092367 Nov 14 16:19 Volume_Partner_X12_Transmission.I14
1092367 Nov 14 16:19 Volume_Partner_X12_Transmission.I15
3189519 Nov 14 16:19 Volume_Partner_X12_Transmission.I16
1092367 Nov 14 16:19 Volume_Partner_X12_Transmission.I17
1092367 Nov 14 16:19 Volume_Partner_X12_Transmission.I18
395585504 Nov 14 16:21 Volume_Partner_X12_Transmission.I19
2140943 Nov 14 16:21 Volume_Partner_X12_Transmission.I20
1092367 Nov 14 16:21 Volume_Partner_X12_Transmission.I21
1092367 Nov 14 16:21 Volume_Partner_X12_Transmission.I22
4238095 Nov 14 16:21 Volume_Partner_X12_Transmission.I23
1092367 Nov 14 16:21 Volume_Partner_X12_Transmission.I24
1092367 Nov 14 16:21 Volume_Partner_X12_Transmission.I25
1092367 Nov 14 16:21 Volume_Partner_X12_Transmission.I26
682012094 Nov 14 16:27 Volume_Partner_X12_Transmission.I27
1092367 Nov 14 16:27 Volume_Partner_X12_Transmission.I28

-What is the map's page size & count settings?
page size 1024
page count 1024

-Is the work area file or memory?
All the input cards we have are MDQ queries connecting to an Oracle databse. The source rule for the FetchAs is set to Integral with WorkArea set to Create and FetchUnit set to S.

-Do you have output cards set to sink?
No. I have only one output card and it's a text file.

-Are you passing large amounts of data to a Functional map, from card to card or a RUN map?
We pass large amounts of data (see above) to a Functional map from card to card

-How many input cards?
38 input cards.

-Can you burst the input data?
All the input cards we have are MDQ queries connecting to an Oracle databse. The ouput file created by this map is an EDI file. Then I don't think we can burst the data. To create the output file, we need to collect all the data before.

I hope that my answers are clear. Feel free if you need more details.
Thanks!!!
#IBMSterlingTransformationExtender
#DataExchange
#IBM-Websphere-Transformation-Extender

Originally posted by: SystemAdmin

Other thing!

It's strange, I tried to run the same job on my laptop and it works! It takes a long time to run but it works. It doesn't fail on the input card 29.
#IBM-Websphere-Transformation-Extender
#DataExchange
#IBMSterlingTransformationExtender

Originally posted by: SystemAdmin

1. Standard build for HPUX will not handle as much data as other platforms. Contact Support and get Exec_Magic build.
2. If you multiply your page size and count, you will see you are trying to allocate way to much space. Try the default first, then increase a bit (2x then 4x) for size and count (changing one at a time).
3. Set StreamMaxMemLimit in the dstx.ini file to 10 meg. and give it a valid path (not the default).
4. Set work area to file & delete.
5. Nopt sure why you need so many different input cards from a DB. You might be able to call a stored procedure that gets more specific data.

How big is the output file?

Based on the size of the work files (up to 600 meg. with 500k rows) it might be better to break up the map into several to create data files.
One way would be to have a main map call a series of RUN maps that would get the DB data and create data files, using the rules in the original map, so that the data files only have the data required.
The last card in the map would create a trigger file for a map that would pick up the specific data and create your EDI file.
#IBM-Websphere-Transformation-Extender
#DataExchange
#IBMSterlingTransformationExtender

Originally posted by: SystemAdmin

We run HP/UX. Our experience regarding 'not enough memory' errors, incidates the map is likely compiled for the Win environment and NOT HP. You need to access MAP/BUILD for Platform/ action bar choice, select HP, then move that binary into your HP/UX system. We process extremely large SAP Idoc files and do not adjust any default map page/size or any ES config. file ---- works fine. Regards, TR
#IBMSterlingTransformationExtender
#IBM-Websphere-Transformation-Extender
#DataExchange

Originally posted by: SystemAdmin

-What is the map's page size & count settings?
page size 1024
page count 1024

So you allocate 1 gig per map. Get a copy of 8.0 and use the page size commend line utility to tune the map.
#DataExchange
#IBMSterlingTransformationExtender
#IBM-Websphere-Transformation-Extender