Decision Optimization

I have written code for a scheduling problem in opl (.mod and .dat files) and I have used doopl python interface for calling those two .mod and .dat code and performing all the statistical operations.
The code is working fine and finding the solution in around 7 minutes time, but after solving(finding the solution), it is taking around 20-25 minutes for writing the data to a csv files and generating them. I have created the csv files and writing to them in .mod file. I have also tried to create those csv files in doopl python instead of .mod, but even then it is taking long time for creating of csv files.
How can I reduce that time?
Please help..
Thank you.

------------------------------
Subhas Chatterjee
------------------------------

#DecisionOptimization

Hi,

can you check that you do not rely on too large loops in the scripting part ?
What would be more efficient is to do the computing in OPL postprocessing part and not do the large loops in the scripting part

regards


------------------------------
ALEX FLEISCHER
------------------------------

Original Message:
Sent: Tue June 30, 2020 02:59 PM
From: Subhas Chatterjee
Subject: time optimization

I have written code for a scheduling problem in opl (.mod and .dat files) and I have used doopl python interface for calling those two .mod and .dat code and performing all the statistical operations.
The code is working fine and finding the solution in around 7 minutes time, but after solving(finding the solution), it is taking around 20-25 minutes for writing the data to a csv files and generating them. I have created the csv files and writing to them in .mod file. I have also tried to create those csv files in doopl python instead of .mod, but even then it is taking long time for creating of csv files.
How can I reduce that time?
Please help..
Thank you.

------------------------------
Subhas Chatterjee
------------------------------

#DecisionOptimization

You will have to show us what exactly you are doing, otherwise we cannot tell how this could be improved.
In general, it is probably faster to do things in Python rather than using OPLScript in .mod. But maybe your Python code is still inefficient. Like I said, please show the code to be improved.

------------------------------
Daniel Junglas
------------------------------

Original Message:
Sent: Tue June 30, 2020 02:59 PM
From: Subhas Chatterjee
Subject: time optimization

I have written code for a scheduling problem in opl (.mod and .dat files) and I have used doopl python interface for calling those two .mod and .dat code and performing all the statistical operations.
The code is working fine and finding the solution in around 7 minutes time, but after solving(finding the solution), it is taking around 20-25 minutes for writing the data to a csv files and generating them. I have created the csv files and writing to them in .mod file. I have also tried to create those csv files in doopl python instead of .mod, but even then it is taking long time for creating of csv files.
How can I reduce that time?
Please help..
Thank you.

------------------------------
Subhas Chatterjee
------------------------------

#DecisionOptimization

#DecisionOptimization

Did you try to directly write the csv file? It looks like unncessary overhead to first create a huge dataframe only to then export it to csv. Also, have you considered creating the dataframe with the right size from the start. If I read your code correctly then you keep adding new rows to the dataframe. This repeated resizing may slow down the code. Maybe things get faster if you create the dataframe with the right size initially?

------------------------------
Daniel Junglas
------------------------------

Original Message:
Sent: Wed July 01, 2020 08:27 AM
From: Subhas Chatterjee
Subject: time optimization

​The lines from 1067 to 1100 is creating 3 csv files and writing to them, the lines from 1067 to 1100 is only taking a lot of time around 20-25 minutes.
So, I tried to create the same csv files in doop python code in the commented lines from 201 to 238, but even that is taking the same time,
My solution is taking just 4-5 minutes, but these csv files' writing is taking a lot of time, 
Please help me in resolving this time issue.
Looking forward for your reply...

------------------------------
Subhas Chatterjee

Original Message:
Sent: Wed July 01, 2020 04:22 AM
From: Daniel Junglas
Subject: time optimization

You will have to show us what exactly you are doing, otherwise we cannot tell how this could be improved.
In general, it is probably faster to do things in Python rather than using OPLScript in .mod. But maybe your Python code is still inefficient. Like I said, please show the code to be improved.

------------------------------
Daniel Junglas

Original Message:
Sent: Tue June 30, 2020 02:59 PM
From: Subhas Chatterjee
Subject: time optimization

I have written code for a scheduling problem in opl (.mod and .dat files) and I have used doopl python interface for calling those two .mod and .dat code and performing all the statistical operations.
The code is working fine and finding the solution in around 7 minutes time, but after solving(finding the solution), it is taking around 20-25 minutes for writing the data to a csv files and generating them. I have created the csv files and writing to them in .mod file. I have also tried to create those csv files in doopl python instead of .mod, but even then it is taking long time for creating of csv files.
How can I reduce that time?
Please help..
Thank you.

------------------------------
Subhas Chatterjee
------------------------------

#DecisionOptimization

#DecisionOptimization

Like I said: it may be better to create the CSV files directly without taking a detour through a dataframe. I suggest to try this and see whether it helps.

------------------------------
Daniel Junglas
------------------------------

Original Message:
Sent: Wed July 01, 2020 09:39 AM
From: Subhas Chatterjee
Subject: time optimization

Yes , I was adding row wise , How could I efficiently do the creation of those 3 csv files from lines 1067 to 1100 in .mod code sir?
I think converting the 2D matrix to tuple in .mod file at commented lines 939​ and then reading the tuples in doopl python using opl.get_table might be taking time..
So, how could I efficiently create those 3 csv files sir?

------------------------------
Subhas Chatterjee

Original Message:
Sent: Wed July 01, 2020 09:22 AM
From: Daniel Junglas
Subject: time optimization

Did you try to directly write the csv file? It looks like unncessary overhead to first create a huge dataframe only to then export it to csv. Also, have you considered creating the dataframe with the right size from the start. If I read your code correctly then you keep adding new rows to the dataframe. This repeated resizing may slow down the code. Maybe things get faster if you create the dataframe with the right size initially?

------------------------------
Daniel Junglas

Original Message:
Sent: Wed July 01, 2020 08:27 AM
From: Subhas Chatterjee
Subject: time optimization

​The lines from 1067 to 1100 is creating 3 csv files and writing to them, the lines from 1067 to 1100 is only taking a lot of time around 20-25 minutes.
So, I tried to create the same csv files in doop python code in the commented lines from 201 to 238, but even that is taking the same time,
My solution is taking just 4-5 minutes, but these csv files' writing is taking a lot of time, 
Please help me in resolving this time issue.
Looking forward for your reply...

------------------------------
Subhas Chatterjee

Original Message:
Sent: Wed July 01, 2020 04:22 AM
From: Daniel Junglas
Subject: time optimization

You will have to show us what exactly you are doing, otherwise we cannot tell how this could be improved.
In general, it is probably faster to do things in Python rather than using OPLScript in .mod. But maybe your Python code is still inefficient. Like I said, please show the code to be improved.

------------------------------
Daniel Junglas

Original Message:
Sent: Tue June 30, 2020 02:59 PM
From: Subhas Chatterjee
Subject: time optimization

I have written code for a scheduling problem in opl (.mod and .dat files) and I have used doopl python interface for calling those two .mod and .dat code and performing all the statistical operations.
The code is working fine and finding the solution in around 7 minutes time, but after solving(finding the solution), it is taking around 20-25 minutes for writing the data to a csv files and generating them. I have created the csv files and writing to them in .mod file. I have also tried to create those csv files in doopl python instead of .mod, but even then it is taking long time for creating of csv files.
How can I reduce that time?
Please help..
Thank you.

------------------------------
Subhas Chatterjee
------------------------------

#DecisionOptimization

#DecisionOptimization