I have seen using a loop and the $iteration variable. Create a Java service to wrap the integer modulo operator (unfortunately, there is no pub.math mod service)

Assuming your data is in list, very rough pseudocode

loop over list of listSize { add current list item to batchInsertlist if ($iteration % 1000 == 0 || $iteration == listSize) {  // current iteration is evenly divisible by 1000 or the final iteration    batchInsert batchInsertlist      commit transaction  }}

Another idea would be to partition your list into smaller lists using pub.document:groupDocuments loop over the returned documentGroups list. But that could be challenging to get groups of predictable fixed sizes

Dear Dave,

Thank you very much for your answer.

After commiting the transaction, should we call the startTransaction again?
We'll soon implement the way you suggested and let you know the results.

I have seen using a loop and the $iteration variable. Create a Java service to wrap the integer modulo operator (unfortunately, there is no pub.math mod service)

Assuming your data is in list, very rough pseudocode

loop over list of listSize { add current list item to batchInsertlist if ($iteration % 1000 == 0 || $iteration == listSize) {  // current iteration is evenly divisible by 1000 or the final iteration    batchInsert batchInsertlist      commit transaction  }}

Another idea would be to partition your list into smaller lists using pub.document:groupDocuments loop over the returned documentGroups list. But that could be challenging to get groups of predictable fixed sizes

Bom dia, Renan -

Out of curiosity: why do you want to do this?

Do you have a plan for what to do you if one of your groups of transactions fail? For example, let's say you have 5,000 records to be inserted. The first group of 1,000 records get committed successfully, but the second group fails. Will you move on to the next group? Will you abort? And what happens with that first group that was successfully inserted? Doing commits mid-batch may complicate your exception scenarios a bit.

And the answer to your question is "yes": the startTransaction and commitTransaction go hand-in-hand. If you're going to do commits within the loop, then startTransaction will also need to be in the loop (and perhaps rollbackTransaction as well depending on how you decide to handle exceptions).

Abs!

Dear Dave,

Thank you very much for your answer.

After commiting the transaction, should we call the startTransaction again?
We'll soon implement the way you suggested and let you know the results.

I have seen using a loop and the $iteration variable. Create a Java service to wrap the integer modulo operator (unfortunately, there is no pub.math mod service)

Assuming your data is in list, very rough pseudocode

loop over list of listSize { add current list item to batchInsertlist if ($iteration % 1000 == 0 || $iteration == listSize) {  // current iteration is evenly divisible by 1000 or the final iteration    batchInsert batchInsertlist      commit transaction  }}

Another idea would be to partition your list into smaller lists using pub.document:groupDocuments loop over the returned documentGroups list. But that could be challenging to get groups of predictable fixed sizes