Fbatch extremely slow

I used Fbatch to batch upload CSV files to a table of size about 50 MB and used go smoothly.
Recently, I used Fbatch in the same way as I used and the file of 50 MB to be uploaded to PSQL, but this time it took about 7 -8 hours to upload for one file that used to get uploaded in 15 to 20 mins maximum. What could be wrong.
I need to upload about 30 file for each month x 4 to the psql and I am trying to figure out the issue but not sure.

thanks in advance

Hi Rajeev,

I guess the issue is related to the PostgreSQL transaction behavior!

FlowHeater usually use one big database transaction to import/update records in a databases. Just for PostgreSQL this isn’t sometime too much and PostgreSQL couldn’t handle big data import or updates as fast as expected. I’ve never seen such database transaction behavior in other databases engines. Currently I’m searching for perhaps a PostgreSQL option to improve the performance during large data imports or updates.

To prevent such slow PostgreSQL imports/updates you have to configure the FlowHeater PostgreSQL Adapter. Open the PostgreSQL Adapter configurator on the WRITE side and switch to the tab “Advanced”. Here are two possible options possible.

• You disable database transactions
• You configure the PostgreSQL Adapter to perform an database auto commit for example every 5,000 records.

I did not get the first step correct. the picture shows tick on the use database transaction, but you suggested ' You disable database transaction'.

The two options are separately.

It´s possible to disable the use of database transactions for this Definition, this I do not recommend.

The better way to do this is the second option. Do not disable database transactions and configure to perform an AutoCommit after 5.000 processed records. I think this will improve your PostgreSQL performance in this case much more. The posted picture shows this option.