Dynamic data import / export


simply more productive
Download the freeware version (10 Mb)
Up to 10,000 records can be processed with the freeware version!
Welcome, Guest
Username: Password: Remember me
Welcome to the FlowHeater support forum.

Questions about using FlowHeater are answered here.

TOPIC: SQL Log files too big.

SQL Log files too big. 4 years 2 months ago #2242

  • Niel Bullock
  • Niel Bullock's Avatar
  • Offline
  • New Member
  • Posts: 6
I have a database where the transaction log grows way out of bounds. I have many FlowHeater batch programs running each night; They empty large tables before appending new data and I believe that SQL 2008 is logging each row as a transaction -- Even thought (I think) I have the logging set to simply for that database. When I try limiting the log size, the database stops and I do not get a error.

1. How do I properly set the database as read only, no/minimal logging and still allow FlowHeater to update it?
2. How do I set error reporting with a batch fails?

Any suggestion will be appreciated. Would dropping and recreating the tables cause less logging?
The administrator has disabled public write access.

Re:SQL Log files too big. 4 years 2 months ago #2243

  • FlowHeater-Team
  • FlowHeater-Team's Avatar
  • Offline
  • Administrator
  • Posts: 233
Hi Niel,

About database transactions
FlowHeater usually import/update data within one big database transaction. You can change this behavior by setting the Adapter property "AutoCommitAfter".
Set this property to -1 (see picture below) tells FlowHeater to not use database transactions. Maybe this makes sense in your case?

NOTE: If something goes wrong during the import/update all data up to this point are written. FlowHeater can’t rollback these changes!

I guess you have only to maintain your database. If your recovery model is set to simple it’s enough to make daily full backups. In other case you can run daily (or weekly) jobs to shrink the transaction log automatically.

Empty table before importing) This option can take a while for big tables. Also this option uses a lot of space in the transaction log as well. It’s possible to use the faster truncate table command. To use truncate table instead of delete table (default) you have to set the Adapter property "UseTruncate" to TRUE (see picture below).

About report errors
If you use a batch command script (.BAT or .CMD) for your daily automated jobs you can here decide whether FHBATCH goes wrong. An error occurs if the ERRORLEVEL less than zero. In this case you just send the redirected output "error.log" via an SMTP Command line utility like Blat.

Example batch CMD script
@echo off
set FHBATCH="C:\Program Files\FlowHeater V2\BIN\FHBatch.exe"
REM > redirects the output to the file error.log
%FHBatch% import-export.fhd > error.log
if %ERRORLEVEL% LSS 0 goto error
echo OK
goto end

REM Send an Email here


Hope this helps?

Best wishes
Robert Stark

Did this answer your question? We would be grateful if you provide a brief comment as feedback. It may also help others who may have encountered a similar problem.
The administrator has disabled public write access.
Moderators: FlowHeater-Team
Time to create page: 0.088 seconds