Skip to main content
simply more productive

The Data Specialist

Thanks to our intuitive transformation designer, you can convert data effortlessly to a variety of formats without knowing how to program. It’s not only efficient, it’s also easy to use.
Flowheater Chart

SQL Log files too big.

  • Niel Bullock
  • Topic Author
  • Offline
  • User
  • User
More
12 years 3 months ago #2242 by Niel Bullock
SQL Log files too big. was created by Niel Bullock
I have a database where the transaction log grows way out of bounds. I have many FlowHeater batch programs running each night; They empty large tables before appending new data and I believe that SQL 2008 is logging each row as a transaction -- Even thought (I think) I have the logging set to simply for that database. When I try limiting the log size, the database stops and I do not get a error.

1. How do I properly set the database as read only, no/minimal logging and still allow FlowHeater to update it?
2. How do I set error reporting with a batch fails?

Any suggestion will be appreciated. Would dropping and recreating the tables cause less logging?

Please Log in or Create an account to join the conversation.

More
12 years 3 months ago #2243 by FlowHeater-Team
Replied by FlowHeater-Team on topic Re:SQL Log files too big.
hi niel,

about database transactions
flowheater usually import/update data within one big database transaction. you can change this behavior by setting the adapter property "autocommitafter".
set this property to -1 (see picture below) tells flowheater to not use database transactions. maybe this makes sense in your case?

note: if something goes wrong during the import/update all data up to this point are written. flowheater can’t rollback these changes!

i guess you have only to maintain your database. if your recovery model is set to simple it’s enough to make daily full backups. in other case you can run daily (or weekly) jobs to shrink the transaction log automatically.

empty table before importing) this option can take a while for big tables. also this option uses a lot of space in the transaction log as well. it’s possible to use the faster truncate table command. to use truncate table instead of delete table (default) you have to set the adapter property "usetruncate" to true (see picture below).

about report errors
if you use a batch command script (.bat or .cmd) for your daily automated jobs you can here decide whether fhbatch goes wrong. an error occurs if the errorlevel less than zero. in this case you just send the redirected output "error.log" via an smtp command line utility like blat .

example batch cmd script
Code:
@echo off set fhbatch="c:\program files\flowheater v2\bin\fhbatch.exe" rem > redirects the output to the file error.log %fhbatch% import-export.fhd > error.log if %errorlevel% lss 0 goto error echo ok goto end :error rem send an email here :end

hope this helps?


Best wishes
Robert Stark

Did this answer your question? We would be grateful if you provide a brief comment as feedback. It may also help others who may have encountered a similar problem.

Please Log in or Create an account to join the conversation.

Time to create page: 0.263 seconds
FlowHeater Logo

FlowHeater - The Data Specialist

Efficient data integration and transformation with FlowHeater – the perfect solution for a seamless transfer of your data.

Legal information

Support & Contact

Contact

Phone:
0951 / 9933 9792

eMail:
This email address is being protected from spambots. You need JavaScript enabled to view it.


Copyright © 2009-2024 by FlowHeater GmbH. All rights reserved.