General error handling

Hello,

Originally we had laid out how we wanted our packages to work. In that concept we never really wanted our packages to fail but to offload the problem data to a file so that we could examine it (finding a problem in 1, 10, or 100 rows is easier than finding problems in 100,000 lines). The 2nd thing we want to achieve is to audit each task at the beginning and at the end. Very similar to how the the number of rows is shown in the dataflow.

Our first problem is that each task would require us to create a new file to offload the problem data. This quickly became a huge concern due to the number of files that we would have to maintain. The second problem is that we haven't figured out a way to capture the row count as it flows through the tasks and send that data to an audit table.

The auditing we want to do will go along way in ensuring that we can truly automate our packages which need to run once an hour.

Are any of you currently doing something of this nature and if so how did you go about handling some of the issues and concerns we have had?

Thank you in advance!

-Krusty

Krusty,

Re: Capturing the rowcount.

Can you not use the rowcount component and then use an eventhandler on the task to log that value to your audit table? I currently do this extensively and it works very nicely. In fact, every row that I log to my audit table populates fields called InsertCount, DeleteCount and UpdateCount (all default to zero).

Its one of the reasons why I'm such a strong advocate of implementing custom logging using eventhandlers (http://blogs.conchango.com/jamiethomson/archive/2005/06/11/1593.aspx).

-Jamie

|||

Hello Jamie,

Thanks for the link and your time. We will most likely look into using the same approach. Hats off to you, again!

-Krusty

|||

Jamie,

We have implemented a flavor of your auditing approach from your post above. However, when we disable the events on containers we don't want them to run in, they still fire. Have you experienced this? Thanks for your assistance.

-Krusty

|||

I think I can understand the problem here.

I'm guessing that you have set DisableEventhandlers=TRUE on a particular task but your eventhandlers at the package level are still firing. Am I correct?

If so, the reason is that DisableEventHandlers=TRUE only disables the eventhandlers scoped to that task (or, more accurately, to that container). This setting does not stop events propogating up the container hierarchy and being caught by eventhandlers scoped to ancestral containers (of which the package container is one).

Instead of setting DisableEventhandlers=TRUE on a particular container, create an eventhandler scped to that container and, within it, set System::Propogate=FALSE. This will stop the event from propogating up the container hierarchy.

The container hierarchy is explained a little here: http://blogs.conchango.com/jamiethomson/archive/2005/07/13/1792.aspx

-Jamie