Saturday, May 3, 2008

Unit Testing Production Exceptions

I have a web site that gets a bit of traffic and I log exceptions on that site. Nothing exciting or special, a sort of Log4Net but home grown and speciliazed for the site. The log recently showed an index out of range exception. When I saw the exception I kicked myself for not having the foresight to also log all the options, parameters and data that the user had input so that I could reproduce the error. This got me thinking...

How about another or supplemental log that stored structured unit test data from the production environment? An exception happens in the production environment but it's an exception that's in a function that is directly unit tested in the development environment. During exception logging you write an XML fragment with all the relevant data to your log, supplemental log or DB.

When it comes to running unit tests you could have a setup script pull all unit test data from the production logs and insert it into the data store that's used for unit testing. You'd obviously have to do some checks to make sure that you don't have duplicate data as this is likely to happen if many users enter the same data that generates the same exception. Filtering this during the setup import would be the best place to cull the duplicates.

I've done a partial implementation of this. My log now has detailed input data and I currently copy/paste this into the unit test data store. Next step is to come up with a clever pattern to allow me to pull data for different types of unit tests. To be continued...

No comments:

Post a Comment