Article Preview

COLUMN

The Source Of Truth

Don't Needlessly Replicate Data

Issue: 10.6 (September/October 2012)
Author: Craig Boyd
Author Bio: Craig Boyd is currently a Sr. Data Modeler for a well-known national retailer. But in his 17 years of IT experience he has been everything from a PC Technician to iSeries System Administrator to iSeries Programmer and Sr. Technical Lead. He lives in the great state of Texas with his wife and two kids.
Article Description: No description available.
Article Length (in bytes): 8,508
Starting Page Number: 79
Article Number: 10612
Related Link(s): None

Excerpt of article text...

In this article I hope to communicate the importance of not replicating data throughout the corporate ecosystem without first preserving the concept of a single source of truth.

In my family, we all love to read, admittedly to varying degrees. My wife and I are both avid readers and our two children both like to read also, just not to the same extent that my wife and I read. Their grandparents on both sides are devoted readers and so are most of their cousins. So books and thoughts about books are always permeating the conversations. The other day though the discussion came up as to "how do we keep track of the books we have?" We don't have a lot by our families standards, but I suspect we have more than the average household. What makes us crazy though is that we still occasionally buy duplicates.

My wife piped up that she has an app on her iPhone for tracking books and I told her that I also had one that I liked. I also know for a fact that we have an application on one of our desktops for tracking book collections. I promptly announced that we should have one source of truth for our book collection. Spreading the information across all of the known devices without having a single spot for it is not going to happen on my watch! No one understood what I meant so I made a point using time. I asked my daughter to look on her watch and then to compare it to all the time pieces she could find in our kitchen area. Three of the clocks in the kitchen were synchronized, but a fourth was way off. My watch reported a slightly different time as did my wife's watch and her iPhone. So I asked her to tell me exactly what time it is. She thought for a moment and said that she guessed it would be pretty close to three that were the same, but she admitted she could not tell me exactly what time it is. I then explained to her that the most accurate time pieces are atomic clocks and the further you get from them the more variation you get in time. I saw that light bulb go on and thus another wonderful moment of understanding was born in a child. But I digress.

The same is true about data. The further you get from the source of truth the more opportunities show up for variances. I have worked places where there was literally a spider web of replication and ETL (Extend Transform Load) jobs that made database change impact analysis a herculean effort. We do not do anyone any favors by letting something like this slide. We create more work for ourselves and/or our peers. We increase the cost of application development and in fact slow it down. The more complex the web the more difficult it is to understand it. Do not underestimate the value of a good clean database design. It is priceless in what it can save, even though it may cost a bit more up front. As professionals, we must protect not just ourselves, but our partners in the business from this sort of thing.

But how?

The first step is starting with a good clean front end application database. If your application database needlessly duplicates data, then you have to start there with the clean up. Front end application databases are referred to as OLTP (On-Line Transaction Processing) databases. If those are in bad shape, then it can really hamstring your efforts to keep the single source of truth undiluted. Most businesses have multiple OLTP applications. Usually there is an accounting application which may be nothing more than a handful of spreadsheets that get sent to the accountant once a month. Or maybe it is some sort of small business accounting application. Or maybe it is something much larger, like SAP. A business will also have some means of tracking payroll information and paying their employees. Most businesses, but certainly not all, need to keep track of physical assets either because they are in the business of selling them or because they need to report the depreciation to the government. Each of these may be from separate vendors or maybe the company splurged a little and bought a suite of applications or implemented an open source suite. The point is that you can have multiple systems that act as the single source of truth for a particular subject area.

...End of Excerpt. Please purchase the magazine to read the full article.