One of the great promises of service oriented architecture is to ameliorate data integration challenges faced by many organizations struggling to integrate disparate technologies and systems.  While adopting SOA can be extremely advantageous in easing data integration woes, it is important to understand that SOA is not a replacement for fundamental information management practices.  Indeed, the benefits of SOA are predicated on intelligent information management practices and policies.  

While there are many information management activities that should be considered, I will share three in this article that are extremely important.  They are:  

•    Information Stewardship
•    Data Quality Procedures and Processes
•    Managed Data Redundancy

Information Stewardship

Information management requires strong leadership and stewardship, especially at the top of the organization.  When implementing information stewardship, it is imperative that the company understands that the chief information steward should be the CIO.  If the CIO is primarily focused on “technology,” then he is functioning as a CTO rather than a CIO.  In this situation, which is quite common, the IT department is not managing its information as a key strategic resource, and information stewardship is going to be difficult to achieve.

The first step in fixing the problem is to re-align the CIO’s mission so that he is truly focused on managing the information and not just the technology.  Begin by updating the CIO’s job description and bonus criteria to reflect information tasks and responsibilities.  In addition, update the mission and vision statements of the IT department so that “data” and “information” are truly mission critical.  This will empower the CIO to manage the company’s information resources as a vital business asset that will benefit the organization.

Once you have your Information Officer re-aligned, develop a stewardship program with key information stewards over each major subject area of the company who report directly to the CIO.  The information stewards are responsible for the accessibility, timeliness, and accuracy of their subject’s information.  

In addition to being the “owners” of the data, the information stewards must possess intimate knowledge of both the business processes and data for their subject.  For this reason, it is best to promote from within, since a new hire simply will not have the knowledge of the business to be effective.  

The stewards must document all of the information resources within their area of responsibility.  This is accomplished through meticulous data modeling activities – both physical and logical – and capturing highly detailed metadata.  Data modeling and metadata management are the purview of the information stewards who are responsible for maintaining the integrity and accuracy of both the models and metadata.
Data Quality Processes and Procedures

The great news about SOA is that it is possible to expose the company’s data in ways that are not possible in traditional architectures.  It can, however, be problematic because it is sure to expose data quality problems as well.  So, one of your chief goals in your information management strategy in SOA is to find, fix, and prevent data quality issues so that data services have access to clean data.  

This will require a comprehensive data quality strategy and team to execute it.  When designing your data quality program, keep in mind that data quality falls into two categories:  reactive and proactive measures.  Most organizations focus on reacting to data quality problems rather than proactively preventing the errors from occurring.  In fact, both are required.  

When assembling your data quality team, remember that data quality is not just an “IT” issue with a technology solution.  Data quality problems often arise from broken business processes that create the data.  Therefore, the solution is often in the business where the problem originates.  To make this happen, the data quality team should be comprised of both IT and business people, with the information stewards leading the charge.
Managed Data Redundancy

One of the major benefits of SOA is service reuse.  That is to say it is important to develop services as reusable components that can be leveraged by many consuming applications across the enterprise.  Maximizing service reuse and minimizing service redundancy should be one of your primary goals in implementing SOA in order to achieve the greatest benefit.

Just as redundancy in services makes absolutely no sense, neither does the redundancy of data.  Consolidate redundant databases where possible, and develop a common understanding of the data (i.e. metadata management) so it can be consolidated.  This will greatly facilitate your move to SOA by simplifying the development, testing, and deployment of services for accessing the company’s “single version of the truth.”

There are several ways an organization can “consolidate” its information.  The most common way to consolidate data is to do it physically in the RDBMS (Relational Database Management System).  Another way to achieve consolidation is to do it virtually through federation, which is gaining considerable momentum thanks to rapidly emerging technology.

Both data consolidation and federation are probably going to be necessary, but they will have to be weighed carefully and part of your larger architectural strategy and goals.  While it is certainly easier to manage the data when it is centralized, consolidating databases may not be feasible in a widely distributed environment.  

Data federation may alleviate some of the database challenges associated with database consolidation; however, performance is a key concern when evaluating federation. That is because federating data from many, heterogeneous systems in a virtual layer can require a lot of complex joins without the benefits of indexing and performance tuning at the database level.    

Managed redundancy, along with stewardship and data quality, are very important aspects of information management.  However, there are many things to consider as your company develops its service oriented architecture.  I have co-authored a two-part white paper on SOA that is available upon request.  For those who are looking for more information on the subject, email me and I will be happy to send it to you.

About the Author

Ken Karacsony

Ken Karacsony has 15 years of consulting and professional experience in IT. He is currently an Information Strategist developing information management practices that support SOA.  He is an author and lecturer who has presented at the international DAMA conference and the international data quality (IAIDQ) conferences and has authored featured articles in both ComputerWorld and DMReview magazines. You can contact him at Ken.Karacsony@Verizon.net.