Data within your database is just that, data. It becomes information only when you can effectively extract and distribute in an understandable form. Using meta-data has come a long way; it allows for free-flowing information and is putting the ability to extract information into the hands of end-users.

When I first started my career in data processing, I began as a COBOL programmer. Back then, meta-data was strictly “data about data” and, in those days, we had to rely on s FDs, copy books, and a scarce set of documentation that comprised our meta-data. Such resources told us where data was located and what particular objects (tables) and columns meant.

Today, meta-data has grown into a complete subject area encompassing such terms as Knowledge Management, Corporate Data Dictionaries, and Enterprise Meta-Data Repositories. All of these classifications have the single goal to categorize the underlying information buried within databases so that end users can get to information faster. They do this by employing tools that allow end users to unlock the mysteries of what information is available and where it is located.

Imagine that you are cornered in the break room by the CEO or called into a sales meeting and asked to produce a report that describes demographic information for current customers. This task seems to be quick and easy, at least at face value. But, you quickly remember that you have distributed databases that control different product lines, and the way objects have been defined within those databases are quite different. To compound the issue, your CEO is starting to describe “customers” as anyone who has purchased products or services from you including potential customers that are not stored in a database yet but are in multiple spreadsheets maintained by your sales reps. No doubt, it is becoming obvious that these requirements will be changing and you begin to feel like you will be trying to hit a moving target.

Now imagine that same CEO and sales team having access to a tool that allows them to search across all corporate databases in a natural query language. And they could seamlessly correlate all data sources together to get all of the information they need. Sounds a bit like fiction to me, but this is where the power of meta-data comes into play. Today, there are a handful of individuals that are effectively rethinking and building tools that are on the edge of re-organizing and categorizing the “data about data” (meta-data) to unlock how we will be “querying” information in the future.

Typically when you and I talk “query” we are talking about the individual “SELECT” statements that we have grown to love. For the future of information retrieval, the term “query” is meant as a SEARCH for information. If you were to correlate this to something we are all familiar with, it could be compared to an Internet search. Think of the meta-tags in Web pages the same way as the meta-data of your data dictionaries. When an end-user of the future does a “query” for information, that user will, in effect. be searching the meta-data for information that “matches” what they are looking for. This search will be able to cross database boundaries much the same way an Internet search crosses domains and will also search other forms of corporate information such as e-mail, Word, PDF, text, and Excel documents. Obviously the reasoning here is that if I can search for information throughout the net, why can’t I search my own corporate information sources? This power allows end-users and decision makers the ability to satisfy questions about direction, trends, and goals, while tailoring the search criteria until valuable results are shown. Next, I’ll cover just a few of the benefits to employing meta-data within a search for information.

Benefits of using meta-data as a search mechanism:

1. Full-view into all information sources
2. Users can retrieve information autonomously
3. Seamless integration of all data sources
4. Natural language search
5. Reduced reliance on systems analysts, developers, and DBAs
6. The underlying meta-data holds the keys to a search
7. No need to manually create or search enormous data dictionaries

The ability to effectively use a particular database’s internal data dictionary can often be limited depending on the system. Many of these are legacy systems where IT shops are fearful of altering them. Solutions may be two-fold. One solution would be to build up the existing data dictionary by employing views, synonyms, or other database dependent objects that add a layer of abstraction to the meta-data and allow the search to go through the abstraction layer. Secondly, you may wish to add in a completely new data dictionary layer so that you need not touch the source database. Whatever the case, there are limiting factors to your ability to use existing meta-data that has one or more of the following list.

Limitations on building / altering meta-data information:

1. Corporations have enormous amounts of objects defined
2. Fear of touching legacy systems
3. Naming conventions and standards are non-existent
4. Missing constrained relationships
5. Business rules are external to the data model

Querying data within our corporate databases should be no different that a search we issue across the Internet. The concepts and reasoning are all the same. We are annually increasing the amount of data we store exponentially with limited tools to pull all the data together and make some information sense out of it. But there is hope in the future. There are a handful of companies that are effectively bringing this information together and placing it in the hands of business users. Through the power of meta-data, we can begin to pull corporate information resources together, understand what data there is, where the data resides, and ultimately place information in the hands of the end-user more quickly. After all, this is where it belongs.

Thanks to Mark Reiners and Dan Oldis at MetaTrieval Software, LLC for giving insight into the power of meta-data.

--

James F. Koopmann is dedicated to providing technical advantage and guidance to companies within information technology. Over the years, James has worked with a variety of database-centric software and tools vendors as strategist, architect, DBA, and performance expert. He is an accomplished author appearing regularly within printed publications across the Web, and speaking at local area User Groups as well as industry conferences. He may be reached at jkoopmann@pinehorse.com or www.pinehorse.com.