Database Administrators (DBAs) do not need metadata. You read that statement right. They don't need metadata. Here - I have been an advocate of metadata management for close to eight years and I come out with a statement like that. Unbelievable … or is it? Database Administrators don't need metadata because they already have what they need. Well … let me just say that they have enough to do their job and do it well. In comparison to data modelers, application developers, and business / data analysts, DBAs are way ahead of the curve.

The database catalog, the sidekick of the DBA, is a metadata database. The data in the catalog defines how the data is structured in the database. Desktop and mainframe tools used by the DBAs read and write metadata directly from and to the database catalog. Data definition language (DDL), an executable script of metadata, is created and executed by the DBA to change the database catalog. The catalog is THE major resource of information (read metadata) about databases.

The database catalog is a perfect example of, to steal a phrase from data modeling / metadata lecturer and author - William G. Smith, "in-line" metadata. By definition, changing in-line metadata directly impacts the definition, design, structure, movement, accessibility, … of the data and the processes that use the data. Changing a table index in the database catalog changes the actual index of the table. Changing the database catalog can be dangerous. This is one reason why, in most cases, only DBAs have the privilege of making changes to the catalog.

DBAs use "in-line" metadata. So what? Why should the DBAs be concerned with the metadata health of the organization when they have what they need? How important is the DBA's metadata to the rest of the organization? How important is the Oracle DBA's metadata to the DB2 or the SQL Server DBA? What role does the DBA play in the overall context of metadata to the organization? The purpose of this article is to demonstrate where DBA metadata fits into the enterprise metadata picture and how the DBAs can become a partner with positive influence on metadata and information resource management success.

DBA as the Enterprise Metadata Hub

Let's start with the first question first. Why should the DBA be interested in metadata? Using basic proof theory: 1) DBAs understand the importance of the data in the database catalog, 2) the data in the database catalog is metadata, therefore, 3) the DBAs understand the importance of metadata. Too easy? Perhaps not. If the DBA can imagine life for one day without access to the catalog, they may understand how others in the organization feel when they don't have information about the data that they need to perform their job function. So how can the DBA help?

The metadata in the database catalog is central to the Core Meta Model displayed in Figure 1. The databases, tables / views of data, indexes, and the columns (represented as metadata types on the diagram) are vital links (or the Hub) in the definition of metadata to support the enterprise. Without DBA metadata, there is no connection between the business definition of the data (in the data model) and the data. Without DBA metadata, there is no map / relationship between source and target data. Without DBA metadata, there is no connection between the SQL and the data. Without DBA metadata, the enterprise structure of metadata (at the technical level) does not exist.

Figure 1: Core Meta Model

It is almost too simple to think that such a small number of metadata types from the database catalog (the databases, tables / views, and columns) can have such an impact on the ability to manage enterprise metadata. But it is true. The rest of the article will analyze the relationships between the DBA and the data modeler, the DBA and the application developer, and the DBA and the business / data analyst. This analysis will focus on the metadata as the driver of these relationships.

The DBA and the Data Modeler

The interaction between the DBA and data modeler typically involves the passing of information from one individual to the other. This hand-off of information always includes data about data, whether it is logical information (model entities, attributes, and keys) coming from the data modeler or physical database information (databases, tables, columns, and indexes) coming from the DBA. The information is metadata. The strength of the relationship between the modeler and the DBA often is based on the quality of the metadata that is passed from one individual to the other.

Changes Made by the Modelers First

In an IT environment where changes in data and data requirements feed through the data modelers to the DBAs, the data model is changed prior to changes to the physical database design. In this environment, the information that is passed from the data modeler to the DBA typically takes on one of three forms:

1. a logical data model that has not yet been forward engineered to a new physical model; in this situation, the information needed by the DBA to update the physical data design is stored in the logical data model
2. a draft physical model that is the result of forward engineering logical data to physical data in a data modeling tool; in this case the information needed by the DBA is stored in the physical data model
3. a completed physical data model that has been transformed by the data modeler into data definition language (DDL); this information can be executed by the DBA to physically change the database design.

This interaction between the modeler and the DBA takes place during the design and development phase of the data life cycle, or when changes to the data model have been requested by the application developers or the business analysts responsible for defining data requirements.

When the logical data model is changed before the database, the physical data model and database catalog need to be changed to reflect the changes. Ideally, steps need to be taken to bring the physical database definition up to date with the changes to logical model and database design. In some shops processes are clearly defined for where the modeler's responsibility ends and the DBA's responsibility takes over. This process, although though not stated as such, is the passing of metadata from the modeler to the DBA.

Changes Made by the DBAs First

In an IT environment where changes in database structure are first given to the DBAs (exactly the opposite of the above scenario), there needs to be a process for the DBAs to communicate the changes to the data modelers (preferable before the DBAs make their changes). In this environment, the information that is passed from the DBA to the data modeler typically takes on one of two forms:

1. a memo or a scratch piece of paper indicating what changes need to be made
2. the physical structure of the database through the catalog or data definition language

Reverse-engineering (as it relates to data modeling) is a term used to describe the movement of metadata from the physical database / catalog towards the (re-) generation of the physical and logical data models. Tools are available that allow DBAs to hand their changes back to the data modelers, however, the process of reverse-engineering database metadata to become data model metadata can become a manual and time intensive process. Companies use this type of functionality to model existing databases that did not originate with a model. Companies seldom perform this type of task on an on-going basis to keep data models and databases in synch.

The DBA and the Application Developer

Application Developers are very creative people (and this comes from a former COBOL and MUMPs developer). Harkening back to my developer days, when a glitch became apparent in a program, screen, batch script, … the developer's first reaction was to solve the problem by fixing the program so it operates "properly" or "stops blowing up". The changes could be as simple as adding another "if-then" statement that allowed for a new value to pass from one variable to another. Little thought was given to the validity of the new value or the impact of the additional value on the rest of the application or business. This was done over and over again, and process is still intact with many "patchwork" legacy applications that exist today. In those days there was seldom a methodology for making a change.

If we look closely at the application development methodologies (ADM) in companies today, we find first that many companies don't have a singular ADM for the entire company or even a fraction of the company. One of the key strengths of the ADM is that it nary leaves a stone unturned. In a well-followed ADM it is documented where one individual's responsibility ends and the next person's responsibility starts. When information is passed from one individual to another, this information is metadata. Without an ADM, there is not a defined procedure for passing metadata between developers and DBAs.

To (over-) simplify matters, lets look at a very basic example:

When the Application Development Manager tells the DBA to add an "exception indicator" to the "customer transaction" table, the problems start. If the DBA adds the column without delay the Manager is happy and the DBA did what he / she was told. If the DBA tells the Manager to see the Data Administration group so that the field can be defined, dissected, and turned inside-out, the change may be a week or weeks or it may not take place at all. Is anybody happy? Is this a good thing or a bad thing? The answer is … it depends (on the validity of the requirement, the definition, the business rule, …).

Without a "structured" method to communicate changes between the DBAs and the application developers, the top-down design can quickly become an "upside-down and side-to-side" application. Improved information and information flow about the application and the data can improve the understanding and the quality of both.

When the application developers and DBAs are on the same page, the DBAs are not viewed as "quick change artists". The DBAs are viewed as a strategic link in the chain that consists of business interests and application development. If DBAs and developers exchange and document quality information (in the form of metadata), the potential for success becomes much greater.

The DBA and the Business / Data Analyst

Back to the example of the "exception code" change to the "transaction table":

Lets say that the "exception code" was absolutely necessary, the DBA told the modelers to change the data model, the modelers forward-engineered the logical model to a physical model, DDL is generated from the physical model, and the DBA executes the DDL to add the column to the "customer transaction" table. The metadata exchange process between the developer, the DBA, and the modeler was followed to precise detail.

BUT … What about the business and data analysts that use the data? Do they know what the "exception" is or when the "exception" became active? Do they know anything other than the fact that a new column called "exception code" (or EXCPT_CD) appears on the table that they use? Do they understand the impact that the column may have on the report that is sent to the COO every quarter, and the one sent to the CFO every week? Do they know where to get this information? Or lets say that the business or data analyst doesn't even know that the column was added?

Business and data analysts are always looking for information about the data. The more they know about the data, the better off they are. This information ranges from data definition and valid values, to business rules relating subjects and entities of data, to databases, tables, views, indexes, columns, … much of this information about the data comes from the database catalog. The DBAs, by default, create the metadata source that supplies much of this information to the business and data analysts.

DBAs also provide the business and data analysts with other information not mentioned here. DBAs provide support to the data user in the form connectivity, authorization, performance, and on and on.

However, most often, the meta data in the DBMS catalog that will be important to individuals other than the DBAs will include information about the database, tables, views, columns, and indexes. There is a tremendous amount of metadata in the DBMS catalog, but when individuals are looking to use the data, the physical characteristics listed [here] cover their most basic needs. (1)

Typically, it is not the DBA's responsibility to inform the business and data analysts of the information that they need to do their jobs. The DBAs are responsible for all of the physical aspects of the databases. However, it has to be someone's responsibility to prevent occurrences like the one caused by the "exception code" field. If the DBA is the front-line, they can prevent occurrences like this from happening.

DBAs Do Need Metadata

This article has given examples of how DBAs play a key role in metadata management, with or without them being aware of it. BUT … Do DBAs really have all the metadata that they need in their DBMS catalogs? The real answer to that question is a resounding "NO!"

Mike Yocca, of Ace Database (Pittsburgh - http://www.acedb.com/) stated it well when he labeled his search for information about the databases to be "the churn". Mike stated that "in the day of increased complexity of databases and database servers, my customers spend a tremendous amount of time researching and identifying IP addresses, server info, contact info, business rules, user IDs and passwords, and much more. DBAs certainly DO NOT have all of the metadata that they need."

Conclusion

Don't go running to the DBAs in hopes that they will lead the charge in support of your metadata project. DBAs understand the importance of metadata because they use it all of the time. They have a database full of metadata that they use on a regular basis (which is more than can be said for data modelers, application developers, or business / data analysts) but they might not have thought of themselves as being a key player in metadata success for the corporation. Hopefully, this article will help them to think otherwise.

(1) Meta Data Themes: The Basics - Robert S. Seiner - The Data Administration Newsletter, June 1998

Originally published at TDAN.com.
© Copyright 2000 - The Data Administration Newsletter

---

Robert (Bob) S. Seiner is recognized as the publisher of The Data Administration Newsletter (TDAN.com), an award winning electronic publication that focuses on sharing information about data, information, content and knowledge management disciplines. Mr. Seiner speaks often at major conferences and user group meetings across the U.S. He can be reached at the newsletter at rseiner@tdan.com or 412-220-9643 (fax 9644).

Mr. Seiner is the owner and principal of KIK Consulting Services, a company that focuses on Consultative Mentoring or simply stated ... teaching company's employees how to better manage and leverage their data, information, content, and knowledge assets. Mr. Seiner's firm focuses on data governance/stewardship, meta-data management, business intelligence and knowledge management. KIK has developed a 4-Step Method© for Consultative Mentoring that involves customizing industry best practices to work in your environment.

For more information about Mr. Seiner, KIK Consulting Services and The Data Administration Newsletter (TDAN.com), please visit www.tdan.com and www.tdan.com/kik.htm.