If you continually gauge database performance from a strictly database-internal point of view, you are missing the boat. Begin watching your database end users and performance will begin to take on a whole new meaning.

database end user (n) -  

Any individual, not directly responsible for the maintenance of the database, that uses computing resources, applications, or database internals such as utilities and internal code for the purpose of extracting and viewing data.

Source: James F. Koopmann (2004)

From day one, we being good DBAs have been told to “watch our systems.” We needed to watch CPU levels, disk capacities, I/O levels, detect access paths, reduce locking ... the list goes on and on. And we must venture into the complete unknown at times, seeking and attempting to destroy the bottlenecks that crippled databases.

But we have learned our lessons far too well. We have sharpened our skills so perfectly that we have developed a kind of “corporate attitude” that stipulates we must have our heads down and buried in manuals, Web sites, white papers, and scripts if we ever want to solve a problem (and justify our jobs). Let’s face the fact: We DBAs are nut-, bolt-, and tinkering-individuals who prefer the fight to the solution. We enjoy wrestling the beastly internals of the database system, maintaining it anonymously, and feeling just a little transcendent when we find some obscure method of shaving a microsecond off nightly batch jobs. We have, over the years, forgotten who the real users of database systems are: those individuals who are the recipients of the data we so diligently keep accessible.

Fast Forward to Changing Times

If you haven’t noticed, the times have changed. Databases have become less threatening to our end users, so they are requesting more from the data and the from database administrators. And the automation of DBA tasks with the intelligence of the toolsets that give DBAs the ability to work smarter and faster make it possible for end users and DBAs to work together more closely than ever. And with that capability, the need to work with the end user has forged to the forefront of our daily database administrator tasks.

In short, we must get our heads out of the databases and actually talk to the people who use them to determine their needs and desires. We are being asked to help unlock the inner secrets of data models (which, in the past, we so closely held to ourselves), help formulate new methods of data extraction and presentation for our customers, help developers, modelers, and architects understand the new techniques and features of our databases, and aid executive management in understanding and seeing the horizons of best database practices and implementations. Remember the old jokes about slipping pizza under the door or through the mail slot to the phantom operational staff? Those days are long gone.

This focus on dealing effectively with end users brings new importance to the role of DBAs to when performance issues arise. Everything changes when viewing problems through an end user’s eyes; the DBA can no longer rely exclusively on concrete internal statistics. End users have their own ideas about what are acceptable levels of performance and even if all of our monitoring mechanisms are telling us that the database is running smoothly, they may still be experiencing poor response times. We now must work through the user’s perceptions about data, applications, and models and relate all this back to database performance.

Get Answers by Asking the Right Questions

If response times are not acceptable, users will assume that the data could be corrupt, lost, invalid, or not be trusted. It’s our job as DBAs to assure the customer that everything is well with their data and that we are diligently working to resolve any and all response issues they are having, and that can only be done through proper communication channels. No internal statistics we could collect from the database will tell us whether performance levels are acceptable for the user. There are some third-party tools that allow us to see true database response at the users’ terminals, but this still does not tell us the level of user satisfaction.

So when you begin opening up those critical communication channels, be sure to ask yourself the following basic questions and take appropriate action:

1. Do your end users know that you are monitoring application performance as it directly impacts their level of satisfaction?
2. Do your end users know they will be notified when you notice response levels are below expected levels, even before the user community notices this dip?
3. Do your end users have expectations about how quickly performance issues should be resolved and they should be notified about resolution?
4. Do your end users have a process or means to tell you when performance levels are not up to their expectation levels?
5. Do your end users understand what you have to do each and every day to maintain the service levels they are currently experiencing, and the issues you encounter when solving their response time problems?

As you begin to deal with your end user community, you will quickly learn that you have pockets of various types of users, each with their own opinions and thresholds of urgency based on the type of response they are getting from their application or database. Dealing with each of the following four groupings presents unique challenges and opportunities. As you read “my take” on these groups, put yourself into the role of a user and how you might feel if you had a certain level of response with the application you were using. It is only through such role playing that I believe you and I can begin to understand the true user experience.

Response Type: Instantaneous

Definition

When a user hits the enter key, clicks a button, or submits a request, the screen is immediately painted with information. The response is so automatic that the end user does not even realize that work has been done in the background, down the pipe, and within a database.

Challenges

1. Complacency: Don’t allow applications or queries that consistently perform well to be taken for granted, and perceived as unimportant.
2. Making “improvements” to applications such as intricate user interfaces or issuing loads of information to the user. When we change things, we introduce more complexity (and more that could go wrong).

Opportunities

1. Highlighting the great performance users experience with their applications or systems can be a great way to open communications with your end users. If you do not already have a relationship, there is nothing better for convincing them that you are committed to excellent response time levels than an application that is already working well.
2. When application or query response time is excellent, document everything you can about how this application or query works within the database. You may need this information when response times deteriorate.
3. Determine what “instantaneous” really means for a particular application or query. You may be able to un-tune and still get by with acceptable levels of response if another application or query needs processing power or access paths to data that this application is using.

Response Type: Questionable

Definition

While waiting for a request to process, a user may grab a sip of coffee or jot something down on a notepad. The system response to their request isn’t instantaneous, but before they finish their interim task, the database has done the work, passed back the information, and presented it on the screen. What’s important to remember is that while an end user may not be as productive when the response is not instantaneous, she will not typically leave her terminal and focus on other tasks. Thus, there is still continuity in the user’s ability to do their work.

Traps

1. The belief that, just because the system isn’t completely down and the user can still work, the slowdown doesn’t require immediate attention.
2. Assuming that a user always prefers an instantaneous response. Sometimes immediate responses are not required, and users themselves may appreciate that brief moment for contemplation between query and response before moving on to the next transaction or job.
3. That providing all users with instantaneous response levels can be simple, or even possible. I have often found that questionable response levels are built into applications and are a function of many applications.

Opportunities

1. Know your user and application. Users can actually put up with this level of response if they do not have high volumes of processing to do or they are not highly motivated to complete their work and move on to more productive items.
2. If you can move them on to instantaneous response the bond you will make through your commitment will leave a lasting impression.

Response Type: Deteriorating

Definition

An acceptable level of response that is slowly worsening. Users may or may not be aware of the deterioration because it could be increasing at very small intervals for any amount of time over days or weeks.

Traps

1. Conveying that the slowdown is is just a momentary lapse in database performance, caused by unknown factors. Users often prefer to hear that response time will remain slower than before rather than hear that you don’t know or understand why performance has been impacted.
2. Assuming that if your database is working just fine, everything else is fine. Remember that users will always assume that the database is the cause of performance degradation since that is where their data lives. Thus, whether or not it is technically your responsibility, you should still be involved in solving the response issue.

Opportunities

1. If you can notify users that they are experiencing performance deterioration before they begin to complain about it, you are more likely to be seen as someone as proactive, committed to customer satisfaction.
2. Use data to prove that the problem is not a database issue if it truly isn’t, but be involved in the problem resolution as part of your commitment to the end user.

Response Type: Delayed / Unresponsive

Definition

The user is experiencing delayed response or no response at all, and they feel they are trapped at the computer terminal, waiting for its response. They are torn between their ability to complete the task on the computer and quitting that task to work on something else. It is hard for them to get their work done, and they wonder if it would be easier to do the work without the computer.

Traps

1. You don’t know how to solve the problem, so you just do nothing. You should do something, even if you don’t know the ultimate solution. Your users already can’t use the system; they will at least see that you are trying to resolve their issues.

Opportunities

1. Here is your chance to be a hero. Give your end users a time frame when you will be able to solve the problem, provide status updates, and get to work.
2. Be aware of when end users will need the system for higher levels of processing and gear your solution or work-around to that date.
3. Get a clear indication of what the acceptable levels of response are for the job or task your end user is doing before attempting to resolve the issue. Only then will you be able to know when your database is tuned adequately.

Conclusion

Have you ever approached a colleague who’s so deep in thought that you are met with a glassy-eyed response about why, on this good green earth, you were interrupting them? The reason is simple. They are stuck in the old world, trying to fix issues with their databases by arduously searching for statistics and patterns that tell them whether there is a problem. Sure, there are internal ways to troubleshoot the database, but let’s not forget that the user experience is as good an indicator as any statistic.

--

James F. Koopmann is dedicated to providing technical advantage and guidance to companies within information technology. Over the years, James has worked with a variety of database-centric software and tools vendors as strategist, architect, DBA, and performance expert. He is an accomplished author appearing regularly within printed publications across the Web, and speaking at local area User Groups as well as industry conferences. He may be reached at jkoopmann@pinehorse.com or www.pinehorse.com.