Ecommerce | Ecommerce News

Benchmarks Are The First Step In Solving System Performance Problems

There is an old adage that says if you want to be successful you should start with the end in mind. In other words, it is a good idea to have a goal. This is especially true when it comes to managing a system's performance. The purpose of this article is to explain how using a benchmark is one simple way to get started in managing the performance of your system.

Obviously, the reason for having a goal is to create a clear view of where we want to be. As it relates to system performance, this would mean having a way to see what good performance looks like. This is where it gets tricky. The first thing to understand is that there are no strict rules which distinguish good performance from bad performance. It is often all about perception. If you don't believe this, just ask someone who uses a web-based email system, "How well did it perform today?" Depending on whom you ask, their likely response would be that it was “slow", "bad", or "the usual". Seldom does a person respond with the actual number of seconds it took for them to log on and retrieve their email. Without consciously keeping track, a person can tell whether the experience was acceptable. For the person who was most disgruntled perhaps there was not a problem with the speed of the system, but rather a pop-up ad got in their way while trying to get their emails. People's perceptions make the assessment of a system’s performance a difficult task. The way around this is to obtain a benchmark of your system's performance which can be used in the process of developing system performance goals.

In daily life we use benchmarks, or points of reference, to manage perceptions. Imagine traveling along the highway at 70 MPH and unexpectedly, you come upon a small town where there is no speed limit posted. Somewhat perplexed, you know that you should drive slow, so you reduce your speed to what you feel is safe. Suddenly, you are being pulled over and the officer tells you that you are going too fast. You ask the officer what is the speed limit and his reply is “slow”. From your perspective, you had been going 70 MPH and then you were going 35 MPH, which you feel was a significant slowing. But the officer explains that he had just come from an accident where a car going 25 MPH had hit a pedestrian. In real life, most places in which you drive do have speed limit signs posted in order to avoid these types of perception issues. However, for the people who use and manage systems without benchmarks which define acceptable system performance, conflicts of perception are a regular occurrence.

In the case of your system’s performance, it is likely that there are no “posted” response times for your system. In order to manage perception, it is necessary to create a benchmark of good system performance. Document everything you can about the system during the times when the system is heavily used and there are no reports of a performance problem. When you capture a profile of metrics which represent a known state of the system, you are essentially benchmarking your system's performance. This would include taking note of the available system metrics, such as the CPU, memory, and disk utilization. Even using a simple stop watch to measure the time it takes to complete a transaction is a good way to way to capture a response time benchmark. If available, get the application metrics such as the transaction throughput or the number of con-current active users. This information can be used to identify if there is abnormal usage of the system which could be responsible for a performance problem. If you acquire these metrics when the system is known to be performing well, this information will be your system's benchmark of good performance.

Benchmarking is the first step in managing the performance of a system. When the system’s performance is reported to be unacceptable, you will now have metrics that were taken at a point in time when the performance was known to be good. Compare the metrics obtained during the time of the reported poor performance to the benchmark metrics when the system was known to be performing acceptably. Identifying the differences will likely provide a good place start troubleshooting your system's performance issue.

Justin Martin has worked over the past 10 years in fields of Business Process Management Consulting, Capacity Planning and Systems Performance Management. His work revolves around the capacity management of distributed platforms and IT infrastructure for a fortune 500 company. Justin holds a Masters of Science degree in Business Information Systems.