In part 1 of this series, we talked in general about the Performance Optimization Center that Mercury Interactive is touting in their new Business Technology Optimization (BTO) product line. Many of us out in the field who use LoadRunner and other QA products from Mercury are wondering if this is just a repackaging of old products to make them look new again, or if this a "revolution" as Mercury claims. We talked about the concept of a "Center of Excellence" and some of the qualities I had in my mind that defines such a thing in the realm of application performance. Click HERE to review that article. I'm happy to report that I have received some good feedback from both Mercury representatives as well as from performance engineers who do this stuff day to day. This article will highlight some of that feedback.

Going back to my original question: What does it mean to have a Performance Optimization Center? The first thing we have to do is separate the terms "Center of Excellence" from "Performance Optimization Center". In the past these were used interchangeably at Mercury, which may have been confusing. An Optimization Center is a product offering from Mercury that would be used in a customer's own Center of Excellence. Optimization Centers are considered to be the next phase of Mercury products. I want to talk about this in a little more detail in today's article.

By Mercury's definition, an Optimization Center consists of 3 types of components:

1. A Dashboard which provides summarized and real-time information as to the health of the area in question (IT Governance, Quality, Performance, Business Availability and Business Recovery). This is done through performance indicators (KPIs) from a business perspective. Dashboards can be configured to provide the views that are needed by all levels of management - an Department Manager can see information specific to his/her area while an Executive's view may encompass the entire enterprise.

2. Technologies for automating and integrating critical business functions. This is the area Mercury has been historically identified with and contains LoadRunner, WinRunner, etc. For the performance center this would be LoadRunner, ProTune, Deep Diagnostics, and Test Center.

3. Foundation Technologies for collecting the data, performing functions, etc. that are used across modules and Centers.

At the 2003 Mercury Worldwide User Group conference, attendees learned the direction of LoadRunner and Test Center within the context of this new Optimization Center. I see it as building additional tools into LoadRunner that you would have to get from 3rd party sources currently. Here is Mercury's vision for the Performance Optimization Center (or POC) according to their web site (as of 12/01/2003):

The Application Delivery Dashboard and Foundation are common and mentioned above. We already know about general load and stress testing with LoadRunner, and production tuning with ProTune. The Mercury Diagnostics refers to the Deep Diagnostics product (gained from the Performant acquisition). The Deep Diagnostics currently appears as an additional tab (with it's own graphs) in LoadRunner which allow you to dig down to the code level for bottlenecks (method calls, memory leaks, etc). Currently this is available only for J2EE. I suspect .NET will not be far behind. The Capacity Planning module is something we have not seen yet. I suspect this is some kind of modeling tool. This would be beneficial when your development or QA environment is nowhere near the specifications of production and you need to "model up" to get some idea of what to expect. It is still a guess, but an educated one.

Amnon Landan (Mercury CEO) says that Mercury has been very broad in the past, but now wants to become deep too. For instance, LoadRunner can reveal what the end user experience is easily. When isolating bottlenecks, usually only a general area for where to start looking is known. Deep Diagnostics is an attempt to replace the need for third party application profilers (like Compuware DevPartner Studio or Intel's VTune). I hope to see an additional module that contains a network sniffer, allowing the network layer to be looked at more closely at the time of load without using an additional tool such as NetMon. When you look at all modules planned and in existence, its obvious Mercury is trying to create a centralized performance tool used throughout the whole development lifecycle with the ability to get more granularity out of each layer (application, network, hardware). The benefit of having these modules within LoadRunner is the ability to correlate all metrics into a single "results" file. This makes for more powerful analysis and diagnostic troubleshooting of applications.

Now back to my original reason for this series of articles: Just because you have the entire POC product offering with every module, does not mean you have automatically have a Center of Excellence. Just as in the past, the tool makers assume you know what to do with the software to prevent it from being "shelf ware". This time around, it has to be different.

Mercury has a model for a Center Of Excellence. It is in use every day. If you have ever used the outsourced offerings of Protune Validation Service (formerly ActiveTest) or the Protune Delivery Service (formerly ActiveTune), you have interfaced with their internal Center. There are teams of people running projects (engagements) for multiple clients concurrently in a centralized framework, utilizing infrastructure in disparate places. They do performance testing and tuning inside and outside the firewall. They address performance at every layer and tier.

The model Mercury Interactive uses to operate and manage their own Center of Excellence should be the foundation for owners of the POC to begin implementing their own. I am sure Mercury has learned a few tips and tricks running multiple projects in a performance lab and has probably documented some best practices around how they use their tools internally. This is the missing piece of the equation that customers of their products needs and deserve. Perhaps you only have 10 projects a year and one LoadRunner lab. You can still provide a Center of Excellence on a smaller scale to your organization. If Mercury is truly committed to their customer base implementing a Center of Excellence, a detailed methodology for using the POC in this manner needs to be made available. I alluded to this last time. I have spoken with a few people within Mercury in recent days who have told me that this is their general direction.

Now that we have distinguished a Center of Excellence from the Mercury Performance Optimization Center, it is time to talk about how a Center of Excellence focused on performance should be implemented. In my next article, I'll try to do just that. As usual, your thoughts are appreciated in the Loadtester.com Forums.

For more information about Scott, check out the rest of this web site at
www.loadtester.com