So, Did That Project Meet Your Expectations?
| Setting Goals |
| Baselining Operations |
| Conduct a Post Mortem |
| Compare Results With Multiple Sources |
Investment firms invest millions of dollars upgrading systems that are calculated to provide ROI through enhanced functionality, reduced operational costs, and lower risk. Most firms create detailed business cases that quantify the benefits and costs of each technology project and identify the exact timeframe that will provide a “break-even” on the investment. In keeping with this practice, after obtaining funds for the project, you will carefully place your business case on the shelf and leave it, undisturbed, for perpetuity.
Most firms, large and small, require some form of financial justification for every significant investment in their infrastructures. It is considered a “best practice” to gain the support and commitment from all stakeholders – those who will benefit from the investment and those who must pay for it (not always the same groups).
Then why is it also true that a large percentage of firms fail to review projects to make sure that they have met both tangible and intangible objectives? Several reasons exist, of which the one most cited is lack of discipline.
Discipline involves behavioral change. All infrastructure organizations – both technology and operations – should develop a practice of benchmarking their capabilities on periodic schedules, e.g., annually, upon major reorganization or acquisition, etc., to know where they stand and to keep on track with their goals. Benchmarking exercises highlight areas of strengths and weaknesses, giving you the information you need to prioritize your activities. Any firm that outsources its capabilities (technology or process) inserts a benchmarking clause into the outsourcing contract to make sure that it is getting the return that it expects from its investment. Why would you treat your internal capabilities any differently? To be effective a good benchmarking program should not require a new layer of overhead.
You can obtain meaningful and actionable results by incorporating a few techniques into your current operations:
| Setting Goals | Top of Page |
Whether you are planning next year’s budget, or your next system upgrade, be sure to specify all objectives in a measurable manner. The adage “If you can’t measure it, you can’t manage it” holds true here. Headcount changes and transactions per day are relatively straightforward metrics. However, you can also establish metrics for “improved customer satisfaction”, e.g., increase retention rate by 10% or increase customer account size by 20%, and other “soft” goals by requiring that stakeholders articulate the real value that these goals will deliver.
| Baselining Operations |
Take a snapshot of your current operation before you initiate any changes. Express baseline metrics in the same terms used in the budget or business case, e.g., operations FTEs, maintenance dollars/month, so you can measure improvement at the end of the project.
| Conduct a Post Mortem |
At the end of the project or year, re-run the baseline activity to determine how far you have progressed. In time you will want to perform this task more frequently to allow for mid-course corrections.
| Compare Results With Multiple Sources | Top of Page |
Results frequently differ from expectations. Some of the reasons for these differences are obvious – lack of staffing or funding to effect the changes required – where others are more difficult to discern. Obtain performance data from your peers to see how effective their processes are. You may find that others are obtaining similar results; this is extremely useful information to have when you are trying to explain any shortfalls to management.
Experiment with these guidelines by applying them to your next small project. As you gain confidence in the approach, expand it to other, more complex initiatives and, eventually, to your entire organization. Over time you will notice that your team will pay closer attention to organizational and/or project targets, and start questioning whether what they are doing is helping to achieve those goals.
Base Lining Best Practices
Base lining is all about metrics. Metrics should be:
| Clear | Top of Page |
State the metrics in quantifiable terms, and include the units you are measuring. Transactions per day is clear; fewer staff is not.
| Relevant |
Identify three to five specific goals that you must achieve with this initiative. These goals should relate to, or be restatements of, the objectives you specified in the business justification that you made to secure approval for the project.
| Measurable |
You must be able to collect the data required to calculate the metric. And you need to be able to capture this data at the level of detail required to properly assess the operation.
| Repeatable |
Are you going to be able to obtain the same data once you have completed the project? If you materially change a process during the project you may find that you cannot make a fair “before and after” comparison.
| Actionable | Top of Page |
You need to be able to act on the information. Just because some measurement deteriorates over time does not mean that you can affect the change.
| June 2006 • Issue 39 |
| Base
Lining Metrics Best Practices |
| Clear |
| Relevant |
| Measurable |
| Repeatable |
| Actionable |
|
| Join
us for Lunch for a Review & Discussion of Benchmarking Thursday, July 13th, 2006 London, England RSVP NOW pmcalpine@cutterassociates.com |
|
| Upcoming Events & Webcasts | ||
|
July 2006 | |
| UK Data Management Affinity Group™ July 5 |
||
| Webcast July 13 Topic: Operations Outsourcing |
||
| Webcast July 27 Topic: Supporting Hedge Funds |
||
|
||
|
August 2006 | |
| UK Data Management Affinity Group™ August 2 |
||
| Webcast August 10 Topic: Macgregor |
||
| Webcast August 24 Topic: Service-Oriented Architecture |
||
|
||
|
September 2006 | |
| UK Data Management Affinity Group™ September 6 |
||
| Webcast September 14 Topic: Implementation in a "Long Only" Shop |
||
| The Technology Alliance™ September 18 & 19, Boston Topics: Managing Your Back Office |
||
| The Technology Alliance™ September 18, Boston Reception & Dinner Cruise |
||
| Webcast September 28 Topic: Eagle PACE |
||
|
||
|
October 2006 | |
| UK Data Management Affinity Group™ October 4 |
||
| Webcast October 12 Topic: Charles River |
||
| Technology Council Update Service™ October 19 Topic: Data Warehousing Products |
||
| Webcast October 26 Topic: Derivatives Systems and Tools |
||
|
||
|
November 2006 | |
| UK Data Management Affinity Group™ November 1 |
||
| Webcast November 2 Topic: IT Resources: Outsourcing and Off Shoring |
||
| The Technology Council™ November 15, London Topics: Client Reporting and Business Intelligence Systems; Supporting Derivatives Processing |
||
| The Technology Forum™ November 15, London Topics: Client Reporting and Business Intelligence Systems; Supporting Derivatives Processing |
||
| Webcast November 30 Topic: MiFID: Level 3 Member Discussion |
||
|
||
|
December 2006 | |
| The Technology Council™ December 5, New York Reception and Dinner Topics: Client Reporting and Business Intelligence Systems; Supporting Derivatives Processing |
||
| UK Data Management Affinity Group™ December 6 |
||
| Webcast December 14 Topic: Data Dictionaries and Meta Data Tools If you have any questions or require help with registering, please contact: |
||