Introduction

Measuring the performance and progress of an organization’s Enterprise Architecture (EA) gives the Enterprise Architecture team an opportunity to identify areas where they need to focus, in order to ensure that EA is functioning properly and delivering the intended value.

Periodic monitoring and measurement of EA metrics lead to a successful EA program across the organization.

Today, most organizations have no mechanism to measure the effectiveness of EA, there being very little guidance on the measurement of EA effectiveness. Most organizations focus on the definition and completeness of their EA and the maturity of their EA development processes rather than measuring the effectiveness of their EA.

Challenges in measuring the effectiveness of EA

Traditionally, Enterprise Architecture has been measured on the immediate benefits that it provides to the organization. Although EA has a significant impact on how the IT organization aligns with Business, if EA metrics are only limited to measuring Business-IT alignment, they will fail to measure the long-term effect of EA, and more importantly, its effect on business, as an enabler and catalyst of change.

Based on the author’s interactions with various customers, the following are the practical challenges that they encountered in measuring the effectiveness of EA:

• Effectiveness of EA is rarely assessed
• No established industry-standard metrics for measuring EA effectiveness
• EA context significantly differs across organizations and also within Organization units
• Different stakeholders have different perceptions and expectations of EA
• Most EA metrics aim to measure EA value in IT terms, not in business terms
• Measurement data concerning EA often does not exist within the organization
• Objectives of EA change during the implementation path – this is not a challenge. In fact, EA should be more agile in adopting changes.
• Most employees of an organization do not know about architecture

Why measure EA

 EA effectiveness is the degree to which the objectives of EA are attained by means of EA. EA metrics not only help gauge the long-term effect of EA, but also help measure the true value of EA. This enables the organization to plan more effectively and efficiently.

Measuring EA effectiveness is necessary to:

• Determine how effectively and efficiently a process or service satisfies the customer
• Improve the organization’s ability to seize new business opportunities
• Make decisions based on data
• Improve agility
  
• Extend systems to meet increasing demand
• It provides opportunity for Innovation

Measurement should:

• Translate customer expectations into goals
• Evaluate the quality of processes
• Track improvements
• Support enterprise strategies

EA metrics measurement framework

Periodic monitoring and measurement of a set of defined EA Metrics leads to a successful EA Program across the organization.

As EA goals and processes are very broad, YoY (Year on Year) measurement comparisons for all EA metrics would be a good indicator of EA program penetration across the organization. The status of EA Metrics is presented and communicated to the relevant stakeholders at regular intervals. The following diagram depicts the EA Metrics Measurement Framework.

The business strategy of the organization is the preliminary input to the EA metrics model consisting of EA Metrics inputs and EA planning inputs.  Use these inputs in all planning activities across the organization. Measuring the planning effectiveness and efficiency would clearly represent the value of EA to business, as these planning activities affect all four aspects of the organization, i.e. customer, finance, growth and processes.

Customer metrics: Focus is on increasing the business value using EA. It helps in improving the business process (quality, costs) efficiency through architecture initiatives. Also, improves the business and IT alignment.

Financial metrics: It covers capital expense reduction through the prevention of non-standard, one-off solutions. It helps in enabling the organization to use the latest technologies and asset reuse. Promotes operational expense reduction in the form of supporitng rationalization to simplify the portfolios, which brings cost transparency to run IT.

Growth metrics: Focus on collaboration, workforce effectiveness, learning and capability factors.

Process metrics: Regulatory compliance and project architecture assurance are covered as part of this metric. It drives behaviors toward adopting relevant IT standards & complying with processes. Also, it helps in improving collaboration through stronger compliance with IT standards and processes.

The Metrics types are:

Short term: These metrics measure the progress of EA during the initial iterations of EA implementation.

Long term: These metrics measure the efficiency and effectiveness of EA to ensure that the true value of EA is delivered to the business.

Summary

Establishing an Enterprise Architecture function helps organizations keep complexity and misalignment at bay. However, it is not inconceivable that Enterprise Architecture may itself fall prey to this complexity trap if it is not well defined, managed and maintained. Even if it is, it may be of little to no value to the organization if the well-defined artifacts are not in use by the organization or if the EA processes not adhered to.

To ensure that Enterprise Architecture is progressing well and is adding value to the organization, there is a need to define metrics for measuring the progress and performance of EA.

The EA metrics should measure the implementation of EA and be governed by the concept of EA maturity. EA metrics should be defined in terms of effectiveness, agility and alignment to business goals and strategies. Once EA architecture is properly implemented and business entities are engaged, it will improve the overall planning activities of the organization and not just IT.