Forgot your password?
New user?
Document Actions
Chapter 10: Monitoring and Evaluation
In this Chapter:
Monitoring and Evaluation Overview
Policy-makers and project managers can use benchmarks, key performance indicators (KPI), user satisfaction surveys and other monitoring and evaluation (M&E) tools to track the performance of e-Government projects and to determine whether mid-course adjustments are needed. Monitoring and evaluation as a process helps measure progress and can create an internal feedback loop, which can help avoid failures and maximize the potential for success.
KPIs and other metrics will differ from project to project, depending on the goals that have been articulated, the intended audience, and how government and user costs and benefits have been defined. Comprehensive M&E uses both financial and non-financial metrics. Metrics can be conceptualized as progress indicators, outcome indicators and impact indicators.
M&E is different from assessment. e-Readiness assessments are performed to determine a country’s capacity to use and apply ICT, and e-Government readiness assessments are focused on the extent to which governments have the capacity to implement – and users have the capacity to take advantage of -- e-Government projects. Designers of e-Government projects, when they are beginning their planning efforts, would benefit from an assessment of the e-Readiness of the nation or the specific intended audience of a project, in order to determine what types of services can realistically be provided and what barriers are likely to be encountered.
In contrast, M&E occurs throughout the implementation of a project, to measure progress, support mid-course corrections and guide resource allocation decisions. Monitoring and evaluation may occur at various levels. On the national level, for example, the US Office of Management and Budget (OMB), which is within the executive branch of the President, tracks agency-by-agency progress towards government-wide e-Government goals. On the regional and international scale, there are several comparative benchmarking projects that are measuring the progress of e-Government, ranking countries from year-to-year. Perhaps most important, however, is monitoring and evaluation at the project level.
Project level evaluation is not something done at the end of a project. Rather, it should be done throughout a project, so that findings can be acted upon and deficiencies corrected. Most effectively, monitoring and evaluation refers back to goals and metrics laid down at the planning phase. Therefore, it is important to invest time and resources in defining KPIs or other metrics at beginning of the project and to create a plan for the monitoring and evaluation process. Too often, governments do not consider evaluation metrics until after project completion. Cost and benefit indicators should be defined in the beginning of a project and built into the design and implementation process, so that the relevant data can be collected as a project moves forward.
Unfortunately, it seems that few governments in developing countries are investing in measuring demand for e-Government. Understanding the interests and needs of users and continually taking them into account in the planning and implementation process will contribute to the success of e-Government projects. A failure to measure benefits and costs and to compare them against desired outcomes will inhibit the capture of lessons learned and the identification of best practices that can be incorporated into program design. Without such a feedback loop, it will be difficult to overcome the high rate of failure of e-Government projects.
Increasingly, donors investing in e-Government are insisting that the funded projects use qualitative and quantitative indicators to measure the results and impact of projects.
KPIs and Benchmarks: A Key Performance Indicator (KPI) is a metric of progress towards or accomplishment of a partial or final objective. KPIs can be used to measure both progress and outcome of a project or a program. Wikipedia defines benchmarking (also "best practice benchmarking" or "process benchmarking") as a process in which organizations evaluate some aspect of their operations in relation to best practices (or the best performer), usually within their own sector. Benchmarking allows organizations to develop plans on how to adopt such best practices or otherwise to improve their performance in relation to the best in the field. Benchmarking is when an organization compares its own performance against the performance of similar organizations, or against some identified best practice. Benchmarking is used mostly to define program objectives and may lead to the identification of KPIs. Progress in project implementation is measured against internally developed objectives or milestones while benchmarking is comparison with other projects.
M&E Resources
M&E Practice Notes
- Monitoring and Evaluation Overview
- Monitoring and Evaluation Methodologies
- Global Benchmarking Studies
- Additional Resources on Monitoring and Evaluation
Monitoring and Evaluation Overview
Policy-makers and project managers can use benchmarks, key performance indicators (KPI), user satisfaction surveys and other monitoring and evaluation (M&E) tools to track the performance of e-Government projects and to determine whether mid-course adjustments are needed. Monitoring and evaluation as a process helps measure progress and can create an internal feedback loop, which can help avoid failures and maximize the potential for success.
KPIs and other metrics will differ from project to project, depending on the goals that have been articulated, the intended audience, and how government and user costs and benefits have been defined. Comprehensive M&E uses both financial and non-financial metrics. Metrics can be conceptualized as progress indicators, outcome indicators and impact indicators.
M&E is different from assessment. e-Readiness assessments are performed to determine a country’s capacity to use and apply ICT, and e-Government readiness assessments are focused on the extent to which governments have the capacity to implement – and users have the capacity to take advantage of -- e-Government projects. Designers of e-Government projects, when they are beginning their planning efforts, would benefit from an assessment of the e-Readiness of the nation or the specific intended audience of a project, in order to determine what types of services can realistically be provided and what barriers are likely to be encountered.
In contrast, M&E occurs throughout the implementation of a project, to measure progress, support mid-course corrections and guide resource allocation decisions. Monitoring and evaluation may occur at various levels. On the national level, for example, the US Office of Management and Budget (OMB), which is within the executive branch of the President, tracks agency-by-agency progress towards government-wide e-Government goals. On the regional and international scale, there are several comparative benchmarking projects that are measuring the progress of e-Government, ranking countries from year-to-year. Perhaps most important, however, is monitoring and evaluation at the project level.
Project level evaluation is not something done at the end of a project. Rather, it should be done throughout a project, so that findings can be acted upon and deficiencies corrected. Most effectively, monitoring and evaluation refers back to goals and metrics laid down at the planning phase. Therefore, it is important to invest time and resources in defining KPIs or other metrics at beginning of the project and to create a plan for the monitoring and evaluation process. Too often, governments do not consider evaluation metrics until after project completion. Cost and benefit indicators should be defined in the beginning of a project and built into the design and implementation process, so that the relevant data can be collected as a project moves forward.
Unfortunately, it seems that few governments in developing countries are investing in measuring demand for e-Government. Understanding the interests and needs of users and continually taking them into account in the planning and implementation process will contribute to the success of e-Government projects. A failure to measure benefits and costs and to compare them against desired outcomes will inhibit the capture of lessons learned and the identification of best practices that can be incorporated into program design. Without such a feedback loop, it will be difficult to overcome the high rate of failure of e-Government projects.
Increasingly, donors investing in e-Government are insisting that the funded projects use qualitative and quantitative indicators to measure the results and impact of projects.
KPIs and Benchmarks: A Key Performance Indicator (KPI) is a metric of progress towards or accomplishment of a partial or final objective. KPIs can be used to measure both progress and outcome of a project or a program. Wikipedia defines benchmarking (also "best practice benchmarking" or "process benchmarking") as a process in which organizations evaluate some aspect of their operations in relation to best practices (or the best performer), usually within their own sector. Benchmarking allows organizations to develop plans on how to adopt such best practices or otherwise to improve their performance in relation to the best in the field. Benchmarking is when an organization compares its own performance against the performance of similar organizations, or against some identified best practice. Benchmarking is used mostly to define program objectives and may lead to the identification of KPIs. Progress in project implementation is measured against internally developed objectives or milestones while benchmarking is comparison with other projects.
M&E Resources
M&E Practice Notes