Request for guidance on monitoring and evaluating e-governance initiatives

Have there been any new research and initiatives on monitoring and evaluating e-governance initiatives?

A UNDP study entitled "Empowering the Poor" (published in 2006) was conducted that provides a framework for monitoring and evaluating the potential of pro-poor e-government projects for upscaling. The variables used to measure project's potential for upscaling include:

Project Design - The initial design of the project will have an impact on its outcomes, its viability and hence the likelihood that it will have the potential for scaling up. A well-conceived project will be more likely to deliver useful benefits to its target audience in a sustainable manner.

Staff Capability - Indicated by its numbers and levels of education and by the opinions of clients with regard to the quality of the service they provide.

Community Acceptance - Indicators of community acceptance would be the benefit that is derived by the community from the services of the project as well as its willingness to continue using the services of the project.

Service Delivery - This factor relates to the ability of the project to maintain the delivery of the services that it is meant to provide over a long-term period, for as long as the host community might wish to receive those services. Key indicators
here would be the nature of the relationship between the project and the content providers, the form of partnership, and the level of their joint commitment to the maintenance of the service.

Technology - Project design incorporates the selection of the most appropriate technology.The awareness of technology among the users of the centres is used as an indicator for this variable.

Community Participation - Four indicators are used: equality in decision-making; equality in benefits; influence on project
management; and leadership development.

Equality in Decision-Making - Unless the sharing of responsibility is genuine, the project is likely to breed dependence and the community will fail to appropriate the technology for its own purposes. Indicators include: participation in meetings and committees, not just attendance, and also acceptance of roles of responsibility. There need to be representatives of all sections of the community within decision-making processes, especially women and disadvantaged groups.

Equality in Benefits - The benefits of the project must be distributed widely throughout all sections of the community.

Influence on Project Management - Has the community has been able to influence the way in which the project has been managed on a day-to-day basis. This would reflect both real influence as well as project managers responsive to their client’s needs.

Leadership Development - Useful indicators include the extent to which the community has been mobilized towards using information within its own development plans and planning for its use of information as a means towards achieving a better life.

and many others.

This framework was used to study 18 development projects in India that make use of ICTs in the form of community telecentres for the benefit of the poor. Questionnaires were completed by 2,156 users of the telecentres and interviews were conducted with project stakeholders and personnel. The study also published details of these 18 development projects and the lessons learned from this study.

For more information, please see http://unapcict.inigo-tech.com/ecohub/resources/empowering-the-poor-information-and-communications

I agree that "Empowering the Poor" is a useful resource.

However, many e-government is implemented at a national level. It is much more difficult to measure e-government at the national level than to evaluate specific projects. Evaluation requires a large degree of compatibility between data from different agencies, but their data are rarely comparable.

As part of OECD's Good Governance for Development in Arab Countries Initiative, report from a 2007 seminar on monitoring and evaluating e-government provides some new insights on this issue.

For example, the report identifies specific measures that can be used for a broader e-government evaluation, including:

• Pre-requisites for online services (e.g. Internet penetration, necessary skills, etc.).
• Level of shared infrastructure (e.g. are different agencies sharing the same infrastructure resources, or is each obliged to build its own?).
• Channel delivery strategy and/or existence of a one-stop shop (e.g. one point of access for all government services, whether national, regional or local and whether all agencies are working through it).
• Level of regulatory framework and enforcement at national level (e.g. privacy and security standards, authentication).
• Prevalence of national standards.
• Extent of co-operation and co-ordination among organisations.
• Level of public-private partnerships.
• Existence of financing mechanisms supporting e-government.

The full report is available at: http://www.oecd.org/dataoecd/55/16/39856235.pdf

Thanks for two very useful resources. Any other?

When we talk about Monitoring & Evaluation, how do you view eGov project assessment? 

On eGov project level assessment I am aware there have been various frameworks being used in Australia (DAM-VAM), USA (Public Value base), etc.  In India also we are using one based on Results and Enablers.