Strengthening the Evaluation System of Nepal

The major issues with the evaluation system in Nepal include a lack of evaluation capacity (evaluation reporting methods), lapses in the evaluation cycle (use of information in planning), a lack of a positive culture of evaluation (political ownership of evaluation), low involvement of stakeholders (both with donors and between ministries), and an inadequate legal framework for evaluation and appropriate organizational structures to support it (data collection mechanisms, complicated evaluation forms, and weak organizational structure) (Shrestha, 2009). In an effort to reform and improve evaluation in Nepal, particularly through the localization of evaluation, several government agencies in Nepal and JICA are coordinating on a project entitled the Strengthening of Monitoring & Evaluation System (SMES) Project. These issues are addressed largely through improvements in evaluation capacity through training and the localization of evaluation to harmonize and organize the levels of policy structure.

The SMES project is also consistent with the poverty alleviation objectives of the government of Nepal, especially in terms of its aim for good governance, which is provided through effective evaluation and the localization of evaluation. The government of Nepal has crafted a national development plan that outlines its strategy for poverty reduction. Nepal’s key strategies for poverty alleviation include increasing economic growth, social development, including previously excluded groups, and good governance. In order to promote good governance, Nepal has made efforts to strengthen its monitoring and evaluation system, as well as the process and outputs of decisions-making, including government expenditure and improving its overall efficiency, reliability, and transparency (Japan International Cooperation Agency & National Planning Commission Secretariat - The Government of Nepal [JICA &

NPCS] , 2007a, p. 1).

Nepal’s tenth five-year plan (2002-2007) for development includes the adoption of bottom-up planning and devolution. This includes capacity development training, participatory monitoring, and the development of linkages between the central and local governments, coordination with various ministries and districts and the verification of evaluation at the different levels as planned programs (JICA & NPCS, 2007a, p. 9). The inclusion of these measures in the development plan of Nepal indicates a desire for the localization of evaluation.

The SMES project is a three year project commencing in October 2006 as a project of technical assistance between Nepal and Japan. The project includes Japanese and local coordinators, as well as representatives from various government agencies in Nepal including the Ministry of Agriculture and Cooperatives, the Ministry of Physical Planning and Works, the National Planning Commission, the Ministry of Education and Sports, the Ministry of Health and Population, the Ministry of Finance, the Ministry of Local Development, and the Ministry of Forests. Most of the participants at the Training of Core Trainers (Tokyo, August 2007) had several years of experience with evaluation and were involved with the planning and evaluation sections of their ministry.

The main strategies of the SMES project include:

1. the training of trainers, core trainers, and government officials at various levels in monitoring and evaluation;

2. training courses at the central and local government level in policy structure planning, appraisal, implementation, project management, and monitoring and evaluation;

3. improvements to training manuals and monitoring reporting documents to strengthen the information management, analysis, communication, and feedback system (JICA &

NPCS, 2007a, p. 2).

The strategies of this project focus on the localization of evaluation, which can be seen through the training of local government representatives and the contents of the training of core trainers course, which further emphasize localization with t

trainings of national trainers. Three additional trainings per year are conduc

local government officials in evaluation. The SMES project has the aim of training several cadres of trainers at all levels of government in Nepal, as well as providing the relevant technical skills for evaluation (evaluation capacity

sharing.

The core trainers training took place in Tokyo from July

designed for government officials involved with evaluation. This group of trainers is charged with the task of leading evaluation in their ministry and/or department

trainers. The Tokyo training included lectures from experts in the areas of evaluation systems and procedures, human resource development for M&E, aid coordination, as well as

view Japanese local administrations in action.

In addition to the lectures and field trips, participants of this training were responsible for formulating action plans for themselves and their ministries to improve their evaluation system The lectures on evaluation emphasized the localization of national policy and the operationalization of the evaluation system in a local context. A practical and operational evaluation system that has been localized can be undertaken through the harmon

outcomes throughout the policy structure by adapting the magnitude and indicators, depending on the level of the policy structure. According to this logic, the end outcomes of the local level of policy structure can be considered to be the

of the national policy structure.

The evaluation lectures noted that in addition to this localization of policy structure, evaluation can itself be a tool for change within an organization by examining the ro

Source: JICA & NPCS, 2007a Figure 24 - SMES Nepal training pyramid

training courses at the central and local government level in policy structure planning, ppraisal, implementation, project management, and monitoring and evaluation;

improvements to training manuals and monitoring reporting documents to strengthen the information management, analysis, communication, and feedback system (JICA &

The strategies of this project focus on the localization of evaluation, which can be seen through the training of local government representatives and the contents of the training of core trainers course, which further emphasize localization with the coordination of evaluation

throughout the levels. These i also facilitate evaluation management cycle.

The trainings on evaluation include philosophy, procedures, and methods, as well as opportunities for the participants t

common problems with evaluation. The aim of the project is to establish a consistent, comprehensive, and functioning evaluation system in Nepal through improved evaluation capacity of the institutions and individuals involved in evaluation. Figure 4 shows the breakdown of the participants and trainees involved in the SMES project.

The core group of trainers is trained in Japan then return to Nepal to conduct yearly trainings of national trainers. Three additional trainings per year are conducted for district and local government officials in evaluation. The SMES project has the aim of training several cadres of trainers at all levels of government in Nepal, as well as providing the relevant technical skills for evaluation (evaluation capacity) as both a tool for improvement and for information The core trainers training took place in Tokyo from July-August 2007. This training was designed for government officials involved with evaluation. This group of trainers is charged

ask of leading evaluation in their ministry and/or department and

trainers. The Tokyo training included lectures from experts in the areas of evaluation systems and procedures, human resource development for M&E, aid coordination, as well as

view Japanese local administrations in action.

In addition to the lectures and field trips, participants of this training were responsible for formulating action plans for themselves and their ministries to improve their evaluation system The lectures on evaluation emphasized the localization of national policy and the operationalization of the evaluation system in a local context. A practical and operational evaluation system that has been localized can be undertaken through the harmon

outcomes throughout the policy structure by adapting the magnitude and indicators, depending on the level of the policy structure. According to this logic, the end outcomes of the local level of policy structure can be considered to be the small components of the aggregate end outcomes The evaluation lectures noted that in addition to this localization of policy structure, evaluation can itself be a tool for change within an organization by examining the ro

Source: JICA & NPCS, 2007a raining pyramid

156 training courses at the central and local government level in policy structure planning,

ppraisal, implementation, project management, and monitoring and evaluation;

improvements to training manuals and monitoring reporting documents to strengthen the information management, analysis, communication, and feedback system (JICA &

The strategies of this project focus on the localization of evaluation, which can be seen through the training of local government representatives and the contents of the training of core he coordination of evaluation throughout the levels. These improvements evaluation in the project The trainings on evaluation include philosophy, procedures, and methods, as well as opportunities for the participants to discuss common problems with evaluation. The aim of the project is to establish a consistent, comprehensive, and functioning evaluation system in Nepal through improved evaluation capacity of the institutions and individuals ure 4 shows the breakdown of the participants and trainees involved in the SMES project.

The core group of trainers is trained in Japan then return to Nepal to conduct yearly ted for district and local government officials in evaluation. The SMES project has the aim of training several cadres of trainers at all levels of government in Nepal, as well as providing the relevant technical ) as both a tool for improvement and for information August 2007. This training was designed for government officials involved with evaluation. This group of trainers is charged and becoming core trainers. The Tokyo training included lectures from experts in the areas of evaluation systems and procedures, human resource development for M&E, aid coordination, as well as field trips to In addition to the lectures and field trips, participants of this training were responsible for formulating action plans for themselves and their ministries to improve their evaluation system.

The lectures on evaluation emphasized the localization of national policy and the operationalization of the evaluation system in a local context. A practical and operational evaluation system that has been localized can be undertaken through the harmonization of end outcomes throughout the policy structure by adapting the magnitude and indicators, depending on the level of the policy structure. According to this logic, the end outcomes of the local level small components of the aggregate end outcomes The evaluation lectures noted that in addition to this localization of policy structure, evaluation can itself be a tool for change within an organization by examining the roles and

157 functions of the organization and building capacity through a participatory evaluation process (Miyoshi & Stenning, 2008). It was pointed out during the August 2007 training that the main weakness of the evaluation system in Nepal was that the system did not incorporate evaluation into the planning process, indicating a lapse in the policy management cycle. Furthermore, it was noted that the information from evaluation was collected by the central government but often was not utilized afterwards for planning or for feedback to the project or community in question; this also problematic.

Through the course of the Tokyo training, the participants were encouraged to discuss issues and ideas in regards to evaluation. Many recurrent issues came up, such as the difficulty of coordinating evaluation with multiple stakeholders, including donors, issues with participatory evaluation, and how to go about effective localization of evaluations. The participants seemed eager to improve their evaluation system and incorporate localization into evaluation; however there was a significant amount of discussion about how to have effective localization of evaluation and involve the local level in the policy management cycle up through national level policy planning. It was clear from both the topics of the training and the discussions of the participants that localization of evaluation is of key importance for evaluation reform in Nepal.

According to the results of a survey conducted by JICA as a part of the mid-term progress report for fiscal year 2007, several issues and challenges to the institutionalization of the evaluation system in Nepal were described including the utilization of results, quality of reports, human resources in M&E, updating and using report formats and guidelines, and the planning of M&E (JICA & NPCS, 2007b, p. 10).

The utilization of results and the planning of M&E activities reflect lapses in the proper functioning of the evaluation cycle. The low quality of the reports, inadequate human resources in monitoring and evaluation, and the lack of updating and utilizing report forms and guidelines demonstrate low levels of evaluation capacity. These issues are consistent with the issues found in the institutionalization of evaluation systems in other developing countries (discussed later in this chapter) and the lack of evaluation capacity and evaluation cycle described by Biju Shrestra (2009). There should be an increased awareness of these issues and the development of strategies for overcoming them. There should also be recognition of the importance of the evaluation process and a functioning evaluation cycle for effective evaluation and its institutionalization.

Although this survey describes some of the complications with evaluation reform in Nepal with the continued localization of evaluation and the evaluation capacity building trainings included in the SMES project, progress with evaluation institutionalization has been made.

However, the additional potential benefits of the SMES project should not be overlooked by either JICA or Nepal. Such benefits include social and institutional development through the utilization of evaluation as an information sharing and capacity building tool.

The strategies of the SMES project facilitate the functioning of the evaluation cycle through the introduction of cyclical policy structure management methods. Ultimately the training of core trainers and the other trainings conducted through the project will serve to increase the evaluation capacity in Nepal. It is also hoped that a culture of evaluation can be developed through the trainings. Discussions on pervasive issues in evaluation, the promotion of a productive atmosphere of evaluation, as well as on participation and ownership of evaluation are encouraged during the trainings, which were observed in the August 2007 training of core trainers in Tokyo. The SMES project has strategies to address the issues with evaluation through localization, human resource development in evaluation skills, manual and reporting

158 improvements, and has the potential to be quite effective in improving the functioning, effectiveness, usefulness, and accountability of the evaluation system in Nepal.

The SMES project concluded in 2009. By the end of the program, it is expected that there will be enough growth of evaluation capacity, a perpetuation of the evaluation cycle, and the preliminary process of effective localization of evaluation that the evaluation reform and good governance can take hold in Nepal.