Background: APEC has 17 fora (working groups, policy partnerships, etc.) which report to the Steering Committee on Economic and Technical Cooperation (SCE). Since 2006, SCE has required each forum to be reviewed every four years, so that every year approximately four fora are assessed. Previously, SCE has assessed the performance of these fora against APEC goals using a number of methods, including support for independent assessments. This lack of uniformity in fora assessments led to unnecessary duplication, created indicators which were not necessarily comparable, and made it difficult to holistically address common issues across APEC fora.
Sustineo was engaged in 2017 to create a standardized framework for fora assessment to address these issues.
Approach: Our team began this project with a systematic review of past fora assessments and other relevant documents. Out of this review, we developed an initial Concept Plan which provided a macro-design of the Standardized Program with a suggested set of qualitative and quantitative indicators, and data collection techniques. Sustineo refined this Concept Plan through consultation with SCE members.
To strengthen and improve performance assessment across APEC fora, Sustineo adopted a ‘capability maturation’ approach – where implementation planning recognizes the need to stage implementation and develop capability over time. While the project brief assumed a fully functioning and effective performance monitoring and evaluation system, our M&E specialists recognized that this would take time to develop. To contribute to this learning process, our team helped APEC pilot and test the proposed Standardized Program to facilitate its wider adoption, focusing on an initial set of key indicators that were more easily measured and that could be expanded and developed over time.
Outcome: The final report set out a standardized system for the conduct of future SCE Fora assessments, including indicators, recommendations of the frequency of data collection and reporting, and data collection instruments such as surveys and report templates. This report and associated indicators and tools were used to guide the conduct of 2018 Fora Assessments.
The key outcome of the project provided the APEC Secretariat with a framework and processes that enabled a common and consistent approach to measuring fora performance. In addition, the approach could be applied across fora that differed in size, scope, and focus of work.
Further, the project resulted in a significant improvement in the understanding of performance measurement principles across APEC, including the use of indicators at the output, outcome, and impact level.