Launching the NAP Review: Results
18 months ago we set out to develop a tool for civil society in OGP countries to assess how open and ambitious their governments were being in developing their National Action Plans. After countless drafts of questions and two rounds of pilots, we’re pleased to be able to share with you what we’ve come up with. In this three part blog post series, we will 1) introduce the tool, 2) share the results from the latest round of pilots, and 3) draw some overarching conclusions from the pilots.
Today we look at the results of the pilot countries.
Seven countries – Argentina, Colombia, Finland, France, Hungary, Israel and Liberia – have just been through the process of completing the review. Here’s how they did.
The table below shows the percentage of the available score each country received overall and in each part of the review (Part A: NAP Implementation; Part B: NAP Creation; & Part C: NAP Quality). A link to the full results can be found at the end of the post.
As we said in the first post, we need to be careful about the conclusions we draw, particularly in comparisons between countries, because how a question is interpreted and answered will depend on a lot of individual, societal and cultural factors. That said, here we outline some high level findings and conclusions from each of the countries.
Argentina
Argentina received 41% of the available score from its civil society reviewers. It did best on the quality of the new action planAction plans are at the core of a government’s participation in OGP. They are the product of a co-creation process in which government and civil society jointly develop commitments to open governmen... (52%) and worst on the openness of the implementation of the previous plan (33%). Across the review, it was judged to have fulfilled just 4 of the 47 criteria to a large extent, compared with 11 not at all.
The scores suggest that the Government made some effort to run an open process, fulfilling most of the criteria to some extent. However, despite there being frequent meetings, Argentina did particularly poorly on its relationship with civil society. For example, civil society felt little influence over the action plan, and judged the partnership with government to be poor. In contrast, commitments were well written, with clear problem statements and milestones, but again they only partially matched the priorities of civil society. It will be interesting to see how the new government will take forward the OGP process.
Colombia
Colombia received 44% of the available score from its civil society reviewers. It did best on the process for developing the latest action plan (65%), and worst by some margin on the openness of the implementation of the previous plan (16%). Across the review, it was judged to have fulfilled just 7 of the 47 criteria to a large extent, compared with 8 not at all.
The scores suggest that the Government made a good effort to run an open process (fulfilling most of the criteria to a moderate extent), though notably it scored poorly on factors that would enable widespread engagement (e.g. widely publicising the details of meetings, and providing for remote participation). Commitments were well written, with clear details, milestones and owners, though they lacked clear problem statements. Colombia scored worst of the participating countries on the openness of the implementation of the previous action plan, which hurt its overall score, and would seem an important area for attention over the coming year.
Finland
Finland received 66% of the available score from its civil society reviewers. It scored evenly on the openness of the process for implementing the previous (70%) and developing the latest plans (68%), but fell down on the quality of the action plan (59%). Along with Liberia, it was the only country judged to have fulfilled all of the criteria to at least some extent, with 8 met to a large extent.
The scores suggest that the Government made a good effort to run an open process, scoring particularly highly on aspects of its transparencyAccording to OGP’s Articles of Governance, transparency occurs when “government-held information (including on activities and decisions) is open, comprehensive, timely, freely available to the pub... More (e.g. publication of a timeline, point of contact details, and key documents), though it did poorly on publishing inputs into the consultation process and communicating why they were or were not taken into account. Finland also fell short on a couple of criteria related to the SMARTness of its commitments (e.g. publication of metrics, and owners), which could be easily rectified in the coming months. Considering its positive performance overall, an area of particular focus would be reviewing where the Government’s broader activities are not consistent with the principles of open government, a criteria on which most countries performed badly.
France
France received 60% of the available score from its civil society reviewers. This was France’s first action plan, so Part A of the review was not completed. In the remaining sections, it scored a little better on the creation (65%) than the quality (54%) of the NAP. Across the review, it was judged to have fulfilled 14 of the 47 criteria to a large extent, compared with just 3 not at all.
The scores suggest that the Government made a good effort to run an open process, like Finland scoring particularly highly on aspects of its transparency (e.g. publication of a timeline, point of contact details, and key decision maker details), but poorly on publishing inputs into the consultation process or communicating why they were or were not taken into account. France performed particularly well on holding regular, open meetings with civil society, though this was not felt by civil society to translate into decision making power, with low scores assigned for the extent to which civil society was involved in developing commitments or influencing the plan. Improving this partnership would seem an important area for attention.
Hungary
Hungary received 30% of the available score from its civil society reviewers. It did best on openness of the implementation of the previous action plan (40%), and worst on the creation of the new plan (26%). Across the review, it was judged to have fulfilled just 2 of the 47 criteria to a large extent, compared with a third (16) not at all.
The scores suggest that the Government made very little effort to run an open process, scoring particularly poorly on consulting or involving civil society in developing the NAP. Hungary scored the worst of the participating countries on the quality of its action plan, with low scores on milestones, metrics and ownership. Perhaps of most concern, civil society judged the government to be particularly poor on the sincerity of its commitmentOGP commitments are promises for reform co-created by governments and civil society and submitted as part of an action plan. Commitments typically include a description of the problem, concrete action... towards open government. In contrast, there does appear to have been some effort to involve civil society in implementing the previous action plan, with the presence of a multi-stakeholder forumRegular dialogue between government and civil society is a core element of OGP participation. It builds trust, promotes joint problem-solving, and empowers civil society to influence the design, imple... or steering group being a particular plus point; this could, perhaps, be a vehicle for improving Hungary’s performance in the future.
Israel
Israel received 28% of the available score from its civil society reviewers, the lowest score among the participating countries. It did best on the quality of its latest action plan (49%), and worst by some margin on the openness of the process to create it (14%). Across the review, it was judged to have fulfilled just 3 of the 47 criteria to a large extent, compared with half (23) not at all.
The scores suggest that the Government made no effort to run an open process, receiving the lowest score of the participating countries on this aspect of the review, and only scoring higher than “to some extent” on one criteria (the extent to which civil society was free to self organise). Scores were also poor across the implementation of the previous action plan, with the one positive being the presence of a joint process to review the consultation and engagement process. In contrast, Israel did comparatively well on the SMARTness of its commitments, though this was offset by weak scores on the extent to which the plan covers civil society priorities and the sincerity of the Government’s commitment to openness. Judging by these scores, it would seem that Israel has significant work to do in implementing the principles and requirements of OGP, and improving its engagement with civil society in the future.
Liberia
Liberia received 77% of the available score from its civil society reviewers, the highest score among the participating countries. Liberia scored remarkably consistently across the review, with just 2 percentage points separating its highest and lowest scores. Along with Finland, it was the only country judged to have fulfilled all of the criteria to at least some extent, with 19 met to a large extent.
The scores suggest that the Government made a very good effort to run an open process, scoring particularly well on the involvement of civil society directly in the decision making process (e.g. developing commitments and forming the action plan), and has successfully developed a strong action plan with civil society. It did comparatively less well on the transparency of the process, though was still judged to have fulfilled each of the criteria to a moderate extent. The only criteria that Liberia was judged to have met only to some extent was on publishing the inputs to its consultation. A couple of areas for improvement next time perhaps.
Conclusions
The analysis above is based on the scores provided by the civil society reviewers. The detailed results for each country can be found here. Let us know in the comments if you disagree with the way we’ve interpreted the results.
In the next post, we’ll draw out some high level conclusions from across the pilot countries.