Skip Navigation

IRM to Raise the Bar for Model Commitments in OGP

The Independent Reporting Mechanism (IRM) is a key means by which all OGP stakeholders can track progress on open government commitments in participating countries. The IRM produces progress reports for each country participating in OGP through an internationally reviewed process, led by IRM researchers in each OGP country. These reports are intended to stimulate dialogue and promote accountability between governments and citizens.

As part of the reporting process, since 2014, the IRM recognizes model commitments and awards them ‘stars’.  In order to receive a star, a commitment must meet the following criteria:

  1. It must be specific enough that a judgment can be made about its potential impact. Starred commitments will have “medium” or “high” specificity.
  2. The commitment language should make clear its relevance to opening government. Specifically, it must relate to at least one of the OGP values of: Access to Information, Civic Participation, or Public Accountability.
  3. The commitment must have “moderate” or “transformative” potential impact, should it be fully implemented.
  4. Finally, the commitment must see significant progress during the action plan implementation period (receiving a ranking of “substantial” or “complete” progress).

Both the mid-term Progress Reports and the final End of Term Reports include stars.  (Last year, the OGP Steering Committee expanded the IRM mandate to produce final, end of term reports for final accounting of OGP commitments, beginning in 2016.) However, since the progress reports primarily focus on learning, the End of Term Reports should be considered the final tally of stars from the entire implementation of the action plan.

Using the criteria above, almost 24% of the commitments analyzed by IRM reports in 2014 received stars.

In order to further incentivize a race-to-the-top in OGP and to ensure that starred commitments are in fact model commitments, starting in 2015 the IRM will raise the standards for what can qualify for a star.  Specifically, only those commitments considered to be potentially transformative in the relevant policy area will be eligible for starred status.  Of course, the commitment will still have to meet the other three criteria above.

Professor Jonathan Fox, one of the members of the International Experts Panel that guides the IRM method and ensures the quality of IRM reports, offered the following explanation for this change:

“Our goal in creating the idea of starred commitments was to draw attention to those government efforts that have made tangible progress toward clearly ambitious goals. Now, after two years of reviewing dozens of reports and coding of hundreds of commitments, we have come to the conclusion that ‘potentially moderate impacts’ are not sufficiently ambitious. In the coding process, we found that many of the starred commitments were close to borderline cases with low impact. We don’t want to water down the idea of starred commitments.”

Had they been considered under the new criterion, about 10% of the commitments analyzed in 2014 would have received stars.

In order to facilitate cross-country learning, the IRM progress reports published in 2015 will include star counts using both systems, before a full transition to the new system is implemented in 2016. 

The IRM process is guided by a publicly available Procedures Manual, which explains the IRM method and provides an introduction to its values and foundations, as well as details of the assessment process. The Procedures Manual also includes complete research guidance for IRM reports as well as the criteria for awarding starred status to commitments.

In order to be as clear, accurate and comprehensive as possible, the Procedures Manual was refined and updated this year. The new manual has made significant improvements in the following four areas:

  • Updated Questions and Guidance:  The IRM has made many revisions and clarifications to the types of questions in its method in order to ensure that national researchers can collect the highest quality of information possible. 
  • Research Ethics:  Recognizing the inherently participatory nature of the IRM’s work, we have added a section on the risks and ethics of participatory research.  We have also developed a series of specific participatory exercises for researchers to use during their consultations with stakeholders.
  • Eligibility Section: In September 2012, OGP decided to begin strongly encouraging participating governments to adopt ambitious commitments in relation to their performance in the OGP eligibility criteria. The OGP Support Unit collates eligibility criteria on an annual basis, and the IRM reports now present these scores in a table at the end of IRM progress reports.  When appropriate, the IRM reports will also discuss the context surrounding progress or regress on specific criteria in their Country Context sections.
  • Simplified Commitment Tables: Finally, to maximize the readability of IRM reports, each commitment will now be analyzed in a simplified table that focuses on the four key variables that determine a commitment’s eligibility for starred status: Specificity, Relevance, Potential Impact, and Completion.  The IRM will continue to collect the rest of its data points for the IRM databases, but these will only be published in separate IRM open data releases and not as part of the written reports.  Starting with the IRM reports scheduled for release in July 2015, all will feature the new table format. For a preview of the new format, see the recent Special Accountability Report published for Indonesia.

As always, please direct any questions for the IRM to irm@opengovpartnership.org     

Filed Under: Research
Open Government Partnership