Skip Navigation

SDG 16 Data Initiative – Making them Count

Thomas Wheeler,John Romano,Jairo Acuna-AlfaroandAlexandra Wilde|

Using indicators and data to strengthen accountability for the SDGs

Summary note of consultation workshop[1]

This note draws upon a consultation on 6 December 2016, which focused on whether and how the indicators and data linked to the Sustainable Development Goals (SDGs) can strengthen accountability on issues of peace, justice, governance and human rights. The consultation workshop was held as an official side-event to the Fourth Open Government Partnership (OGP) Summit in Paris. There were around 30 participants from governments, multilateral organisations, and civil society organisations from around the world. The workshop was co-hosted by Saferworld, the Transparency, Accountability and Participation (TAP) Network, and the United Nations Development Programme (UNDP).

The SDGs, indicators and data

The 2030 Agenda, agreed in September 2015, sets out a new global framework of 17 SDGs and 169 targets. This includes SDG16 on issues of peace, justice, governance and human rights. As well as establishing norms and acting as a coordination framework at the global level, evidence suggests that voluntary frameworks like the 2030 Agenda can incentivise governments to take action for the sake of their global and regional image.[2] International frameworks can also be leveraged by national level actors – inside and outside of government – who are already working for reforms of contested issues.[3] However, whether and how international frameworks interact with change dynamics will vary from context to context.  

As with the Millennium Development Goals (MDGs), and other international development frameworks such as the New Deal for Engagement in Fragile States, the SDGs include a set of global indicators that will be used to monitor progress. These global indicators, which are being finalised by the Inter-Agency and Expert Group on Sustainable Development Goal Indicators (IAEG-SDGs), are expected to be accompanied by complementary regional and national indicators.[4]  Regarding SDG16 in particular, the 2030 Agenda agreed on ten targets, two means of implementation and 23 indicators. These indicators have been grouped in three tiers according to the levels of conceptualisation, methodological aspects and availability (see Annex). However, workshop participants highlighted that agreement on indicators is not merely a technical exercise: they crystallise what a goal or target means in practice and thus are far from neutral or apolitical. As one government official at the workshop noted, “where you stand will shape how you see indicators”.

Once the indicators are agreed, quantitative data needs to be gathered for each indicator. National Statistical Offices (NSOs) are intended to be the primary source of official SDG data, which will be collected by custodian UN agencies and then aggregated into a single metric. The MDG period saw a considerable increase in data availability on the development issues they focused on.[5] The SDGs hold the same promise; however, with a total of 230, there are far more global indicators included in the SDG framework, and precise methodologies for many of them are yet to be agreed. Furthermore, even if they wanted to – which should not be assumed – few countries will have the capacity or resources to gather data for every global indicator. At the same time, calls are being made for an investment in a ‘data revolution’ that harnesses the huge amount of data being produced by official and non-official actors across the globe.[6] In the development sector more broadly, numerous initiatives, platforms and partnerships are being formed to generate and gather new data that could potentially be linked to SDG indicators.

Recognising that a global indicator framework will inevitably be limited, and that monitoring at the national level will be of critical importance, many workshop participants agreed on the importance, from an accountability perspective, of developing national level indicators for Goal 16. In many contexts, monitoring SDG16 progress at the national level could include three categories of indicators: (i) official global indicators, as agreed by the UN Statistical Commission and National Statistics Systems; (ii) internationally comparable and supplementary indicators; and (iii) country-specific indicators developed either by government or by non-governmental sources.

     

It is expected that each country will identify SDG16 priorities based on its own national realities and the main gaps in the global indicator set relative to its particular priorities.  For example, the global indicators for target 16.3 will only measure limited aspects of criminal justice relating to victims of crime and numbers of detainees. It is likely that many countries will want to measure other aspects, such as levels of dispute resolution and access to justice, in line with the ambition and balance of the 2030 Agenda on justice and the rule of law more broadly. In this case they will need to identify more meaningful supplementary and complementary indicators at the national level.  Similarly, the indicator for target 16.9 only addresses birth registration, whereas there are many other aspects of legal identity that different countries will wish to consider and measure with a range of indicators.

The significance of data and indicators

But what is all this data for? How does it actually contribute to change? One answer to this question is that better data could radically improve decision-making through providing more granular information on where resources and attention can be most effectively focused. To take a simplified example linked to SDG4 on education, comprehensive national data on enrolment in public schools might reveal a specific region of a country where enrolment is particularly low. The solution for the national government might appear to be more schools in the region; however, corresponding data on low public transport in this region may suggest to decision makers that an investment in transport infrastructure is a more effective solution to increasing enrolment. This theory of change, however, is based on the assumption that the primary problem is an information gap. In fact, the problem may well be more to do with a lack of incentives and political capital. Elite policymakers in the capital may decide, for example, that investing in education in a marginalised part of the country is not a political priority.

Another answer to the question of how data can be used for change focuses more directly on incentives. Specifically, if indicators and data are used to monitor progress towards meeting the SDGs then those responsible for meeting them – primarily national governments – can be held to account for what they have signed up to, thus shifting their incentives. Indeed, the SDGs without a set of indicators would simply be a voluntary set of statements of intent. Their strength is that they lay out measurable goals and targets for the world to work towards. Take, for example, target 16.1 on ‘significantly reducing’ violence. If there is no data available it is an easy promise to break. But with data on the number of homicides, conflict deaths or other forms of violence – whether at global, regional or national levels – those responsible for signing up to this commitment can be said to have failed or succeeded by 2030 (albeit depending on interpretation of what is ‘significant’). In this regard, data has the potential to be used as an accountability currency that can shift incentives.

International accountability

Important questions remain about who is accountable to whom, and for what? As signatories and representatives of their countries, it is clear that governments hold the primary responsibility for collectively meeting all the SDGs and the 169 targets. It is less clear, however, whether they are accountable to one another or to their citizens.

The 2030 Agenda affirms that the goals are global in nature, i.e. they are meant for the world to meet collectively. One government official noted that his landlocked country could do little to advance SDG14 on the oceans, but he affirmed that meeting the SDGs “is like building a house: different states contribute different things. One helps with the roof, another with the foundations”. This approach suggests that even though the 2030 Agenda is universal, not every country should be expected to meet every goal and target. It also suggests that accountability is primarily inter-state. This requires global indicators and data that are comparable across countries, allowing for a single global metric to be aggregated and for differences between countries and regions to be identified. Inter-state accountability also requires a forum in which states engage with one another. The High Level Political Forum (HLPF),[7] held annually at ministerial level and at the level of Heads of State and Government every four years, provides a space where countries undertake voluntary reviews of progress in meeting the SDGs, drawing on both qualitative narratives as well as quantitative data.

However, the effectiveness of such an inter-state accountability process is questionable. To start with, while international reputation and legitimacy matters, it is not clear how much this impacts on the choices of national decision makers. When it comes to comparison, evidence shows that the influence on governments of global rankings and indices is lower than often assumed (although comparisons and review processes at the regional level may have greater impact).[8] Moreover, on the evidence of the first HLPF, much greater efforts will be required if it is to serve its purpose as a credible and comprehensive accountability forum. In the words of one official, “member states can get up there and pretty much say what they like”. One civil society activist worried about box-ticking:  “Governments are driven by two forms of legitimacy: international and local. The latter is more important than the international, because they can just cook the books for an international audience”. Workshop participants also worried that if uniform global indicators do influence decision makers, they may skew where attention is focused: away from national priorities to generic issues, deemed important by more powerful countries in the international community.

National accountability

What then, are the prospects for states being accountable to their own citizens for meeting the SDGs? The 2030 Agenda makes clear that countries will be responsible for aligning the SDGs with their own national development strategies. The universality of the agenda – whether it applies in equal measure to each country – is open to interpretation since the 2030 Agenda states that the “SDGs and targets are … global in nature and universally applicable, taking into account different national realities, capacities and levels of development and respecting national policies and priorities”.[9] In reality, few countries can be expected to meet all 17 goals and 169 targets, and countries will need to prioritise. The Georgian government, for example, has harmonised 94 targets from 16 of the goals into a national plan. Others countries have chosen to prioritise issues already addressed by their national development plans. The extent to which the process of prioritising SDGs is inclusive and consultative will shape how much they are owned beyond the government. The process of alignment could also provide an opportunity to reflect on whether existing national strategies are ambitious enough and focused on the right issues.    

The 2030 Agenda also commits member states to carrying out ”regular and inclusive” reviews at national level, noting that they are a means to ”support accountability to our citizens” and that parliaments have a role to play. While the form of reviews is to be decided at national level, this does potentially create a space for intra-state accountability. Review and accountability processes may already exist for national development strategies, for example through scrutiny by parliament. With regard to public engagement on the SDGs, workshop participants emphasised that awareness of the SDGs and the commitments that every government has signed up to is currently very low: “The public can’t hold a government to account for something they don’t know about”. On the other hand, it was stressed that many of the substantive issues addressed by the SDGs – including SDG16 – are well-known and part of the public discourse. For example, the public or civil society organisations may not be aware of target 16.5 to ‘substantially reduce corruption and bribery’, but they do know about the corruption their societies face. Indeed, it is critical that backers of the SDGs remain focused on how their targets are relevant to different contexts, and avoid pursuing process for the sake of process. This requires that the SDGs are translated from abstract global concepts into the language and discourse of what is currently being contested in a particular context. As one activist noted, “contextualisation is where the issues in the SDGs can become politicised”.

How data and indicators can support action at the national level

Where do indicators and data fit in? In general, outside of international frameworks, data has the potential to be a powerful tool in the activist’s toolkit. For example, in Nepal one civil society organisation set up a means of gathering data on people’s grievances about the 2015 earthquake recovery effort, which were fed into specific forums for engagement with officials. In Nigeria, activists drew on data related to gender equality in their advocacy with the government. Data allows accountability actors to challenge official narratives of progress – based on clear evidence – and to demand change.

There is some evidence that MDG data was used by accountability actors at the national level for these purposes; for example, pushing governments to meet MDG targets on education. However, according to one academic, research on the use of international framework indicators found that “within civil society there was not much appetite for using global indicators for accountability purposes”.[10] Indicators need to have local resonance, legitimacy and buy-in if they are to galvanize action at the national level. For example, in Romania civil society felt that EU accession indicators on the rule of law – focused on corruption and judicial reform – did not speak to the rule of law issues that they were concerned with. Civil society therefore set their own indicators and “more local ownership made a huge difference – people see the needs, then create indicators to address them, rather than looking outside”. An activist from Sudan agreed, arguing that “unless I contextualise an indicator with people on the ground, it won’t mean anything to them”; while an activist from Nigeria stated that indicators “don’t mean anything until they speak to personal welfare, improvements in our lives and people’s own experience of development”. In Nepal, public surveys on government services found that teacher absenteeism was considered the biggest challenge, which could then serve as an indicator generated from the bottom up. This perspective may be shared by government officials. For example, as one official at the workshop noted, “comparison at global level is important, but it is not the starting point or the priority – that is the people”.

Participants felt strongly that nationally-created and nationally-owned indicators thus had greater potential for driving accountability and changing the incentives of decision makers. This is not to say that global indicators have no role to play: they may speak very directly to the issues that matter for the public. Indeed, they can open up debates at national level and support those seeking to advance particular global norms in their own contexts. Participants felt that it need not be one or the other: national indicators can complement global ones and need not necessarily be weaker, indeed they can significantly strengthen them. Moreover, setting national indicators can help contextualise the SDGs as a whole by linking them to those issues actually being contested at the time in a particular country. As one activist noted, “when I engage with a minister, I might not talk about meeting one SDG or another, or one specific target, but I will talk about the indicators and data that are linked to them, and link these to the issues that matter in the country”. In this way, localised data produced to monitor SDGs may prove useful for accountability actors, even if broader awareness of the SDGs is low.

Setting national indicators and data sources

The process to set national indicators is not defined in the 2030 Agenda. Some countries, such as Georgia and El Salvador, have already taken the initiative to identify their own indicators. Participants stressed the importance of an inclusive process so that the indicators represent genuine national ownership, not just government ownership. The forums for agreement on national indicators can be SDG-specific, such as Georgia’s multi-stakeholder SDG Council, or they may be related to existing national development strategies that have been aligned with the SDGs. Alternatively, they could use space created by other initiatives, such as the OGP’s process for the co-creation of national action plans between civil society and government.

While the process of setting national indicators should not ignore global SDG indicators that are relevant for the country concerned, a genuinely bottom-up approach could start by gathering data on people’s concerns, and building up from there. A final element that may add extra ‘bite’ to the indicators is the establishment of quantified baselines and benchmarks for indicators. For example, this could mean setting a specific percentage decrease in the number of bribes paid by 2030. The risk with such an approach is that governments simply work towards meeting the benchmarks, or seek to manipulate the indicators, rather than addressing the broader issues they speak to.

Selecting indicators is one thing, producing and gathering the data required is quite another. To start with, participants made clear that in every context more action is needed to make existing data open and available (itself an SDG16 target – 16.10). For example, homicide data may exist but not be shared between government departments, let alone made available to the public. However, the reality is that – especially on SDG16 issues – significant investments still need to be made to increase data-gathering capacity. This includes efforts to generate data disaggregated by identity markers such as sex, location, age and ethnicity. Official statistical systems certainly need to play a role in this and should have their capacity strengthened to do so. However, it is imperative that NSOs are independent and considered credible. Furthermore, non-official actors – such as research organisations or civil society groups – may also merit support as data producers. Indeed, another approach may be to foster the development of diverse data producers, which leads to healthy debate and contestation rather than reliance on a single metric from a single official source.[11] 

Setting national indicators and identifying data sources is only part of the accountability story. So-called ‘info-mediaries’, such as the media, social media users or civil society groups, may also need support so that they are able to translate data into meaningful narratives for a broader set of actors, including the public. This entails engagement with a broad set of accountability actors – including parliamentarians, activists, the media, and political parties – so that they buy in to the potential of the data as an accountability tool, whether linked to the SDGs or to related concerns. Basic data literacy among these actors, and within the broader public, will likely need to be strengthened. Finally, consideration should be given to deploying data at critical junctures in the political process where it may have more impact, for example leading up to elections.

In an effort to increase accountability, there are plans to complement the 2030 Agenda indicator framework with supplementary indicators produced according to a country’s context and priorities. This is to be accompanied by a series of consultation and validation processes aimed at developing country-specific periodic score-cards (preferably annually) with specific policy options and recommendations.

There are a number of other important issues to consider. First is the essential role of civil society and a free and open media. Workshop participants stressed that civic space is a necessary precondition for any accountability process; but also cautioned that, even when it is afforded space to operate, civil society is not always active or necessarily representative. Second, is to consider how the SDGs link up to other international frameworks, such as the OGP and the New Deal. Otherwise there is a risk of duplication or siloing into parallel accountability processes on cross-cutting issues that need to be addressed with a holistic approach. Finally, more needs to be done to ensure that the 2030 Agenda’s 15-year time horizon can be institutionalised so that commitments made are not lost every time a government changes. This could, for example, be achieved through enacting legislation or by creating independent bodies that provide institutional continuity.    

Priorities going forward

In order to maximise the potential impact of the SDGs, the evidence base on how indicators and data contribute to substantive change needs to be strengthened, especially when it comes to issues of peace, justice, governance and human rights. With regard to shaping incentives, we still have much more to learn about how accountability actors use data in general, above and beyond that which is linked to international framework indicators. We also need to explore how quantitative data can most effectively complement qualitative narratives and other stories of progress. It is critical to understand these issues – which should underpin any theory of change – before resources are invested in developing new data sets or portals.  

There are also questions about the relative advantages of global and national monitoring. Resources are limited. Is it more important for the international community to generate comparable data for the world to review progress – and each country’s contribution to it – or country-specific data for national stakeholders to mobilise around? We may be able to strike a balance, but it will not always be possible to have our cake and eat it. On the one hand, the centralising tendencies of multilateral processes – and the power dynamics they embody – need to be acknowledged and challenged. On the other hand, those who campaigned long and hard for the inclusion of peace, justice and inclusive societies in the SDG framework should be under no illusion that all governments – and especially those in countries where the goal matters most – will welcome accountability processes at the national level that are seen to challenge the political status quo.      

The development, peace, justice, governance and human rights communities are still working out how to optimise the impact of the SDGs. The way that indicators and data strengthen accountability at the national level is an important part of this evolving story. There will be no single answer or template approach: we will not be able to drive accountability in the same way on the same issues in every context. Exploration, testing and learning will be critical in every context, and will need to focus on what particular combination of pressures and incentives can motivate change in each case.  

About the SDG16 Data Initiative: Launched in July 2016 by a consortium of 14 partner organisations, the SDG16 Data Initiative seeks to support the open and holistic tracking of the commitments made by 194 countries captured in Sustainable Development Goal 16 (SDG16), to promote peaceful, just and inclusive societies. The indicators and data used for monitoring progress towards SDG16 are critical for accountability and policy-making purposes, and thus play a central role in helping to deliver action and change towards meeting SDG16. See http://www.SDG16.org 

ANNEX:

Global indicators for SDG16: Promote peaceful and inclusive societies for sustainable development, provide access to justice for all and build effective, accountable and inclusive institutions at all levels

10 TARGETS

23 INDICATORS

TIER

16.1 Significantly reduce all forms of violence and related death rates everywhere

16.1.1  Number of victims of intentional homicide per 100,000 population, by sex and age

1

16.1.2  Conflict-related deaths per 100,000 population, by sex, age and cause

3

16.1.3  Proportion of population subjected to physical, psychological or sexual violence in the previous 12 months

2

16.1.4  Proportion of population that feel safe walking alone around the area they live

2

16.2 End abuse, exploitation, trafficking and all forms of violence against and torture of children

16.2.1  Proportion of children aged 1-17 years who experienced any physical punishment and/or psychological aggression by caregivers in the past month

3

16.2.2  Number of victims of human trafficking per 100,000 population, by sex, age and form of exploitation

2

16.2.3 Proportion of young women and men aged 18-29 years who experienced sexual violence by age 18

2

16.3 Promote the rule of law at the national and international levels and ensure equal access to justice for all

16.3.1  Proportion of victims of violence in the previous 12 months who reported their victimization to competent authorities or other officially recognized conflict resolution mechanisms

2

16.3.2  Unsentenced detainees as a proportion of overall prison population

1

16.4 By 2030, significantly reduce illicit financial and arms flows, strengthen the recovery and return of stolen assets and combat all forms of organized crime

16.4.1 Total value of inward and outward illicit financial flows (in current United States dollars)

3

16.4.2 Proportion of seized small arms and light weapons that are recorded and traced, in accordance with international standards and legal instruments

2

16.5 Substantially reduce corruption and bribery in all their forms

16.5.1 Proportion of persons who had at least one contact with a public official and who paid a bribe to a public official, or were asked for a bribe by those public officials, during the previous 12 months

2

16.5.2 Proportion of businesses that had at least one contact with a public official and that paid a bribe to a public official, or were asked for a bribe by those public officials during the previous 12 months

2

16.6 Develop effective, accountable and transparent institutions at all levels

16.6.1 Primary government expenditures as a proportion of original approved budget, by sector (or by budget codes or similar)

1

16.6.2 Proportion of the population satisfied with their last experience of public services

3

16.7 Ensure responsive, inclusive, participatory and representative decision-making at all levels

16.7.1 Proportions of positions (by sex, age, persons with disabilities and population groups) in public institutions (national and local legislatures, public service, and judiciary) compared to national distributions

3

16.7.2 Proportion of population who believe decision-making is inclusive and responsive, by sex, age, disability and population group

3

16.8 Broaden and strengthen the participation of developing countries in the institutions of global governance

16.8.1 Proportion of members and voting rights of developing countries in international organizations

1

16.9 By 2030, provide legal identity for all, including birth registration

16.9.1 Proportion of children under 5 years of age whose births have been registered with a civil authority, by age

1

16.10 Ensure public access to information and protect fundamental freedoms, in accordance with national legislation and international agreements

16.10.1 Number of verified cases of killing, kidnapping, enforced disappearance, arbitrary detention and torture of journalists, associated media personnel, trade unionists and human rights advocates in the previous 12 months

3

16.10.2 Number of countries that adopt and implement constitutional, statutory and/or policy guarantees for public access to information

2

16.a Strengthen relevant national institutions, including through international cooperation, for building capacity at all levels, in particular in developing countries, to prevent violence and combat terrorism and crime

16.a.1 Existence of independent national human rights institutions in compliance with the Paris Principles

1

16.b Promote and enforce non-discriminatory laws and policies for sustainable development

16.b.1 Proportion of population reporting having personally felt discriminated against or harassed in the previous 12 months on the basis of a ground of discrimination prohibited under international human rights law

3

 

Note: Tier classification

 

 

Number of Indicators

Tier 1: Indicator conceptually clear, established methodology and standards available and data regularly produced by countries

6

Tier 2: Indicator conceptually clear, established methodology and standards available but data are not regularly produced by countries

9

Tier 3: Indicator for which there are no established methodology and standards or methodology/standards are being developed/tested.

8

 

 

 

[1] Drafted by Thomas Wheeler, Saferworld; John Romano, TAP Network; and Jairo Acuna-Alfaro and Alexandra Wilde, UNDP. The views expressed in this note do not necessarily reflect the official position of UNDP or the United Nations.

[2] Sarwar, M (2015) National MDG Implementation: Lessons for the SDG era Overseas Development Institute:  London

[3] See for example Miller-Dawkins, M (2015) Global Goals and International Agreements: Lessons for the SDGs ODI: London, ODI (2015), Binat Sarwa, M (2015) National MDG Implementation: Lessons for the SDG era, ODI: London, Vernon P & Baksh, D (2010) Working with the Grain to Change the Gain International Alert: London.

[4] Global indicators are to be used to track global progress towards the SDGs, aggregating data provided to the UN system from each member state (or from other sources in the case of transnational issues). Though countries can draw on global indicators, national indicators are to be identified at the country level and used to track issues of particular relevance to each context. Regional bodies will also potentially create further indicators to track regional issues.

[5] Cassidy, M (2014) Assessing Gaps in Indicator Availability and Coverage Sustainable Development Solutions Network: Washington DC

[6] See United Nations Secretary-General Independent Expert Advisory Group (2014) A World that Counts: Mobilising the data revolution for sustainable development United Nations: New York

[7] See https://sustainabledevelopment.un.org/hlpf

[8] Custer, S et al (2016) Governance Data: Who Uses It and Why? Williamsburg, VA: AidData and the Governance Data Alliance

[9] The 2030 Agenda for Sustainable Development – https://sustainabledevelopment.un.org/post2015/transformingourworld 

[10] Serban, M. (2015) ‘Rule of Law Indicators as a Technology of Power in Romania’ in Merry, S, Davis, K & Benedict Kingsbury, B. eds (2015) The Quiet Power of Indicators: Measuring Governance, Corruption, and Rule of Law, Cambridge University Press: Cambridge

[11] The question of whether this data is of a sufficient standard is important. Non-official data producers will need to be able to demonstrate that they meet high enough standards. There may be a role for NSOs to play in assessing this. See Saferworld (2015) Who should measure the Sustainable Development Goals?  – http://www.saferworld.org.uk/news-and-views/comment/174-who-should-measure-the-sustainable-development-goals 

Filed Under: Research
Open Government Partnership