Measuring Scotland’s Progress (SCO0002)

Issued to be addressed: The development of a robust framework which enables Scotland’s progress towards the SDGs to be measured. Background: The 17 Sustainable Development Goals were formally agreed by the UN in September 2015. The Goals are an inter-governmentally agreed set of targets relating to international development. They are global, high level priorities. In July 2015, the First Minister announced Scotland’s intention to sign up for the goals as well as our plans for measuring progress through the National Performance Framework (NPF). Introduced in 2007 and refreshed in 2011 and 2016, the National Performance Framework (NPF) sets out a clear, unified vision for the kind of Scotland we want to see and how our actions will improve the quality of life for the people of Scotland. The 66 measures in the NPF provide a broad measure of national and societal wellbeing, incorporating a range of economic, social and environmental measures. Scotland’s commitments to Human Rights are set out in The Scottish National Action Plan (SNAP) of the Scottish Human Rights Commission and in other places such as the Children and Young People (Scotland) Act 2014 which will deliver the UNCRC this commitment seeks to draw these commitments together so they can be used in conjunction with the NPF to measure Scotland’s progress. Primary objective: To develop a robust framework which enables Scotland’s progress towards the SDGs to be measured in an effective and transparent way; ensuring that the commitments made under national and international treaties covering human rights are aligned with NPF. Short description: The development of a robust framework which enables Scotland’s progress towards the SDGs to be measured. OGP challenge: Given the nature and content of the SDGs, the NPF and SNAP, robust measurement of progress address all five of the OPG grand challenges. The NPF is a key tool by which the SG is held to both public and parliamentary scrutiny and accountability. Using these frameworks will ensure that the measure of Scotland’s progress towards the SDGs is open and robust.

Commitment 2. Measuring Scotland´s Progress

Commitment Text

The development of a robust framework which enables Scotland's progress towards the Sustainable Development Goals (SDGs) to be measured.

Milestones

1. A programme of public, civil society and stakeholder engagement on the development of a measurement framework

2. Measurement framework in place (Autumn 2017) ^{For the purposes of this review, this milestone has been added by the IRM researcher. This is because, although it does not appear in the action plan as a discreet milestone, the wording under the 'end date' for milestone 1 states 'framework is in place Autumn 2017' which would suggest a second milestone.}

Commitment Overview

Commitment Aim

Overall Objective & Relevance

The Scottish Government has recently committed to signing up to the United Nations Sustainable Development Goals (SDGs), an inter-governmentally agreed set of targets relating to international development. The targets set out under the SDG framework overlap, to some extent, with existing commitments which the Scottish Government has made under various domestic plans, including the National Performance Framework (NPF) and Scotland´s National Action Plan for Human Rights (SNAP).

The Scottish Government (then the Scottish Executive) introduced the National Performance Framework (NPF) in 2007 (revised in 2011 and 2016) as an agreed set of national outcomes to improve the quality of life for the people of Scotland. The 66 measures in the NPF provide a broad measure of national and societal wellbeing, incorporating a range of economic, social and environmental measures. The Scottish Government tracks and reports on progress on the NPF through Scotland Performs, as and when the data are available ^{Scottish Government: Scotland Performs (last accessed 14 July 2017) http://www.gov.scot/About/Performance/scotPerforms} . Scotland´s National Action Plan for Human Rights (SNAP), meanwhile, along with legislation such as Children and Young People (Scotland) Act 2014, set out the government´s commitments to upholding human rights.

This commitment aims to align these different frameworks, enabling Scotland's progress towards the SDGs and human rights obligations to be measured in an effective and transparent way in conjunction with the NPF. According to the action plan, using these frameworks helps ensure that the measure of Scotland´s progress towards the SDGs is open and robust.

More specifically, the commitment aims to engage the public, civil society and other stakeholders in the development of the framework. This is an important step forward. For example, in response to the Scottish Parliament's Finance Committee consultation on the NPF in 2013, the Royal Society of Edinburgh (RSE), Scotland's National Academy, noted that:

'there has been limited opportunity for those outwith [sic] Government and the public sector to influence the development of the NPF and Scotland Performs. This not only applies to external technical input and analysis, but also to engagement with the public. (€¦) It is important that a process of deliberative dialogue is initiated to ensure that civic society is engaged in the development of the NPF' ^{The Royal Society of Edinburgh (2013) The Scottish Government´s National Performance Framework: A response to the Scottish Parliament's Finance Committee (last accessed 14 July 2017) https://www.rse.org.uk/wp-content/uploads/2016/09/AP13_08.pdf} .

Furthermore, a number of civil society organisations welcome the inclusion of the Sustainable Development Goals into Scotland's OGP framework through this commitment. According to the SCVO, the OGP process in Scotland has coincided with increasing interest in work around the SDGs among civil society. The link between the SDGs and Scotland's NPF is clear to many CSOs in Scotland (e.g. Oxfam Scotland, HIV Scotland), and has served as a mechanism to also engage other 'unusual suspects' such as the Church of Scotland. The SDGs have therefore acted as an important linchpin to connect OGP to the more immediate priorities of those working on social justice and social services ^{Interview with Lucy McTernan and Ruchir Shah, SCVO, 11 July 2017} .

This commitment is relevant to the OGP value of civic participation insofar as it opens up decision-making on the development of the proposed measurement framework on the SDGs to interested members of the public. It is also relevant to the value of Access to Information as it aims to inform the public about progress towards meeting the SDGs. According to the action plan, the NPF is also a key tool by which the Scottish Government is held to both public and parliamentary scrutiny and accountability. However, according to OGP guidelines, this commitment is not considered directly relevant to the value of public accountability, as it does not include a mechanism whereby citizens can actively seek answers or justification from government regarding their performance under the framework.

Specificity and Potential Impact

The level of specificity for this commitment is low. The commitment is narrowly focused and defines two outputs (a programme of engagement and a framework), as well as both government and non-governmental actors responsible for implementation. However, the commitment and milestone text is vague and there is insufficient detail to enable the milestones to be verifiably measured. For example, it is unclear what form the programme of engagement will take, who will be invited to participate, or how stakeholder views will be incorporated. Thus, while it may be possible to ascertain whether some form of engagement took place, there is little to indicate what successful engagement might look like. Likewise, it is unclear what is meant by the framework being 'in place', for example, whether this means that the content has been finalised, the mechanisms to enact the framework have been developed, or whether the framework is actively being used to measure progress by the intended completion date. Finally, the wording of the commitment leaves some doubt as to whether the government envisages an entirely new measurement framework for the SDGs or a joint framework for measuring all existing commitments including those currently under the NPF and the SNAP as well as the SDGs.

The potential impact of this commitment is minor. While the Scottish Government's commitment to implementing the SDGs, as well as the NPF, is expected to contribute to improving the economic, social and environmental wellbeing of the people of Scotland, the potential impact of having a robust framework in place is a small, albeit important step, in that process. The alignment of Scotland's various commitments could potentially lead to greater efficiency in terms of monitoring and reporting on Scotland's progress, while greater transparency would allow citizens to better understand how government is performing on various measures of social progress. Achieving more meaningful impact is only likely if the resulting measurement framework ultimately reflects the priorities of civil society and other non-governmental stakeholders, and the government adapts its policy priorities to meet the redefined targets. However, the text does not state this outcome as an explicit aim of the commitment.

Completion: Limited

There has been limited progress on the implementation of this commitment. While the government commissioned a programme of engagement on the development of the framework during the year as planned (milestone 1), the publication of the framework itself (milestone 2) has been delayed until 2018. This is largely due to the decision to conduct an additional round of engagement on the NPF indicators (rather than just the outcomes, as had been originally planned) before the revised framework is submitted to parliament for review in the spring of 2018 ^{Interview with Roger Halliday, Scottish Government, 2 November 2017, Edinburgh} . While consultation with parliament on the outcomes is a legal requirement of the Community Empowerment Act, consultation on the indicators is not. Instead the decision to consult more widely on the indicators was made by the NPF Roundtable (see below) in order to ensure that the full framework is in place before being submitted to parliament and to promote greater buy-in to the revised NPF ^{Interview with Roger Halliday, Scottish Government, 2 November 2017, Edinburgh} .

Work on the development of the measurement framework to date has involved two strands. On the one hand, the External Affairs Directorate and Children and Families Directorate, the Scottish Human Rights Commission and others undertook an internal exercise to map how the 220 Sustainable Development Goals (SDG) indicators and Scotland´s human rights targets align with the current set of measures in the NPF ^{Interview with Roger Halliday, 2 November 2017, Edinburgh} , including working with UK Office of National Statistics to look at progress towards the SDGs ^{Anne-Marie Conlong, OGP Steering Group meeting, 23 June 2017 https://forum.opengovernment.org.uk/conversations/968} . The results of the exercise were not published. At the same time the government commissioned the Carnegie UK Trust and the Children´s Parliament to run a series of public events to feed into the update of the NPF outcomes. The Carnegie UK Trust sub-contracted Oxfam Scotland to conduct 10 street stalls in both deprived and affluent areas covering each of the eight electoral regions, which engaged more than 300 participants ^{Interview with Jamie Livingston and Francis Stuart, Oxfam Scotland, 2 November, Edinburgh} , while the Carnegie UK Trust also facilitated 20 discussion groups with a total of 196 participants between December 2016 and February 2017 ^{Carnegie UK Trust (forthcoming) What Sort of Scotland Do You Want to Live In? Report on Discussion Groups and Street Stalls to inform the review of the National Performance Framework (unpublished)} . Both Oxfam and Carnegie UK Trust incorporated elements of the SDGs into their consultations to support the alignment work ^{Interview with Jamie Livingston and Francis Stuart, Oxfam Scotland, 2 November, Edinburgh} . The Children´s Parliament, meanwhile, engaged with 102 children aged between 7 and 12. In addition, the government also undertook a series of discussions with business groups to discuss the NPF outcomes ^{Interview with Roger Halliday, Scottish Government, 2 November 2017, Edinburgh} .

Based on these exercises, the government developed, but did not publish, a statement of 11 draft national outcomes, some of which are similar to existing outcomes in the NPF, some of which are new with greater emphasis on issues such as fair work, inclusive growth, celebrating culture, and human rights ^{Interview with Roger Halliday, Scottish Government, 2 November 2017, Edinburgh} . As noted above, the government is now planning a further series of workshops to discuss and develop indicators for each of the outcomes ^{OGP Scotland October 2017 update http://www.gov.scot/Topics/Government/OGP} . According to Roger Halliday, the Scottish Government´s Chief Statistician, the plan is to form groups of interest in order to build cross-sectoral support for the delivery of the outcomes, as well as cross-party and cross-ministerial support ^{Interview with Roger Halliday, Scottish Government, 2 November 2017, Edinburgh} . However, the government has not yet shared the draft outcomes or any concrete plans on this additional consultation phase with civil society, other than providing an indication that some form of additional engagement is planned ^{Interview with Jamie Livingston and Francis Stuart, Oxfam Scotland, 2 November, Edinburgh; Interview with Jennifer Wallace, Lauren Pennycook and Rebekah Menzies, Carnegie UK Trust, 6 November 2017, via telephone} . Nor has the government yet published the findings of the Carnegie UK Trust, Oxfam and Children´s Parliament consultations, although they plan to do so once the revised framework is ready for consultation with Parliament ^{Interview with Roger Halliday, Scottish Government, 2 November 2017, Edinburgh} .

Early results: did it open government?
Access to Information: No change
Civic Participation: Marginal

This commitment aims to enable Scotland's progress towards the SDGs and human rights obligations to be measured in an effective and transparent way through a robust framework (which was, in practice, conceived as a revised version of the NPF, framework that incorporates elements of the SDGs and HR commitments), leading to greater efficiency in terms of monitoring and reporting. In addition, the commitment aims to engage the public, civil society and other stakeholders in the development of the framework, building an improved understanding among citizens of Scotland´s progress. This is an important development given the fact that the government´s work on developing and revising the NPF to date has largely been conducted behind closed doors. Given the delays in finalising the revised NPF, it is not possible to tell whether these effects have been achieved although there are some early signals of progress.

Through the activities undertaken as part of this commitment, the government did create opportunities for the public to inform the revision of the NPF, albeit through a relatively small engagement exercise. The extent to which the process also led to improved quality of information is limited however, especially given the government´s failure to properly communicate the process beyond the few organisations directly involved.

As acknowledged by all stakeholders, both within and external to government, the way the NPF was initially conceived was not participatory and very much reflected government structure at the time. To this extent, Jamie Livingston from Oxfam Scotland, believes that the form of consultation chosen by government (via civil society organisations experienced in citizen engagement as opposed to via more standard public consultation documents) was positive, albeit limited due to a small budget. In his view, commissioning Oxfam and Carnegie to undertake the consultations shows some openness and a willingness beyond the Fairer Scotland process to outsource engagement and hand over a limited degree of control to CSOs ^{Interview with Jamie Livingston and Francis Stuart, Oxfam Scotland, 2 November, Edinburgh} . Likewise, Carnegie recognised the government´s genuine desire to rectify the lack of engagement in the NPF ^{Interview with Jennifer Wallace, Lauren Pennycook and Rebekah Menzies, Carnegie UK Trust, 6 November 2017, via telephone} .

However, Jamie Livingston also found the process for consultation on the NPF outcomes and indicators to have been haphazard with the government consulting on various elements at different points. He also noted that the timelines and process for revising the NPF remain opaque, which speaks to a power imbalance between government and civil society, whereby the terms of the engagement are set by the government ^{Interview with Jamie Livingston and Francis Stuart, Oxfam Scotland, 2 November, Edinburgh} . The lack of information on how the NPF revision process is progressing was also noted by participants at the OGP Network meeting on 3^rd August 2017, as was the need for clearer evidence that the Sustainable Development Goals (SDGs) are not only a bolt-on to the National Performance Framework (NPF). According to participants, the process could be improved through providing more funding to consult properly, better communications in the consultation process and connecting with international peers to learn from experience ^{OGP Network meeting, 3rd August 2017.} .

Moreover, as noted earlier, for this commitment to achieve more meaningful impact would require the resulting measurement framework to ultimately reflect the priorities of civil society and other non-governmental stakeholders, and for government to adapt its policy priorities to meet the redefined targets. With regards to the former, Roger Halliday noted the gradual inclusion of outcomes reflecting themes emerging from engagement with civil society, including areas such as culture and human rights, fair work and inclusive economic growth ^{Interview with Roger Halliday, Scottish Government, 2 November 2017, Edinburgh} . With regard to the latter, as noted by both Oxfam Scotland and Carnegie UK Trust, the NPF is not currently sufficiently used to inform the budget process. While parliamentary committees and Audit Scotland are provided with updates on the NFP, they are rarely the focus of debate ^{Interview with Jennifer Wallace, Lauren Pennycook and Rebekah Menzies, Carnegie UK Trust, 6 November 2017, via telephone; Interview with Jamie Livingston and Francis Stuart, Oxfam Scotland, 2 November, Edinburgh} . In a similar vein, Alison Hosie from the Scottish Human Rights Commission noted the need to ensure that the outcomes under the NPF (and the SNAP) are accompanied by concrete actions. In her words, 'we have come a long way with how we´re going to measure progress, but I´m seeing a deficit at the moment in how we´re going to deliver that progress.' ^{Interview with Alison Hosie, Scottish Human Rights Commission, 7 November 2017, via telephone}

Recommendations

· A number of stakeholders voiced concerns about the lack of communication around the purpose, timelines and process for revising the NPF. Any future action plan should seek to more clearly explain the planned structure of the measurement framework, and set out a clear and well-resourced consultation process outlining who will be involved in its further development and through which mechanisms and activities (beyond the NPF Roundtable). This would enhance accountability of the process and enable more accurate monitoring of the commitment´s implementation. The government should also publish its own mapping exercise of the alignment between the SDGS indicators and the measures set out in the NPF, its statement of 11 draft national outcomes, and the findings of the Carnegie UK Trust, Oxfam and Children´s Parliament consultations on the NPF as soon as possible in order to dispel mistrust and ensure greater buy-in to the process from civil society stakeholders.

· Strengthen the role of civil society and other non-governmental organizations in the development of the framework to reflect a broad and inclusive set of priorities. Civil society actors could consider, for example, developing a simple shadow reporting mechanism to complement the government´s own monitoring framework.