Breaking static impact practice
Organisation: Children 1st

The organisation

Children 1st is Scotland’s National Children’s charity. They help Scotland’s families to put children first, with practical advice and with support in difficult times. And when the worst happens, they support survivors of abuse, neglect, and other traumatic events in childhood, to recover.

Children 1st helps Scotland’s communities to play their part in looking-out for and protecting children. They listen to Scotland’s children and families. And when government or society fails to hear them or respect their rights, they speak out.

Over the last few years, Children 1st has been engaged in a major change programme to develop the way work within front-line services is recorded and reported on. A key driver for this work has been the need to evidence the impact of these services on the lives of children, young people and families.

In 2011 the charity received funding from Social Investment Scotland to develop a new integrated information system. This allowed the charity to move away from paper recording and also provided the mechanism to adopt a new outcomes measurement framework based on the Scottish Government’s Getting it right for every child (GIRFEC) SHANARRI wellbeing indicators.

Since the launch of the new system, the focus has been on embedding the information system and monitoring and evaluation practice, achieving consistency in recording and developing a set of useful reports for operational and business reporting purposes. Children 1st is already realising the benefits from this work including improved visibility of activity across the charity and the ability to turn insight into improvement. Being able to show a standard set of reports to potential funders so that they know what level of reporting to expect for any new service has also benefitted Children 1st during tendering processes, with one funder commenting that they would like to see a similar approach from other charities.

The invitation to join the Embedding Impact Practice Group facilitated by Evaluation Support Scotland (ESS) came at a good time for Children 1st as they started to look more closely at how to move things on to the next level. The charity was looking for answers to some important questions, such as:

• ‘How do we make sure our recording practices support the delivery of high quality services to children and families?’
• ‘We’ve been collecting a lot of data but do we need it all?’
• ‘What can we learn from the information we are collecting?’
• ‘What could be done better?’

Eileen Scouller, Information Systems Manager for Children 1st, joined the Embedding Impact Practice Group in November 2015.

Resources and products used

Inspiring Impact’s Measuring Up! (MU!) tool helped senior staff to identify where concentrated effort was needed and also gave an idea of where Children 1st could most benefit from improvements in monitoring and evaluation practices. A very positive insight was how invested the board and senior management are in the importance of monitoring, evaluation and impact measurement across Children 1st.

By using this tool it became clear that although a great deal of consultation and consideration of activities and outcomes for children and families had taken place when the information system was initially developed, since then, the focus had been on making this system work on an operational level and that some momentum had been lost in tying data collection in with the wider strategic agenda.

MU! also showed that the charity is strong in some areas in front line services, but weaker on a national level. For example, in services staff are skilled at interpreting other factors which might be influencing outcomes for a child or family, but this information is often lost when looking at reports for the whole organisation.

The logic model that Eileen created in consultation with her colleagues has also been a useful tool for Children 1st. Feedback on the draft logic model from ESS showed that it is easy to use standard phrases without thinking about what they really mean. If an outcome is to put children first, how would this be evidenced? In parallel with the work that Eileen was doing in the group, the Children 1st management team created a draft logic model which was then discussed and revised at roadshows with staff and volunteers across Scotland. Going forward, it is anticipated that this will help clearly tie monitoring and evaluation practices to Children 1st activities and overall outcomes.

Challenges and results

The group was made up of a mix of people from direct service and funding organisations. Eileen feels that this made for lively discussion and gave her a better understanding of the challenges faced by funders in interpreting the information they receive from beneficiaries. A few key points of interest for Eileen were:

• An appreciation of the challenges faced by different organisations. Smaller organisations can lack resources to dedicate to monitoring and evaluation work and larger organisations with complex service provision can take time to respond to changes in direction.
• Think more about what supporting evidence is collected for outcomes - ideally supporting evidence should come from two sources.
• Be creative! - Tools for collecting feedback and evidence don’t always have to be online surveys or complicated paper forms. Think about how to make these as engaging as possible and tailor the tool to suit the audience and situation. Make sure that the voice of the respondent doesn’t get lost in a box ticking exercise.
• The importance of being open with funders about any challenges around monitoring and evaluation. For example, funder expectations vs the reality of tracking the progress of children and families after work has finished to measure long term impact.
• When something hasn’t gone so well, be open about it with a funder as early as possible and learn from it.

Actions and impact

A review of the Children 1st outcomes framework is now underway to establish whether the outcomes list can be reduced in order to streamline data entry and make the information collected more consistent and meaningful. The ultimate aim for this piece of work will be to improve the charity’s ability to evidence impact.

Discussions are ongoing as to how Children 1st can better interrogate the data that is being collected.

‘Management information reports’ are already used in monthly service reviews, quarterly board reports, annual impact and funding reports but the challenge for the charity will be how to move this on to the next level by looking at this information in more depth and bringing together the staff who hold different pieces of the puzzle to help interpret the data. For example, if there is a change in the number of families affected by welfare related issues, is this related to an increase or decrease in Children 1st services being offered in this area or is it linked to external factors such as changes to the welfare system or a shift in awareness of how to access welfare benefits?

In line with the recognition that there was a risk of becoming too ‘systems focussed’, a group which had been meeting to address operational issues for the information system has been changed to have a more strategic remit. It is anticipated that the MU! tool will be very useful to help keep an eye on progress and to help identify further areas for improvement.

Please indicate the extent to which you agree or disagree with this statement: I would recommend others to use Inspiring Impact resources/products?

Strongly agree