We need to talk about outcomes

The title of this post will most likely send shivers down the spines of many youth practitioners, and I can understand why. Youth work has it’s foundation in volunteering, providing safe spaces, creating opportunities, building relationships and providing a compassionate ear for young people, who choose to participate.

Ever since 18th Century chapels and churches opened their doors for Sunday schools, the informality of what the sector has been doing is what many see as it’s strength, . The rigours of targets and key performance indicators doesn’t seem to sit comfortably along side an informal, adaptable and needs-led approach. The very language…..accountability, tracking, impact assessment…….seems incongruous and intimidating to the sector.

But, like it or not, this is a very real hurdle that youth provisions must successfully surmount if they are to survive. Times have changed and are still continuing to change more, and for youth services to flourish there really isn’t much choice but to adapt or die. This is inevitable, and clinging on to the past luxury of being funded locally without the need to evidence value is a sure route towards even deeper cuts.  It’s simply not good enough to know that we have an impact and provide essential services, we need to show it as well.

Imagine for a moment you have a member of staff who you sense is competent, reliable and good at their job, but you can’t quite tell how much they are actually doing and whether results are really a consequence of their work or not.  So what do you do with this staff member when their contract is up for renewal and you have budgets to balance?  Trust your instincts and re-employ them, or expect them to prove their value before making a decision on whether to invest in them again?

The same is true for the youth service which is currently being decimated all over the country. We need to prove the value of what we do to show that we deserve public money, which will also have the added consequence of opening up possibilities of other funding streams from private and philanthropic sources.

And is this really such a terrible thing? All we’re really talking about here is being clear and specific about what it is we aim to achieve, work out a way to honestly assess whether we’re doing a good job or not, and then show that we have achieved the things that we set out to do.  Forget all the intimidating jargon and scare mongering, it really isn’t rocket science, it just requires some forethought, planning, open reflection and a robust way to record and evidence what we do and the impact we have.

People fear that this will mean provisions adapt to such a degree as to be unrecognisable, marketised, barely even youth work anymore, but this is a cul-de-sac of an argument. The definition of what encompasses youth work and how it is resourced is broad and varied, and anyone that tries to contain it in a box labelled ‘how I do youth work’ needs to go out and visit the hundreds of creative and dynamic projects that are happening throughout the country, impacting positively on young people and their communities.

As a youth worker for more than 20 years my ultimate aim has always been to have a positive impact on young people, improve their life chances and enhance communities, and there are so many amazing and creative ways of doing this, with money out there for those open-minded and creative enough to develop ways of achieving these things.

So, if we can accept that outcomes and measurements are not the enemy and are here to stay, with those that fail to adopt them not being around for much longer, then how do we go about it?

Well, one thing is clear, it will not only take some work, but also a willingness to be open-minded and honest in our evaluation of what we do and how we do it.

I want to reflect a bit on the process that ThinkForward has progressed through recently, under the guidance of David E.K. Hunter,  not in vague terms, but a specific process that all youth focused organisations can go through. So here are 5 steps to developing a good foundation to move forward in being more evaluative and evidence-based.

1. Mission statement

Start with the mission statement. Does it clearly state what it is you aim to achieve? Is it measurable and does it hold you to account? When you look back at your work with a young person will you unequivocally be able to say ‘yes’ or ‘no’ to having achieved it? If not then re-work it.  State clearly who you will be working with, followed with one line to define the context of your work, and then specific outcomes as a result of your engagement with young people.

Here is ThinkForward’s new re-worked (draft) mission statement:

ThinkForward ensures that young people who are disengaged from or under-performing at school receive the individualised support they need so that they transition successfully into sustained employment or higher education.

Note that the above doesn’t describe how we do it, just what we do, and who we do it with, clearly including a specific measurement of success.

2. Target Population

Next, be clear who you are working with, be as specific as possible, and equally be clear who you won’t be working with, i.e what would exclude a young person from engaging with you?

Every youth work project has limited capacity, and so every time you work with someone outside of your target population you rob someone inside of your target population of the opportunity to engage in your work and support.

In other words, who are you really in ‘business’ to help?

3. Outcomes

OK, so here is where we really get down to it. What do you aim to achieve?

Start with the long-term outcomes, be they independent living, sustainable employment, community engagement, youth leadership, achieving qualifications, reduced STIs, increase in youth volunteering, or whatever. Ultimately, this will how you know if you’ve succeeded or not, this should be inline with your mission statement.

Your intermediate outcomes focus on the achievements of the young people towards the long-term outcomes, and when they will be achieved in their timeline of engagement with you?

Again, specificity is needed here, while also appreciating that all young people progress at a different pace, and will need varying support in order to achieve these benchmarks of progress.

At ThinkForward, we collect streams of data and evidence to show changes in school attendance, behaviour incidents and attainment, and them in partnership with the schools evidence any improvements the young people have made as a consequence of their engagement with us.

Finally, your short-term outcomes. How will you know that the young people are moving in the right direction, and you’re doing the right things to support them? To be certain of this you need to establish regular ways of assessing their progress, as often as possible.

These may include, for example, levels of engagement and an assessment of their soft skills development.

At ThinkForward we have a set of 14 mind-sets and employability skills (work readiness) that we are always working with our young people on, and ask them to assess themselves every six months against these. This is not nearly enough, and we are currently designing a method to implement this assessment much more regularly, to be carried out in conjunction with their closest staff member and others involved in their life.

Think of this as a staircase, with the steps made up of short-term outcomes, leading up towards ‘intermediate outcome’ landings, with an exit door at the top of the stairs which is your ultimate long-term outcome. Young people will inevitably move up and down the staircase at differing paces, and they’ll be different interventions needed to help them progress at different stages (see programme design below), just so long as over extended periods of time they are moving up towards the exit at the top of the stairs, and you can prove it.

4. Programme design

If you know you are doing something which works then why wouldn’t you ensure these opportunities are available to all your young people? How would this look and how would you go about delivering it?

In other words, what’s the experience, from the young person’s point of view, from the first day they walk through your doors to the very last day that they say goodbye?

There will quite obviously be variance per young person, dependent upon need, but you need to agree what delivery will get the outcomes that you have set for your initiative, and when will each element of the programme experience be received by the young people, i.e. at what point on the staircase do they get the required interventions to support them to progress higher?

Again, in very broad terms, ThinkForward is gradually moving towards the below model for young people over a five year period.

14 years old – intensive one to one coaching relations, support them to become engaged in their own education, value learning and to attend school. Additional family engagement and support as required.

15 years old – Continued one-to-one support, with additional needs-led group work and world of work experiences, designed to broaden horizons, raise aspirations and develop work-readiness skills and mindsets. Supported to develop goals for themselves and establish a clear pathways to success.

16 years old – Business mentoring relationships developed, continued work experience opportunities, develop social capital, varied and needs-led cultural enrichment and life skills development. Focus on GCSE’s and intensive support with transition toward Further Education or apprenticeships. Opportunity to become ThinkForward Ambassadors.

17 years old – Ongoing support to ensure they remain in education or employment, intervening more intensively through one-to-one coaching, group work and advocacy when required. Opportunity to become near peer-mentors and ThinkForward Apprentices.

18 years old – Continued one-to-one support, link with potential employers and Higher Education institutions, support transition to sustainable employment and programme exit. Opportunity to become Alumni Ambassadors and join the youth board.

Clearly, a thoroughly ‘codified’ programme will require significantly more detail to be included.

5. Performance management

One simple question, with an undoubtedly complex answer.

Who, specifically, is responsible for ensuring all elements of what is achieved throughout the youth programme, and how are they supported effectively to deliver to an exceptionally high quality?

————————————————————————————————————————-

To read more about this outcome-focused approach, the above Theory of Change and successful implementation, I highly recommend David E.K. Hunter’s Working Hard, Working Well.

It’s important to continue this debate, so please feel free to post any reflections or comments below.

 

 

 

Advertisements
This entry was posted in Matt Lent. Bookmark the permalink.

6 Responses to We need to talk about outcomes

  1. Diane says:

    I’d certainly recommend you read this article by Tony Taylor http://indefenceofyouthwork.files.wordpress.com/2009/05/threatening-yw-and-illusion-final.pdf Its much more enlightening.

  2. You make a good case for at least building out one’s theory of change. This seems like a reasonable thing to do as it at least provides one with some clarity. However, once the rubber hits the road it all falls apart, as it should if one is doing youth work. That is, if one sees youth work as a co-created space with young people (as I do) then you cannot define the outcomes ahead of the practice at all because the youth will help shape it. The model you prescribe works when adults define the services to be provide a priori but this does not work for youth work that aims to be more inclusive and participatory of youth. Then the trick for measurement is how to develop a mathematics that is far more like calculus than addition or multiplication (though this is another blog altogether).

  3. No shivers down any spine. I have been doing outcomes based evidencing of youth work successes for many years. When done well it is easy to understand, simple to implement and clearly evidences the success of groups. See evaluation and replication reports at http://www.ansoncabin.co.uk for real world examples.

    I have also developed a new Assessment Wheel for one-to-one work that, again, is easy to understand, simple to implement and, if wanted, can be used to evidence outcomes across a cohort of clients. It is now being implemented by several organisations and, unlike Outcomes Stars, can be created bespoke for groups and has no ongoing charge.

  4. Matt – I enjoyed reading your in-depth explanation of the ThinkForward model/framework. I wish you well with that and thank you for sharing it. I also read the other commentators’ comments, including following the link to Tony’s diatribe, which I knew I shouldn’t have done but couldn’t resist. Having said that, I did gain immense respect for the interviewer.

    Youth work has become, for me, over-complicated … and even Tony contributes to that agenda. I am clear about youth work ~ it is about process and it is not about pre-determined outcomes, ‘a priori’ or otherwise. Youth work interventions, however they are determined, preferably co-created or fully determined by young people, nevertheless deliver outcomes and, in some cases, even long-term impact for those young people. That these can then be captured in any number of ways is also possible, if desired, and your model describes very well some of your thinking and practice on this.

    The heart of this debate therefore seems to me not to be about youth work or even outcomes per se but more to do with professional accountability. Whether or not people agree with the local government sector performance agenda introduced by the Labour Government of 1997, the reality of modern professional life is that they did; it was absorbed by, modified and continued by the Coalition government of 2010; and will continue to do so ad infinitum ~ simply because politicians like to know what they are getting for their money. It doesn’t matter if they are national, regional or local politicians, they have the same agenda! Depressingly, in the current economic climate, this also drives their choices even more fiercely, as witnessed by the swathe of devastation experienced by many former colleagues in public and third sector youth services/organisations.

    The discussion leaves me wondering whether engaging in either the ‘mystical/trust me I’m a youth worker’ process of youth work or the ‘bean counting’ approach of recorded and accredited outcomes has had any real impact, especially when it comes to mitigating political decisions. For many of our colleagues it clearly hasn’t made a jot of difference. Then again, I’ve always been a pragmatist. Despite the apparent lack of bolstering youth services’ position either approach might serve, I’d venture that anyone interested in the continuance of their service provision is going to move hell and high water to deliver evidence of outcomes and impact to bolster their position or to continue to compete in commissioning sweepstakes.

    Your post has therefore raised a number of interesting debating points and definitely given people one option for taking forward whatever they view as their accountability agenda. Thanks for sharing.

  5. Pingback: It’s time for youth workers to make some NOISE! | The Youth Sector Blog Site

  6. Pingback: We still need to be making much more noise about youth work | The Youth Sector Blog Site

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s