Making it easier to produce evidence

By Jim Vine - on 03/02/2016

For a sector that has historically had more experience of case study evidence than evidence of effectiveness, producing this new type of evidence can be daunting. The newly-published Standards of Evidence make it easier to produce evidence by providing a consistent process to follow.

The evidence agenda is gaining steam in the housing sector. Increasingly, people in housing associations and the wider sector are approaching us to discuss the possibilities of conducting studies to establish the effects of various activities, projects and services. This growing appetite for evidence-based housing is why we have published the Standards of Evidence, to help the sector answer these questions.

Across the sector, people facing the reality of constrained resources want to know what works, so they can invest their time, money and effort into things that really make a difference. Sometimes this takes the form of wanting to know what the impacts are for tenants and communities directly; in other cases it is about establishing what the bottom-line impact is for a housing association business, because the ongoing delivery of a social mission is dependent upon continued financial viability. In the case of tenancy sustainment, success might mean better outcomes for both the tenant and for the landlord’s rent account. We call this 'evidence of the effectiveness of interventions', a phrase we have chosen to accommodate breadth in both the range of outcomes that might be considered 'effectiveness' and the range of services, products and activities that housing providers want to study.

Whichever outcomes are of interest though, the fact remains that producing this type of evidence is relatively new for the housing sector. We have historically relied on other types of evidence, such as case study-based evaluation reports. Whilst some of these methods can deliver valuable insights of other types, they will not normally provide much evidence of the effectiveness of an intervention: you might find out that your tenants have a warm feelings about your service, but you won’t know whether the improvements in their circumstances are really a result of it.

So, what are aspiring housing professionals to do, when pulled between a desire to learn what works so they can do more of it, and knowing that they and their colleagues have much more experience of generating other types of evidence? When faced with something unfamiliar that you recognise would do good if you could pull it off, what would help you to get over that unfamiliarity to start reaping the benefits?

We think that part of the answer to that is to be able to follow a process that has been well thought-out, tested in a range of settings and is known to work. Crucially, a process that can be applied consistently, and that means that you don’t need to worry that you are forgetting any vital steps. When travelling in an unfamiliar place you would want a map (or equivalent app on your smartphone!); similarly, when using an unfamiliar method for producing evidence you want a defined process to guide your route.

The Standards of Evidence – recently published by HACT, and available for free download – provide just such a map. They provide a standardised process that anyone can follow to identify what the impacts are of their activities.

There are two main documents* that you may want to look at to start building this type of evidence. There is an 8 page summary guide that talks through all the major steps for producing evidence of this type. And there is the specification’ – the part of the standards that spells out in detail all of the various requirements and recommendations for producing evidence of the effectiveness of interventions. The specification is definitely quite a ‘dry’ read – its focus is more on making sure everything is thorough and unambiguous than on being a page-turner – but it deliberately takes that form so that anyone picking it up will know that there is a comprehensive set of steps they can follow.

As with most tools, making use of the Standards of Evidence is optional. You could produce evidence of the effectiveness of your interventions without following these standards. But making use of the right tool can make your life a lot easier, especially if you are new to the approach.

We hope that these Standards will make a valuable contribution to the housing sector’s ability to produce robust evidence of its impacts. They are certainly not the only thing that could be helpful and will be looking to provide further support in the future. In the meantime, if you have any queries about the standards themselves or about producing robust evidence, please get in touch.

 

To be discuss the Standards of Evidence or enquire about support for producing evidence, please contact Jim Vine (jim.vine@hact.org.uk).

 

* As well as the summary guide and the specification for producing evidence there are also a couple of other documents that you might want to refer to if you want further information. A vocabulary provides a glossary of many of the terms used within the Standards, which might be helpful if you want to check how a term is being used. And an Explanation and elaboration document explains why each of the steps in the process has been adopted – it is provided for those who want to delve more into the whys and wherefores, and is not required reading for people who just want to follow the process.)

Tags: 
evidencewhat works