As our Community Investment and Bottom Line research project concludes, our Data Scientist Christina Knudsen reflects on the project and the initial key findings.
The ambitious Community Investment and Bottom Line project aimed to investigate how the bottom line of an organisation is impacted by community investment activities. This blog will briefly outline the methods used to answer this question.
In order to consider the bottom line impact of community investment on housing providers fully, the Community Investment and Bottom Line project was broken down into two parallel investigations. The first branch considered the outcomes of participation in community investment activities. The purpose of these activities is to change a tenant’s status. For example, employment training aims to get unemployed tenants into work. So is there a difference in costs incurred to housing providers between tenancies with people out of work compared to those with people in work? After careful examination of the data received from project partners, propensity score matching (PSM) was considered to be the most appropriate method to answer this question. Using PSM allowed us to investigate the association between a tenant’s status and the costs incurred to the housing provider, while controlling for other characteristics of a tenancy. By controlling we can unpick the effect of tenant’s status on bottom line costs.
To look at the employment example already mentioned, here we are wanting to identify the difference in bottom line costs associated with being out of work compared to being in work. Using the PSM methodology, we first build a predictive model to determine the probability a tenant is in work based upon characteristics of a tenancy other than their working status (eg. household composition or tenancy length). This probability is known as the propensity score. To allow us to include all the data in our analysis, the propensity scores are converted into weights. Using the weights results in an average equivalence between the two groups on all observed characteristics other than their working status. To illustrate this, say the mean tenancy length for those in work was 5 years, then the mean for those out of work group should also be 5 or close to that. The weighted groups are then compared using regression to establish the difference in cost to the housing provider.
The second branch considers tenants who went through a community investment scheme, and determining whether the costs associated with their tenancy reduced as a result. The method used was, again, determined after inspection of the data received. The approach used is based on the idea that those who completed a community investment activity earlier would have lower costs than those who concluded their activity later in the year. This was tested through regression analysis looking at the trend of costs for the last financial year on the date the tenant completed their community investment activity.
The methodologies outlined above have produced a useful framework for determining the potential costs savings for individual housing providers through community investment activity. For housing providers interested in understanding the bottom line impact from community investment activities it is worth noting that there is potential for this work to be enhanced based upon more detailed data being available, such as:
- Statuses, eg. working status, recorded as a times series
- Costs incurred to housing providers recorded on a per tenancy per instance basis
- Recording attendance (dates) of all tenants attending community investment activities in a way that can be linked back to tenancies
- And more generally, ensuring records are up to date
The Community Investment and Bottom Line project is drawing to a close and a report will be released in due course providing greater detail on the methods used, data requirements and the final outputs from the investigations.