Tuesday, February 01, 2011

Measuring Equality Performance - Or how to tell an input from an output

The watch word for Equality and Diversity is 'outcomes'.

By that I mean measuring the effects on people of how you design and deliver a service, or a make a policy decision .. as opposed to simply totting up the work you've put in.

Unfortunately people often get confused and focus on the latter.

They may focus on measures such as the number of staff they've trained; the number of consultation meetings they've held; the number of documents they've written (such as Equality Impact Assessments); or the number of boxes they've ticked. They might even count the amount of money they've spent on these initiatives.

That can lead to people thinking they're getting somewhere yet maybe achieving less than they thought.

Beware false Gods

I can understand the allure of measuring what can broadly be described as 'inputs' or 'processes'.

For one thing, inputs are generally quite easy to measure.

You can count staff trained in classroom events or through 'e-learning'. It's easy to show paper evidence of meetings. Documents can be catalogued.

Counting inputs is reassuring too.

Making lists of what you've done (or paid to have done) means you can feel worthwhile. You can show your employer how hard you've worked.

What you put in is not necessarily what you get out

Yet, for many practitioners in Equality and Diversity, there seems to be an article of blind faith that putting all that work in will just magically produce something worthwhile out the other end.

You wouldn't invest money in a bank without considering the return. So why would anyone invest time on an equality and diversity programme without having some way of knowing where to put the effort and of verifying if it's effective?

I'm not against recording and analysing the work that people put in. However, if we're to be objective, the only thing that counts -- especially in the eyes of both stakeholders and regulators -- is the effects.

Outcomes are the statutory measure

The new public sector equality duty can sound as though it's just concerned with the processes you carry out. It talks of a duty to eliminate discrimination and to look for opportunities to promote greater equality.

The old Race, Disability and Gender Equality Duties also encouraged a focus on processes because of things such as the requirement to produce an Equality Scheme. People fell into the trap of thinking that once they'd produced such a document and put it on their web site they could tick the box and consider the duty to have been satisfied.

Yet, if discrimination is to be tested objectively then the only way to do so is to look for the adverse outcomes that are occurring in your organisation and then do things to minimise them.

No matter how much effort you put in, nobody on the outside is going to be very impressed if the outcomes for particular groups of people remain less favourable than for everyone else.

Outcomes are harder to measure

One obvious reason why people may not focus on outcomes as much as they should is because they take more effort to measure.

To measure an outcome you need to first decide how to measure the appropriate aspect of the status quo. For instance, if you're considering the diversity of your workforce then you need to decide what categories you're going to count; and how you're going do the counting. Depending on the way you break the numbers down, you may get very different answers.

To give an example of this, we know that roughly 75% of the 220,000 people who work for the NHS in North West England are women. Hurrah, you may say. Yet, that's not the complete picture. It's only when you break the numbers down to look at pay grades and occupations that you begin to see where the inequalities .. the adverse outcomes .. are located.

Having those measures isn't the end of the story either. If you've done an exercise like the one above then all you'll know is where the problems lie. It's on that evidence that you'll need to decide what actions to take. At some point afterwards you then have to come back and perform the same measurement again, using the original numbers as your baseline.

It's the difference between the before and after numbers that shows whether your actions have produced an outcome.

See why people might prefer not to have the hassle?

See why outcome measures are crucial though?

How to measure outcomes objectively

In future posts I'll explain how our team pioneered an approach to measuring the outcomes that matter within healthcare commissioning and provision. That includes service delivery as well as workforce measures too.

It is this work which also told us that there was a big gap between what people thought they were achieving (i.e. the work they'd put in) versus the ability to objectively demonstrate that they were equalising outcomes.

Before I can explain that, I'll need to explain the surrounding philosophy and strategy though. It's what we call our 'Eco-system' approach.

I'll talk about the evidence gathering we did, the conclusions we drew, the strategy we formed, the tools to support our goals, and then finally the framework for measuring the results. Each of those is a subject in itself. And the reason we talk of an 'Eco-System' is because they are all co-dependent.

But all that's for subsequent posts.

For now, however, think about how you would set about measuring equality outcomes -- and importantly the differences for various groups -- and what you might then need to do to address those.

No comments: