A Course of Leeches
If you have been following me on Twitter or reading my blogs you will know that I have raised a few issues with regards Evidence Based Policing (EBP)
To begin with, I would like to clarify that the concept of EBP – actually knowing what works and using it – is something I have absolutely no issue with at all.
In theory it makes total sense – if there is science or evidence which suggests that a certain thing is effective then why on earth would you not adopt it as best practice.
No – it is not the principle of EBP which concerns me.
It is not the theory – but the practice.
There is much talk at the moment about the medicalisation of policing. Not only should we strive to be as Evidence Based as the clinical world but certain quarters are even adopting the language: “treatment” “dosage” are just two examples.
To my mind there are a couple of linked issues with this overall ambition.
Firstly, policing is several hundred years behind medicine in terms of an evidence base. Medicine has gone from the days of spiritual healing, through blood-letting and eventually onto the discovery of incredible treatments, equipment to scan the body, vaccines and cures. Policing is currently at the “I recommend a course of leeches” stage of clinical enlightenment.
It’s not that policing doesn’t have a vast range of tactical options and responses to various problems – it’s just that we actually have no idea on whether any of them are effective or efficient.
The second, linked, problem is the typical police rush to put something in place. It has taken medicine centuries to get where it is today; much trial and error; many false hopes and eventually a position where the understanding of the human body is massively advanced and the knowledge of how to treat various ailments and injuries leaves us with only a few major diseases left to truly conquer. More is known of these diseases than ever before and more is known every day but it has taken a very long time to get to this point.
When it comes to policing – we seem to want to have it all in place by next Tuesday so we can claim it is embedded and we too can call ourselves professional.
This is patently impossible and it is likely to end up rushed and a pale imitation of what it should be.
There are any number of problems which stand in the way of EBP. That is not to say that we shouldn’t confront them and chip away at them but this will take decades – maybe even centuries.
What we seem to be doing, as per usual, is nailing something to the promotion, lateral development, annual appraisal processes and then claiming we have embedded it.
This doesn’t make it embedded – it makes it compulsory. They are not the same thing.
By extension of the same argument we could make it compulsory to paint a picture if you wish to proceed in policing and then claim that the police are all now artists. To extend the metaphor still further you have to then ask who is judging the artwork and from what frame of reference and expertise?
Who will be assessing the contribution to advancing the evidence base or the use of the evidence base when there are so few knowledgeable practitioners in the police. Simply having a rank or two above the candidate does not suddenly make you an expert. In fact – if the candidate is more qualified than the assessor then it can’t even really be a peer review.
Simply making something necessary does not ensure full buy in or understanding. It also runs the risk of becoming a box ticking exercise with the blind leading the blind.
Policing is full of this – take risk assessing a missing person for example. The ultimate decision on whether someone is classified as high, medium or low risk usually sits with an Inspector or Sergeant. How much more training have they had in risk assessing missing people than the constable? None at all. But they have extra rank and so that assumes they are the more appropriate decision maker. Doesn’t matter if the Constable has 28 years service and the Inspector is direct entry with 18 months – the Inspector wins.
One of my concerns about insisting that you must demonstrate the use of development of EBP for what is effectively a personnel procedure is “who is assessing this?”
When you actually look at the gold standard of evidence – the peer review – you know that in academia the peer is going to be a recognised subject matter expert.
My fear in policing is that this “expertise” will simply fall to person on the next rung of the ladder. Who may have no idea at all about how to do it properly and determine whether the contribution is valid or not.
Another problem with this approach is sheer volume. Take an average force with, say, 2000 officers – if the requirement is that they must demonstrate employment of or development of the evidence base for the purposes of annual appraisal or development then this is an awful lot of experiments. The assessor must be aware of the existence of the evidence the person is applying or must be able to properly, systematically and correctly assess whether the new contribution withstands scientific scrutiny. The workforce is not currently sufficiently skilled to do this.
Thinking back to my earlier years in the service, the concept of problem oriented policing was all the rage. I was a key supporter and proponent of it.
With the absolute best of intentions I embarked on a series of Scanning, Analysis, Response and Assessment to try and deal with some of the intractable problems on my beat.
At the conclusion of these initiatives I was able to demonstrate a fall in demand and a fall in crime ergo the initiatives were a success. On paper they looked impressive. 80% drop in calls and crime down by 25%
My mistake – these were binary comparisons. I wasn’t trying to deceive anyone but the numbers seemed to speak for themselves. I never did prove causality. I didn’t have to – the awards nominations soon followed.
But the thing is – we tried to replicate the work the following year and couldn’t get it off the ground. I had more money and resources available as a result of the apparent success of the previous year but for whatever reason the interest wasn’t there.
The national media coverage had meant I was invited to a few other forces to explain what we’d done and I don’t know if they were able to replicate it.
That first year looked like “what worked” but actually it was really “might work.” It was very much of the moment. It clearly relied on factors beyond the actual response itself. There was a human factor here which simply couldn’t be accounted for, predicted or easily copied.
And this is the thing with a social science like policing compared to a pure science like medicine. Drugs do what drugs do – there are predictable outcomes and side effects. Policing relies heavily on human nature and human whim which are far less predictable.
That “treatment” I put in place worked once. I think. It seemed to. But it worked at that time and in that place. Does that mean it isn’t worth trying somewhere else – no but success is not guaranteed at all.
I had the pleasure of hearing Gavin Hales from the Police Foundation give a talk this week on two programmes which neatly demonstrate some of the problems with Evidence Based Policing.
These were detailed, scientifically designed and evaluated and ran over five years. One looked at a response to recurrent domestic violence and the other looked at burglary reduction.
If you look at the responses which were put in place you would recognise them because a lot of forces are still doing them or things very similar to them. In some cases they would be the text-book or standard responses. The perceived wisdom and common sense approach.
Except they didn’t work.
Neither of them achieved what they set out to achieve. There was no drop in domestic violence and there was no drop in burglary.
The fact that they didn’t work isn’t actually the problem what both programmes illustrated were some of the issue which will stand in the way of successful implementation of EBP.
By way of example:
1. The length of time they took. The police are a “do now” organisation and frequently operate in a short term world. Five years is a long time in policing and during that time other factors had come into play. People had moved on and priorities had changed. What was an important project in 2011 had been superceded by something new and more important by 2015 and resources had to be shifted elsewhere. By the end of one of the projects it was getting increasingly hard to get people to respond to emails. There was a huge difference between the initial ambition and what was ultimately implemented and senior managers were impatient for results. What was important was forever being overtaken by what was urgent.
2. The force of personality. One of the projects had a very dynamic force lead. Their drive and enthusiasm played a very crucial part in the operation and even though it didn’t work their commitment kept it afloat. This shows how important an individual can be in the success of any initiative. It makes it hard to replicate. If that person moves on then it is likely that the drive will go as well. People don’t sit still for long in policing.
3. One of the projects involved multi-agency strategy meetings. This led to diffused responsibility and some confusion. At the end of the project people were lauding the improved partnership working and collaboration. We hear this a lot in policing as well – it is frequently cited as a success outcome. However, it is a means and not an end in and of itself. It’s all great if the various agencies can get along nicely with each other but if they aren’t actually achieving anything can it truly be said that the partnership working has improved? Relationships may have – but that is not enough.
4. If you were to look at the measures that were put in place and see how ineffective they are it should give pause for thought. An opportunity to look at similar responses (there are, for example, many different multi-agency strategy meetings) and ask questions as to whether they are also achieving what they are intended to achieve. Lessons should be learned from this but, with the greatest of respect to Gavin and the Police Foundation, if I were to ask how many people had ever heard of this work the answer would be “not very many people.” I wonder how many people in the actual forces involved have heard of or even remember these programmes. The lessons are important but I would imagine forces around the country are still doing the things which were shown to be ineffective and are still saying they are an essential and effective response. The futility of this is actually quite staggering.
Another factor which will inhibit the development of EBP is that policing is not a Petri dish. It is not a safe, sterile and controlled environment for experimentation. You can only work with live subjects – you cannot simulate. These are real people and the consequences are real.
Look at DASH forms for example – we are being told that they are essential and life saving tool to assist in the risk management of domestic violence. They are mandatory. However, they are now being properly evaluated and the early findings are that they are about as accurate a predictor as a horoscope. But we will still get messages from the Superintendent reminding us of their value and that they are compulsory. They are, in fact – to steal again from the medical parlance, a Placebo Policy. Something we do to show that we are doing something.
This is the other problem with implementation of EBP – accountability. We are hearing a lot that it is “ok to fail” but is this really true? Is it actually ok to fail in policing? Will the IPCC, HMIC or media take a sympathetic view to a failed initiative especially if it had unwanted consequences?
I could go on but this blog is already long enough. EBP is a laudable objective but my fear is that it is going to be rushed and implemented poorly.
Professionalisation of policing is not something that can be achieved by putting labels on things to copy others. We need to live and we need to breathe it.
Yes – we have to start somewhere but please recognise where we are starting from, the barriers there are to overcome and dont make the mistake of thinking that by making something compulsory we have successfully implemented it.
In other words – slow down to speed up.