A new inspection regime - the comprehensive area assessment - is to be rolled out for a range of local services. HSJ and sister title LGC collected the great and the good to discuss how it will affect their work

A new inspection regime - the comprehensive area assessment - is to be rolled out for a range of local services. HSJ and sister title LGC collected the great and the good to discuss how it will affect their work

Richard Vize: Gareth, could you give us a resume of where we are with comprehensive area assessments and where the Audit Commission hopes to get?

Gareth Davies: We are at the early stages of developing this framework. This flows from the local government white paper published at the end of last year and is reflected in the Local Government and Public Involvement in Health Bill currently going through Parliament.

The framework will replace the existing comprehensive performance assessment, which was an assessment of councils and council services. The new assessment will be more outcome focused, and it will reflect not just the Audit Commission's judgements or performance indicator results, but the views of local people, users of services, taxpayers, citizens and also the judgements of other inspectorates.

Integrating all this into a single assessment framework for local services presents lots of challenges, and integration with the assessment of policing and community safety is only one of them.

But our ambition for the new assessment is to come up with a system of assessment that is less expensive and less intrusive than the previous version, but more sophisticated and helpful in driving the right kind of service improvement and user engagement set out in the white paper. We are moving away from central, target-driven approaches to assessing how well the local partnership is dealing with issues that confront it and giving more attention to local priorities.

RV: Abena, how should this manifest itself for the citizen?

Abena Dadze-Arthur: From the National Consumer Council's perspective, the primary purpose of the regulator is to serve the public service users - so in the case of the Audit Commission, its primary responsibility would not be to the local authorities but to the public service users.

And a regulator, as opposed to being a stick to beat authorities into line, would be an effective tool to help local authorities eventually provide services people want.

One of the reasons that has not happened so far is because regulation is focused on targets, league tables. What we are advocating is that the soft outcomes, such as customer satisfaction, are integrated in the performance-measurement system in a way that they can be weighted in terms of performance measurement.

There are challenges. It is not easy to quantify social justice, social capital and community cohesion, but there are ways to do it, which is what we are working on.

There are some great examples in the pipeline, such as local involvement networks [which gather information and make recommendations to trusts and councils].

Our vision is a regulator that works with the local authorities with the ultimate purpose of providing better services to those sitting at the receiving end - the users.

RV: Paul, does that match your vision?

Paul Walker: Yes. For the past seven years, we have been concentrating on a small collection of councillors, the executive and, in our case [Hartlepool borough council], the elected mayor, and how well they have done. As we have gone through the comprehensive performance assessment, we have certainly improved standards on that.

The move to the comprehensive area assessment will bring into focus the good work the frontline councillors do, because they all work as champions in their areas. They are all on neighbourhood forums, on resident forums and they do a lot of work. It helps to raise standards, but it is not really recognised.

My hope is in a few years' time, this will be seen as a stepping stone to fully trusting councils and councillors to do the work that they want to do. And this is a really good step in that direction.

Corin Thomson: There is a real advantage to the new assessment in bringing in councillors more. The focus on customer satisfaction is something councillors can take an interest in and will be involved in.

But there are risks - we have shown real improvements in performance over the past few years and need to make sure that the new system will continue to allow us to do that.

RV: David, what do you think are the barriers to success in terms of implementing this system and how can they be overcome?

David Fry: There is a lot of difficulty because satisfaction is only one measure of the quality of service being delivered.

Often people are not satisfied with a service, but actually it is a good service because there is always a balance between need and demand.

Central government will have to let go. Imagine getting them to agree between departments on what the 200 indicators are. Those are all shifts in the way people behave.

In local government, the good areas have done well, but the evaluation of the first phase of local area agreements is patchy and where it works well it should continue to work well. Where it doesn't, you are relying on a much greater ability to develop a more complex local area agreement.

Also, the Audit Commission is best placed [to manage the new system] but there is a significant change in role here.

The capture and management of the data of performance indicators across a range of services - the analysis of that which allows you to form a risk-based view of the future and the ability to play that back in a way that is meaningful to consumers and to the local partners - is quite a shift in emphasis.

Game of risk

RV: One of the most ambitious aspects of the new framework is that it is going to be risk-based, which could mean those high performers doing the most innovative projects could get more attention than poor performers. So I would like to bring in our health and police colleagues who have perhaps a better understanding of the issue of risk than most. Roger?

Roger Baker: Definition is a key issue because people speak about community safety, but what does it mean? You have really got to know what a good job looks like before you can get into what risk looks like.

Also in moving from the old assessment, for example, to the new one, there is a risk of letting go of things that have worked and the staff and the consumer understanding of it.

But the risk for me is that on the level of assessment, we question ourselves and try to define risk too much. There is a difference between the local area agreement assessment and what the assessment of policing and community safety is saying about the same issues. And that is going to leave most of us very confused.

The other aspect is while measuring outcomes is key to the service deliverer, if you are going to go into this partnership with a different mindset, you have got to be clear on inputs as well, because what may happen is that arguments start to develop when you start to risk assess [in relation to] who has contributed what to this particular party.

You have got a number of bodies engaged who, as good as they are, never inspect crime and disorder reduction partnerships There is no inspection, no intervention. So there is a degree of walking before we try to break out into a trot, let alone running. These are the negative things, but all can be dealt with.

RV: Chris - you are in an area where working across boundaries should be second nature. How do you see the new comprehensive area assessments working?

Chris Born: At North Somerset primary care trust we have a coterminous relationship with the local authority. This is real progress, and when I read the vision [for the assessment] I thought 'fantastic'. But my question is whether we really are taking it into the other agencies and across government departments.

We have to find the measures for consumer satisfaction and engagement because there is more and more work being done, including in the health service, to measure it. We always say it is too difficult to measure; we do surveys and they are not very good. So we have got to focus on the service user and the citizen more than we have on finding measures that are right.

There is little mention here of equalities. One of the scandals for all of us is the gap between the people who get outcomes and those who don't, who are usually associated with social deprivation.

But my main concern is, has [the Audit Commission] gone far enough in translating the vision, because it still feels largely like a local authority approach rather than a place approach? We have got something called Standards for Better Health - most of it is plain obvious enough stuff about making sure patients and the public get a good deal from us and we give good value for money. So is it beyond the wit of mankind to say let's have a set of common standards such as information governance, or involving the public, or clinical outcomes or service outcomes?

I read the number of targets and thought: 'Great, 200 targets for the lot of us.' But I then see -'Oh, an extra 35 for education, can we have an extra 35 for the police and we had better have an extra 35 or 250 for health.' So my concern is are we taking it far enough or is it just too difficult to go that far in terms of government ability to integrate and our own ability?

I love what the new assessment says, but I want to take it as far as we can take it.

Jo Webber: Coming back to the targets, the local ones are really interesting because what does this mean in terms of how much is nationally prescribed and how much is local? At some point you hit the dreaded postcode word don't you?

How far is that going to be allowed and how far is that really going to enable people to consider services locally, because if you really get into place-shaping, as set out in the local government white paper, you are going to start having areas that look very different in terms of the public services that they deliver through their functions.

RV: Do your members perceive the assessment to have local government written through the middle?

JW: It would be nice to see more of a move towards this being about the local strategic partnership, bearing in mind the local authority is the lead of the partnership. But it would be interesting to broaden the scope of this so long as, picking up Chris Born's point, you are not over-regulating.

We need to integrate the comprehensive area assessment, otherwise I think that the buy-in may be related to how much more burden there is associated with it. PCTs are already out there delivering local partnerships and building long-term sustainable relationships, but if this is seen as an extra burden, then that's more difficult.

There is also the change in mindset for health organisations that are used to a risk-control rather than a risk-management set-up - local authorities are managed risk services, whereas a lot of health services are controlling risk.

Soft focus

RV: Erin, what changes do public sector operators and commissioners of services need to deliver to make the assessment work?

Erin McFeely: The most interesting thing for us is the idea of intelligent commissioning and relationship-building. Coming out of that will be the sorts of concepts people are talking about here - long-term sustainability of partnerships, particularly in terms of commissioners understanding the needs of voluntary sectors in relation to longer-term contracts, full cost of covering funding, all those needs.

And that is a two-way thing between the sector and the commissioners. Each needs to understand where the other is coming from. That will bring with it the sort of softer outcomes we were talking about, and the need to measure them is something that we would welcome, because we have often said this is what the third sector can bring to the table, these sorts of less-definable outcomes rather than outputs.

RV: Paul, in your area, how can you have an intelligent conversation with the public about risk? Is it possible to have it at all or is it a fool's errand trying to get it through?

PW: I don't think the public are interested in what the risks are. They are interested in the outcomes and how you deal with them. An important thing is who is assessing the risk? And who is applying the risk factors in an area?

For example, I represent North East councillors on the regional resilience forum [for emergency preparedness and response]. We drew up the risk register for the region and some of the indicators put in nationally bore no relationship to what was happening in the North East. We potentially ended up with an incredibly high risk that nobody had even heard of or thought of in the North East.

AD-A: From the intelligence we have from the public as well as public service users, this is not true. There is quite a large segment of people who are interested [in risk]. And by involving them through whatever appropriate means, you are already working at the risk.

There is something interesting about explaining this to the public, getting their input, debating that with them. Then even if the risk turns out to happen, because the public has been involved they feel they had a stake in this and the risk has already diminished. And the consequences of the risk, they have already diminished as well. So the public are not only interested, they are part of the risk management.

PW: I think that is a different and more important issue. What you are talking about is process. What you are saying is correct - you mention talking to people.

Most of our staff are out every day, in communities, talking to people. But they are not interested in process. It is what the outcomes can be for that area or those people and how you get them - that is more important.

AD-A: Obviously you have to make it relevant. You have to connect processes - they are only a means to achieve an outcome. So it should not be viewed as a separate entity. And when talking to the public or public service users you have to make the process relevant to the outcome.

To give you an example, in the health sector, in our work with the Healthcare Commission, the annual healthcheck is actually perceived as more or less a complicated thing, the risk process, third-party assessment, it is a whole jungle of terminology that nobody understands.

But interestingly, the forums - which were made up of people who are actually very informed and interested and wanted to have that discussion about process - wanted to talk to the Healthcare Commission and to all the other stakeholders about how the process doesn't make any sense to them, how the process is not meaningful to them. And through having a dialogue about the process, it linked to the outcome and had interesting effects on the whole way the assessment process was done.

Citizens' band

RV: Citizen engagement is a key facet of the new assessment. Roger, in terms of police operations, this is an extraordinarily high-profile issue. How do you make that work?

RB: There is an issue of us being on a journey because people do want to engage with you at particular levels. And what they do ask you for, whichever sector you work in, are usually the fairly basic things in life.

My customer, if I could talk parochially, is not that demanding. We have managed to set - not deliberately - particularly low expectations. So if you can actually get through on the telephone you are on a winner. So if you deliver a reasonable basic service they do also engage with you.

But the downside to that is the postcoding, which becomes an issue in the assessment field because people will compare and contrast.

The other thing that would give us all an idea of what a good job looks like is greater clarity about what the potential sanctions are so you can inform your customer that if we fail to do this, this is the comeback through multi-agency work.

Everyone likes to talk when you have done really well. When you have failed it all becomes a case of definition becoming a bit loose.

CB: What is critical is linking people's experience and preferences with all these measures and targets. Because if we talk about things that matter to people, then there is the whole issue of access - can I get hold of the service I want and can I get it reasonably quickly and conveniently whether it is through telephone helplines or whatever? And that drives a lot of all our targets differentially and when we come together on them. So why can't we get that link?

Similarly, most people want health and well-being. And that includes safety, whether the streets are clean, as well as getting social care. If we listen to people, we will be able to set that in measures.

Also, is there an opportunity to integrate some of the more formal processes? We have given scrutiny quite a lot of power. We know it has worked differentially and sometimes not as well as we would have liked around the country. But if we are giving scrutiny more powers, then how can we make sure it represents public views?

CT: If you look at the next phase of public sector reform and where the focus is now, there is a lot more talk about customer satisfaction. And that has to be right when we have come from a period of time where we have put a lot of money into public services and they have improved in terms of the outputs, but there is this ongoing mismatch between customer satisfaction overall and satisfaction with individual services. And focusing on what the customer wants has to be a way forward to addressing that mismatch.

We are putting in place here a framework and a process to make that much easier than perhaps it has been in the past across all of the partners. A lot of local government does do customer-focused services quite well and is working well with local people and finding out what they want. The new stronger local area agreements are a route to making it easier for people to engage with all their local services, not just with local authorities.

DF: There is a fundamental difference between customer satisfaction and customer input to service quality. Most of the research shows a mismatch between satisfaction and the reality of service delivery - it is often quite astounding. In the last Ipsos MORI surveys of local authorities, there were some areas particularly where some of the worst-performing councils were actually getting in the upper quartile of satisfaction figures.

Using people to influence service policy means allowing them to understand what that policy is in the first place and how it compares. So with the issue you have raised, Roger, it is quite right to get people to accept if they are happy with a certain level of service - why spend more on it?

As soon as comparative information becomes available in a way that people understand, then they will start to get much more engaged in the agenda. So making information about service policy available will make a significant difference to people's behaviour. And that will create quite a tension for local service providers. This debate about what is national and what is local - it will have to be both because people will say 'why aren't I as good as the rest of the country?' and 'I want my local priorities as well please'. It cannot be either/or.

RV: Gareth, there is a huge amount of stuff for you to respond to. How do you ensure that everyone is getting access to these better services and not just a few? Has the assessment got local government written right through it? How can you balance the problems or risk?

GD: The [local government] white paper, first of all, did get signed up to by every department with an interest in local services, which is most of them, and the formal structures we have put in place in developing the comprehensive area assessment to senior representatives from Ofsted, from the Healthcare Commission and so on. So this is a joint effort and the future consultation events will reflect that. This is not just about a new framework for local government. It is a place-based agenda.

I wanted to say something about the differences across areas. Hopefully, this will only work if we do arrive at quite different things that we are looking at place by place. And so - this is a central bit of this - for the assessment to have any chance of being a constructive and intelligent contribution, the local area agreements have to be a high-quality piece of work.

That dialogue between government office representing central government departments and the local strategic partnership has to be really high quality. It is a big opportunity to be grabbed here by partnerships because they can say 'we understand our area. We are not just dealing with an immediate crisis, we've got a 30-year vision.'

If what comes to that table is high-quality analysis locally and really thoughtful partnership work, then central government will be forced to deliver on the commitments in the white paper about devolution. Why would you interfere with somebody who knows what's clearly needed here and who is getting on with it?

Government is serious about the opportunities it is giving in the white paper. That local area agreement point is really important because we are not just talking about an assessment framework, it is what goes into those agreements in the first place.

On the business about engaging local people, [we have to ensure] the maximum incentives are there for local service providers to engage local people in imaginative and effective ways.

The incentives have to be for people to do their job and that means engaging local people rather than to say 'well I'm going to do my best, but the Audit Commission is going to come and convene a panel of people locally and find out what they want'. Clearly, there will be evidence of local satisfaction played in but that should just be the final stage in what has been a way in which everybody does their job locally.

David is right about the mismatch between objective performance measures and what local people feel. But having said that, one note of hope is when we did the analysis with Ipsos MORI - the information David referred to, which was very recent - against our most recent comprehensive performance assessment results for councils. At the level of aggregation - when you do the averages - they match pretty closely.

So this is not an area of complete unreality where the technocrats come up with one set of assessments that are not recognised by the people who experience the services. So there is some hope in there that we are not so far away from where we could be in our aims.

Final thoughts

GD: On risk: for the comprehensive performance assessment, it is the risk that the priority outcomes that the partnership set for the area are not achieved on the timescale that is set out. And that deals neatly with the concern about risk aversion that some people fear this might create, because the serious risk will be that we are not innovating enough to achieve this ambitious trajectory we have set for ourselves as a partnership. We are going to be criticised for it. So that is the bit we have to get right.

There is more focus on outcomes in our work and it is not about ways to innovation - you will have to innovate to be able to achieve suitably stretching and ambitious targets around that. That is going to be the challenge, sticking to that. If we are not seeing local strategic partnerships taking risks then there is probably something going wrong - they have to be well-measured risks, but there is probably a lack of ambition too.

The other point is about incentives. We have been surprised over the development of the comprehensive performance assessment how much of an incentive a simple clear performance publication has been and the sheer peer pressure on people like Paul. They do not want an assessment whether they agree with the detail of it or not - they do not want the public assessment that is risking the reputation of the authority. So keeping that kind of parity without the simplistic scoring system - that is a real challenge.

The real incentive is reputation and career development. I would be concerned to get away from the little pockets of money for this or that, that I know have a place but they work directly against local priority setting. By definition, some people have a national initiative and want to give chunks of money for that particular thing - it needs a lot of careful thinking through to get away from the small performance pots of money.

RV: Abena, some closing thoughts?

AD-A: What came out really well in this discussion is that to make the assessments an effective tool there is a need for clarity and for definition: clarity of roles, tasks, stakeholders, the stakeholders involved not only from the private sector, the public sector, the third sector but also the leadership - what is the role of the leadership?

What is the role of the staff, the people, the citizens, the public service users?

Also, while taking risks and innovating and key drivers of customer satisfaction are new and good tools, nevertheless they are tools to improve services, and we have to be careful this framework does not turn these tools into perverse incentives that we have just moved away from.

Ultimately, what we have to keep in mind is that the assessment is meant to be an enabling framework that also has to adjust to a very quickly changing and dynamic world.

What we are moving towards is a system where local authorities will have to establish contractual arrangements with local involvement networks so these are active citizen hubs. And these hubs will actually have commissioning powers and take key decisions in terms of public service delivery, at the moment only in health and social services, but ultimately the vision is for all public services. This citizen hub will be tapped into and have this active role in commissioning and deciding on services. So this framework does not have to appeal only to local strategic partnerships, all the different sectors, all the different levels, but it has to encompass all of that and take into account that the powers and the responsibilities are actually shifting downwards.

The panel

  • Richard Vize, HSJ editor and discussion chair
  • Roger Baker, chief constable, Essex police
  • Chris Born, chief executive, North Somerset primary care trust
  • Abena Dadze-Arthur, senior policy advocate, National Consumer Council
  • Gareth Davies, managing director for local government, housing and community safety, the Audit Commission
  • David Fry, head of local government consulting, PA Consulting
  • Erin McFeely, media and communications executive, Association of Chief Executives of Voluntary Organisations
  • Corin Thomson, programme director for improvement and performance, Local Government Association
  • Paul Walker, chief executive, Hartlepool borough council
  • Jo Webber, deputy policy director, NHS Confederation