Three weeks ago Cabinet Office Minister Francis Maude formally kicked off an open policy making process to explore if government should allow more data to be shared between government departments, and if so under what circumstances. He stood in front of a group of civil society organisations and government officials to give his commitment to this open process (full meeting note here). He described the benefits he believed would come with better processes for sharing data between government departments.

However, he acknowledged the strong opposition that exists to such data sharing. He stated his belief that an open process could help to develop a clearer understanding of the circumstances under which data sharing should be allowed, thus allowing the UK to reap the benefits expected from such sharing. He was clear, it seemed to me, that for such a process to develop policy to be successful it must be a genuinely consensual process, and that if data sharing as a solution in some areas was rejected he would respect the outcome.

This is the week that those involved in this open policy process start to work on the detail of the government’s proposals. It is unfortunate then that reports over the weekend suggest that HMRC releasing data about VAT to credit reference agencies on questionable grounds and that it also has plans to release aggregated personal tax data.

A number of members of the civil society network have already begun to raise questions about what this means for the data sharing process launched by Francis Maude less than a month ago. Is the government committed to the process or not? Is it worth civil society organisations spending time and energy engaging with the process at all?

On the face of it there would now appear to be reason to be far more cynical about the intentions of government around the whole issue of data sharing. This will lead some people to conclude that the data sharing process launched by Maude is nothing more than a cynical attempt to absorb the energy of civil society to prevent it causing too much of a fuss as government shares data elsewhere from the public estate.

My own view is that it is too early to draw this conclusion, relying as it does on the view that government is a single entity with one view and objective on every policy issue. This view of monolithic government is false; we know that there is often significant disagreement between government departments (and even within them) about a wide range of policy areas. We saw this during the CSO-government process to develop a joint action plan for the Open Government Partnership, and the policy issues around data and data sharing are no different.

The CSOs who took part in the meeting with Maude went into it with their eyes open. They have known from the start that it will be hard to find agreement about when, under what circumstances, and with what safeguards data sharing between departments should be allowed to happen.

The datasets that the reports over the weekend suggest that HMRC plans to sell are not part of the discussions under the current CSO-Government data sharing process. We are seeing the fractured nature of government in action; it doesn’t result, I believe, from any bad faith on the part of Maude or the officials involved.

While I’m personally appalled by what I’ve heard about the plans to sell public data in the way it is alleged, I’m equally worried that it might distract both government and CSOs from reaching for two bigger prizes.

The first is the prize of a strong set of policies that facilitate data sharing between government departments to ensure that citizens have more effective and efficient delivery of services. This can only happen through an open process between government and a range of CSOs working together to identify the relevant safeguards that are needed to give citizens confidence in the way it happens. Without such a process it seems certain that opposition will be so strong government will be prevented from sharing any data at all.

The second prize is in the demonstration that genuine collaboration between CSOs and Government, through an open policy making process, can lead to better, stronger and more popular policy, even on one of the most controversial issues.

The recent reports about HMRC’s plans to sell tax data will undoubtedly make all the CSOs involved more vigilant and questioning about the intentions of Maude and the departments taking part in the collaborative data sharing policy process. They also remind us just how important it will be to ensure that the safeguards we develop together are strong enough to prevent risky data sharing in the future.

The potential of the two prizes I highlight, and the lack of evidence of bad faith by those from government involved in the process, mean that we shouldn’t throw in the towel just yet. I hope that others involved are making the same calculation.

Simon Burall, director of Involve