Disruptive Proactivity

No one* goes to work for the NHS intending to cause harm.

It’s easy to argue that Palantir are evil, and that they should have no place in the NHS, but that’s not going to change anything. Especially in a public health emergency.

Companies like Palantir – and especially ‘AI mercenaries’ Faculty – should be kept at barge pole distance (~20ft), not because they’re bastards, but because they’re seen by the public as bastards.

The job we face is not to build tech, but to help patients.

And that will be impossible if your product works best with a continuous high body count.

Unfortunately, the extent to which this Secretary of State and his staff will care is unclear. Their primary measure is to protect the system, not about staff or patients.

Just like all of the NGOs rushing into the health field, everyone else has shown up to help too – each one of them bringing the toolkit they were holding at the time the pandemic started.

Arguments against Faculty, Google DeepMind, or any of the other companies involved cannot and should not rest on morals; they must be substantive, properly evidenced and articulated in real world terms. It isn’t enough to be legally right.

People choices

Just as there is a range of competencies in civil society, there’s a range in all other sectors. 

When Facebook makes decisions, everyone knows that they’re doing what’s good for Facebook, whatever they claim their reasons to be.

DHSC/NHSE make no better decisions than Facebook, Palantir, or DWP – people are, after all, only human. 

People do not make better decisions due to their public office. Witness the unreformed, ‘institutionally ignorant’ and racist view which is slowly pervading everywhere in government…

DHSC makes decisions that are good for DHSC; Number 10 what’s best for Number 10 (today) – and Number 10 always wins.

The deaths of one in ten people in care homes are just one consequence of that approach.

We’ve met civil servants who are exceptionally talented at their (current or previous) jobs. There are also some who should never have been let into the building. 

But, given the immediate crisis, there are too many decisions to make and there’s a massive overstretch of political leadership within decision-making inside DHSC, so appointing people who can work with the existing team may be helpful. But the unit of accountability is this team.

While it is legitimate to put in place Ministerial (political) accountability to lead the testing operation in DHSX, they gave it to the Chair of NHS Improvement.

And when individuals do make decisions, often they would prefer to be entirely insulated from the consequences of those choices – for which the best way is by not knowing, or by not believing those who do know.

For the best example of that, see PMQs today

It’s notable from the NCSC blog and detailed paper which user needs it makes harder, resulting from the DHSC’s public statements the new app doesn’t expect to be true.

What should the pre-vaccine-normal look like for a user?

Here’s your main interaction with the app after setting it up:

• Your phone goes ping as you’re walking to the tube, and alerts you that you may have been exposed. 

As you have no symptoms, and because alerts or advice do not constitute decisions, what should you do?

1. You stop by (ideally outside) the local pharmacy and take a covid test (by the time it will be normal to walk down the street to the tube again, there should be enough covid tests available for you to do this; but DHSC have to deliver, and NHSX doesn’t seem to think they will).

2. A (socially distanced) long coffee later, you know your result and have evidence on which to take a clearly defined action (either carry on to the tube if negative, or turn round and go (/stay) home if positive). 

3. If you tell your app that you’ve had a negative test, it shouldn’t notify you again if the exposure it is notifying you about is much older than your most recent negative test.

If those steps can’t happen, the overall covid response remains terrible; if those steps can happen, then many of the protections and threat models that NCSC/NHSX cite as their reason for their approach are irrelevant (but NCSC/GCHQ do like their central authorities for other political reasons).

Now we have seen the app screenshots, as built, NHSX is not yet at the point of facilitating that process: you’ll have no way to know whether you need to take another test tomorrow despite the one you took yesterday…

If No10/DHSC’s public statements were treated as accurate inside NHSX, then there would be an expectation of enough testing for people to take an informed action on a notification – instead NHSX/NCSC have created a complicated threat model designed by computer people rather than public health people. (There are things they could do about that, but there is no evidence of those things, and it would breach their promises about only holding anonymised data etc)

— 

An aside on the ICO’s role: they may be in a complex position, but handing the advisory role to the Centre for Data Exploitation and Intrusion (with their track record of cheerleeding anything anyone in Government wants to do) and the sycophantic aspects of civil society craving corporate recognition (and the funding) would be catastrophic for non-pandemic times. The ICO is far from perfect at time, but a new model set up in a rush would be even worse. 

—

Edits: point 2 was clarified and links added.