Take any card

Probably the most clichéd of card tricks is the one where the magician asks a member of the audience to "pick any card". It's called forcing a card, making the poor sucker take the card the magician wants them to take. I felt something similar was going on when I chanced to get a glimpse of the Jersey Annual Social Survey.

How it works is this: instead of presenting discrete options, you present paired options, so that there is an inbuilt degree of bias in the question; it assumes the result has to be either / or. Here is an example:

Suppose the government had to choose between increasing taxes and spending more on services, or decreasing taxes and spending less on services, which one should you think it should choose?

- reduce taxes and spend less on health, education and social benefits
-  keep taxes and spending on these services at the same level as now
- increase taxes and spend more on health, education and social benefits


Or perhaps - apply something like John Seddon's Systems Thinking to improve the efficiency of public services without having to make such choices, and looking for ways to get the trading local companies who effectively do not pay tax on profits on an even footing with those who do. But these aren't options we are given, and it is unlikely that many local politicians have ever read "Systems Thinking in the Public Sector: Case Studies: Delivering Public Services that Work (Volume 1)".  These are case studies of public sectors which have been turned around to provide more with less staff. It can be done, but this is not one of the options given in the survey.

And take immigration. One question asks about how concerned you are about immigration - very, fairly, not very, not at all. But the next brings back the paired options once more - choose (for each option) between very acceptable, fairly acceptable, not very acceptable, not at all acceptable.

How acceptable is future immigration to Jersey if it...

- keeps taxes for individuals as low as possible
- helps to maintain businesses and job opportunities
- makes sure we have enough workers to support the ageing population
- means our pensionable age can be kept as low as possible
- leads to better public services such as education and health


If the section on taxes and spending showed some bias, here is naked bias. Those are leading choices, almost designed to bias the recipient into giving the answer than immigration is, after all, acceptable. Who wouldn't want lower taxes, more job opportunities, better public services? But by linking these in a paired option with immigration, the survey is playing the forced card - we are back in magician territory again.

Of course this is nothing new - the same design faults were a major flaw in the Imagine Jersey forums some years ago. But it is rather disquieting to learn that nothing seems to have been learned from that except to follow the same pattern. What it does NOT do is to ask the following questions (which are assumed):

Do you think it is likely that inward migration can provide enough extra taxation to keep taxes low?
Do you think that inward migration improves job opportunities?
Do you think that inward migration provides enough workers to support the ageing population, or more aged people later on (and more pension demands)?
Do you think inward migration gives us more money for better public services, or are the migrants create an extra demand on schools and the hospital?

When we ask these questions, we can see core assumptions and biases made by the survey. Of course it could be answered that there is the option of giving "not at all acceptable" in response to the questions it asked, but the way that they are phrased, the unintentional consequences of each choice is hidden.

For instance, we could see that inward migration may lead to a vicious spiral needing more migration as the migrants themselves age and add to the demands on health services and pensions. That's not considered, or even placed in front of the average individual answering the question. Yet it is a perfectly obvious thing not to overlook - migrants, after all, don't stay preserved in amber at the same age they arrive in the Island. They age. They will be pensioners themselves. They will be an ageing population.

Is this a fair survey? It seems very close to the specimen survey given by Sir Humphrey Appleby in "Yes Minister". Part of that will depend on whether all the questions are published as well, so that the bias is apparent to any reader of the results. But even that is not enough - by using paired options, and by neglecting the possibility that those options might not be paired, the survey is actually excluding what people might think. There is a comment box, however, and I suggest it is used to note that.

Sir Humphrey: "You know what happens: nice young lady comes up to you. Obviously you want to create a good impression, you don't want to look a fool, do you? So she starts asking you some questions: Mr. Woolley, are you worried about the number of young people without jobs?"
Bernard Woolley: "Yes"
Sir Humphrey: "Are you worried about the rise in crime among teenagers?"
Bernard Woolley: "Yes"
Sir Humphrey: "Do you think there is a lack of discipline in our Comprehensive schools?"
Bernard Woolley: "Yes"
Sir Humphrey: "Do you think young people welcome some authority and leadership in their lives?"
Bernard Woolley: "Yes"
Sir Humphrey: "Do you think they respond to a challenge?"
Bernard Woolley: "Yes"
Sir Humphrey: "Would you be in favour of reintroducing National Service?"
Bernard Woolley: "Oh...well, I suppose I might be."
Sir Humphrey: "Yes or no?"
Bernard Woolley: "Yes"
Sir Humphrey: "Of course you would, Bernard. After all you told you can't say no to that. So they don't mention the first five questions and they publish the last one."