I've been looking at the addendum with the Hassell Blampied Associates information on pay benchmarking which is available at:.
http://www.statesassembly.gov.je/documents/reports/22574-18689-892009.pdf
From their website I notice the blurb:
Because of the simple, straightforward but robust methodology adopted in comparing jobs, we believe we have the best range of market information available in the Offshore Islands to ensure that businesses can take strategic decisions about their reward policies for their staff. Our survey data takes account of the differing needs in organisations from the most senior staff to the most junior, together with the clear reports that pinpoint the market information (1)
I can't see any explanation of this methodology, which means there is no way of checking what they mean by this at all! The survey itself says that:
Forty-five private sector employers participated in the survey, providing data on 3,583 jobs, and these were drawn from a variety of sectors, including Finance, Retail, Construction and the Utilities.(2)
Quite how there is a chief officer comparison with anyone in those areas is beyond me, especially as there are no details of what counts as "like for like". This is a complex problem which is extremely important. As the statistician Professor Joel Best notes:
Statistical comparisons promise a bit more-at least two numbers that might reveal a pattern: things are getting worse; or things are worse in one place than another; or this group has it worse than that one; or even this problem is more important than some others. But comparison depends on comparability. Unless each number reflects the same definitions and the same methods of measurement-unless each number is an apple, and not something else-comparisons can be deceptive. Unless the numbers are comparable, the pattern apparently revealed through comparison may say more about the nature of the numbers than it does about the nature of the social problem (3)
The report gives different grades of job, but nowhere does that state what the tie up is.
On Talkback (13/09/2009) Senator Sarah Ferguson had a solid figure for a policeman with two years training in London and Jersey, which showed a differential of £1,000 of Jersey over London in like for like occupations with the same experience and training. That is a clear like for like comparison. In fact, police, doctors, nurses and teachers are excluded from this survey, which compares private sector to public sector pay.
But where do you compare, for example cleaners when there are a variety of rates and part time jobs in the private sector? Is a clerical worker at the hospital equivalent of a doctor's surgery receptionist? How do you compare catering staff (where there are staff canteens)? Is there an equivalent of a hospital porter in the private sector?
With a sample, normally some degree of stratification needs to be in place, over perhaps gender and age, so that the comparison reflects the private population in employment as a whole. The accompanying table gives no indication of gender, and yet other studies have shown this is important. In the USA, for example, studies noted that:
Men earned about the same, or less, in state and local government as comparable men earned in the private sector, but women earned more in state and local governments than in private firms.(4)
This means that a sample which is not stratified, and which has more women in the public sector sample than the private sector sample might well be skewed to show the employees in general in the public sector are paid more, whereas this may reflect the prevalence of better paid jobs in the public sector, where salary is awarded on the basis of grades, while in the private sector it is well known that women often receive less pay than men in comparable jobs.
Moreover some studies have noted that:
Marital status was an important influence in pay rates in the private sector, but not the public sector, and was more important for men than for women.(6)
Another area is part-time work, where a UK survey showed that:
Part-time workers received higher pay in the private sector. (6)
There is no breakdown of part-time employees in the table, nor is there any breakdown of the type of work, just the grade of work. And yet this too can be significant:
For some occupations, such as accountant, attorney, engineer, and personnel supervisor/manager, the data appear to support pay compression: the private sector pay advantage indeed rises with grade level. But for other occupations, such as computer programmer, computer systems analyst, and computer systems analyst supervisor/manager, comparisons do not show any wage compression.(6)
Because the table just shows the grade, and not the kind of occupation, it is impossible to see if the differentials between public and private sector are across the board in a grade, or whether part-time, female workers, different occupations may all give different results, some being more highly paid than the equivalent in the private sector, some less so.
Until the mid-1980s, all studies implicitly assumed that workers chose whether to work in the public or the private sector without considering pay differences between the sectors. A variety of studies since then have argued that differing pay structures attract different types of workers (e.g., if the public sector pays minorities better than the private sector does, it will attract more minority applicants) and that this will in turn affect the pay differences observed.(4)
This, of course, also counts for legal positions like that of the Solicitor General, Attorney-General, Deputy Bailiff and Bailiff. A lawyer may earn more in the private sector, but if they want both to be in public service, and also perhaps (because they are human, after all - and why not!), enjoy the limelight and unpaid kudos that this involves, perhaps with a knighthood, the opportunity to meet Royalty etc, the choice may not be entirely pay related.
Without some detail, the nice table we are given looks very solid and professional, but may simply be reflecting in part a social disparity between pay. We just do not know how the comparisons of like for like are made, and if we are in fact looking at apples and pears, both fruit, but only marginally comparable.
The words "simple, straightforward but robust methodology" used by the survey bring to mind the deficiencies that are often noted in surveys of this kind. The lack of transparency means we cannot tell what this is, but here is an example of how other surveys have been conducted with what one person may describe as "simple" but another as "crude", like a blunt instrument:
The crude measures that are used to establish comparability of individuals provide for only gross equivalence. The measures of work experience are often rough estimates that cannot separate unemployment or time out of the labor force from paid employment; they never distinguish between related and unrelated employment. Education typically is assumed to have a log-linear impact on salary, which builds in the assumptions that an additional year of education has the same percentage impact on salary whether it is at the high school, college, or postgraduate level, and that a bachelor's degree in literature increases salary as much as one in engineering does.(4)
The size of the firms in the sample may also be significant. Again other critical studies of comparisons have shown that:
Most studies do not control for the size of the employing establishment, although large companies typically pay better than small ones.(4)
Once more, the lack of any transparency in the published results means that it is impossible to see the size of the companies in the survey of private firms. And yet, as Joel Bests notes, this is extremely important:
Whenever there is disagreement about the statistical evidence, it is possible to look more closely, to discover how different measurement choices, different definitions, or other factors can explain the disparities. But, of course, this can be a lot of work; few people will make the effort to examine original sources. And, even when it is possible to clarify a specific statistical disagreement, that clarification will not resolve the larger debate about the broader social issue. Again, debates over broad social issues have their roots in competing interests and different values. While advocates for different positions tend to invoke statistics as evidence to bolster their arguments, statistics in and of themselves cannot resolve these debates. This is important because we often equate numbers with "facts." Treating a number as a fact implies that it is indisputable. It should be no surprise, then, when people interested in some social problem collect relevant statistics and present them as "facts", this is a way for them to claim authority, to argue that the facts ("It's true!") support their position
The table provided looks very authoritative, but how were the comparisons of like for like made? How were the private sector jobs chosen? What proportion of the whole, and how representative are they? Were there any refusals in the survey? What total percentage of jobs in the private sector does 3,583 represent? How many of the lower ones were on income support?
Once those questions are asked, and there is no information available that I can see for them, it seems that a lot is being taken on trust regarding the "objectivity" of the survey.
I have only considered the statistics, and not the causal factors behind them. But that might also be worth research. Deputy Daniel Wimberley has suggested that the higher rates of pay in the survey (given its limitations) may be due to the fact that the market goes for a race to the bottom, so that there are more private sector workers at the bottom of the scale on the minimum wage. It is not clear how many of the sample were on the minimum wage, and as he pointed out, possibly costing the State as a result of claims for income support which higher paid public sector workers, still at the bottom of the pay scale, do not. This illustrates, however, how a survey on pay alone can overlook other factors which should be considered, and begs the political question over whether low scale public sector pay should be reduced given that it may lead to more payments under income support. A single focus on pay and not income support may miss this completely.
I would like to end by noting Joel Best's comments on the problems that beset single studies:
Single studies, then, can't do the job. Absolutely every study every test, every piece of research-has limitations and flaws in its methods that make it a target for legitimate criticism. Studies should be replicated, and they should also inspire further research that uses different methods (with, presumably, different limitations and flaws). When replication and differing methodologies confirm the same result, confidence in that finding grows. The results of' a lone study, particularly if the research raises serious methodological concerns, should not, in most scientists' view, be treated as authoritative. Only time and further research can sort out the erroneous findings from the more reliable.(5)
Now I am not saying that the results have been deliberately spun, and other surveys may well come to the same conclusion about pay. I am not aiming to discredit Hassell Blampied, merely to point out that one survey, with limited transparency, and no scientific peer review, can hardly be taken as authoritative, and there are weaknesses which should be addressed.
All I would say is that in the studies I have reviewed, the sample data and the sampling methodology has all been transparent, and published in peer reviewed journals so that other statisticians can check the results and replicate them, or test the assumptions of equivalence and parity. The note by Hassell Blampied that they have been doing these surveys for some time is not relevant to this - they could have been prone to the same methodological flaws every time through no fault of their own, and despite their complete professionalism.
Links:
1) http://www.hassellblampied.com/Salary%20&%20Benefits%20Surveys.htm
2) http://www.statesassembly.gov.je/documents/reports/22574-18689-892009.pdf
3) Dammed Lies and Statistics, Joel Best, 2001
(4) Pay, Productivity, and the Public Sector: The Case of Electrical Engineers., Langbein, Lewis, "Journal of Public Administration Research and Theory."Vol 8. Issue: 3. , 1998
(5) More Dammed Lies and Statistics, Joel Best, 2004
(6) "The Public-Private Pay Debate: What Do the Data Show?. Michael A. Miller, Monthly Labour Review. Vol 119. Issue: 5. ,1996
http://www.statesassembly.gov.je/documents/reports/22574-18689-892009.pdf
From their website I notice the blurb:
Because of the simple, straightforward but robust methodology adopted in comparing jobs, we believe we have the best range of market information available in the Offshore Islands to ensure that businesses can take strategic decisions about their reward policies for their staff. Our survey data takes account of the differing needs in organisations from the most senior staff to the most junior, together with the clear reports that pinpoint the market information (1)
I can't see any explanation of this methodology, which means there is no way of checking what they mean by this at all! The survey itself says that:
Forty-five private sector employers participated in the survey, providing data on 3,583 jobs, and these were drawn from a variety of sectors, including Finance, Retail, Construction and the Utilities.(2)
Quite how there is a chief officer comparison with anyone in those areas is beyond me, especially as there are no details of what counts as "like for like". This is a complex problem which is extremely important. As the statistician Professor Joel Best notes:
Statistical comparisons promise a bit more-at least two numbers that might reveal a pattern: things are getting worse; or things are worse in one place than another; or this group has it worse than that one; or even this problem is more important than some others. But comparison depends on comparability. Unless each number reflects the same definitions and the same methods of measurement-unless each number is an apple, and not something else-comparisons can be deceptive. Unless the numbers are comparable, the pattern apparently revealed through comparison may say more about the nature of the numbers than it does about the nature of the social problem (3)
The report gives different grades of job, but nowhere does that state what the tie up is.
On Talkback (13/09/2009) Senator Sarah Ferguson had a solid figure for a policeman with two years training in London and Jersey, which showed a differential of £1,000 of Jersey over London in like for like occupations with the same experience and training. That is a clear like for like comparison. In fact, police, doctors, nurses and teachers are excluded from this survey, which compares private sector to public sector pay.
But where do you compare, for example cleaners when there are a variety of rates and part time jobs in the private sector? Is a clerical worker at the hospital equivalent of a doctor's surgery receptionist? How do you compare catering staff (where there are staff canteens)? Is there an equivalent of a hospital porter in the private sector?
With a sample, normally some degree of stratification needs to be in place, over perhaps gender and age, so that the comparison reflects the private population in employment as a whole. The accompanying table gives no indication of gender, and yet other studies have shown this is important. In the USA, for example, studies noted that:
Men earned about the same, or less, in state and local government as comparable men earned in the private sector, but women earned more in state and local governments than in private firms.(4)
This means that a sample which is not stratified, and which has more women in the public sector sample than the private sector sample might well be skewed to show the employees in general in the public sector are paid more, whereas this may reflect the prevalence of better paid jobs in the public sector, where salary is awarded on the basis of grades, while in the private sector it is well known that women often receive less pay than men in comparable jobs.
Moreover some studies have noted that:
Marital status was an important influence in pay rates in the private sector, but not the public sector, and was more important for men than for women.(6)
Another area is part-time work, where a UK survey showed that:
Part-time workers received higher pay in the private sector. (6)
There is no breakdown of part-time employees in the table, nor is there any breakdown of the type of work, just the grade of work. And yet this too can be significant:
For some occupations, such as accountant, attorney, engineer, and personnel supervisor/manager, the data appear to support pay compression: the private sector pay advantage indeed rises with grade level. But for other occupations, such as computer programmer, computer systems analyst, and computer systems analyst supervisor/manager, comparisons do not show any wage compression.(6)
Because the table just shows the grade, and not the kind of occupation, it is impossible to see if the differentials between public and private sector are across the board in a grade, or whether part-time, female workers, different occupations may all give different results, some being more highly paid than the equivalent in the private sector, some less so.
Until the mid-1980s, all studies implicitly assumed that workers chose whether to work in the public or the private sector without considering pay differences between the sectors. A variety of studies since then have argued that differing pay structures attract different types of workers (e.g., if the public sector pays minorities better than the private sector does, it will attract more minority applicants) and that this will in turn affect the pay differences observed.(4)
This, of course, also counts for legal positions like that of the Solicitor General, Attorney-General, Deputy Bailiff and Bailiff. A lawyer may earn more in the private sector, but if they want both to be in public service, and also perhaps (because they are human, after all - and why not!), enjoy the limelight and unpaid kudos that this involves, perhaps with a knighthood, the opportunity to meet Royalty etc, the choice may not be entirely pay related.
Without some detail, the nice table we are given looks very solid and professional, but may simply be reflecting in part a social disparity between pay. We just do not know how the comparisons of like for like are made, and if we are in fact looking at apples and pears, both fruit, but only marginally comparable.
The words "simple, straightforward but robust methodology" used by the survey bring to mind the deficiencies that are often noted in surveys of this kind. The lack of transparency means we cannot tell what this is, but here is an example of how other surveys have been conducted with what one person may describe as "simple" but another as "crude", like a blunt instrument:
The crude measures that are used to establish comparability of individuals provide for only gross equivalence. The measures of work experience are often rough estimates that cannot separate unemployment or time out of the labor force from paid employment; they never distinguish between related and unrelated employment. Education typically is assumed to have a log-linear impact on salary, which builds in the assumptions that an additional year of education has the same percentage impact on salary whether it is at the high school, college, or postgraduate level, and that a bachelor's degree in literature increases salary as much as one in engineering does.(4)
The size of the firms in the sample may also be significant. Again other critical studies of comparisons have shown that:
Most studies do not control for the size of the employing establishment, although large companies typically pay better than small ones.(4)
Once more, the lack of any transparency in the published results means that it is impossible to see the size of the companies in the survey of private firms. And yet, as Joel Bests notes, this is extremely important:
Whenever there is disagreement about the statistical evidence, it is possible to look more closely, to discover how different measurement choices, different definitions, or other factors can explain the disparities. But, of course, this can be a lot of work; few people will make the effort to examine original sources. And, even when it is possible to clarify a specific statistical disagreement, that clarification will not resolve the larger debate about the broader social issue. Again, debates over broad social issues have their roots in competing interests and different values. While advocates for different positions tend to invoke statistics as evidence to bolster their arguments, statistics in and of themselves cannot resolve these debates. This is important because we often equate numbers with "facts." Treating a number as a fact implies that it is indisputable. It should be no surprise, then, when people interested in some social problem collect relevant statistics and present them as "facts", this is a way for them to claim authority, to argue that the facts ("It's true!") support their position
The table provided looks very authoritative, but how were the comparisons of like for like made? How were the private sector jobs chosen? What proportion of the whole, and how representative are they? Were there any refusals in the survey? What total percentage of jobs in the private sector does 3,583 represent? How many of the lower ones were on income support?
Once those questions are asked, and there is no information available that I can see for them, it seems that a lot is being taken on trust regarding the "objectivity" of the survey.
I have only considered the statistics, and not the causal factors behind them. But that might also be worth research. Deputy Daniel Wimberley has suggested that the higher rates of pay in the survey (given its limitations) may be due to the fact that the market goes for a race to the bottom, so that there are more private sector workers at the bottom of the scale on the minimum wage. It is not clear how many of the sample were on the minimum wage, and as he pointed out, possibly costing the State as a result of claims for income support which higher paid public sector workers, still at the bottom of the pay scale, do not. This illustrates, however, how a survey on pay alone can overlook other factors which should be considered, and begs the political question over whether low scale public sector pay should be reduced given that it may lead to more payments under income support. A single focus on pay and not income support may miss this completely.
I would like to end by noting Joel Best's comments on the problems that beset single studies:
Single studies, then, can't do the job. Absolutely every study every test, every piece of research-has limitations and flaws in its methods that make it a target for legitimate criticism. Studies should be replicated, and they should also inspire further research that uses different methods (with, presumably, different limitations and flaws). When replication and differing methodologies confirm the same result, confidence in that finding grows. The results of' a lone study, particularly if the research raises serious methodological concerns, should not, in most scientists' view, be treated as authoritative. Only time and further research can sort out the erroneous findings from the more reliable.(5)
Now I am not saying that the results have been deliberately spun, and other surveys may well come to the same conclusion about pay. I am not aiming to discredit Hassell Blampied, merely to point out that one survey, with limited transparency, and no scientific peer review, can hardly be taken as authoritative, and there are weaknesses which should be addressed.
All I would say is that in the studies I have reviewed, the sample data and the sampling methodology has all been transparent, and published in peer reviewed journals so that other statisticians can check the results and replicate them, or test the assumptions of equivalence and parity. The note by Hassell Blampied that they have been doing these surveys for some time is not relevant to this - they could have been prone to the same methodological flaws every time through no fault of their own, and despite their complete professionalism.
Links:
1) http://www.hassellblampied.com/Salary%20&%20Benefits%20Surveys.htm
2) http://www.statesassembly.gov.je/documents/reports/22574-18689-892009.pdf
3) Dammed Lies and Statistics, Joel Best, 2001
(4) Pay, Productivity, and the Public Sector: The Case of Electrical Engineers., Langbein, Lewis, "Journal of Public Administration Research and Theory."Vol 8. Issue: 3. , 1998
(5) More Dammed Lies and Statistics, Joel Best, 2004
(6) "The Public-Private Pay Debate: What Do the Data Show?. Michael A. Miller, Monthly Labour Review. Vol 119. Issue: 5. ,1996
Posts
No comments:
Post a Comment