What to look for when choosing salary surveys
They look like they should be straightforward, but compensation surveys, in the words of experts in the field, constitute “a mix of art and science.”
As a result, it falls on the compensation specialist to use judicious interpretation of the results they have on hand.
“It’s not necessarily in black and white. There are many shades of grey and it really depends on the interpretation and the judgement that you bring to the data. Prior to using the information, you have to know what you want to use it for,” said Ingrid Huss, senior consultant at Aon Consulting.
With a variety of surveys and survey methodologies available in the market, choosing which ones to consult depends on the needs of the company, said compensation experts. Compensation surveys can range from the standardized national published surveys that draw data from hundreds of companies, to the more customized surveys that match for industry, company size, location and sometimes even level of unionization.
The advantages of each depend on the particularities of the market in which a company competes. First off, organizations participating in a survey have to know what market they’re in and decide which organizations they want to measure themselves against, said Sandy Weeks, managing director at Human Resources Partner, Inc.
Regardless of the type of survey compensation specialists might choose to participate in and consult, there are common variables that they should look for to gauge the usefulness of the data. Some survey compilers might collect average salaries, while others gather data for all incumbents, the advantage of which is to allow for a certain granularity of the data, said Huss.
“If you have a company that has 100 engineers, and they give you the average pay, you don’t know what’s actually happening within that average number. If you collect for incumbent number — the salary of every person in that position — you would be able to tell whether someone is paid abnormally high versus everyone else.”
Some adjust the data by discarding these extremely high or low salaries to better provide a realistic salary range for a given job, said Christopher Banks, consultant at Rochon Associated. On the other hand, in doing so “you lose the highs and lows,” said Merrilyn Earle, consultant at Mercer. “You don’t see what the best performers are paid and what the low performers are paid.”
One major factor in methodologies that users should look out for is how surveys are weighted. “If you have a number of companies, and one of them has 50 clerks, and the next largest has four, that big company will over-weigh the sample, and there’s a chance that the big company is an over-payer, if it’s on the bigger side,” said Randy Dutka, National Director, Expertise and Market Development at Hay Group.
One common solution is company-weighting, in which the same number of data points would be granted to each company, regardless of the number of incumbents they have. The counterpart of that is incumbent-weighting, which grants a data point for each incumbent measured.
“If you do incumbent-weighted, you’re pretty much getting what the market pays. If you do company-weighting, it gives you another type of distortion because it doesn’t give enough weight to that big company.” At Hay Group, added Dutka, analysts use a combination-weighting system which inputs up to five points, but no more than five, for each company surveyed.
One difficulty most often cited by survey suppliers is the accurate matching of job descriptions. Said Banks: “An administrative assistant at organization A and an administrative assistant at organization B might be called the same thing, but do they do the same things? As the economy moves toward knowledge work, this will become more of a problem, because positions will be more individualized.”
The problem of matching jobs is particularly acute in the information technology sector. Simply matching up “system administrator” jobs, for example, would result in surveys that gloss over differences that exist between someone who works with Unix and another who uses Windows — important differences when it comes to pay, said David Foote, president and chief research officer of Foote Partners. In IT, he added, it’s important to tabulate results based on worker certifications.
Level of experience can also be instructive, said Mairead MacLure, compensation consultant at Towers Perrin. That’s why she lines up job categories against functionality levels.
One form of survey that several of the above consultants look askance on is self-reported surveys, commonly offered by Web sites and professional associations. In such a format, there’s too great a chance that employees inflate their own salaries.
Weeks at Human Resources Partners, Inc would go further and cast doubt on salary surveys conducted by some headhunters.
Apart from an absence of job-matching based on job duties, said Weeks, headhunters also deal with the cream of the crop, so when they report their clients’ last salaries or the salaries offered by new employers, their data may not be representative of what most people holding similar positions are paid.
“This kind of reporting is based on hearsay, so it doesn’t come from the same rigorous approach. But they can be useful as a reference point.”
But responding to this line of argument, Gerald Walsh, of Atlantic executive recruiting firm Gerald Walsh Associates, said he doesn’t believe that there is that great of a tendency to embellish among candidates.
“When candidates are being interviewed by us and they’re asked how much you’re currently paid, they know that we can easily verify the data,” he said. “For a recruiter who has some experience, it’s easy to get a handle on what people are worth.”
Walsh said he bases his Atlantic salary survey on what job-seekers report as their current earnings, but also on the actual salaries offered to new hires, as well as a survey of about 350 employers. “A lot of the employers ask us to come in as a third-party when there are salary disputes and arbitration. So we’re well-respected by a lot of employers.”
As a result, it falls on the compensation specialist to use judicious interpretation of the results they have on hand.
“It’s not necessarily in black and white. There are many shades of grey and it really depends on the interpretation and the judgement that you bring to the data. Prior to using the information, you have to know what you want to use it for,” said Ingrid Huss, senior consultant at Aon Consulting.
With a variety of surveys and survey methodologies available in the market, choosing which ones to consult depends on the needs of the company, said compensation experts. Compensation surveys can range from the standardized national published surveys that draw data from hundreds of companies, to the more customized surveys that match for industry, company size, location and sometimes even level of unionization.
The advantages of each depend on the particularities of the market in which a company competes. First off, organizations participating in a survey have to know what market they’re in and decide which organizations they want to measure themselves against, said Sandy Weeks, managing director at Human Resources Partner, Inc.
Regardless of the type of survey compensation specialists might choose to participate in and consult, there are common variables that they should look for to gauge the usefulness of the data. Some survey compilers might collect average salaries, while others gather data for all incumbents, the advantage of which is to allow for a certain granularity of the data, said Huss.
“If you have a company that has 100 engineers, and they give you the average pay, you don’t know what’s actually happening within that average number. If you collect for incumbent number — the salary of every person in that position — you would be able to tell whether someone is paid abnormally high versus everyone else.”
Some adjust the data by discarding these extremely high or low salaries to better provide a realistic salary range for a given job, said Christopher Banks, consultant at Rochon Associated. On the other hand, in doing so “you lose the highs and lows,” said Merrilyn Earle, consultant at Mercer. “You don’t see what the best performers are paid and what the low performers are paid.”
One major factor in methodologies that users should look out for is how surveys are weighted. “If you have a number of companies, and one of them has 50 clerks, and the next largest has four, that big company will over-weigh the sample, and there’s a chance that the big company is an over-payer, if it’s on the bigger side,” said Randy Dutka, National Director, Expertise and Market Development at Hay Group.
One common solution is company-weighting, in which the same number of data points would be granted to each company, regardless of the number of incumbents they have. The counterpart of that is incumbent-weighting, which grants a data point for each incumbent measured.
“If you do incumbent-weighted, you’re pretty much getting what the market pays. If you do company-weighting, it gives you another type of distortion because it doesn’t give enough weight to that big company.” At Hay Group, added Dutka, analysts use a combination-weighting system which inputs up to five points, but no more than five, for each company surveyed.
One difficulty most often cited by survey suppliers is the accurate matching of job descriptions. Said Banks: “An administrative assistant at organization A and an administrative assistant at organization B might be called the same thing, but do they do the same things? As the economy moves toward knowledge work, this will become more of a problem, because positions will be more individualized.”
The problem of matching jobs is particularly acute in the information technology sector. Simply matching up “system administrator” jobs, for example, would result in surveys that gloss over differences that exist between someone who works with Unix and another who uses Windows — important differences when it comes to pay, said David Foote, president and chief research officer of Foote Partners. In IT, he added, it’s important to tabulate results based on worker certifications.
Level of experience can also be instructive, said Mairead MacLure, compensation consultant at Towers Perrin. That’s why she lines up job categories against functionality levels.
One form of survey that several of the above consultants look askance on is self-reported surveys, commonly offered by Web sites and professional associations. In such a format, there’s too great a chance that employees inflate their own salaries.
Weeks at Human Resources Partners, Inc would go further and cast doubt on salary surveys conducted by some headhunters.
Apart from an absence of job-matching based on job duties, said Weeks, headhunters also deal with the cream of the crop, so when they report their clients’ last salaries or the salaries offered by new employers, their data may not be representative of what most people holding similar positions are paid.
“This kind of reporting is based on hearsay, so it doesn’t come from the same rigorous approach. But they can be useful as a reference point.”
But responding to this line of argument, Gerald Walsh, of Atlantic executive recruiting firm Gerald Walsh Associates, said he doesn’t believe that there is that great of a tendency to embellish among candidates.
“When candidates are being interviewed by us and they’re asked how much you’re currently paid, they know that we can easily verify the data,” he said. “For a recruiter who has some experience, it’s easy to get a handle on what people are worth.”
Walsh said he bases his Atlantic salary survey on what job-seekers report as their current earnings, but also on the actual salaries offered to new hires, as well as a survey of about 350 employers. “A lot of the employers ask us to come in as a third-party when there are salary disputes and arbitration. So we’re well-respected by a lot of employers.”
