A Critical Review of Compensation Data from Surveys and Websites. Do You Know the Quality of the Data You are Using?

“I’m updating the organization’s compensation plan and need reliable market data. Will I need to purchase a survey or find information on the Internet to get quality compensation data?” No! To get reputable compensation data does not mean you need to purchase an expensive survey or join an online web service. As an Employers Council member, you have access to the most reliable source of compensation survey data in the Arizona, Colorado, Utah, and Wyoming markets. Employers Council has been collecting survey data since 1947!

HR professionals have access to many more sources of compensation data today than ever before. Today you can find compensation information through surveys produced by associations or work-related websites such as Glassdoor, Indeed, LinkedIn, PayScale.com, etc. However, it is essential to know the source of the compensation data before using it in your compensation plan.

Following are some critical factors an employer should consider when evaluating the quality and reliability of compensation data from surveys and websites.

1. Methodology

a. Who is reporting the compensation data?
Employers Council collects compensation data from human resource or finance professionals in member organizations, not self-reported data by individuals. Many of the work-related websites noted above are self-reported data by individuals vs. employer-reported data.
b. How was the data collected?
Employers Council collects compensation data through online questionnaire, Excel spreadsheet, paper copy – whatever is most convenient for the participant. It is important to understand how the data is collected through other compensation sources you use.
c. What was the validation process?
Employers Council has a 3-step validation process to ensure the publication of accurate data for our members. 1st – we compare an organization’s data to its previous survey submission. 2nd – we compare all data by job from low to high. 3rd – we compare the data to the previous survey report. It is important to understand how other sources validate the data submitted or if they even validate data!
d. What data are collected and displayed?
Employers Council collects individual base pay, formal ranges, incentive pay, exemption status, as well as number of organizations reporting and number of incumbents by job. All data collected through Employers Council’s surveys are displayed in the published report along with the effective date of the data. What data are the other sources collecting and publishing? Does the resource disclose the date of data collection or just date of publication?
e. What should you know about survey methodology?
If I am updating my organization’s compensation plan, I need to know the survey methodology as I may be questioned by senior leadership for detail. I should be prepared to explain the statistical data and know how the organization positions itself relative to the market data. In other words, if the organization wants to pay above market, will I age the data and by what factor? Do I also know the organization’s labor market(s) we compete in and the corresponding data lines in the survey?

2. Job Matches

a. Does the survey or website contain job descriptions or job titles only?
Employers Council collects salary data based on job descriptions. Chances are other surveys or websites that are collecting data based solely on job titles will have erroneous job matching – meaning bad data.
b. Survey data should be used based on job content, not job title. What percent of the job description should be matched for a valid match?
Employers Council defines that participants can report an employee who spends 70% or more of their time in the described function. What criteria are used to match jobs with website sources or association surveys?
c. What if a job in my organization matches more than one job in a survey?
There may be jobs that are blends of two or more jobs found in the survey. If the job in your organization has duties in more than one functional area, it is matched to two survey jobs. Once the match is approved and salary survey data is pulled, the weighted averages count equally and an aggregate average is calculated.

3. Participant Base

a. How large is the participation and how consistent is it from the previous survey?
Employers Council collects data from organizations throughout Arizona, Colorado, Utah, and Wyoming. Sixty-five percent of participants in the Benchmark Compensation Surveys contribute data two years in a row. What is the consistency of the data reported in website sources or association surveys?
b. What types of industries are included?
Employers Council collects data across a wide array of industries such as manufacturing, government, non-profit, healthcare, financial, retail/wholesale, ski, etc. What types of industries are reporting data in website sources or association surveys?
c. What is the employment size of organizations included?
Employers Council collects data from employers of all sizes. What is the employment sizes of the organizations represented in website sources or association surveys?

4. Customized Reports/Extracts

a. Does the website resource or survey publisher offer customized reports of the data?
Employers Council publishes data by industry type, employment size and geographic location on Employers Council’s website. Employers Council offers Custom Survey Analysis (CSA) by peer group or custom demographic cuts from our surveys. And, members who participate by the survey deadline are eligible for a free CSA!

Final thoughts – It is critical to know the source of the compensation data being reported (employer or individual) and if the data is validated when using compensation information from association surveys or website sources. Employers Council’s survey data meet both of these two most important criteria. Employers Council publishes over 30 compensation and benefit surveys each year with the data being reported by Employers Council member organizations. Employers Council’s compensation data is extensively analyzed and validated; 80% of participating employers are contacted to validate their input.

Many organizations want to utilize as many survey sources as possible. While that is considered a good practice, focus on survey quality over quantity.

Survey data is an important tool when updating a compensation plan but it is not the only tool. Reliable survey data is meant to provide an indicator of external competitiveness. It must be analyzed and balanced with other elements of a successful plan design. These elements include an articulated compensation philosophy based on the organization’s business strategy and objectives, consideration of compensable factors that survey data doesn’t provide, and consistent pay practices.

Contact Employers Council’s Surveys Department at Surveys@employerscouncil.org, or Compensation Consulting Services at Compensation@employerscouncil.org, if you have any questions on survey resources or consulting services available to Employers Council members.

Thank you for your support and participation in Employers Council’s Surveys. Your participation does matter!