Canadian Council says some marks disappointing but is confident exam for designation is appropriate
When she sat down in September to write the national standards knowledge exam for the Certified Human Resources Professional designation (CHRP), Suzanne Gautreau felt pretty good about her chances.
She’d been studying for months, giving up family time and weekends and spending money on the preparation courses recommended by Human Resources Professionals Association of Ontario (HRPAO). She thought she was ready for the test.
“I went in there so confident. I just couldn’t wait to tackle the exam because I knew everything,” she said.
Three hours later: “I left there feeling humiliated, frustrated, demoralized,” she said.
“If I did pass, it would be pure luck. I will not be able to take any credit for my months of preparing and studying. If I had to write the exam again, I wouldn’t even know what to study. No schooling would have prepared you for this exam and the courses we took seemed irrelevant.”
The first-ever national knowledge exam, soon to be a prerequisite for all CHRP candidates, was introduced as part of the national standards. Rolled out last March, the new standards were created to ensure HR professionals across the country meet similar requirements for the designation.
Gautreau, who took the exam at Mohawk College in Hamilton, was one of 320 students to write the exam. She is not alone in feeling the exam was unfair. Along with four other exam-takers, Gautreau sent a letter to the Canadian Council of Human Resources Association (CCHRA), the body responsible for drafting the exam.
They also sent the letter to the HRPAO, the body responsible for administering the exam in Ontario and preparing students. (The letter was also sent to Canadian HR Reporter and is reprinted in part on page 27.)
Kathy Bocking, a certified general accountant with three years of HR management experience, finished the post-graduate HR course at Sheridan College in August. She said all of the classmates she talked with were very disappointed in the exam.
“We all walked out of there saying, ‘What was that?’ The HRPAO indicated on all of their literature that if you were qualified to write the provincial exam you were qualified to write the national. So many of us chose to write the national,” she said. She added that the prep material provided by HRPAO didn’t seem to match what was on the exam. The Sheridan curriculum, for example, didn’t cover pensions with as much depth as the exam, she said.
“We weren’t taught the things — in the college course — that were on the exam,” said Carmella Ciotti, the drafter of the letter signed by Gautreau.
As this was the first exam, she expected some problems. But the colleges and the CCHRA don’t appear to be on the same page when it comes to deciding what should be on the knowledge exam, said Ciotti, an experienced HR practitioner.
“We were truly the guinea pigs.”
The CCHRA said that after an early review of the marks and of the complaints from some students, it found the exam to its satisfaction. It suggested that problems may have been with the way students were prepared.
“We acknowledge that some people have been unhappy,” said Regina Delovitch, executive director of the CCHRA. “We do take all of those concerns seriously. We are reviewing the exam and the exam process,” she said.
The council received the results of the exam from the Independent Board of Examiners, the arm’s length group responsible for creating the exam and tabulating the marks. However, the council won’t release the results until completing a thorough analysis. (Students were expected to receive their marks by the end of November.)
The council wants to know if there are any correlations between marks and other factors, such as which provincial association was responsible for preparation, education and experience levels. That information would be used to make improvements to the process for the future, Delovitch said.
Some people were disappointed by the difficulty of the exam, but that is inevitable, said Anne Charette, president of the CCHRA. “We should have failures.”
“We are not unhappy,” she said, about the marks. “If everyone thinks they failed, I think people are going to be surprised.
“The whole idea is to establish a standard. How would we establish a standard if everyone passed the exam?” she asked. In the United States, the pass rate for people writing the entry-level national exam for the Society of Human Resources Management designation is about 65 per cent.
“We are going to come in about that area,” she said. The goal of the first exam is to establish a benchmark and then make improvements to the system so that more students hit that mark, she said.
But she also said she was surprised by the marks for Ontario students, as well as those for college students nationally. Those with university degrees did significantly better than college graduates. Similarly, there was considerable difference between regions, with Nova Scotia and B.C. students doing relatively well.
Asked how Ontario students did, Charette said, “Not as well as I would have liked to see them do.” About 200 of the 320 exam takers came from Ontario.
There may have been a misconception among Ontario students, she noted. They may have felt that just by taking the eight courses needed to write the old Ontario exam, they would be fine for the national standards exam.
That was not the case, she said. There were some knowledge gaps they needed to fill through self study and use of a Professional Assessment Resources Centre (PARC) Web site, which was set up by CCHRA to assist in exam preparation. A review of the feedback forms filled in by students who wrote the exam showed many never went to the PARC site.
“Our experts feel that when all is said and done, while it needs some improvements, it is overall a good exam.” The IBE has learned some things and will make some adjustments, but the board feels the exam is fundamentally sound, she said.
If changes are in order, they would likely be in the way HR students are educated and prepared for the exam, she said. For example, the difference in performance between university and college grads bears further analysis. There even appears to be a wide range in performance between colleges.
“I expected the colleges to do better than they did,” she said. Students from some colleges did very well, she said. “On the other hand, some colleges had a lot of writers who did very poorly, so it should be a wake up call for them.”
If the results do come in too low, “they would look at bell curving the marks,” she said. And because this was the first effort and things weren’t perfect, the council is considering taking steps to smooth some of the feathers ruffled in the first attempt.
Though it isn’t finalized, the council is considering allowing anyone who was close to passing the exam, a chance to write it again for free. She also said a new online course is being designed to help writers prepare for the next exam.
Monica Belcourt, president of HRPAO, said there will always be some people who say the exam was too difficult or not fair. That does not mean there are fundamental problems with the exam.
“The first time you give an exam, it is entirely fair to say let’s learn from this,” she said. If everyone finds some questions unreasonable, then those questions can be removed the next time.
And some students may have underestimated the difficulty and not prepared sufficiently, she said.
“I would think for the first couple of exams, until word gets out that these exams are tough, the failure rate might be a bit higher.”
If some students feel the exam did not test them on what they learned in the college courses, that is something the individual colleges have to explain, she said.
HRPAO agrees to the body of knowledge upon which students are tested and then approves the course outline for each college to ensure the body of knowledge is being taught. “What we don’t know is if the instructor actually gets into the classroom and teaches what the college has committed to being taught,” she said. “That is where HRPAO has no control.”
Vic Catano, a professor of psychology at Saint Mary’s University in Halifax and chair of the Independent Board of Examiners, said based on an early review of exam results, there is no evidence to suggest the test needs significant change. “We don’t see any major problems with the content of the exam,” said Catano. There were a range of questions from difficult to easy and on the whole the exam was appropriate, he said.
“I think some of the people who may have difficulty with it may reflect some inadequate preparation on their part,” he said. “People who prepared by going through the provincial association workshops, who made use of the PARC site and extensively looked at the resources on the PARC site will have done fairly well.”
Said Charette: “We will get to where we need to be, but everybody needs to adjust (to the new system). I just don’t want to create an air of pointing fingers and laying blame. The whole goal here is to make (students) be successful. The tone of the complaints is that we are out to get them and it is a money grab. Nothing could be further from the truth.”
She’d been studying for months, giving up family time and weekends and spending money on the preparation courses recommended by Human Resources Professionals Association of Ontario (HRPAO). She thought she was ready for the test.
“I went in there so confident. I just couldn’t wait to tackle the exam because I knew everything,” she said.
Three hours later: “I left there feeling humiliated, frustrated, demoralized,” she said.
“If I did pass, it would be pure luck. I will not be able to take any credit for my months of preparing and studying. If I had to write the exam again, I wouldn’t even know what to study. No schooling would have prepared you for this exam and the courses we took seemed irrelevant.”
The first-ever national knowledge exam, soon to be a prerequisite for all CHRP candidates, was introduced as part of the national standards. Rolled out last March, the new standards were created to ensure HR professionals across the country meet similar requirements for the designation.
Gautreau, who took the exam at Mohawk College in Hamilton, was one of 320 students to write the exam. She is not alone in feeling the exam was unfair. Along with four other exam-takers, Gautreau sent a letter to the Canadian Council of Human Resources Association (CCHRA), the body responsible for drafting the exam.
They also sent the letter to the HRPAO, the body responsible for administering the exam in Ontario and preparing students. (The letter was also sent to Canadian HR Reporter and is reprinted in part on page 27.)
Kathy Bocking, a certified general accountant with three years of HR management experience, finished the post-graduate HR course at Sheridan College in August. She said all of the classmates she talked with were very disappointed in the exam.
“We all walked out of there saying, ‘What was that?’ The HRPAO indicated on all of their literature that if you were qualified to write the provincial exam you were qualified to write the national. So many of us chose to write the national,” she said. She added that the prep material provided by HRPAO didn’t seem to match what was on the exam. The Sheridan curriculum, for example, didn’t cover pensions with as much depth as the exam, she said.
“We weren’t taught the things — in the college course — that were on the exam,” said Carmella Ciotti, the drafter of the letter signed by Gautreau.
As this was the first exam, she expected some problems. But the colleges and the CCHRA don’t appear to be on the same page when it comes to deciding what should be on the knowledge exam, said Ciotti, an experienced HR practitioner.
“We were truly the guinea pigs.”
The CCHRA said that after an early review of the marks and of the complaints from some students, it found the exam to its satisfaction. It suggested that problems may have been with the way students were prepared.
“We acknowledge that some people have been unhappy,” said Regina Delovitch, executive director of the CCHRA. “We do take all of those concerns seriously. We are reviewing the exam and the exam process,” she said.
The council received the results of the exam from the Independent Board of Examiners, the arm’s length group responsible for creating the exam and tabulating the marks. However, the council won’t release the results until completing a thorough analysis. (Students were expected to receive their marks by the end of November.)
The council wants to know if there are any correlations between marks and other factors, such as which provincial association was responsible for preparation, education and experience levels. That information would be used to make improvements to the process for the future, Delovitch said.
Some people were disappointed by the difficulty of the exam, but that is inevitable, said Anne Charette, president of the CCHRA. “We should have failures.”
“We are not unhappy,” she said, about the marks. “If everyone thinks they failed, I think people are going to be surprised.
“The whole idea is to establish a standard. How would we establish a standard if everyone passed the exam?” she asked. In the United States, the pass rate for people writing the entry-level national exam for the Society of Human Resources Management designation is about 65 per cent.
“We are going to come in about that area,” she said. The goal of the first exam is to establish a benchmark and then make improvements to the system so that more students hit that mark, she said.
But she also said she was surprised by the marks for Ontario students, as well as those for college students nationally. Those with university degrees did significantly better than college graduates. Similarly, there was considerable difference between regions, with Nova Scotia and B.C. students doing relatively well.
Asked how Ontario students did, Charette said, “Not as well as I would have liked to see them do.” About 200 of the 320 exam takers came from Ontario.
There may have been a misconception among Ontario students, she noted. They may have felt that just by taking the eight courses needed to write the old Ontario exam, they would be fine for the national standards exam.
That was not the case, she said. There were some knowledge gaps they needed to fill through self study and use of a Professional Assessment Resources Centre (PARC) Web site, which was set up by CCHRA to assist in exam preparation. A review of the feedback forms filled in by students who wrote the exam showed many never went to the PARC site.
“Our experts feel that when all is said and done, while it needs some improvements, it is overall a good exam.” The IBE has learned some things and will make some adjustments, but the board feels the exam is fundamentally sound, she said.
If changes are in order, they would likely be in the way HR students are educated and prepared for the exam, she said. For example, the difference in performance between university and college grads bears further analysis. There even appears to be a wide range in performance between colleges.
“I expected the colleges to do better than they did,” she said. Students from some colleges did very well, she said. “On the other hand, some colleges had a lot of writers who did very poorly, so it should be a wake up call for them.”
If the results do come in too low, “they would look at bell curving the marks,” she said. And because this was the first effort and things weren’t perfect, the council is considering taking steps to smooth some of the feathers ruffled in the first attempt.
Though it isn’t finalized, the council is considering allowing anyone who was close to passing the exam, a chance to write it again for free. She also said a new online course is being designed to help writers prepare for the next exam.
Monica Belcourt, president of HRPAO, said there will always be some people who say the exam was too difficult or not fair. That does not mean there are fundamental problems with the exam.
“The first time you give an exam, it is entirely fair to say let’s learn from this,” she said. If everyone finds some questions unreasonable, then those questions can be removed the next time.
And some students may have underestimated the difficulty and not prepared sufficiently, she said.
“I would think for the first couple of exams, until word gets out that these exams are tough, the failure rate might be a bit higher.”
If some students feel the exam did not test them on what they learned in the college courses, that is something the individual colleges have to explain, she said.
HRPAO agrees to the body of knowledge upon which students are tested and then approves the course outline for each college to ensure the body of knowledge is being taught. “What we don’t know is if the instructor actually gets into the classroom and teaches what the college has committed to being taught,” she said. “That is where HRPAO has no control.”
Vic Catano, a professor of psychology at Saint Mary’s University in Halifax and chair of the Independent Board of Examiners, said based on an early review of exam results, there is no evidence to suggest the test needs significant change. “We don’t see any major problems with the content of the exam,” said Catano. There were a range of questions from difficult to easy and on the whole the exam was appropriate, he said.
“I think some of the people who may have difficulty with it may reflect some inadequate preparation on their part,” he said. “People who prepared by going through the provincial association workshops, who made use of the PARC site and extensively looked at the resources on the PARC site will have done fairly well.”
Said Charette: “We will get to where we need to be, but everybody needs to adjust (to the new system). I just don’t want to create an air of pointing fingers and laying blame. The whole goal here is to make (students) be successful. The tone of the complaints is that we are out to get them and it is a money grab. Nothing could be further from the truth.”
