'It has this well-known issue where it hallucinates answers'
ChatGPT and other similar AI programs are becoming more ubiquitous by the day but for one employment lawyer, there is a lot of improvement still to come.
“I drafted an article about a month ago where I had asked ChatGPT version 4 a series of questions on employment matters and reviewed its responses so I got a bit of a sense of what its limitations are,” says Spencer Knibutat, associate at Filion Wakely Thorup Angeletti in Toronto.
“The main ones I found were it has this this well-known issue where it hallucinates answers so basically [it may] constantly produce an incorrect answer.”
“If somebody is not that familiar with a general topic that they’re asking questions on, they can’t really assess whether this is in the ballpark of reasonableness,” he says.
While some organizations are using AI tools for a variety of menial topics, such as writing job descriptions, drafting policy documents or other administrative tasks, the overall uptick is low, according to a recent survey of HR professionals.
Generative AI is used most often to generate text after being given a series of inputs from a user in the form of questions or notes.
Spotting the use of generative AI
For those who are more familiar with the tools, spotting AI output is relatively simple, according to an academic.
“What you tend to find with AI tech is it tends to be less complex, and it tends to have a very uniform length. I can often, with a naked eye, have a good idea if something has been generated with AI because when the students submit the work, it tends to be quite uniform in terms of the sentence structure and sentence lengths,” says Louis Volante, professor of education governance and policy analysis at Brock University in St. Catharines, Ont.
And for HR, candidates might submit documents created by AI when applying for jobs.
“It’s probably going to be a huge challenge for people in HR, in terms of ‘How do I evaluate a candidate based on a portfolio that’s generated with the use of AI?’ You want to be hiring a person you think has the requisite skills and if writing is one of those requisite skills, then it behooves you to try to figure out how authentic [was] the sample of work that was submitted,” says Volante.
For those HR departments who are keen on fully exploring its potential, “do your homework, first and foremost,” he says.
“That doesn’t mean I expect you to go read a book or read a bunch of journal articles but do some do some research on it depending on what type of application you want to use it.”
Generative AI and performance evaluation
One area that might be intriguing for HR professionals to consider is AI generating performance review evaluations based on collecting employee data and notes — but employers should tread carefully when doing this.
“My concern with the present generative AI is it just doesn’t have that transparency that you would hope for if you’re going to be relying on it for performance reviews because you have to be able to justify the decision, and it’ll be difficult to do that if you’re relying almost exclusively or primarily on generative AI,” says Knibutat.
“If they can’t really explain it, it leaves the door open for accusations that the generative AI produced the bias or incorrect output, which can definitely create liability and make it difficult for the employer to defend that decision.”
But generative AI could be employed in other ways related to performance.
“I can also see it being used in a more tangential ways right now: it could summarize lengthy performance review documents for reviewers to cut down on the time that it takes to conduct those reviews,” he says.
From Volante’s perspective, using AI to review performance is “is particularly front and centre right now.” But accurately judging an employee’s overall performance remains elusive.
“The question is, will it lead to companies and HR departments maybe making selection processes or promotion decisions based on unsound evidence? You want some assurance that the individual in front of you can actually perform the tasks that they need to perform on a daily basis and isn’t completely reliant on technology,” he says.
‘Do your homework’
There are also legal considerations when it comes to using generative AI to write a job description, according to Knibutat.
“When generative AI creates a job posting that includes qualifications that would unjustifiably exclude certain groups, it can be content that is generated or a decision or recommendation or predictions made by an AI that adversely differentiates directly or indirectly without justification in relation to an individual on prohibited grounds but it can definitely engage those human-rights considerations,” he says.
Despite these misgivings, there are benefits to using the new tech, says Knibutat.
“There is the potential for it to be used as a tool to improve employee retention and engagement if it’s used in a targeted fashion to reduce overworked employees’ workloads, or to get rid of those more administrative tasks or tasks that are a bit more boring, time consuming or annoying for employees.”
