New research finds chatbots give less polished workplace writing when prompts use language more common among women
As AI use has advanced dramatically in workplaces, there have been a growing number of concerns and data highlighting differences when it comes to gender. The gaps can involve the actual design of the tools, the way the tools are used, or the results they produce.
For example, a recent study of 133 AI systems found that 44 per cent demonstrated gender bias, while more than a quarter showed both gender and racial bias.
“Large language models have repeatedly associated women with the home, family and childcare, while linking men to business, leadership and career success. In some cases, AI systems have generated responses portraying women as sexual objects or as subordinate to men,” says Ana Carmo writing for UN News.
“When researchers asked large language models to simply complete a sentence that began with a person's gender, about one in five responses came back sexist or misogynistic. Some even described women as property, as objects.”
Jayathma Wickramanayake, UN Women lead on digital technologies, said that AI models “pull bias from decades of text written by people, about people, in a world where women were filed under home and family, and men were filed under business and career”.
We’ve also heard that while training increases AI adoption among men, it has no measurable effect on adoption among women, according to the Prosperity Project, an organization focused on advancing women in the economy.
And other research found that women are more likely to question the quality, ethics and downstream consequences of AI-generated work more so than men.
Less sophisticated responses for women
Yet another study is providing a new concern: Since women and men often communicate differently, the AI responses they receive are also different.
As part of the research, when prompts were used containing language commonly used by women, AI chatbots including ChatGPT offered less sophisticated responses compared to those requested using language associated with men, found Johns Hopkins University.
"In American English, men and women just talk differently, and there's a number of features that are well-documented as being good discriminators between male speech and female speech," said lead author Katherine Van Koevering, an inaugural postdoctoral fellow with the university's Data Science and AI Institute.
The team took prompts for workplace correspondence — emails, job applications, and resignation letters — and added language associated with women. These included hedging ("maybe," "I think"), collective phrasing ("we," "our team") and expressive adjectives ("lovely," "wonderful"), according to a university article.
The team fed the prompts into four AI systems — GPT-4, Llama, Gemma and Mistral — and each time, the woman-associated language returned content that was less sophisticated and less formal while language associated with men produced longer, more complex and more formal responses.
"I was just so surprised by how different the responses were," Van Koevering said in the article. "Some responses were so bad I couldn't believe the model would suggest it. And sometimes I just thought, "Wow I should be more careful in my emails.'"
‘Language is hard to control’
The results raise the question: Are women in their everyday work being given different responses than men when using AI?
The researchers make one notable point: When the team tested if using a traditionally male or female name changed the AI's response, it had virtually no effect.
That suggests the issue isn't only about gender. It's about the words people use. Anyone who tends to write cautiously, with phrases like "I think" or "maybe," could get weaker results. The researchers plan to look next at whether similar effects show up across age, race and ethnicity.
However, Van Koevering is not keen to put the onus on employees.
"Language is hard for people to control," she said. "The companies need to fix the models, rather than putting all of the burden on the user."
The AI companies have more than a few challenges to cope with, so it’s possible this apparent bias won’t be going away anytime soon. But being aware of this potential issue and the impact on corporate content — from annual reports to PowerPoint presentations to RFPs — is an important step.
AI users may want to try varying the language in their prompts to ensure a variety of responses. Or consider reviewing and editing AI drafts before sending them, comparing outputs, or including the potential for this kind of bias in AI training and guidelines.
