For years, research firms such as Gartner, McKinsey and Accenture have predicted a shortage of data scientists in the near future — ranging from 100,000 to 190,000 in the United States.
What’s behind the scarcity? Big data continues to evolve and, as a result, big data professionals continue to specialize.
This has created an increased demand, according to Stan Matwin, director of the Institute for Big Data Analytics in Halifax, adding there could be a shortfall of as much as 60 per cent by 2018 in the U.S. and Canada’s shortages will be about 10 per cent of those in the U.S.
To fill the void, organizations are turning to existing employees. Analysts, statisticians and economists are particularly well-suited to working with big data, said Matwin.
“Employees already inside the enterprise can be trained,” he said. “People with degrees in computer science, math and even physics make good data scientists.”
This is significant, said Matwin, because big data — the acquisition, storage, processing and analysis of large data sets — is increasingly being used by HR professionals to develop a deeper understanding of an organization’s operations, employees and customers.
Everything from efficiency to collaboration is being examined through the use of big data, according to Peter Smit, founder and CEO of Collabogence in Toronto.
“There are multiple uses that are very effective,” he said. “Different companies use this data for very different things.”
For example, the same metadata used to measure collaboration within a company could also be used to examine the unauthorized access of information within an organization, he said. Or that data could be run through sentiment analysis to determine how employees are feeling about an announcement such as a divestiture or a restructuring.
Big data is an important tool for HR professionals, but there is still a lack of understanding, said Smit.
“It’s the issue of the difference between data science and analytics.”
Data science is the actual nitty gritty of working with numbers while analytics derive insight or intelligence from that data.
“In a human resources application, for instance, the data is looking at issues like how many employees take all of their vacation days or how many people get sick and how many times each year,” said Smit. “It’s in the application that you derive some sort of conclusion or insight from that data.”
The growing concern about data scientist shortages refers to data science more so than analytics, he said.
“If you run a search on data scientists, you’ll hear there’s a shortage. These are the people who actually have the technical skills to pull data sets out and create data lakes,” said Smit.
“Before you throw your data in the lake, you want to normalize it and cleanse it so it makes sense. So you really need a data scientist to do the extract, transform and load — or what we call the ETL.”
While the shortage must be addressed by increasing awareness of and interest in STEM disciplines — science, technology, engineering and mathematics — the issue of analytics can be addressed within an organization, he said.
“Once you have your data, then you want to derive meaning. You need to manage it, you need to develop algorithms to address particular questions,” said Smit. “You cannot derive data without an understanding of the business problem. We have to have people who understand our business and our procedures, how we do things and the dynamics of the business in order to be able to formulate the issue that needs to be addressed or needs to be investigated.”
By training existing employees in analytics, organizations can guarantee their data will be handled by someone with a deep understanding of the company’s specific needs and goals, he said.
The importance of building a data science team within an organization cannot be underestimated, said Abidin Akkok, project director for Canada’s Big Data Consortium in Toronto.
“Considering that data science is a multi-disciplinary domain, most of the time, it may not be possible to have one person who will be the jack-of-all-trades,” said Akkok. “Therefore, companies need to build data science teams consisting of employees with expertise in different fields such as math, computer science, statistics, operations research, machine learning and domain expertise. Companies may employ a variety of strategies depending on the size and urgency of the problem.”
A focus on citizen data scientists will also alleviate the growing skills shortage, said Akkok.
“This requires collaboration between government, industry and academia. Labour market clarity needs to be improved by establishing common professional definitions and career pathways. Employer demands need to be better met by building more of the right types of talent, and the existing talent needs to be leveraged by sourcing and growing talent internally rather than only recruiting new talent.”
While the world of big data can be intimidating, the tools associated with data science and analytics, specifically, are becoming more user-friendly, said Smit.
“The tools with which we gather the data and examine the data have become better and more sophisticated and simpler,” he said. “Once the data is formatted and it gets thrown into a repository on a regular basis, somebody like a marketing analyst could sit down with a tool and visualize the data… Interpreting the data and looking for patterns in the data has become more sophisticated and actually requires less specialization. Less smarts are required.”
Organizations should focus on training employees to be data-literate even if they aren’t planning on taking advantage of everything big data has to offer, said Smit.
“People want more flexibility in terms of when and where they work, so more people are working remotely and doing all sorts of things,” he said. “If you’ve got people scattered across multiple locations, they’re never going to see each other but they have to work together. That’s all technology-dependent. If you’re in a work environment, you need to understand what the capabilities are.”
Another reason many employers are focusing on training employees to be citizen data scientists is to increase security.
“Everybody is worried about breaches and privacy and all sorts of issues around that,” said Smit.
Companies that hire a consulting firm or data specialization firm to handle data science are increasingly turning to employees to analyze that data rather than let information out of the in-house data centre, into the hands of an outside party.
Additionally, more employees are expressing a desire to work with big data, he said.
“On a much broader basis, people are feeling more comfortable with all the big data that is around. The edge of ‘Big Brother’ has come off. There’s corporate governance, information governance and access control. So there’s less concern about it,” said Smit.
“The second piece is that there’s a stronger appreciation for the value that can be extracted from the data. Big data is actually allowing us to develop a path with which we can help organizations get better. If you can’t measure it you can’t manage it, and you certainly can’t get better at it if you don’t know where you start.”
© Copyright Canadian HR Reporter, Thomson Reuters Canada Limited. All rights reserved.