Researchers develop a new AI-powered tool to identify and recommend jobs

To make the job transition process easier, and increase the chances of success, researchers from the University of Technology Sydney (UTS) and UNSW Sydney have developed a machine learning-based method that can identify and recommend jobs with similar underlying skill sets to someone's current occupation.

The system can also respond in real-time to changes in job demand and provide recommendations of the precise skills needed to transition to a new occupation.

Developed by Dr Nikolas Dawson and Dr Marian-Andrei Rizoiu from the UTS Data Science Institute and Professor Mary-Anne Williams, the Michael J Crouch Chair in Innovation at UNSW Business School, the system is based on findings from their new study, "Skill-driven Recommendations for Job Transition Pathways," published in the international journal PLOS ONE.

What are the benefits of using AI to find a job?

Dr Dawson says while workplace change is inevitable, if we can make the job transition process easier and more efficient, there are significant productivity and equity benefits not only for individuals, but also for businesses and government.

"It can be a daunting proposition to switch to a new career, particularly for those who have been in the same job for a long time. Successful transitions typically involve workers leveraging their existing skills, and acquiring new skills, to meet the demands of the new occupation," he said.

Professor Williams says the new recommender system can help to reduce the inevitable stress during times of job loss by lowering the costs of job transitions and providing evidenced-based recommendations that better met the needs of individuals with specific skill sets that often transcend their occupation.

"By focusing on skill sets, rather than occupations, this new approach helps workers, organisations and businesses like retraining advisory services discover the new skills a person would need to acquire to obtain a new in-demand job and assess the associated training investment required," she said.

"In addition, organisations can use our skill similarity measure to design completely new or hybrid occupations that increase the likelihood of finding people with the necessary skill set.

"In the current rapidly changing job market the need to continuously upskill is a challenge for individuals and organisations. Our recommender system can help individuals embrace change by proactively designing their lifelong learning journey and to react to new more exciting job opportunities as they arise by determining the next best skill to acquire."

Dr Rizoiu added: "If we can move towards skills-based hiring, rather than defining an occupation by its job title, then we can help people identify the specific skills they have, or need to develop, in order to find productive and meaningful work."

How was the job-search method created?

The researchers used valuable data from Burning Glass Technologies, an analytics software company that provides real-time information on jobs and labour market trends, to examine and parse the underlying skill sets of more than 8 million jobs advertised in Australia between 2012 and 2020.

They then compared the job transition predictions with data from the Household, Income and Labour Dynamics in Australia (HILDA) survey, which tracks participants over the course of their lives, to validate these predictions with nearly 3000 real-life examples.

The jobs recommender system accurately predicted job transition probabilities and was also able to show whether it is easier to move in one direction than another.

The methods developed in the study can be leveraged by educators, government and business, potentially with data from the Australian Bureau of Statistics, to support industries and sectors undergoing significant upheaval to transition workers at scale.

As part of the study, the researchers also built an early warning indicator of emerging technologies (such as artificial intelligence) that have the potential to disrupt labour markets. This information could allow policymakers and businesses to better prepare for future structural shifts.

Dr Dawson undertook the study as part of his PhD in computational economics at UTS with Professor Williams and Dr Rizoiu. He now works as a senior data scientist at FutureFit AI, a company that partners with industry and government to provide an AI-powered tool to help workers navigate career transitions.

"If you look back in history, it's almost never the case that there are fewer jobs due to automation, but rather new jobs are created at the same time old ones disappear. So it is fundamental that people have the ability to build the requisite skills and transition smoothly into these new jobs," Dr Dawson said.

"The ability to undertake micro-credentials in specific skill areas, customised for the individual, will likely be a key part of this future."