Reviewing job applications can be hard, time-consuming work – especially when you have a long list of applicants.

And in the last couple of years, AI large language model (LLM) tools like ChatGPT, Claude and Google’s Gemini have developed to the point of being excellent at analysing text, which can make them appealing options to cut down the hours and effort of reviewing and shortlisting your job applications.

(Many Applicant Tracking Systems (ATS) have also introduced AI shortlisting features with the same goal.)

But while AI can help automate many once-manual processes, should reviewing applications be one of them?

Experts warn that AI tools have particular biases baked into how they work, so using AI to decide who gets shortlisted for jobs can introduce regressive biases, potentially expose your organisation to legal risks and can lead to otherwise excellent candidates being overlooked or excluded.

Professor Alysia Blackham from Melbourne Law School is one of the world’s leading experts in age discrimination law. In a recent interview, she explained why employers should be cautious about relying on AI to screen job applicants.

1. AI can reinforce age bias

Blackham’s research found that LLMs don’t simply process information objectively – they can reproduce existing stereotypes and assumptions about age.

“In my study, when I asked ChatGPT to recommend age groups for employees in the tech industry with ‘enthusiasm’ and ‘new ideas’, it focused on younger workers and excluded consideration of older workers,” she said.

She also found that while Australian employers often describe “older” workers as those aged 50 and over, ChatGPT suggested someone could be considered “old” from around age 45.

“If these [applicant shortlisting] tools are based on large language models, there is a very real risk that that bias is built into the tool.”

Rather than removing bias from recruitment, Blackham warned that AI can reproduce and amplify the assumptions already present in the data it has been trained on – making those biases harder to spot and easier to apply at scale.

2. AI can reinforce gender bias

AI can also reinforce existing gender biases against women, especially women who have had a career break for parental leave. Blackham notes that this isn’t unique to LLMs: “Essentially, they are amplifying biases that already exist in society.”

A 2025 study by Stanford academics and published in the journal Nature found that inaccurate stereotypes about older women are perpetuated and amplified by large language models (LLMs).

The researchers asked ChatGPT to evaluate the quality of more than 34,500 unique resumes for 54 occupations that were generated for the research. It gave older men the highest ratings – even when they were based on the same initial information as women’s resumes.

That suggests that “AI-based tools employers may use to review resumes may give older men an advantage while putting older women and younger job seekers at a disadvantage.”

Other research from the University of Melbourne found that “when it comes to jobs, AI does not like parents” – especially women who are much more likely to take career breaks to have children.

The researchers found that

“When we added in a gap for parental leave, we found that ChatGPT ranked our parents lower in every single occupation. This was true for fathers and mothers – a gap for caregiving leave told the algorithm that this person was less qualified for the job.”

“It’s not that the AI knows someone’s age or gender,” Blackham said. “It’s inferring those characteristics from other information in the application.” That means employers can’t assume removing a candidate’s date of birth or other identifying details is enough to eliminate bias if an AI tool is making decisions.

3. Discrimination is illegal, and could expose your organisation to legal risks

Australian employers cannot discriminate against applicants on the basis of protected characteristics such as age or gender.

While Blackham believes Australia’s discrimination laws are theoretically capable of discouraging AI-assisted discrimination, proving that discrimination has occurred can be incredibly difficult.

As a jobseeker, “how do you know if you’ve been discriminated against?” she asked. “And how do you know that it’s been done by an AI tool if you don’t even know that an AI tool is being used?” Unlike some countries and states, Australia currently has no requirement for employers to disclose when AI or automated recruitment tools are being used.

She also pointed to a class action underway in the United States against recruitment software company Workday over allegations of age discrimination by an AI recruitment tool against people over the age of 40. While the case has yet to be decided, Blackham said it could have significant implications for employers and software providers by clarifying who is responsible when AI-driven hiring decisions discriminate.

Using AI doesn’t transfer responsibility to the technology provider. If your recruitment process produces discriminatory outcomes, your organisation could still be legally at risk if challenged by an applicant who has been discriminated against.

4. You could miss out on your best candidates

Perhaps the biggest risk isn’t legal – it’s practical.

If AI filters out applications from experienced workers, women with kids and even people with a disability, you may miss out on some of your strongest applicants, putting your organisation at a real disadvantage.

For NFP employers, that could mean overlooking candidates with valuable lived experience, transferable skills or non-traditional career paths – qualities that don’t always fit into the patterns AI has been trained to recognise.

The people who could bring the greatest value to your organisation may be exactly the ones an algorithm ranks lowest.

Plus, the wrong hire can seriously cost your organisation, with estimates of the cost of hiring the wrong person ranging between 15%-21% of that employee’s salary, depending on the seniority of the role. That’s a lot to pay for saving a few hours reading CVs and cover letters.

—

As time pressed as many NFP hiring and HR managers are, recruitment isn’t simply about efficiency. It’s about finding people with the skills, values and lived experience to deliver on your organisation’s mission.

AI is likely to become a bigger part of recruitment as organisations look for ways to reduce administrative workloads. But Blackham’s research suggests these tools are best used to support hiring – not to make decisions. They can help draft job descriptions, prepare interview questions or summarise notes, but deciding who progresses through a recruitment process should still be a task for human judgement.

—

Not-For-Profit People is an initiative of EthicalJobs.com.au – Australia’s top job-search site for the not-for-profit sector and beyond. More than 10,000 Australian charities, not-for-profits and social enterprises use EthicalJobs.com.au to find dedicated and passionate staff and volunteers to help them work for a better world. Find out more at EthicalJobs.com.au/advertise.