While many U.S. workers fear that artificial intelligence is coming for their jobs, researchers say the rapid advancements of tools like ChatGPT and Claude haven't had a major impact on the overall labor market – at least not yet. 

The effectiveness of generative AI platforms to instantly write emails, analyze data and create travel plans can be wildly impressive, but a recent Remote Labor Index study that put these bots to the test showed just how far they still have to go to match the capabilities of humans. 

'LIKE A DAVID VS. GOLIATH FIGHT': Communities take on Big Tech, government over AI data centers 

Researchers tested how effective different company's AI models were at doing hundreds of freelance projects that people had been paid to do. Tasks included product design, architecture, data analysis and video animation. 

The top-performing system (Manus) successfully completing only 2.5% of the jobs, according to the study from Scale AI and the Center for AI Safety.

"Current models are not close to being able to automate real jobs in the economy," Jason Hausenloy, one of the researchers, told the Washington Post.

Despite these shortcomings, employers are scrambling to find ways to implement these tools and a Reuters/Ipsos poll from August found that nearly three-quarters of Americans believe AI will be "putting too many people out of work permanently."

In Philadelphia, the City Council held a hearing in October on the potential government applications of AI. Officials said they hope to release guidelines this spring on how city employees can use the technology on their jobs.

"If anyone in the world would say that they were an AI expert, they would be lying, because it's continuously transforming. It's morphing into new technology," Melissa Scott, the chief information officer for the city's Office of Innovation and Technology, said at the hearing. "So you can be an expert on where AI is today, but three months from now it can be another way."

'Only so far you can go'

Trying to understand the future impact of artificial intelligence might come down to who you ask. While computer scientists tend to be maximalists, saying AI is going to change everything, most economists believe this is just the latest advancement in tech, and jobs will adapt just like they have with invention of the steam engine, assembly line, computer, smartphones and everything in between. 

University of Pennsylvania professors Ioana Marinescu, an economist, and Konrad Kording, a computer scientist, sought to bring these two conflicting ideologies together in their research published in November. "We're trying to explain where these two camps are coming from to have such radically opposite conclusions" Marinescu said. "So we did theoretical work in economics to try to ask what assumptions are they making that are different."

The professors developed an interactive model that split the economy into two fields: the intelligence sector and physical jobs, which includes construction workers, nurses and restaurant staff. The necessity of both and the fact that AI isn't really being deployed in the physical sector limits the technology's overall impact on the economy, they said. 

"One of the key messages of our paper is that if you have complementarity – both the physical and the intelligence – then the impact of intelligence on growth and on wages is going to saturate, meaning there's only so far you can go," Marinescu said.

If tech jobs become more automated because of the use of AI, it could cause a major shift from intelligence to physical jobs. Marinescu said there hasn't been much evidence yet that this job migration is happening.

What jobs are being affected?

Looking at translators and customer service representatives can show the distinction between what AI is currently capable of doing and what it does well. While creating subtitles on TV and movies, for example, is a static task that can be done through automation, having AI handle online chats can prove problematic, even if it has become a common practice for many companies. 

"The bot can do well in many of the more standard cases, but when you have a more outside-the-box case, businesses have found that the robots aren't very good at handling those," Marinescu said. "... It's not that easy to replace customer service because of these edge cases. Even though there's not many of them, it can end up eating the business out of money if you treat them the wrong way. 

"I suspect that eventually they might figure it out, like maybe only get the real people to deal with those complicated cases. ... But let's put it this way, it's not so easy."

As employers navigate how to implement this new technology into their industries, the ability to effectively use AI is becoming a highly sought-after attribute for hiring managers. 

Locally, there were 10,815 job postings in the area that required AI skills in 2024, according to a July report from Brookings. The study ranked nearly 200 metro areas in the country based on their readiness to benefit from AI, and Philly came in at No. 14. The region was included in the second tier behind Silicon Valley and was given the designation of a "Star Hub" with "uniformly strong AI ecosystems, balancing top‑tier talent, research, and enterprise uptake." 

In some jobs, coexisting with these emerging technologies has proven challenging. Sorelle Friedler, a professor of computer science at Haverford College who served as the assistant director for data and democracy in the Biden administration, said one sector AI has been very disruptive is her own. She said she doesn't allow her students to use these platforms at all, unless it's a class on building or learning about AI.

"Part of the learning process is taking that time to struggle ... and I think that stands really opposed to what AI systems can do well and how these systems work because the goal of the system is to really efficiently get an outcome right," she said. "... Students need to learn all of these steps on their own."

As for Marinescu's field of work, she said AI "can do OK" in some aspects of conducting and analyzing research, but it can't quite effectively complete an entire paper. 

"That's one of the elements why I think we don't see a quicker effect (of AI)," she said. "Because it's one thing to say, 'Oh, in theory, could AI do your job?' Yes, maybe it can do part of it, but if you can't cover that last mile, it can still be hard to deploy."

Human-AI collaboration in the workforce

Researchers aren't just interested in finding out if AI can replace humans in the workforce, but also how these platforms are impacting people who rely on them to complete mundane tasks.

The Harvard Business Review published a study last year analyzing workers' production when they use AI on common duties, including crafting emails, writing social media posts and brainstorming. The study, which included more than 3,500 participants working with and without the tools, had mixed results.

While researchers said AI enhanced some quality and efficiency, they warned that there were psychological costs, reporting an 11% drop in intrinsic motivation and 20% increase in boredom on average.

"I think that it can be really hard to know what your own efficiency is, let alone what a full company's efficiency is when using these tools without really careful measurement," said Friedler, who was not involved in the study.

One example of an organization that is successfully implementing AI into its day-to-day operation is Veterans Affairs, Friedler said. The federal agency is starting to utilize the technology as a way to address its infamous backlog of disability claims.

To determine if a veteran is eligible for benefits, VA staffers have to go through loads of paperwork that include service records, medical history and testimonies. There are hundreds of thousands of backlogged claims, meaning those veterans and their dependents are caught in bureaucratic limbo waiting to find out if and when they will receive payments. 

To help chip away at this stockpile, the VA has turned to AI. The agency said that "computer-based tasks, such as summarizing documents, routing information and responding to routine inquiries," will be automated within the next few years.

"Obviously, the VA is itself an important use case, but I also think that many of the other successful use cases have this same type of characterization," Friedler said. "Someone has looked and said, 'How can we sort of easily say yes to a lot of people for the things that we know we're going to approve anyway, and reserve human time for these more complex judgments?'"

AI is here to stay, and understanding its full potential might be the best way to prepare for its impact on the workforce.

If tech jobs become more automated because of the use of AI, it could cause a major shift from intelligence to physical jobs. Marinescu said that while there are early signs it might be getting harder for younger people to get jobs in the intelligence sector, there hasn't been much evidence yet that this job migration is happening.