How AI-Assistants Affect Skilled Workers: The Case of Customer Service Representatives

Introduction

New research suggests that generative AI could affect workers at different skill levels differently. In a recent study conducted by researchers at Stanford and MIT, over 5,000 customer service representatives at an unnamed Fortune 500 company were given an AI chat program to enable them to provide better and quicker answers to customers. However, while it drove an average productivity increase of 14%, the researchers found that the higher-skilled agents saw almost no improvement to their productivity from using the program.

What the Study Found

According to Erik Brynjolfsson, a professor at the Stanford Institute for Human-Centered AI and one of the study’s authors, there were “close to 0%” productivity gains for the high-skilled workers in the study. “What’s happening is that the system is actually capturing a lot of the expertise, the tacit knowledge of those high-skill workers, and making it available to the less-experienced workers, and that’s bringing them up,” Brynjolfsson told Insider. “

The study found that the AI tools are trained to mimic the effective customer conversation styles that highly-skilled workers are demonstrating. The AI tool was trained to understand features like the duration and outcomes of calls, and how top performers fared in those aspects. The AI model is trained on human-generated data in a setting where there is high variability in the abilities of individual agents. As a result, when the model identifies patterns that distinguish successful from unsuccessful calls, it is implicitly learning the differences that characterize high- versus low-skill workers.

Implications for Skilled Workers in Customer Service

Although the study shows potential benefits in less-skilled customer support agents using AI assistance, Brynjolfsson said he could “imagine ways that high-skill workers would also benefit.” Customer service is a particularly ripe area where companies are deploying artificial intelligence technology. From an AI’s perspective, customer-agent conversations can be thought of as a series of pattern-matching problems in which one is looking for an optimal sequence of actions. The AI in the case of the study involved a system focused on customer conversations, which had incorporated a recent version of GPT with additional machine learning algorithms.

The Future of Work with AI-Assistants

Brynjolfsson said that generative AI could be helpful for brainstorming ideas, or for creative purposes. Although there are still problems like hallucination and factuality, which refer to a phenomenon where AI tools authoritatively provide false responses, he believes having a human in the loop is essential. The collaboration of human and machine enables better outcomes than either working alone.

Editor Notes

As labor markets are continuously evolving, technological advancements pose threats to the livelihoods of skilled workers. However, the study shows the potential of generative AI to help less-skilled workers improve their skills and productivity, thus increasing their job security. The collaboration of human and machine is necessary for optimal outcomes, and GPT News Room aims to keep you updated with the latest news and insights in the world of AI. To learn more, visit GPT News Room.