Navigating the Jagged Technological Frontier: Field Experimental Evidence of the Effects of Artificial Intelligence on Knowledge Worker Productivity and Quality
AI helps consultants on some tasks and hurts on others within the same workflow: a 'jagged technological frontier'.
Summary
In a preregistered experiment, 758 BCG consultants (about 7% of its individual contributors) were randomly assigned to work without AI, with GPT-4, or with GPT-4 plus a prompt-engineering overview. On 18 realistic consulting tasks inside the technology's capabilities, consultants using AI completed about 12% more tasks, worked about 25% faster and produced work rated over 40% higher in quality, with the largest gains for the weaker performers (43% against 17%). On a task deliberately set outside its capabilities, integrating subtle data with interview evidence, AI users were 19 percentage points less likely to reach the correct answer (60 to 70% against 84.5%) while still producing persuasive recommendations. Tasks of seemingly similar difficulty could therefore go either way, which the authors call a jagged technological frontier. Successful users behaved either as centaurs, dividing work between human and AI, or as cyborgs, interweaving the two at a fine grain. AI-assisted output also showed less variety of ideas.
Editor's opinion
AI does not lift every task: it makes knowledge workers markedly better on some tasks and worse on others that look just as hard, and workers cannot easily tell which is which. Organisations should map their workflows to see where the frontier lies, train people to judge it, and keep human checks where AI output is persuasive but may be wrong.
Key concepts
jagged frontier · GenAI · knowledge workers · field experiment · consulting