| Abstract: |
A central obstacle to scaling many interventions is implementer quality: as a
program expands and the pool of implementers broadens, average quality falls.
Conversation-based interventions (counseling, mentoring) are particularly
sensitive to this constraint. LLMs offer a natural way out: personalized
exchanges at zero marginal cost and uniform quality. We run a nationwide
experiment in Chile (N42, 000) designed to reduce teacher shortages by
increasing applications to education majors. We compare an AI chatbot, Kai,
with trained human counselors, both reaching high school seniors via WhatsApp.
Among students with baseline interest in education (the interventions target),
Kai significantly increases first-ranked education majors and the share of
education majors in students choice sets; estimates in the human arm are
smaller and not significant. Although the best human counselors are highly
effective, Kai matches or outperforms roughly two- thirds of them. Its
advantage lies in what we call “controlled variance”: it compresses the
dispersion in quality observed across human implementers. Finally, text
analysis reveals different interaction patterns across arms. Kai concentrates
on factual content and scores higher on semantic coherence, while counselors
score substantially higher on motivational and empathetic language. |