Published paper available on arXiv:
A paradox of AI fluency

Bigspin analyzed 27K richly annotated transcripts from the WildChat-4.8M dataset and found that high-fluency users fail 64% of the time, while low-fluency users fail just 24% of the time.

The catch: 86% of novice failures are invisible, and 59% of expert failures are caught and recovered. Fluency is about engaging critically with AI – pushing back, asking questions.

For teams building AI products, this shifts the understanding of what success user success looks like within these products.