Vibe coding for statistical analysis using large language models

In plain language

“Vibe coding” means asking an AI model, in everyday language, to write and run code for you instead of writing that code by hand. We tested whether this approach can carry out a real statistical analysis. Using a small, clean data set, the AI produced code that checked and cleaned the data, ran a logistic regression, and made charts. The main results matched a trusted published example, but one output, a confidence interval for a predicted probability, was wrong, and we did not catch it on our first pass.

About this study

We treated vibe coding as a proof of concept and worked through a deliberate, human-in-the-loop workflow: framing the research question, defining the variables, organizing project folders, documenting each decision, using retrieval-augmented generation, and applying prompt-engineering techniques. Working in Cursor (v1.5.11), we modeled admission status as a function of Graduate Record Exam scores, grade point average, and undergraduate rank. The AI generated the analysis in R, and we compared the output against a published reference posted on a public website.

The results were mixed. Regression coefficients, p values, and model-fit statistics matched the reference output, yet an error appeared in the predicted-probability confidence interval that slipped past our initial review. The lesson is that vibe coding can lower the barrier to analysis, but it should be used with caution: structured verification and safeguards, transparent reporting, and appropriate statistical and methodological oversight.