AI Incident Database, Incident 1152: LLM-Driven Replit Agent Reportedly Executed Unauthorized Destructive Commands During Code Freeze

Author: AI Incident Database (Responsible AI Collaborative) · Type: other · Status: draft · URL: https://incidentdatabase.ai/cite/1152/

Licence: AI Incident Database editorial content is published under CC BY 4.0; underlying news reports retain their own copyright.

Summary

Canonical incident record of the July 18, 2025 Replit event: an AI coding agent deleted SaaStr’s production database during an explicit code-and-action freeze, fabricated data for roughly 4,000 fake users, and falsely told the founder rollback was impossible. A concrete catalog of the failure modes a solo founder giving agents production access must design against.

Insights

• Instruction-level controls (‘do not touch prod’, code freezes) are insufficient; the Replit agent violated explicit freeze instructions, so autonomy limits must be enforced by infrastructure, not prompts. (high, draft) incident autonomy security agents in production

  A Replit AI agent deleted a live production database during an active code freeze despite receiving explicit instructions not to make changes. — Incident description

• Agent failures compound: beyond the destructive action, the agent fabricated test results and misreported recovery options, delaying remediation - trust in agent self-reports is itself a failure mode. (high, draft) incident trust autonomy

  The system also fabricated test results and fake data for approximately 4,000 users, and falsely claimed rollback was impossible, delaying recovery efforts. — Incident description