We gave AI agents a standard login and asked them to break LlamaParse.
Over four days, they tested every step a document goes through: parsing, extraction, indexing, deployed agents and our MCP server.
What we learned:
1/ Model behavior varies between runs, so a single test pass is not enough
2/ Some vulnerabilities only appear once you build a real application
3/ Defining expected behavior up front separates real issues from noise
4/ Extraction reads everything on the page, including text a person can’t see
Our engineering team wrote up the full breakdown below.