Skip to contentStart your project
All insights

Auditing an AI-built app for production: what the prompt misses

A working demo is a starting point. This worked example shows how to review access, payments, data and recovery before an AI-built app meets real customers.

A cutaway pavilion reveals its foundations under a focused inspection light.
Editorial illustration generated with AI for Hilvy.

The demo works. You can sign up, create a project and invite a colleague. The interface is polished enough that the next question feels obvious: can we launch it?

That question needs a different kind of review. A prompt usually describes the experience you want to create. A production review also asks what must never happen, what happens twice and what happens when part of the system stops responding.

This is a worked example of an audit approach, not a claim that we audited a particular client’s app. Imagine a small, AI-assisted client portal with separate company accounts, file uploads and a paid subscription. The findings below are checks to perform, not vulnerabilities we have discovered in a named product.

Start with the boundaries, not the screens

Draw the actual system: browser, application server, identity provider, database, file storage and payment service. Note which component decides who can do what. A convincing interface can conceal an uncertain answer to that question.

Use separate test organisations and ordinary member accounts in a staging environment. Write down the expected access rules before testing them. A member of one company should not gain access to another company’s records simply because they know a record identifier.

OWASP’s authorisation guidance recommends least privilege, denying access by default and checking permissions on every request. In this example, the important evidence is server-side enforcement, not whether a button disappears.

Look for promises made only by the interface

Perhaps the app displays “Admins only” beside an export button. That is a useful cue, but it is not an access rule. The operation itself must reject a request from someone without permission.

The same distinction applies to limits. If the free plan allows three projects, where is that limit enforced? If a file must belong to the current organisation, which layer verifies that relationship? If an invitation expires, can the backend still accept it?

For each promise, record the controlling code or configuration, the test and the result. “Looks right” is not an audit finding.

Ask what happens when an event arrives twice

Payment and integration workflows need to tolerate events that are repeated, delayed or delivered while another dependency is unavailable. In our example, processing the same subscription event twice should not create duplicate entitlements or send contradictory account messages.

Stripe’s webhook documentation covers signature verification and handling duplicate events. For a portal using Stripe, those belong in the implementation review. A successful checkout screen alone does not demonstrate that subscription state stays correct.

Trace an event from receipt to the resulting account change. Check how it is identified, what is stored and how a failed attempt can be retried safely. Test with provider test tools and non-production data, not by creating real charges.

Follow the data beyond the database

A privacy review cannot stop at the primary records. Uploaded documents may appear in temporary storage, error logs, notification emails and support tools. If AI features process them, another provider may enter that path.

List the information collected, its purpose, where it travels and who can access it. Decide how long it is retained and how deletion works across relevant systems. Avoid sending real customer files through a prototype while these decisions are still undefined.

Then inspect logging. You need enough information to investigate a failed request without routinely recording secrets, complete documents or unnecessary personal details. An error message should help an operator, not expose internal information to a visitor.

Run a recovery rehearsal

Backups are only one part of recovery. Can someone identify the last good deployment, restore the required data and verify that the application is usable afterwards? Who makes that decision?

In a safe test environment, rehearse a failed deployment and a representative restore. Record the steps, dependencies and observed recovery time. Do not promise a recovery target simply because a hosting dashboard contains a backup setting.

Also test the quieter failures: a notification service is unavailable, a form takes too long, an upload is rejected. Users should receive a clear state and a safe next step.

Turn findings into a launch decision

Each finding needs a consequence, evidence, an owner and a retest. Separate a launch blocker, such as broken access isolation, from an improvement that can safely follow launch, such as a clearer empty state.

  • Block: unresolved access, payment integrity or sensitive-data exposure risks.
  • Resolve before wider use: missing failure handling, monitoring or a usable recovery process.
  • Schedule: non-critical usability and maintainability improvements with an accountable owner.

This is not a complete security assessment or a guarantee of safety. The scope should expand with the product’s data, users and consequences of failure. Specialist review may be necessary.

AI-assisted development can get a useful idea in front of people quickly. Production readiness comes from evidence about the less visible parts. Our app development work and seven product guardrails start from that distinction.

Want a hand with your website?

We design, build and look after Webflow websites. Tell us what you’re working on.

See our work