Counterweight Labs

Research

Security review for AI-generated code: what to check before shipping

A review sequence for authentication, authorization, data handling, dependencies, and deployment changes in code written with AI assistants.

A policy check allows an authorized request to reach data and sends a denied request to a stop.
In this guide

Review AI-generated code against the application's security requirements before you review its explanation. Identify the operation being added, the identity performing it, and the data or authority it touches. Then follow the implementation through its callers, storage, and deployment configuration. A convincing summary of a change is not evidence that the required checks exist.

Keep the change small enough to understand. If a feature patch also changes middleware, dependency versions, test fixtures, and CI permissions, ask for the purpose of each change. Review the complete diff, including deleted tests and generated configuration. The most consequential edit may sit outside the feature file.

Start with the permissions the feature needs

Write the expected behavior for an allowed user and a denied user. For a report-download feature, a useful requirement is that an authenticated member can download only reports their current organization permits them to read. That requirement raises questions the endpoint implementation must answer: where the organization comes from, whether membership is current, and how the report is bound to it.

OWASP recommends denying access by default and checking permissions on every request. Apply that guidance to the server operation that returns or changes the protected object. Hiding a button in the interface does not make the operation inaccessible through its API.Source: OWASP authorization guidance

An illustrative report-download review
Implementation detailReview question
Report lookup by identifierDoes the query constrain access to the authenticated user's permitted records?
Organization from request inputIs that organization independently authorized for the current user?
Cached download linkCan the link outlive the user's access or be reused by an unintended recipient?
Background export jobDoes the worker preserve the requesting identity and recheck required permissions?
A verified user, an authorized tenant, and a requested report ID feed a server-side read-permission check. An in-scope report can be returned; another tenant's report is denied.
Illustrative report-download review. A client-supplied identifier selects a requested resource; the backend must independently establish identity, tenant access, and permission to read that resource.

Follow data to the operation that consumes it

Trace external input through parsing, validation, transformations, storage, and output. Check what the receiving operation expects. A field accepted by the user interface may still be unsafe in a database query, shell invocation, template, or outbound URL. Review errors and logs as well: an otherwise rejected request should not expose secrets through a diagnostic response.

Use concrete examples from the feature. If a user can supply a callback URL, determine which destinations the service is allowed to contact and where that restriction is enforced. If the feature accepts uploads, check the allowed types, storage location, download permissions, and downstream processors. Give each sensitive operation an explicit owner in the review.Sources: OWASP: server-side request forgery prevention; OWASP: file upload validation and storage

Inspect what the assistant added around the code

OWASP's Secure Coding with AI guidance calls attention to risks in the development process itself, including untrusted repository content, tool permissions, dependency suggestions, and changes to tests. Review the assistant's resulting artifacts and the authority it had while producing them; these are separate from the behavior of the shipped application.Source: OWASP Secure Coding with AI

  • Inspect new package names, maintainers, versions, and lockfile changes before accepting an added dependency.
  • Check for credentials in source, test fixtures, examples, generated files, and logging statements.
  • Review CI and deployment changes for broader permissions, skipped checks, or new network dependencies.
  • Compare changed tests with the original requirements, including tests the patch removed or weakened.

Test the boundary independently

For the report example, prepare one user with access and another user without it. Test the server operation directly, including an identifier belonging to another tenant. Verify both the response and whether any background work or data change occurred. An error response can conceal a side effect that happened before the error was returned.

Ask a reviewer to derive the denied cases from the requirement before reading the generated tests. Otherwise the test suite can inherit the same missing assumption as the implementation. Preserve a regression test for each confirmed security issue and check that the test detects the issue in the original version.

Make the release decision explicit

Record the reviewed commit, the sensitive paths examined, the checks performed, and the remaining uncertainty. Separate confirmed findings from questions that need a broader assessment. If a critical access path could not be exercised because its dependencies were unavailable, give it an owner and a follow-up decision instead of treating the review as complete.

Keep the handoff short enough for the release owner to use: the blocked issues, the accepted constraints, and the evidence supporting the paths that are ready. For changes spanning several services or tenant boundaries, a focused code review can be paired with a penetration test of the deployed behavior.

Sources

  1. OWASP authorization guidance cheatsheetseries.owasp.org
  2. OWASP: server-side request forgery prevention cheatsheetseries.owasp.org
  3. OWASP: file upload validation and storage cheatsheetseries.owasp.org
  4. OWASP Secure Coding with AI cheatsheetseries.owasp.org

Bring the repository and the workflows you are about to ship. We can review the security boundaries, validate findings, and help your engineers address them.

Explore application security
Book a consultation