AI code review: what GitHub, Endor Labs, IBM and Wiz say to check

One phrase, two directions

The phrase points two ways, and the pages we read split along that line. IBM describes tools that evaluate code for quality, style and functionality with machine learning models, and says many platforms begin with static analysis: linters and scanners that follow predefined rules. Wiz describes the same kind of tool, AI models that read a code change and flag bugs, style problems and security issues, usually inside a pull request. It calls the result a complement to human review, “not a swap for it”.

GitHub and Endor Labs face the other way. GitHub’s tutorial is about verifying and validating code that tools such as Copilot, ChatGPT or other agents produced, and it gives eight steps. Endor Labs, a security vendor whose page closes with an offer of a demo, gives six steps for the same case; its page is dated 2 September 2026. Wiz is also a security vendor, and IBM’s page promotes its own Bob, so each of the four has a product behind its text.

Read side by side, they share a stance. IBM calls the output of a review tool a proposal that still needs a person to verify it, and Wiz says to read each finding as a suggestion to check, not a verdict to obey. GitHub tells the reader to be skeptical of code that “looks right” but does not match the intent, and Endor Labs treats generated code as untrusted until it passes the same security checks as code a person wrote.

This page takes the second direction first, because that is the check waiting when a model has written the change, and gives the tool-side view in its own section further down. How such a tool decides what to read, and what it keeps in view, is a different question; our page on AI code generators covers what Claude Code, Cursor and GitHub Copilot say about it.

Before anyone reads a line: tests and scanners

GitHub’s first step is mechanical. It says to run automated tests and static analysis tools first, to make sure the code compiles and every test passes, and to look for new warnings or errors. For vulnerabilities and dependency problems it names CodeQL and Dependabot, and it names GitHub Code Quality for reliability and maintainability issues and for code coverage. Only then does it ask for judgement: its second step is to check that the code fits the purpose and architecture of the project, with the question “Does this code solve the right problem?” and a second one about whether it follows the team’s conventions.

For that second step GitHub suggests using the README, the docs and recent pull requests as context, and it offers an example prompt that asks what assumptions about business logic, design preferences or user behaviour the code has made. The third step is about the reading itself: readability, maintainability and clear naming, and a warning against accepting code that would take longer to refactor than to rewrite.

Endor Labs starts earlier and from the other end. Its first step comes before the code exists: put the security requirements in the request, such as input validation, parameterized queries, authentication checks and no hardcoded secrets, because its page says a vague prompt invites gaps. Its second is a scan on every change, in two places. In the IDE the developer gets instant feedback, and in continuous integration the same checks act as a gate before the merge. It names three kinds: static application security testing of your own source, software composition analysis of the open-source dependencies, and secrets scanning for keys and passwords that should not be in the repository.

So the two pages differ on where to begin: GitHub with tests and static analysis, Endor Labs with the prompt and then the scans. Endor Labs also ranks findings by reachability, meaning whether the flawed code can actually be reached, and asks the team to write its rules down; GitHub ends in the same place, with its own rules in CONTRIBUTING.md. GitHub puts the automated checks before the human reading; Endor Labs scans every change and keeps human review for the risky ones, so for both the checks sit beside the person and do not replace one. IBM and Wiz do not give an order for these checks in the text we read.

Mistakes that belong to a model

GitHub’s fifth step opens with the sentence that AI tools can make unique mistakes. The list is short: hallucinated APIs, ignored constraints and incorrect logic, and “tests that are deleted or skipped” instead of being fixed. It adds the instruction to be skeptical of code that looks right and does not match the intent. Its example prompts turn two of these into questions to ask the tool: what was the reasoning behind a change that deletes a failing test, with alternatives that would fix the test, and which edge cases or scenarios the code might not handle correctly.

Endor Labs lists a different set. When a prompt does not ask for security, it says, the model often leaves it out, which can mean missing authentication, missing access control or missing input checks. It gives two examples of flaws in how the parts fit together: a new endpoint with no authentication, and broken protection against cross-site request forgery, with “the code still looks valid”. Its page says rule-based scanning looks for known bad patterns, so a normal static scan can miss that kind of flaw, and that is the reason it gives for keeping context-aware review.

Where should a person look hardest? Endor Labs says to keep human review for high-risk changes, and defines them as changes that touch authentication, cryptography, sensitive data handling or a new external endpoint. It gives a reason for the extra care that is about people, not code: the output is polished, so it is easy to trust, and developers tend to review code that reads cleanly less closely than code a teammate wrote. Its remedy is process: clear review thresholds, so that trust does not depend on how confident the output appears.

Wiz names the same human risk on the tool side, and the next figure sets these mistakes side by side with the page that names each one.

Packages and keys

GitHub’s fourth step is about dependencies. It asks whether each suggested package exists and is actively maintained, where it comes from and who contributes to it, and whether its licence fits: it gives the example of AGPL-3.0 in an MIT-licensed project, or a dependency with no declared licence. It also warns about packages that do not exist, and about slopsquatting, which it describes as “a theoretical attack” that uses fake or malicious packages. Endor Labs says the same to check, before installing anything the AI suggested: that the package exists, is maintained and has no known serious issues. It adds a list of the components of the software, an SBOM, so you know what you shipped, and it says this step deserves extra care because AI dependency picks are often unsafe.

The two pages disagree about how real the danger is. Endor Labs describes slopsquatting as attackers registering the fake names that models tend to invent, so that whoever installs the suggested package runs the attacker’s code, and it calls this “a real supply-chain path”. GitHub’s word is theoretical. The pages we read do not settle it between them.

The number Endor Labs leans on comes from USENIX Security 2025, which its page calls a peer-reviewed study; GitHub does not cite it. A paper by Joseph Spracklen and colleagues at the University of Texas at San Antonio, the University of Oklahoma and Virginia Tech, which won the symposium’s Distinguished Paper Award, used 16 code-generating models and two sets of prompts in two programming languages. Its abstract reports that the average share of hallucinated packages is at least 5.2% for commercial models and 21.7% for open-source models, and that the authors also tried mitigation strategies that reduced the number while keeping code quality. That is a count of invented package names in generated code. We read the abstract page, not the full paper, and the abstract page does not describe an attack that succeeded.

Endor Labs adds figures of its own: it says its research found that only about 1 in 5 of the dependency versions that AI recommended were safe, with 49% carrying known vulnerabilities and 34% hallucinated. We did not read that research. It counts versions, while USENIX counts package names, so the two sets of numbers do not measure the same thing and should not be placed in one line.

Keys are the other half of this section, and only one page raises them. Endor Labs says the assistant sometimes hardcodes secrets such as API keys straight into the code, and defines a hardcoded secret as a password or key written into source files, where anyone with access to the repository can read it. Its explanation is that the model optimizes for a working answer to the prompt, not a secure one, unless it is asked for both. Its advice is to ask the assistant not to hardcode credentials and to run secrets scanning in the IDE and in continuous integration. The GitHub page we read does not mention keys left in code.

When the reviewer is a tool

IBM says such tools often give suggestions or even automated fixes, and can sit in the IDE and in version control. In a continuous integration pipeline a commit or a new pull request can trigger them, and they examine the diff and the code around it. Wiz adds that the strongest tools look past the changed lines and pull in related files, configuration and history. Both give the same list of limits. False positives flag code that is fine, false negatives miss a real flaw, and Wiz adds alert fatigue, where people start ignoring the notes. IBM suggests custom rules to cut false positives and says that testing remains crucial to catch what was missed.

Context is the second limit. IBM says these tools often struggle with the specifics of a project: the intricacies of APIs and frameworks, complex business logic, domain-specific scenarios, edge cases and the overall architecture. Wiz says that when a tool cannot see enough of the surrounding code, it makes wrong calls and flags things that are actually fine, and that giving it more repository context cuts the misses.

The third limit is the reader. IBM warns that developers may become overly dependent on the tools and give less weight to their own expertise, and Wiz says people “rubber-stamp whatever it approves”. Its habits for avoiding that: treat the output as a draft, keep a human reviewer, run the tool on every pull request, and rank the findings by real risk. One of its habits links the two directions of the phrase: “Separate generation from review.” If an AI wrote the code, Wiz says to have a different tool or model review it, so the same blind spots do not pass twice.

GitHub goes another way on who looks first. In its seventh step it suggests that AI agents with reasoning capabilities may help automate part of the review, and gives the example of a self-reviewing agent that checks draft pull requests against your standards before a human reviews them. Wiz proposes a different tool or model, GitHub an agent that screens drafts, and IBM says a person must still verify what the tool proposes.

IBM lists tools by name: the Code Review feature of Anthropic’s Claude Code, Codacy, CodeRabbit, Cursor’s Bugbot agent, GitHub Copilot code review and IBM Bob. Since IBM sells Bob, we give the list as a list of what exists. The pages we read do not rank them.

What the pages do not say, and what we left out

We reviewed no code and tried no tool. The pages we read do not give a time for reviewing a generated change (IBM’s line that an hour-long bug hunt for a person can take minutes for a tool is about tools, not about checking generated code), and they do not agree on one order for the checks. The GitHub page, the IBM page, the Wiz page and the Endor Labs page do not compare the tools with each other, and the only count of invented package names we read comes from the USENIX abstract; Endor Labs gives a share of hallucinated dependency versions, which counts something else and whose research we did not read.

We did not open the vendor pages from DeepSource, Aikido, Early, CodeAnt and Mend, or a Reddit thread on the subject. We also did not read the full USENIX paper or the Endor Labs research behind its 1 in 5 figure. Endor Labs and Wiz both quote other studies, among them a Veracode report, a CodeRabbit study, a 2025 preprint and the Stack Overflow survey; we read them only second-hand, so we left them out.