How Builder Reports are made

Two kinds of report

• Project reports. For a project imported from GitHub or uploaded. A static reading of the stored snapshot. Nothing in it is ever run.
• Work sample reports. For a submission to a simulation an employer invited you to. They cover that one task and may cite runs Fydell executed in its sandbox.

The two are built as separate result types, so a project report cannot carry a verified run.

A fixed revision

A GitHub import reads one commit, shown to you before the import starts. An upload is identified by a fingerprint of its files, and saving only goes ahead if the files match the preview you confirmed. Every finding records the revision and a content fingerprint it was derived from, so everyone who reads the report refers to the same code.

Each import is a new version of the project; earlier versions stay. When a project is analyzed again, the report lists findings that are new since the previous revision and findings no longer shown.

Code is never run during import

Importing or uploading a project reads files and nothing else. Fydell does not install dependencies, build the project or run its tests. A finding about a test says the test exists and what it checks, not that it passes. The fydell CLI also never runs your code or git; it reads commit details from the files in .git.

What is read

• Source and configuration text. Package manifests, CI workflows and top-level configuration come first, then the README, then source files, with up to 12 test files guaranteed a place before the file limit. Files are taken in turn from each top-level folder, so one large folder cannot crowd out the rest.
• Left out, with the reason shown. Dependency, vendored and build folders, lockfiles, binaries and minified files, symbolic links, submodules, and files that look like credentials, such as .env files and private keys.
• Limits. From GitHub, GitLab or an upload, every eligible source file is read, with no file-count or total-size ceiling, even for repositories with tens of thousands of files. Binaries, vendored output, lockfiles and secret-like files are left out with the reason shown.
• Secrets redacted. Text in the files that looks like a secret is redacted before anything is stored or analyzed.

The report states how many of the project's files were analyzed and why the others were left out.

How findings are produced

Pattern rules read the files and record what specific lines show, for example how an error is handled or what a test asserts. When a language model is configured, it also reads the numbered source and proposes up to eight findings. It is told that file contents are untrusted data, that nothing was run, and to describe the code rather than the person.

Each finding is marked as an observation, which the cited lines show directly, or an inference, which goes further and lists the findings it rests on. Capability summaries are inferences from observations, and each says that it describes the code, not who wrote it.

Citation checks

Every proposed finding, whether from a pattern rule or the model, is kept only if all of these hold:

• It cites at least one line range, in a file that is in the stored snapshot.
• Each range is at most 40 lines and lies inside the file.
• A quoted fragment of at least six characters appears in the cited lines, and the lines are not blank.
• When a claim is checked again later, the cited lines still have the same fingerprint they had when the claim was made.
• It does not state a run outcome, such as tests passing, unless a Fydell run applies to this exact code.
• It does not use judgment or score words such as senior, expert, robust, production-ready, best practice, or a mark out of ten.
• A model-written claim names only functions, variables and values that appear in the cited lines or path.

A claim that fails is never shown as a finding. It is kept with the reason it failed, for audit, and the report states how many claims were rejected. A check confirms that the cited lines exist and contain the quoted text; it does not confirm that the model's interpretation of them is right, and model findings say so.

Verified and reported results

Every fact carries one of three labels:

Execution results are listed in two groups, Fydell-verified and reported by the candidate. When the two disagree, both are shown and the verified run is the execution result. Statements, decisions, corrections and handoff notes from the engineer appear as their own words, labeled as reported by them.

Sandboxed execution for submissions

Only work sample submissions are run, and only by Fydell:

• Each run starts a fresh, time-limited virtual machine from a versioned image, with no network access, no application secrets, no repository checkout and no storage mounts. It is deleted afterwards.
• A run counts as verified only if the files it executed have the same fingerprint as the submission. Runs on other files are listed as excluded, with the reason. If the stored files do not match the submission's manifest, nothing is evaluated.
• Each run is labeled with the runner it used and whether that runner was isolated.
• Candidates see their public test results. Protected test names and counts stay with the hiring team, and a candidate sees the evaluation's public tests only after the team releases the report.

Tests a candidate runs on their own computer are not sandboxed. They run with the candidate's own permissions and are reported as theirs.

Limitations

Every report ends with the limitations that apply to it. The ones you will see most often:

• Files that could not be read are named, and no finding covers them.
• For a project report, no code, tests or builds were executed.
• Inferences go beyond what the cited lines literally show.
• If the model step fails, only findings from pattern rules are shown, and the report says so.
• Results reported by the candidate were not reproduced by Fydell.
• The absence of a finding says nothing about the engineer. The analysis recognizes specific patterns only.
• Owning a repository does not show who wrote each line, and a report never says it does.

What a report never claims

• There is no overall score. Each finding stands on its own evidence.
• It does not judge seniority, skill or potential, or predict job performance.
• It does not say who wrote the code.
• It does not say imported code works or that its tests pass.
• It does not detect AI use.
• It does not recommend, rank or make a hiring decision. People on the hiring team decide.

See Trust and security for who can read each report, and Builder Reports for an example.