openpencil/tools/generate/visual-oracles/src/operations/compare/interpreted-document.ts
Danila Poyarkov e2a3aa3f80
fix: validate parsed JSON at untrusted boundaries with Valibot (#855)
* fix: validate parsed JSON at untrusted boundaries with Valibot

Clipboard HTML, library revisions from shared storage, MCP and automation
WebSocket messages, the MCP discovery file, sidecar output and AI/MCP tool
arguments were JSON.parse'd and cast to their expected types, so a
malformed payload reached the document or crashed paste. They now go
through v.pipe(v.string(), v.parseJson(), Schema), which reports bad JSON
and a wrong shape as the same validation failure.

The path_set tool rejects an invalid VectorNetwork and shares its parser
with create_vector. The CLI library catalog validates its files and runs
revisions through the same size, identity and content-hash checks as the
app; reading image bytes as index-keyed records also stops them coming
back empty. Hand-rolled typeof readers for plugin data, document metadata,
caches and preferences become schemas with their behaviour preserved, and
readCacheJSON takes a schema for its payload.

open-pencil/no-unvalidated-json-parse rejects type assertions on
JSON.parse results other than `as unknown` in src and packages/*/src.

* refactor: validate parsed JSON in tests and tooling

Extend open-pencil/no-unvalidated-json-parse beyond source: tests, helpers and repo tooling now parse JSON through Valibot schemas instead of asserting a type. The shared fixture reader returns a validated object; its old array annotation never matched the fixtures.

* fix: validate clipboard geometry bytes, library images and model catalogs

Clipboard geometry blobs and library image bytes must be bytes at contiguous indexes, so out-of-range or gapped values are rejected instead of silently becoming different geometry or images; serialized library nodes must carry source metadata. The models.dev and OpenRouter responses are validated like their cached copies, and activate-tab rejects a CDP frame it cannot read instead of hanging.

* refactor: extend the JSON validation lint to .json() results

no-unvalidated-json-parse now also rejects type assertions on Response, Bun.file and shell .json() results, the same unchecked parse in another form. MCP server tests read /health through a validated readHealth helper and discovery files through parseDiscoveryInfo; the remaining tooling reads its JSON through schemas.

* test: validate the RPC request body in the CLI app export test

* test: validate CLI JSON output in the tool and app command tests

* test: compare the malformed models.dev fallback with the curated list
2026-10-04 17:01:50 +00:00

89 lines
3 KiB
TypeScript

#!/usr/bin/env bun
import { mkdir } from 'node:fs/promises'
import { join } from 'node:path'
import { parseArgs } from 'node:util'
import { captureGraphOracle, figmaOracleScript } from '#visual/capture/scene'
import { summarizePathDiagnostics } from '#visual/path-diagnostics'
import { compareSceneOracle, SceneOracleNodeSchema } from '#visual/scene-oracle'
import { $ } from 'bun'
import * as v from 'valibot'
import { materializeDocument, parseFigBuffer } from '@open-pencil/fig'
import type { InstancePathDiagnostic } from '@open-pencil/fig/instance-overrides'
const { values } = parseArgs({
options: {
file: { type: 'string' },
node: { type: 'string' },
'figma-key': { type: 'string' },
'allow-partial-assignments': { type: 'boolean', default: false },
output: { type: 'string' }
}
})
if (!values.file || !values.node || !values['figma-key'] || !values.output) {
throw new Error('Required: --file FILE --node ID --figma-key KEY --output DIR')
}
await mkdir(values.output, { recursive: true })
const script = figmaOracleScript(values['figma-key'], values.node)
const capture = await $`figma-use eval ${script} --json`.quiet().text()
const oracle = v.parse(
v.pipe(
v.string(),
v.parseJson(),
v.object({ fileKey: v.string(), rootId: v.string(), nodes: v.array(SceneOracleNodeSchema) })
),
capture
)
if (
oracle.fileKey !== values['figma-key'] ||
oracle.rootId !== values.node ||
!Array.isArray(oracle.nodes)
) {
throw new Error('Invalid oracle capture identity')
}
const { nodeChanges, blobs, images } = parseFigBuffer(await Bun.file(values.file).arrayBuffer())
const diagnostics: InstancePathDiagnostic[] = []
const assignmentDiagnostics: unknown[] = []
const { graph, sources } = materializeDocument(nodeChanges, blobs, {
images: new Map(images),
derivedBounds: true,
onUnresolvedAssignment: values['allow-partial-assignments']
? (diagnostic) => assignmentDiagnostics.push(diagnostic)
: undefined,
onUnresolvedProperty: (diagnostic) => diagnostics.push(diagnostic)
})
const root = sources.get(values.node)
if (!root) throw new Error('Missing assembled root')
const actual = captureGraphOracle(graph, root, sources)
const differences = compareSceneOracle(oracle.nodes, actual)
const groupedDiagnostics = summarizePathDiagnostics(diagnostics)
for (const [name, value] of Object.entries({
oracle,
actual,
differences,
diagnostics,
assignmentDiagnostics,
groupedDiagnostics
})) {
await Bun.write(join(values.output, `${name}.json`), JSON.stringify(value, null, 2))
}
const counts: Record<string, number> = {}
for (const difference of differences)
counts[difference.category] = (counts[difference.category] ?? 0) + 1
console.log(
JSON.stringify(
{
partialAssignmentCallbacks: assignmentDiagnostics.length,
expectedNodes: oracle.nodes.length,
actualNodes: actual.length,
differences: counts,
uniqueUnresolvedPaths: groupedDiagnostics.length,
unresolvedCallbacks: diagnostics.length
},
null,
2
)
)
if (differences.length || assignmentDiagnostics.length) process.exitCode = 1