Compare commits
4
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
0da354f040
|
||
|
|
249b54e8c4
|
||
|
|
d5375254d7
|
||
|
|
fed19b260f
|
@@ -6,5 +6,6 @@ Shared, dependency-light building blocks used across the Continuum cloud and edg
|
|||||||
|
|
||||||
- **`packages/ts-core`** - TypeScript package consumed by `continuum-backend` (and other Bun/Node services). Provides a structured logger, a Supabase JWT verification helper, and ElysiaJS-compatible HTTP error transformers.
|
- **`packages/ts-core`** - TypeScript package consumed by `continuum-backend` (and other Bun/Node services). Provides a structured logger, a Supabase JWT verification helper, and ElysiaJS-compatible HTTP error transformers.
|
||||||
- **`packages/rust-core`** - Rust crate consumed by `continuum-proxy` and `continuum-ai-worker`. Provides a shared `thiserror`-based error type, `tracing` initialization, and G-code checksum/header parsing helpers.
|
- **`packages/rust-core`** - Rust crate consumed by `continuum-proxy` and `continuum-ai-worker`. Provides a shared `thiserror`-based error type, `tracing` initialization, and G-code checksum/header parsing helpers.
|
||||||
|
- **`packages/rust-schema-instance`** - Rust crate consumed by `continuum-proxy` and `continuum-ai-worker`. Builds a `serde_json::Value` instance from a JSON Schema file (resolves `$ref`/`allOf`, fills `const`/`enum`/`default`, only builds `required` properties) - for code that has to construct a raw JSON message against some other system's schema instead of a typed struct.
|
||||||
|
|
||||||
Each package is versioned and published independently; see their own READMEs (inline doc comments) for usage.
|
Each package is versioned and published independently; see their own READMEs (inline doc comments) for usage.
|
||||||
|
|||||||
@@ -0,0 +1,36 @@
|
|||||||
|
# Licence policy for a closed-source commercial product.
|
||||||
|
# See docs/LICENSES.md in continuum-common for the rationale and the
|
||||||
|
# non-cargo dependencies (external binaries, ML models) this file cannot see.
|
||||||
|
#
|
||||||
|
# Run: cargo deny check licenses
|
||||||
|
|
||||||
|
[licenses]
|
||||||
|
# Permissive only. Anything copyleft reaching a shipped binary would make
|
||||||
|
# the product undistributable without releasing our source.
|
||||||
|
allow = [
|
||||||
|
"MIT",
|
||||||
|
"Apache-2.0",
|
||||||
|
"Apache-2.0 WITH LLVM-exception",
|
||||||
|
"BSD-2-Clause",
|
||||||
|
"BSD-3-Clause",
|
||||||
|
"ISC",
|
||||||
|
"Zlib",
|
||||||
|
"Unicode-3.0",
|
||||||
|
"Unlicense",
|
||||||
|
"MPL-2.0",
|
||||||
|
"CC0-1.0",
|
||||||
|
"BlueOak-1.0.0",
|
||||||
|
]
|
||||||
|
confidence-threshold = 0.9
|
||||||
|
|
||||||
|
# Our own crates declare `LicenseRef-Proprietary`, which is deliberately not
|
||||||
|
# an SPDX identifier and must never appear in the allow list above - a
|
||||||
|
# third-party crate claiming it would be a real finding. They are recognised
|
||||||
|
# as first-party by `publish = false` in their manifests.
|
||||||
|
private = { ignore = true }
|
||||||
|
|
||||||
|
[bans]
|
||||||
|
deny = []
|
||||||
|
|
||||||
|
[advisories]
|
||||||
|
yanked = "deny"
|
||||||
@@ -0,0 +1,217 @@
|
|||||||
|
# Licence policy
|
||||||
|
|
||||||
|
Continuum is a closed-source commercial product. We intend to sell it, so
|
||||||
|
every dependency must permit distribution in proprietary software without
|
||||||
|
publishing our own source. This document is the policy; `deny.toml` in each
|
||||||
|
repo is the machine-enforced half of it.
|
||||||
|
|
||||||
|
## The rule
|
||||||
|
|
||||||
|
**Permissive only.** A dependency is allowed if it is one of:
|
||||||
|
|
||||||
|
| Licence | Notes |
|
||||||
|
| --- | --- |
|
||||||
|
| MIT, ISC, 0BSD, BSD-2-Clause, BSD-3-Clause | The common case |
|
||||||
|
| Apache-2.0 (incl. `WITH LLVM-exception`) | Carries a patent grant, which is a plus |
|
||||||
|
| Unlicense, CC0-1.0 | Public-domain dedications |
|
||||||
|
| Zlib, BlueOak-1.0.0 | Permissive, OSI/Blue Oak approved |
|
||||||
|
| Unicode-3.0 | Unicode data tables |
|
||||||
|
| MPL-2.0 | File-level copyleft only; fine as long as we do not edit the dependency's own files |
|
||||||
|
|
||||||
|
**Forbidden, no exceptions:** GPL, LGPL (when statically linked), AGPL, SSPL,
|
||||||
|
RSAL, Commons Clause, "source available", CC-BY-NC, and any licence with a
|
||||||
|
field-of-use or non-commercial restriction.
|
||||||
|
|
||||||
|
Two rules that are easy to get wrong:
|
||||||
|
|
||||||
|
- **Dual-licensed dependencies are fine** - we elect the permissive option.
|
||||||
|
Record the election here so it is not re-litigated (see `node-forge` below).
|
||||||
|
- **LGPL is conditionally fine** but only via *dynamic* linking with the
|
||||||
|
library replaceable by the user. Rust static linking does not qualify.
|
||||||
|
Assume LGPL is forbidden unless someone writes down why a specific case
|
||||||
|
is not.
|
||||||
|
|
||||||
|
## How it is enforced
|
||||||
|
|
||||||
|
Rust, per repo:
|
||||||
|
|
||||||
|
```sh
|
||||||
|
cargo deny check licenses # proxy, ai-worker
|
||||||
|
cargo deny --manifest-path packages/rust-core/Cargo.toml check licenses # common, schemas
|
||||||
|
cargo deny --manifest-path apps/desktop/src-tauri/Cargo.toml check licenses # app
|
||||||
|
```
|
||||||
|
|
||||||
|
Our own crates declare `license = "LicenseRef-Proprietary"` with
|
||||||
|
`publish = false`. `LicenseRef-Proprietary` is deliberately *not* a real SPDX
|
||||||
|
identifier and must never be added to a `deny.toml` allow list - a
|
||||||
|
third-party crate claiming it would be a genuine finding, and `publish =
|
||||||
|
false` is what marks ours as first-party. Our npm packages use
|
||||||
|
`"license": "UNLICENSED"` with `"private": true`, which is npm's equivalent
|
||||||
|
convention.
|
||||||
|
|
||||||
|
JavaScript has no `cargo deny` equivalent wired up yet; the audit below was
|
||||||
|
done by walking installed `node_modules` manifests.
|
||||||
|
|
||||||
|
## What the tooling cannot see
|
||||||
|
|
||||||
|
`cargo deny` only reads the cargo graph. Three categories sit outside it and
|
||||||
|
have to be checked by hand:
|
||||||
|
|
||||||
|
1. **External binaries we spawn or ship.** Invoking a GPL program as a
|
||||||
|
separate process is not itself a derivative work, but *shipping* it in our
|
||||||
|
installer means distributing it under its own terms.
|
||||||
|
2. **ML model weights.** A model's licence is independent of the runtime that
|
||||||
|
loads it. This is the single biggest trap in this project (see
|
||||||
|
`continuum-ai-worker` below).
|
||||||
|
3. **Managed/self-hosted services.** The client library's licence is not the
|
||||||
|
server's licence. `ioredis` being MIT says nothing about Redis.
|
||||||
|
|
||||||
|
## Audit, 2026-09-17 (Redis resolved 2026-10-03)
|
||||||
|
|
||||||
|
All six repos audited. **No copyleft code is currently compiled into any
|
||||||
|
Continuum binary.** Every Rust crate across `continuum-proxy`,
|
||||||
|
`continuum-ai-worker`, `continuum-app/apps/desktop`, `continuum-common` and
|
||||||
|
`continuum-schemas` passes the policy above. The JS graphs are MIT/Apache/ISC/
|
||||||
|
BSD/BlueOak throughout.
|
||||||
|
|
||||||
|
Fixed in this pass: all six of our own crates and all nine of our npm
|
||||||
|
packages were missing a licence field (`cargo deny` reported them as
|
||||||
|
`unlicensed` errors), and no repo had a `deny.toml`.
|
||||||
|
|
||||||
|
### Findings carried forward
|
||||||
|
|
||||||
|
**Redis - RESOLVED 2026-10-03, we use Valkey.** Redis 7.4+ is RSALv2/SSPLv1
|
||||||
|
and Redis 8.0+ added AGPLv3 as a third option. All three are non-permissive,
|
||||||
|
and SSPL §13 specifically targets offering the software's functionality as a
|
||||||
|
service - which is what a cloud control plane does. The *clients* were never
|
||||||
|
the problem (`ioredis` and the `redis` crate are both MIT); the server we
|
||||||
|
deploy was.
|
||||||
|
|
||||||
|
We now deploy **Valkey** (BSD-3-Clause, Linux Foundation, forked from Redis
|
||||||
|
7.2.4). What changed: `REDIS_URL` -> `VALKEY_URL` and
|
||||||
|
`CONTINUUM_REDIS_URL` -> `CONTINUUM_VALKEY_URL`, the exported `redis` client
|
||||||
|
in `continuum-backend/src/queue/` renamed to `valkey`, `Config.redis_url` ->
|
||||||
|
`Config.valkey_url` in `continuum-ai-worker`, `/health` now reports a
|
||||||
|
`valkey` field, and `continuum-backend/docker-compose.yml` pins
|
||||||
|
`valkey/valkey:8-alpine`. Both tiers still typecheck/compile unchanged
|
||||||
|
otherwise, because Valkey speaks the same protocol.
|
||||||
|
|
||||||
|
Two things deliberately did **not** change, so nobody "fixes" them later:
|
||||||
|
the `redis://` URL scheme (it names the RESP wire protocol, and no client
|
||||||
|
library understands a `valkey://` scheme), and the client libraries
|
||||||
|
themselves (`ioredis`, the `redis` crate) - they are RESP clients, and their
|
||||||
|
names refer to the protocol's origin, not to the server we run.
|
||||||
|
|
||||||
|
**AI model weights - blocks `PLAN.md` step 4.** `src/inference/mod.rs`
|
||||||
|
currently stubs a `"spaghetti"` detection. The two obvious off-the-shelf
|
||||||
|
sources are both **AGPL-3.0**: Ultralytics YOLO (v5/v8/v11) and Obico / The
|
||||||
|
Spaghetti Detective's print-failure model. Neither can ship in a commercial
|
||||||
|
product without a paid enterprise licence. The `ort` crate itself is MIT and
|
||||||
|
fine. Permissive alternatives to evaluate when step 4 comes up: YOLOX
|
||||||
|
(Apache-2.0), RT-DETR (Apache-2.0), RF-DETR (Apache-2.0), or training our own
|
||||||
|
weights on a permissively-licensed architecture. **Decide the architecture
|
||||||
|
before collecting training data** - retraining because of a licence is the
|
||||||
|
expensive version of this mistake.
|
||||||
|
|
||||||
|
**go2rtc and ffmpeg - currently fine, keep it that way.**
|
||||||
|
`continuum-proxy/src/go2rtc.rs` spawns go2rtc (MIT) from `PATH` and expects
|
||||||
|
the operator to have installed it, so we distribute nothing. If that ever
|
||||||
|
changes: go2rtc shells out to ffmpeg, and ffmpeg is LGPL-2.1+ *only* when
|
||||||
|
built without `--enable-gpl`. Most distro and prebuilt ffmpeg binaries are
|
||||||
|
GPL builds (they bundle x264). Shipping one would be a GPL distribution.
|
||||||
|
|
||||||
|
**`node-forge` - dual-licensed, we elect BSD-3-Clause.** It appears in
|
||||||
|
`continuum-app` as `(BSD-3-Clause OR GPL-2.0)`. The election is ours to make
|
||||||
|
and is recorded here so the GPL half does not trigger a false alarm later.
|
||||||
|
|
||||||
|
**`caniuse-lite` - CC-BY-4.0, attribution required if redistributed.** A
|
||||||
|
build-time browserslist dependency in `continuum-app`; its data is not
|
||||||
|
bundled into the Nuxt output, so no attribution obligation today. It would
|
||||||
|
acquire one if anything started shipping the dataset itself.
|
||||||
|
|
||||||
|
**Infrastructure already clear.** PostgreSQL and pgvector (PostgreSQL
|
||||||
|
licence), PgBouncer (ISC), pgCat (MIT), Supabase (Apache-2.0), ONNX Runtime
|
||||||
|
(MIT), Cloudflare R2 (a service, governed by contract not licence) all
|
||||||
|
permit commercial use.
|
||||||
|
|
||||||
|
### Not findings
|
||||||
|
|
||||||
|
A `node_modules` walk turns up packages named `baz`, `browser_field`,
|
||||||
|
`invalid_main`, `false_main` and `nw-pre-gyp-module-test` with no licence
|
||||||
|
field. These are test fixtures *inside* other packages, not dependencies.
|
||||||
|
Ignore them.
|
||||||
|
|
||||||
|
## Audit, 2026-10-15 - continuum-proxy dual-licence elections
|
||||||
|
|
||||||
|
Found while adding a zip reader to `continuum-proxy` (see below). All three
|
||||||
|
were already in the tree, all three pass `cargo deny check licenses`, and
|
||||||
|
none had its election recorded - which rule one at the top of this document
|
||||||
|
says to do, so that a `grep` for "GPL" does not restart the argument.
|
||||||
|
Verified from `cargo metadata`, not from memory.
|
||||||
|
|
||||||
|
**`unescaper` 0.1.10 - `MIT OR GPL-3.0-only`, we elect MIT.** Reached via
|
||||||
|
`tokio-serial` -> `serialport` -> `unescaper`, so it is in the Linux build.
|
||||||
|
The MIT half is ours to take.
|
||||||
|
|
||||||
|
**`r-efi` 6.0.0 - `MIT OR Apache-2.0 OR LGPL-2.1-or-later`, we elect MIT or
|
||||||
|
Apache-2.0.** Reached via `native-tls` -> `tempfile` -> `getrandom`, but only
|
||||||
|
on UEFI targets: it is not compiled for any platform we ship. The LGPL option
|
||||||
|
would be the problem one if it ever were, and electing around it is free.
|
||||||
|
|
||||||
|
**`serialport` 4.10.0 - MPL-2.0, allowed with a condition.** The table above
|
||||||
|
permits MPL-2.0 because its copyleft is per-file, which holds only while we
|
||||||
|
do not edit the dependency's own files. We do not and should not: if
|
||||||
|
`serialport` ever needs a patch, vendor the change as a fork with its own
|
||||||
|
notice, or wrap it, rather than editing files in place.
|
||||||
|
|
||||||
|
**An HTTP client in `continuum-proxy`, and the version we could not take.**
|
||||||
|
Added 2026-10-15 so a gateway can download a job file from a presigned URL.
|
||||||
|
The choice is `reqwest` 0.12.28 with `default-features = false` and
|
||||||
|
`rustls-tls-native-roots`, and the version matters:
|
||||||
|
|
||||||
|
- **reqwest 0.13 is out.** Its `rustls` feature pulls
|
||||||
|
`rustls-platform-verifier` -> `webpki-root-certs` under
|
||||||
|
**CDLA-Permissive-2.0**, which is not in the table above, and
|
||||||
|
`cargo deny` rejects it outright (`error[rejected]: license is not
|
||||||
|
explicitly allowed`). CDLA-Permissive-2.0 is in fact a permissive licence
|
||||||
|
and could reasonably be added - but it is a *data* licence applied to a
|
||||||
|
certificate bundle, which is a different question from a code licence and
|
||||||
|
deserves a deliberate decision rather than being waved through to unblock
|
||||||
|
a dependency bump. Left out until someone wants it.
|
||||||
|
- The other 0.13 route brings `aws-lc`, which contains OpenSSL-licensed C.
|
||||||
|
Also out.
|
||||||
|
|
||||||
|
What 0.12 with that feature set brings, all verified from `cargo metadata`
|
||||||
|
rather than from memory, and all permissive: `reqwest` 0.12.28, `hyper`
|
||||||
|
1.12.0, `hyper-util` 0.1.21, `http-body` 1.1.0, `tower` 0.5.3,
|
||||||
|
`tower-service` 0.3.3 (all MIT); `rustls` 0.23.46, `hyper-rustls` 0.27.10,
|
||||||
|
`rustls-native-certs` 0.8.5 (`Apache-2.0 OR ISC OR MIT`); `rustls-webpki`
|
||||||
|
0.103.15 and `untrusted` 0.9.0 (ISC); `ring` 0.17.14
|
||||||
|
(**`Apache-2.0 AND ISC`** - note the AND: both apply, and both are fine);
|
||||||
|
`sync_wrapper` 1.0.2 (Apache-2.0); `subtle` 2.6.1 (BSD-3-Clause);
|
||||||
|
`zerocopy` 0.8.56 (`BSD-2-Clause OR Apache-2.0 OR MIT`); plus `http`,
|
||||||
|
`httparse`, `base64`, `url`, `ipnet`, `rustls-pki-types` and `tokio-rustls`
|
||||||
|
(`MIT OR Apache-2.0`).
|
||||||
|
|
||||||
|
`openssl-probe` 0.2.1 appears and is only a certificate-path lookup, no C.
|
||||||
|
Note that `openssl-sys` is still in the tree regardless, via the existing
|
||||||
|
`native-tls` the Bambu MQTT and FTPS clients use - rustls here does not
|
||||||
|
remove it.
|
||||||
|
|
||||||
|
**New in `continuum-proxy` on 2026-10-15, all clear:** `zip` 9.0.3 (MIT),
|
||||||
|
`flate2` 1.1.10, `crc32fast` 1.5.2, `typed-path` 0.12.3 (all
|
||||||
|
`MIT OR Apache-2.0`), `miniz_oxide` 0.9.1 (`MIT OR Zlib OR Apache-2.0`),
|
||||||
|
`adler2` 2.0.1 (`0BSD OR MIT OR Apache-2.0`), `simd-adler32` 0.3.10 (MIT),
|
||||||
|
`zlib-rs` 0.6.8 (Zlib). Added for reading `.3mf` project archives. The
|
||||||
|
feature set is deliberately `deflate-flate2` only - `zip`'s other codec
|
||||||
|
features pull in bzip2, lzma/xz and zstd bindings, none of which are needed
|
||||||
|
and each of which is another licence to check. Confirmed none of them entered
|
||||||
|
the graph.
|
||||||
|
|
||||||
|
## When adding a dependency
|
||||||
|
|
||||||
|
1. Check the licence against the table above **before** writing code against
|
||||||
|
it. Replacing a library after it is load-bearing is the costly path.
|
||||||
|
2. Run the repo's `cargo deny check licenses`.
|
||||||
|
3. If it is an external binary, a model, or a service, none of the tooling
|
||||||
|
will catch it - add it to the audit section above by hand.
|
||||||
@@ -1,5 +1,6 @@
|
|||||||
{
|
{
|
||||||
"name": "continuum-common",
|
"name": "continuum-common",
|
||||||
|
"license": "UNLICENSED",
|
||||||
"private": true,
|
"private": true,
|
||||||
"version": "0.1.0",
|
"version": "0.1.0",
|
||||||
"workspaces": [
|
"workspaces": [
|
||||||
|
|||||||
@@ -3,6 +3,10 @@ name = "continuum-core"
|
|||||||
version = "0.1.0"
|
version = "0.1.0"
|
||||||
edition = "2021"
|
edition = "2021"
|
||||||
description = "Shared error handling, tracing setup, and G-code helpers for Continuum Rust services"
|
description = "Shared error handling, tracing setup, and G-code helpers for Continuum Rust services"
|
||||||
|
# Proprietary: deliberately not an SPDX id, so cargo-deny treats it as
|
||||||
|
# first-party. See docs/LICENSES.md in continuum-common.
|
||||||
|
license = "LicenseRef-Proprietary"
|
||||||
|
publish = false
|
||||||
|
|
||||||
[dependencies]
|
[dependencies]
|
||||||
thiserror = "1.0"
|
thiserror = "1.0"
|
||||||
|
|||||||
@@ -0,0 +1,16 @@
|
|||||||
|
[package]
|
||||||
|
name = "continuum-schema-instance"
|
||||||
|
version = "0.1.0"
|
||||||
|
edition = "2021"
|
||||||
|
description = "Builds a JSON instance from a JSON Schema - fills const/enum/default, only builds required properties, everything else is left out"
|
||||||
|
# Proprietary: deliberately not an SPDX id, so cargo-deny treats it as
|
||||||
|
# first-party. See docs/LICENSES.md in continuum-common.
|
||||||
|
license = "LicenseRef-Proprietary"
|
||||||
|
publish = false
|
||||||
|
|
||||||
|
[dependencies]
|
||||||
|
serde_json = "1"
|
||||||
|
thiserror = "1.0"
|
||||||
|
|
||||||
|
[dev-dependencies]
|
||||||
|
jsonschema = "0.53.0"
|
||||||
@@ -0,0 +1,268 @@
|
|||||||
|
//! Builds a `serde_json::Value` instance from a JSON Schema file - the
|
||||||
|
//! Rust equivalent of what `json-schema-faker` is used for in JS/TS,
|
||||||
|
//! adapted to what's actually useful in Rust: this crate doesn't invent
|
||||||
|
//! random data for you (a typed struct already forces you to supply every
|
||||||
|
//! field it requires at compile time - there's nothing left for a faker
|
||||||
|
//! to do there). It's for code that has to build a raw `serde_json::Value`
|
||||||
|
//! against a schema it doesn't own - some other system's wire format -
|
||||||
|
//! instead of a typed struct: fill in the one or two fields you actually
|
||||||
|
//! have a value for, and let this fill in everything else the schema
|
||||||
|
//! already determines on its own.
|
||||||
|
//!
|
||||||
|
//! What it does, walking a schema node:
|
||||||
|
//! - `$ref` - loads the referenced file, resolved relative to the
|
||||||
|
//! directory of whichever file is doing the referencing (so a chain of
|
||||||
|
//! refs across several files each resolves against its own location,
|
||||||
|
//! not the original caller's). Only whole-file refs are supported - a
|
||||||
|
//! `#/json/pointer` fragment is an error, not silently ignored.
|
||||||
|
//! - `allOf` - merges every branch's `properties`/`required` into one
|
||||||
|
//! combined object schema before building it. Branches are resolved
|
||||||
|
//! (following their own `$ref`s) before merging, but a merged
|
||||||
|
//! property's *own* schema is then built relative to the outermost
|
||||||
|
//! file's directory, not the branch it came from - harmless for every
|
||||||
|
//! schema in this project today (no property schema anywhere uses
|
||||||
|
//! `$ref`), but worth knowing if you add one that does.
|
||||||
|
//! - `oneOf`/`anyOf` - deterministically picks the *first* listed
|
||||||
|
//! variant. If you need a specific one, point at that variant's own
|
||||||
|
//! schema file directly instead of the dispatcher.
|
||||||
|
//! - `const` / `enum` / `default` - used as-is (enum takes its first
|
||||||
|
//! value).
|
||||||
|
//! - `object` - builds a value **only** for properties listed in
|
||||||
|
//! `required`; anything not required is left out of the result
|
||||||
|
//! entirely. This is the "skip the rest" behavior - equivalent to
|
||||||
|
//! `json-schema-faker`'s `alwaysFakeOptionals: false` (its default).
|
||||||
|
//! - `array` - builds `minItems` copies of `items` (zero if unset).
|
||||||
|
//! - anything else - a simple placeholder for the declared `type`
|
||||||
|
//! (respecting `minLength`/`minimum` where trivial), so the result is
|
||||||
|
//! always schema-shaped even before you fill in the fields you actually
|
||||||
|
//! care about.
|
||||||
|
//!
|
||||||
|
//! Typical use: build the skeleton, then set the one or two fields you
|
||||||
|
//! actually have a value for with plain `serde_json::Value` indexing -
|
||||||
|
//! there's no separate "overrides" API here, because `value["field"] =
|
||||||
|
//! json!(...)` already does that job in one line:
|
||||||
|
//!
|
||||||
|
//! ```no_run
|
||||||
|
//! # use std::path::Path;
|
||||||
|
//! # use serde_json::json;
|
||||||
|
//! let mut message = continuum_schema_instance::build_instance_from_file(
|
||||||
|
//! Path::new("schemas/some_event.schema.json"),
|
||||||
|
//! ).unwrap();
|
||||||
|
//! message["some_field"] = json!("some value");
|
||||||
|
//! ```
|
||||||
|
|
||||||
|
use std::path::{Path, PathBuf};
|
||||||
|
|
||||||
|
use serde_json::Value;
|
||||||
|
|
||||||
|
/// Guards against a `$ref` cycle (or just a schema nested deeper than any
|
||||||
|
/// real one here goes) turning into infinite recursion.
|
||||||
|
const MAX_DEPTH: usize = 64;
|
||||||
|
|
||||||
|
#[derive(Debug, thiserror::Error)]
|
||||||
|
pub enum BuildError {
|
||||||
|
#[error("failed to read schema file {0}: {1}")]
|
||||||
|
Io(PathBuf, #[source] std::io::Error),
|
||||||
|
#[error("failed to parse schema file {0}: {1}")]
|
||||||
|
Json(PathBuf, #[source] serde_json::Error),
|
||||||
|
#[error("invalid schema: {0}")]
|
||||||
|
InvalidSchema(String),
|
||||||
|
#[error("schema nested too deep (possible $ref cycle)")]
|
||||||
|
TooDeep,
|
||||||
|
#[error("schema is `false` - always fails validation, nothing to build")]
|
||||||
|
AlwaysInvalid,
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Loads `path` as a JSON Schema and builds an instance of it. `$ref`s
|
||||||
|
/// inside it resolve relative to `path`'s own directory.
|
||||||
|
pub fn build_instance_from_file(path: &Path) -> Result<Value, BuildError> {
|
||||||
|
let schema = load_schema_file(path)?;
|
||||||
|
let base_dir = path.parent().unwrap_or_else(|| Path::new("."));
|
||||||
|
build(&schema, base_dir, 0)
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Same as [`build_instance_from_file`], but for a schema you've already
|
||||||
|
/// parsed - `base_dir` is where any `$ref` inside it resolves from.
|
||||||
|
pub fn build_instance(schema: &Value, base_dir: &Path) -> Result<Value, BuildError> {
|
||||||
|
build(schema, base_dir, 0)
|
||||||
|
}
|
||||||
|
|
||||||
|
fn build(schema: &Value, base_dir: &Path, depth: usize) -> Result<Value, BuildError> {
|
||||||
|
if depth > MAX_DEPTH {
|
||||||
|
return Err(BuildError::TooDeep);
|
||||||
|
}
|
||||||
|
let depth = depth + 1;
|
||||||
|
|
||||||
|
if let Some(allowed) = schema.as_bool() {
|
||||||
|
return if allowed { Ok(Value::Null) } else { Err(BuildError::AlwaysInvalid) };
|
||||||
|
}
|
||||||
|
let obj = schema
|
||||||
|
.as_object()
|
||||||
|
.ok_or_else(|| BuildError::InvalidSchema("a schema must be a JSON object or boolean".into()))?;
|
||||||
|
|
||||||
|
if let Some(r) = obj.get("$ref").and_then(Value::as_str) {
|
||||||
|
let (resolved, new_base) = load_ref(r, base_dir)?;
|
||||||
|
return build(&resolved, &new_base, depth);
|
||||||
|
}
|
||||||
|
|
||||||
|
if let Some(branches) = obj.get("allOf").and_then(Value::as_array) {
|
||||||
|
let merged = merge_all_of(branches, base_dir, depth)?;
|
||||||
|
return build(&merged, base_dir, depth);
|
||||||
|
}
|
||||||
|
|
||||||
|
if let Some(variants) =
|
||||||
|
obj.get("oneOf").and_then(Value::as_array).or_else(|| obj.get("anyOf").and_then(Value::as_array))
|
||||||
|
{
|
||||||
|
let first =
|
||||||
|
variants.first().ok_or_else(|| BuildError::InvalidSchema("oneOf/anyOf with no variants".into()))?;
|
||||||
|
return build(first, base_dir, depth);
|
||||||
|
}
|
||||||
|
|
||||||
|
if let Some(c) = obj.get("const") {
|
||||||
|
return Ok(c.clone());
|
||||||
|
}
|
||||||
|
|
||||||
|
if let Some(values) = obj.get("enum").and_then(Value::as_array) {
|
||||||
|
return values.first().cloned().ok_or_else(|| BuildError::InvalidSchema("enum with no values".into()));
|
||||||
|
}
|
||||||
|
|
||||||
|
if let Some(d) = obj.get("default") {
|
||||||
|
return Ok(d.clone());
|
||||||
|
}
|
||||||
|
|
||||||
|
let ty = obj.get("type").and_then(Value::as_str);
|
||||||
|
|
||||||
|
if ty == Some("object") || obj.contains_key("properties") {
|
||||||
|
return build_object(obj, base_dir, depth);
|
||||||
|
}
|
||||||
|
|
||||||
|
if ty == Some("array") {
|
||||||
|
return build_array(obj, base_dir, depth);
|
||||||
|
}
|
||||||
|
|
||||||
|
Ok(placeholder_for_type(ty, obj))
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Builds a value only for properties listed in `required` - everything
|
||||||
|
/// else in `properties` is left out entirely.
|
||||||
|
fn build_object(
|
||||||
|
obj: &serde_json::Map<String, Value>,
|
||||||
|
base_dir: &Path,
|
||||||
|
depth: usize,
|
||||||
|
) -> Result<Value, BuildError> {
|
||||||
|
let properties = obj.get("properties").and_then(Value::as_object);
|
||||||
|
let required = obj.get("required").and_then(Value::as_array);
|
||||||
|
|
||||||
|
let mut result = serde_json::Map::new();
|
||||||
|
if let Some(required) = required {
|
||||||
|
for name in required {
|
||||||
|
let Some(name) = name.as_str() else { continue };
|
||||||
|
let prop_schema = properties.and_then(|p| p.get(name)).ok_or_else(|| {
|
||||||
|
BuildError::InvalidSchema(format!("'{name}' is required but has no schema in properties"))
|
||||||
|
})?;
|
||||||
|
result.insert(name.to_string(), build(prop_schema, base_dir, depth)?);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
Ok(Value::Object(result))
|
||||||
|
}
|
||||||
|
|
||||||
|
fn build_array(obj: &serde_json::Map<String, Value>, base_dir: &Path, depth: usize) -> Result<Value, BuildError> {
|
||||||
|
let Some(items_schema) = obj.get("items") else {
|
||||||
|
return Ok(Value::Array(Vec::new()));
|
||||||
|
};
|
||||||
|
let min_items = obj.get("minItems").and_then(Value::as_u64).unwrap_or(0);
|
||||||
|
|
||||||
|
let mut items = Vec::new();
|
||||||
|
for _ in 0..min_items {
|
||||||
|
items.push(build(items_schema, base_dir, depth)?);
|
||||||
|
}
|
||||||
|
Ok(Value::Array(items))
|
||||||
|
}
|
||||||
|
|
||||||
|
/// A schema with no `const`/`enum`/`default` still needs *something*
|
||||||
|
/// there if it's required - the simplest value satisfying its declared
|
||||||
|
/// type (and, where trivial, `minLength`/`minimum`).
|
||||||
|
fn placeholder_for_type(ty: Option<&str>, obj: &serde_json::Map<String, Value>) -> Value {
|
||||||
|
match ty {
|
||||||
|
Some("string") => {
|
||||||
|
let min_length = obj.get("minLength").and_then(Value::as_u64).unwrap_or(0) as usize;
|
||||||
|
Value::String("x".repeat(min_length))
|
||||||
|
}
|
||||||
|
Some("integer") => Value::from(obj.get("minimum").and_then(Value::as_i64).unwrap_or(0)),
|
||||||
|
Some("number") => Value::from(obj.get("minimum").and_then(Value::as_f64).unwrap_or(0.0)),
|
||||||
|
Some("boolean") => Value::Bool(false),
|
||||||
|
_ => Value::Null,
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Resolves each `allOf` branch (following its own `$ref`/nested `allOf`)
|
||||||
|
/// down to a plain object schema, then unions their `properties` (later
|
||||||
|
/// branches win on a key clash) and `required` (deduplicated) into one
|
||||||
|
/// combined object schema.
|
||||||
|
fn merge_all_of(branches: &[Value], base_dir: &Path, depth: usize) -> Result<Value, BuildError> {
|
||||||
|
if depth > MAX_DEPTH {
|
||||||
|
return Err(BuildError::TooDeep);
|
||||||
|
}
|
||||||
|
|
||||||
|
let mut properties = serde_json::Map::new();
|
||||||
|
let mut required: Vec<Value> = Vec::new();
|
||||||
|
|
||||||
|
for branch in branches {
|
||||||
|
let resolved = resolve_schema_object(branch, base_dir, depth + 1)?;
|
||||||
|
if let Some(props) = resolved.get("properties").and_then(Value::as_object) {
|
||||||
|
for (k, v) in props {
|
||||||
|
properties.insert(k.clone(), v.clone());
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if let Some(req) = resolved.get("required").and_then(Value::as_array) {
|
||||||
|
for r in req {
|
||||||
|
if !required.contains(r) {
|
||||||
|
required.push(r.clone());
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
Ok(serde_json::json!({ "type": "object", "properties": properties, "required": required }))
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Follows `$ref`/`allOf` (without building anything yet) until it lands
|
||||||
|
/// on a plain object schema - the schema-level equivalent of `build`,
|
||||||
|
/// used only while merging `allOf` branches.
|
||||||
|
fn resolve_schema_object(schema: &Value, base_dir: &Path, depth: usize) -> Result<Value, BuildError> {
|
||||||
|
if depth > MAX_DEPTH {
|
||||||
|
return Err(BuildError::TooDeep);
|
||||||
|
}
|
||||||
|
let obj = schema
|
||||||
|
.as_object()
|
||||||
|
.ok_or_else(|| BuildError::InvalidSchema("an allOf branch must be a JSON object schema".into()))?;
|
||||||
|
|
||||||
|
if let Some(r) = obj.get("$ref").and_then(Value::as_str) {
|
||||||
|
let (resolved, new_base) = load_ref(r, base_dir)?;
|
||||||
|
return resolve_schema_object(&resolved, &new_base, depth + 1);
|
||||||
|
}
|
||||||
|
if let Some(branches) = obj.get("allOf").and_then(Value::as_array) {
|
||||||
|
return merge_all_of(branches, base_dir, depth + 1);
|
||||||
|
}
|
||||||
|
Ok(schema.clone())
|
||||||
|
}
|
||||||
|
|
||||||
|
fn load_ref(r: &str, base_dir: &Path) -> Result<(Value, PathBuf), BuildError> {
|
||||||
|
if let Some(fragment) = r.split('#').nth(1) {
|
||||||
|
if !fragment.is_empty() {
|
||||||
|
return Err(BuildError::InvalidSchema(format!(
|
||||||
|
"'{r}' - in-document JSON Pointer refs aren't supported, only whole-file refs"
|
||||||
|
)));
|
||||||
|
}
|
||||||
|
}
|
||||||
|
let file_part = r.split('#').next().unwrap_or(r);
|
||||||
|
let target = base_dir.join(file_part);
|
||||||
|
let schema = load_schema_file(&target)?;
|
||||||
|
let new_base = target.parent().map(Path::to_path_buf).unwrap_or_else(|| PathBuf::from("."));
|
||||||
|
Ok((schema, new_base))
|
||||||
|
}
|
||||||
|
|
||||||
|
fn load_schema_file(path: &Path) -> Result<Value, BuildError> {
|
||||||
|
let text = std::fs::read_to_string(path).map_err(|err| BuildError::Io(path.to_path_buf(), err))?;
|
||||||
|
serde_json::from_str(&text).map_err(|err| BuildError::Json(path.to_path_buf(), err))
|
||||||
|
}
|
||||||
@@ -0,0 +1,64 @@
|
|||||||
|
use std::path::PathBuf;
|
||||||
|
|
||||||
|
use continuum_schema_instance::build_instance_from_file;
|
||||||
|
use serde_json::json;
|
||||||
|
|
||||||
|
fn fixture(relative: &str) -> PathBuf {
|
||||||
|
PathBuf::from(env!("CARGO_MANIFEST_DIR")).join("tests/fixtures").join(relative)
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Validates `instance` against the real schema file - closes the loop:
|
||||||
|
/// this confirms the builder's output is actually accepted by a real JSON
|
||||||
|
/// Schema validator (the `jsonschema` crate), not just "looks right".
|
||||||
|
fn assert_valid(schema_path: &PathBuf, instance: &serde_json::Value) {
|
||||||
|
let schema_text = std::fs::read_to_string(schema_path).unwrap();
|
||||||
|
let mut schema: serde_json::Value = serde_json::from_str(&schema_text).unwrap();
|
||||||
|
// These fixtures have no `$id` (real schemas in continuum-schemas do,
|
||||||
|
// as an absolute https:// URL) - the `jsonschema` crate resolves a
|
||||||
|
// relative `$ref` against the schema's own `$id`, so give it one
|
||||||
|
// pointing at this file's real location for the relative `$ref`s
|
||||||
|
// inside to resolve from during this check.
|
||||||
|
let absolute_path = schema_path.canonicalize().unwrap();
|
||||||
|
schema["$id"] = json!(format!("file://{}", absolute_path.display()));
|
||||||
|
|
||||||
|
let validator = jsonschema::validator_for(&schema).expect("schema itself should compile");
|
||||||
|
let errors: Vec<_> = validator.iter_errors(instance).collect();
|
||||||
|
assert!(errors.is_empty(), "instance {instance} did not validate: {errors:?}");
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn required_fields_are_filled_and_optional_fields_are_left_out() {
|
||||||
|
let path = fixture("events/item_created.schema.json");
|
||||||
|
let mut instance = build_instance_from_file(&path).unwrap();
|
||||||
|
|
||||||
|
// "type" has no const at the envelope level, but item_created's own
|
||||||
|
// allOf branch overrides it to one - that const wins.
|
||||||
|
assert_eq!(instance["type"], json!("item_created"));
|
||||||
|
// "event_id"/"item_id" are required with no const - each gets a
|
||||||
|
// placeholder, ready to be overridden.
|
||||||
|
assert!(instance.get("event_id").is_some());
|
||||||
|
assert!(instance.get("item_id").is_some());
|
||||||
|
// "note" is optional and correctly left out entirely.
|
||||||
|
assert_eq!(instance.as_object().unwrap().len(), 3);
|
||||||
|
|
||||||
|
assert_valid(&path, &instance);
|
||||||
|
|
||||||
|
// The one field the caller actually has a value for - plain
|
||||||
|
// serde_json::Value indexing, no special "overrides" API needed.
|
||||||
|
instance["item_id"] = json!("item-42");
|
||||||
|
assert_valid(&path, &instance);
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn allof_merge_lets_a_later_branch_override_an_earlier_ones_property() {
|
||||||
|
let path = fixture("events/item_deleted.schema.json");
|
||||||
|
let instance = build_instance_from_file(&path).unwrap();
|
||||||
|
|
||||||
|
// "type" is defined twice across the allOf: a plain string in the
|
||||||
|
// shared envelope, then overridden to `const: "item_deleted"` in this
|
||||||
|
// event's own branch - the later branch has to win.
|
||||||
|
assert_eq!(instance["type"], json!("item_deleted"));
|
||||||
|
assert_eq!(instance.as_object().unwrap().len(), 2); // event_id, type
|
||||||
|
|
||||||
|
assert_valid(&path, &instance);
|
||||||
|
}
|
||||||
@@ -0,0 +1,11 @@
|
|||||||
|
{
|
||||||
|
"$schema": "https://json-schema.org/draft/2020-12/schema",
|
||||||
|
"title": "EventEnvelope",
|
||||||
|
"type": "object",
|
||||||
|
"properties": {
|
||||||
|
"event_id": { "type": "string" },
|
||||||
|
"type": { "type": "string" },
|
||||||
|
"note": { "type": "string" }
|
||||||
|
},
|
||||||
|
"required": ["event_id", "type"]
|
||||||
|
}
|
||||||
@@ -0,0 +1,15 @@
|
|||||||
|
{
|
||||||
|
"$schema": "https://json-schema.org/draft/2020-12/schema",
|
||||||
|
"title": "ItemCreatedEvent",
|
||||||
|
"allOf": [
|
||||||
|
{ "$ref": "../common/envelope.schema.json" },
|
||||||
|
{
|
||||||
|
"type": "object",
|
||||||
|
"properties": {
|
||||||
|
"type": { "const": "item_created" },
|
||||||
|
"item_id": { "type": "string" }
|
||||||
|
},
|
||||||
|
"required": ["item_id"]
|
||||||
|
}
|
||||||
|
]
|
||||||
|
}
|
||||||
@@ -0,0 +1,13 @@
|
|||||||
|
{
|
||||||
|
"$schema": "https://json-schema.org/draft/2020-12/schema",
|
||||||
|
"title": "ItemDeletedEvent",
|
||||||
|
"allOf": [
|
||||||
|
{ "$ref": "../common/envelope.schema.json" },
|
||||||
|
{
|
||||||
|
"type": "object",
|
||||||
|
"properties": {
|
||||||
|
"type": { "const": "item_deleted" }
|
||||||
|
}
|
||||||
|
}
|
||||||
|
]
|
||||||
|
}
|
||||||
@@ -1,5 +1,6 @@
|
|||||||
{
|
{
|
||||||
"name": "@continuum/ts-core",
|
"name": "@continuum/ts-core",
|
||||||
|
"license": "UNLICENSED",
|
||||||
"version": "0.1.0",
|
"version": "0.1.0",
|
||||||
"private": true,
|
"private": true,
|
||||||
"type": "module",
|
"type": "module",
|
||||||
|
|||||||
Reference in New Issue
Block a user