capnp-janet architecture #

1 Layers #

capnp_kinds / capnp_pointer   (headers: constants + word codecs)
         |
   capnp_message              (segment arena view, resolve, readers)
         |
   capnp_builder              (multi-segment arena + far/double-far)
         |
   janet_mod                  (Janet native module + register for embeds)
         |
   capnpc-janet (planned)     (capnp compile -o plugin)

2 Design choices #

2.1 Native C core, Janet on top #

Janet embeds as a C amalgamation. The wire runtime is C so:

libgrok_policyd can link Cap’n readers without the Janet VM for host-side framing, and
the same readers register into the sealed Janet pack environment for zero-copy field access on the Cap’n message the host already built.

This mirrors capnp-fortran (native language runtime, not a thin FFI over C++) more than pycapnp (C++ wrapper). c-capnproto remains the golden-master peer for byte parity, not a required runtime dependency.

2.2 Zero-copy contract #

capnp_message_view_flat aliases the caller’s stream-framed buffer. Janet message-view-buffer keeps the Janet buffer GC-marked for the message abstract lifetime. message-from-buffer copies so the buffer may be reused.

2.3 What packs see #

Packs never receive parallel C DTO tables. Host builds a Cap’n ShellView (or other schema type) with c-capnproto or this builder; the pack opens the message with capnp/* and decides. Named readers live in janet/policy.janet (compiled into pack images). capnpc-janet will emit that file from schema.

2.4 Compiled images (`.jimage`)#

make-image / janet -c freeze top-level PEGs and functions (Janet for Mortals ch. 2). Native capnp/* CFUNS cannot sit inside the image; the host re-registers them then unmarshals with janet_env_lookup / capnp_janet_lookup_into so bytecode calls resolve. Pure law libs (no Cap’n) unmarshal with core lookup only. The optional capnp.so module is for the Janet CLI compile path: (import capnp) then janet -c.

3 Embed sketch (policyd)#

C host:
  build Policy.ShellView with c-capnproto
  serialize or pass segment pointer
  janet_init; capnp_janet_register(env); load pack
  pack reads Cap'n; returns decision (Janet values or Cap'n builder)

Janet pack:
  (defn check-shell [msg]
    (def root (capnp/root msg))
    ... read argv / pathProbes via getp + get-text ...
    @{:action :deny :reason "..."})

4 Testing surface (Janet)#

C wire tests (meson test) cover capnp_get_u64 / capnp_builder_set_u64 and list-u64. Codegen smoke (test/test_codegen.sh) asserts generated UInt64=/=Int64 getters call capnp/get-u64.

**Testament / Judge suite for generated modules.** Not wired in-tree yet: product hosts amalgamate Janet (no standalone janet pkg-config / jpm on the remote builder path this library uses). A pure Testament suite would need either (1) a system Janet + jpm install of Testament as a test-only dep outside product packs, or (2) a small C host that links the amalgamation, registers capnp_janet_register, and =dofile=s generated helpers. Prefer (1) when a builder image gains Janet without pulling product amalgamation into this repo. Until then, C + codegen smoke is the regression gate for the get-u64 path.

5 List upgrade and bit/void lists #

Supported schema-evolution list views match capnp-fortran parity tests (t_list_upgrade_views / t_list_downgrade_views):

Upgrade: prim (byte/two/four/eight) and pointer lists via capnp_list_get_struct (synthetic struct; data_bits limited; bit/void refuse CAPNP_ERR_KIND).
Downgrade: composite List(Struct) as List(u*) field @0 via capnp_list_get_u*; composite as List(Text) via capnp_list_get_text.

List(Bool): capnp_list_get_bool / capnp_builder_set_list_bool (LSB-first bits). List(Void): capnp_builder_set_list_void + capnp_list_len only.

Full esize×esize upgrade matrix is an explicit non-goal; see README.

6 Builder segment policy #

See capnp_builder.h: default first segment 1024 words; spill when full (max(need, 2*prev_cap)); far landing pads grow the target segment; double-far when the target segment is at max_seg_words. force_single grows in place (canonicalize).