capnp-janet architecture #

1 Layers #

capnpc-janet                 (schema -> typed Janet helpers)
         |
   janet_mod                 (native module + register for embeds)
         |
   capnp_builder             (multi-segment arena + far/double-far)
         |
   capnp_message             (segment arena view, resolve, readers)
         |
capnp_kinds / capnp_pointer  (constants + word codecs)

2 Design choices #

2.1 Native C core, Janet on top #

Janet embeds as a C amalgamation. The wire runtime is C so:

a C application can link Cap’n readers without the Janet VM for host-side framing, and
the same readers register into the sealed Janet pack environment for zero-copy field access on the Cap’n message the host already built.

This mirrors capnp-fortran (native language runtime, not a thin FFI over C++) more than pycapnp (C++ wrapper). The official Cap’n Proto CLI and live C++ peers provide compatibility checks without becoming runtime dependencies.

2.2 Zero-copy contract #

capnp_message_view_flat aliases the caller’s stream-framed buffer. Janet message-view-buffer keeps the Janet buffer GC-marked for the message abstract lifetime. message-from-buffer copies so the buffer may be reused.

2.3 What packs see #

Packs can open a message through capnp/* without a parallel C DTO table. Named readers may live in hand-written Janet code or an importable module emitted by capnpc-janet from the schema.

2.4 Compiled images (`.jimage`)#

make-image / janet -c freeze top-level PEGs and functions (Janet for Mortals ch. 2). Native capnp/* CFUNS cannot sit inside the image; the host re-registers them then unmarshals with janet_env_lookup / capnp_janet_lookup_into so bytecode calls resolve. Pure law libs (no Cap’n) unmarshal with core lookup only. The optional capnp.so module is for the Janet CLI compile path: (import capnp) then janet -c.

3 Embed sketch #

C host:
  build an application schema with c-capnproto or this builder
  serialize or pass segment pointer
  janet_init; capnp_janet_register(env); load pack
  pack reads Cap'n; returns decision (Janet values or Cap'n builder)

Janet pack:
  (defn check-shell [msg]
    (def root (capnp/root msg))
    ... read argv / pathProbes via getp + get-text ...
    @{:action :deny :reason "..."})

4 Testing surface (Janet)#

C wire tests (meson test) cover every scalar width, typed primitive lists, arena ownership, packed/canonical transforms, and live C++ interoperability. test/test_codegen.sh generates modules from multiple schemas, checks their typed surface, then runs test/generated_builder_smoke.janet through the native module. The executable test covers scalar defaults, exact 64-bit values, unions, groups, nested structs, and primitive/struct lists.

5 List upgrade and bit/void lists #

Supported schema-evolution list views match capnp-fortran parity tests (t_list_upgrade_views / t_list_downgrade_views):

Upgrade: prim (byte/two/four/eight) and pointer lists via capnp_list_get_struct (synthetic struct; data_bits limited; bit/void refuse CAPNP_ERR_KIND).
Downgrade: composite List(Struct) as List(u*) field @0 via capnp_list_get_u*; composite as List(Text) via capnp_list_get_text.

List(Bool): capnp_list_get_bool / capnp_builder_set_list_bool (LSB-first bits). List(Void): capnp_builder_set_list_void + capnp_list_len only.

Other esize×esize combinations return CAPNP_ERR_KIND; see README.

6 Builder segment policy #

See capnp_builder.h: default first segment 1024 words; spill when full (max(need, 2*prev_cap)); far landing pads grow the target segment; double-far when the target segment is at max_seg_words. force_single grows in place (canonicalize).