Self-hosted keybr with a TOTEM/PUQ30 geometry #3

New issue

Closed

opened 2026-08-29 18:11:23 +02:00 by ma3s7r0 · 5 comments

ma3s7r0 commented 2026-08-29 18:11:23 +02:00

Owner

Copy link

Tracking issue for standing up a local keybr instance that can drive the PUQ30
relearn. Follows on from #2 §5, which priced this at "roughly half a day" and
left it as an option rather than a decision.

Why

The relearn currently drills through Monkeytype custom text fed from
training/stage-*.txt. That works, but it is a static staircase: the stage
files are hand-generated letter sets and advancing between them is a manual
accuracy judgement.

keybr does the same thing adaptively — it introduces letters one at a time keyed
to per-key accuracy, and generates pseudo-words from a language model for both
en and de. That is precisely what training/README.md builds by hand.

Two further wins, both currently listed as disabled settings in
training/README.md:

• The on-screen keymap is off today because "it draws a staggered ANSI board
  that does not match this keyboard." A TOTEM geometry fixes that.
• The finger colouring in training/finger-map.svg (from tools/finger_map.py)
  has to be read from a second window. keybr colours its own board by finger.

keybr.com's hosted site has no custom-layout upload — upstream #463 is
that exact request, still open — so this means running a modified instance.

The hypothesis to verify first

This is probably not a keybr layout problem.

PUQ30 lives in firmware. The host receives an ordinary US keyboard: pressing the
right-hand middle home key sends HID usage T, which the browser reports as
code: "KeyT". Character and physical code always agree on the base layer.
keybr's layout files (packages/keybr-keyboard-generator/layout/, the Layout
class) exist for host-side layout emulation — the same thing already switched
off in Monkeytype, and for the same reason.

What is actually missing is a keybr geometry: a 38-key column-staggered
TOTEM board that places each US key code at its real physical position, with the
correct finger/zone per key. Layout stays plain US, emulation stays off.

If that holds, the fork is one geometry file plus a registration rather than a
layout definition and a generate-layouts run.

Fallback: if keybr's lesson generator draws its letter ordering from
Layout rather than Geometry, add a PUQ30 layout entry as well and commit the
generated output.

Inputs already exist: key positions in
config/boards/shields/totem/totem.dtsi, column-to-finger assignment in
tools/finger_map.py.

Decisions

Decision	Choice	Why
Where the fork lives	ma3s7r0/keybr-totem	The change edits upstream source in more than one place. A nix applyPatches patch against a moving master rots; a rebasable fork branch lets git do the merging. nix-config stays consumer-only.
Host	errata	Already terminates TLS for *.cl12.de with Caddy + ACME, already runs a Forgejo runner and podman. heimdall is LAN/tailscale-only; drills happen from work machines too.
Packaging	plain nix first (buildNpmPackage + a systemd unit), container as documented fallback	keybr has a single root package-lock.json, so buildNpmPackage on the workspace is the normal path and the unit is small. If the workspace build fights back, switch to virtualisation.oci-containers against upstream's own Dockerfile, image built by Forgejo Actions in the fork.
Exposure	keybr.cl12.de, public	It is a typing trainer; progress is per-browser unless an account is made. Tailscale-only is a two-line vhost change if that turns out to be preferred.

Work

• Verify the geometry-vs-layout hypothesis against packages/keybr-keyboard/lib/ and packages/keybr-lesson/
• Mirror aradzie/keybr.com onto git.cl12.de, as zmkfirmware/zmk and caksoylar/keymap-drawer already are
• keybr-totem: TOTEM geometry — 3×5 + 3×5 column-staggered alpha block, 2 outer keys, 6 thumbs
• keybr-totem: flake.nix exposing packages.default and nixosModules.default, so nix-config never sees npm
• nix-config: flake input, hosts/errata/modules/keybr.nix, Caddy vhost, ACME cert, host import
• DNS keybr.cl12.de → errata
• Rewrite the daily-loop section of training/README.md

Prerequisite, already met

#2 flagged niri not setting the altgr-intl variant as a blocker — without it
every German lesson would be untypable on the TOTEM. Fixed in nix-config
19775ac (2026-08-27); see #2 (comment).

Out of scope

• Symbol and number training. The symbol layer does not exist yet (#2 phase 2).
• PUO30 as a second geometry. The escape hatch stays offline in training/puo30/
  unless it is actually taken.
• Accounts and multi-user setup. Single user, single browser profile.

Notes

Keep training/stage-*.txt. They are the offline fallback, and the corpus
analysis behind them in tools/layout_score.py is still the layout's
justification. The worst-bigram drill list it produces is also directly reusable
for a targeted-drill mode:

sfb:      ct  e.  ui  tc  gt  rl  ph  mb  wr  rw  kp  cd  dc  oa  tg
scissor:  bl  yp  zm  py  ku

Tracking issue for standing up a local keybr instance that can drive the PUQ30 relearn. Follows on from #2 §5, which priced this at "roughly half a day" and left it as an option rather than a decision. ## Why The relearn currently drills through Monkeytype custom text fed from `training/stage-*.txt`. That works, but it is a static staircase: the stage files are hand-generated letter sets and advancing between them is a manual accuracy judgement. keybr does the same thing adaptively — it introduces letters one at a time keyed to per-key accuracy, and generates pseudo-words from a language model for both `en` and `de`. That is precisely what `training/README.md` builds by hand. Two further wins, both currently listed as *disabled* settings in `training/README.md`: - The on-screen keymap is off today because "it draws a staggered ANSI board that does not match this keyboard." A TOTEM geometry fixes that. - The finger colouring in `training/finger-map.svg` (from `tools/finger_map.py`) has to be read from a second window. keybr colours its own board by finger. keybr.com's hosted site has no custom-layout upload — [upstream #463][463] is that exact request, still open — so this means running a modified instance. [463]: https://github.com/aradzie/keybr.com/issues/463 ## The hypothesis to verify first This is probably **not** a keybr *layout* problem. PUQ30 lives in firmware. The host receives an ordinary US keyboard: pressing the right-hand middle home key sends HID usage `T`, which the browser reports as `code: "KeyT"`. Character and physical code always agree on the base layer. keybr's layout files (`packages/keybr-keyboard-generator/layout/`, the `Layout` class) exist for host-side layout *emulation* — the same thing already switched off in Monkeytype, and for the same reason. What is actually missing is a keybr **geometry**: a 38-key column-staggered TOTEM board that places each US key code at its real physical position, with the correct finger/zone per key. Layout stays plain US, emulation stays off. If that holds, the fork is one geometry file plus a registration rather than a layout definition and a `generate-layouts` run. **Fallback:** if keybr's lesson generator draws its letter ordering from `Layout` rather than `Geometry`, add a PUQ30 layout entry as well and commit the generated output. Inputs already exist: key positions in `config/boards/shields/totem/totem.dtsi`, column-to-finger assignment in `tools/finger_map.py`. ## Decisions | Decision | Choice | Why | |---|---|---| | Where the fork lives | `ma3s7r0/keybr-totem` | The change edits upstream source in more than one place. A nix `applyPatches` patch against a moving `master` rots; a rebasable fork branch lets git do the merging. `nix-config` stays consumer-only. | | Host | `errata` | Already terminates TLS for `*.cl12.de` with Caddy + ACME, already runs a Forgejo runner and podman. `heimdall` is LAN/tailscale-only; drills happen from work machines too. | | Packaging | plain nix first (`buildNpmPackage` + a systemd unit), container as documented fallback | keybr has a single root `package-lock.json`, so `buildNpmPackage` on the workspace is the normal path and the unit is small. If the workspace build fights back, switch to `virtualisation.oci-containers` against upstream's own Dockerfile, image built by Forgejo Actions in the fork. | | Exposure | `keybr.cl12.de`, public | It is a typing trainer; progress is per-browser unless an account is made. Tailscale-only is a two-line vhost change if that turns out to be preferred. | ## Work - [ ] Verify the geometry-vs-layout hypothesis against `packages/keybr-keyboard/lib/` and `packages/keybr-lesson/` - [ ] Mirror `aradzie/keybr.com` onto git.cl12.de, as `zmkfirmware/zmk` and `caksoylar/keymap-drawer` already are - [ ] `keybr-totem`: TOTEM geometry — 3×5 + 3×5 column-staggered alpha block, 2 outer keys, 6 thumbs - [ ] `keybr-totem`: `flake.nix` exposing `packages.default` and `nixosModules.default`, so `nix-config` never sees npm - [ ] `nix-config`: flake input, `hosts/errata/modules/keybr.nix`, Caddy vhost, ACME cert, host import - [ ] DNS `keybr.cl12.de` → errata - [ ] Rewrite the daily-loop section of `training/README.md` ## Prerequisite, already met #2 flagged niri not setting the `altgr-intl` variant as a blocker — without it every German lesson would be untypable on the TOTEM. Fixed in `nix-config` `19775ac` (2026-08-27); see #2 (comment). ## Out of scope - Symbol and number training. The symbol layer does not exist yet (#2 phase 2). - PUO30 as a second geometry. The escape hatch stays offline in `training/puo30/` unless it is actually taken. - Accounts and multi-user setup. Single user, single browser profile. ## Notes Keep `training/stage-*.txt`. They are the offline fallback, and the corpus analysis behind them in `tools/layout_score.py` is still the layout's justification. The worst-bigram drill list it produces is also directly reusable for a targeted-drill mode: ``` sfb: ct e. ui tc gt rl ph mb wr rw kp cd dc oa tg scissor: bl yp zm py ku ```

ma3s7r0 commented 2026-08-29 18:14:51 +02:00

Author

Owner

Copy link

Source read: the hypothesis was wrong, and upstream already supports this case

Read against aradzie/keybr.com@master. The "geometry, not layout" framing in
the issue body does not survive contact with the source. Both are needed, and
the mechanism is one upstream already built.

Emulation.Reverse is exactly the TOTEM

packages/keybr-keyboard/lib/settings.ts defines three emulation modes, and the
third is this case verbatim — from the doc comment on reverseEmulation in
packages/keybr-textinput-events/lib/emulation.ts:

Keyboard layout switching is done in hardware. It changes physical key
locations to the QWERTY equivalents. So if the A key is pressed in a custom
keyboard layout, the hardware will send the physical key location of the A
letter in the QWERTY layout. We use a layout table and a character code as
reported by the OS to fix the physical key location.

Surfaced in the UI as "Keyboard hardware emulates layout"
(packages/page-practice/lib/settings/KeyboardSettings.tsx:156). So this is not
the Monkeytype situation where emulation must be off — keybr has a third state
for firmware-side layouts, and it is the one to select.

That inverts the plan: Emulation.Reverse takes the character the OS reports
and looks up which key the selected layout assigns it to, so a PUQ30
character dict is required — it is the lookup table that makes the highlight
land on the right key.

Why the geometry is still needed

Ordering weight comes from the geometry's row zone, not the layout —
Keyboard.getCodePoints() in packages/keybr-keyboard/lib/keyboard.ts:

switch (shape?.row) {
  case "home": weights.set(combo.codePoint, 1); break;
  case "top":  weights.set(combo.codePoint, 2); break;
}

Home row weighs 1, top row 2, everything else 1000 — that is the adaptive
letter-introduction order, and it is only correct if the geometry says which
keys are home. KeyShape derives finger, hand and row from the zone list,
and those drive ZonesLayer (finger colouring) and PointersLayer (which
finger to use) in keybr-keyboard-ui.

Good news for a 38-key board: the SVG frame is computed, not hardcoded —
getFrameSize() in packages/keybr-keyboard-ui/lib/shapes.tsx takes the max
x + w / y + h over the shapes. Fractional coordinates work, so column
stagger is just a per-column y offset. An arbitrary geometry renders and
auto-sizes.

German needs a second Layout entry, not a second dict

Layout carries its language, and KeyboardOptions.selectableLayouts() filters
by it — so one Layout serves one language. Two entries sharing one
CharacterDict is the fix, and upstream already does exactly that:
[Layout.EN_JP, LAYOUT_JA_JP_JIS] in load.ts reuses the JIS dict under an
English entry.

The umlauts fall out of the AltGr level the dict format already has. Source
format is plain JSON, up to four characters per key — base, shift, AltGr,
shift+AltGr (packages/keybr-generators/layouts/*.json):

"KeyQ": "ьЬъЪ"

The GER layer puts € ü ä ö ß on the right home row, so those become the third
character on the five right-home keys. This mirrors the host's altgr-intl
variant one-for-one, which is the same table the keymap was verified against.

Revised shape of the work

Three pieces, not one:

1. packages/keybr-generators/layouts/puq30.json — the character dict, umlauts
   at the AltGr level. Run npm run generate-layouts, commit the output.
2. Two Layout entries sharing it, Language.EN and Language.DE, both
   emulate: true, both listing the TOTEM geometry.
3. packages/keybr-keyboard/lib/geometry/totem.ts — 38 keys, positions from
   totem.dtsi, zones from tools/finger_map.py.

Then in the UI: layout PUQ30, geometry TOTEM, emulation "hardware emulates
layout".

Requirement noted

The on-screen board with finger colouring and next-key pointers is a
requirement, not polish — it is the main thing Monkeytype could not give. That
rules out shipping against Geometry.MATRIX as a first cut: the board has to
match the real TOTEM for the finger positions to be readable at a glance.
keyboard.colors and keyboard.pointers both default to true.

## Source read: the hypothesis was wrong, and upstream already supports this case Read against `aradzie/keybr.com@master`. The "geometry, not layout" framing in the issue body does not survive contact with the source. Both are needed, and the mechanism is one upstream already built. ### `Emulation.Reverse` is exactly the TOTEM `packages/keybr-keyboard/lib/settings.ts` defines three emulation modes, and the third is this case verbatim — from the doc comment on `reverseEmulation` in `packages/keybr-textinput-events/lib/emulation.ts`: > Keyboard layout switching is done in hardware. It changes physical key > locations to the QWERTY equivalents. So if the A key is pressed in a custom > keyboard layout, the hardware will send the physical key location of the A > letter in the QWERTY layout. We use a layout table and a character code as > reported by the OS to fix the physical key location. Surfaced in the UI as **"Keyboard hardware emulates layout"** (`packages/page-practice/lib/settings/KeyboardSettings.tsx:156`). So this is not the Monkeytype situation where emulation must be off — keybr has a third state for firmware-side layouts, and it is the one to select. That inverts the plan: `Emulation.Reverse` takes the character the OS reports and looks up which key *the selected layout* assigns it to, so a **PUQ30 character dict is required** — it is the lookup table that makes the highlight land on the right key. ### Why the geometry is still needed Ordering weight comes from the geometry's row zone, not the layout — `Keyboard.getCodePoints()` in `packages/keybr-keyboard/lib/keyboard.ts`: ```js switch (shape?.row) { case "home": weights.set(combo.codePoint, 1); break; case "top": weights.set(combo.codePoint, 2); break; } ``` Home row weighs 1, top row 2, everything else 1000 — that is the adaptive letter-introduction order, and it is only correct if the geometry says which keys are home. `KeyShape` derives `finger`, `hand` and `row` from the zone list, and those drive `ZonesLayer` (finger colouring) and `PointersLayer` (which finger to use) in `keybr-keyboard-ui`. Good news for a 38-key board: the SVG frame is computed, not hardcoded — `getFrameSize()` in `packages/keybr-keyboard-ui/lib/shapes.tsx` takes the max `x + w` / `y + h` over the shapes. Fractional coordinates work, so column stagger is just a per-column `y` offset. An arbitrary geometry renders and auto-sizes. ### German needs a second `Layout` entry, not a second dict `Layout` carries its language, and `KeyboardOptions.selectableLayouts()` filters by it — so one `Layout` serves one language. Two entries sharing one `CharacterDict` is the fix, and upstream already does exactly that: `[Layout.EN_JP, LAYOUT_JA_JP_JIS]` in `load.ts` reuses the JIS dict under an English entry. The umlauts fall out of the AltGr level the dict format already has. Source format is plain JSON, up to four characters per key — base, shift, AltGr, shift+AltGr (`packages/keybr-generators/layouts/*.json`): ```json "KeyQ": "ьЬъЪ" ``` The GER layer puts `€ ü ä ö ß` on the right home row, so those become the third character on the five right-home keys. This mirrors the host's `altgr-intl` variant one-for-one, which is the same table the keymap was verified against. ### Revised shape of the work Three pieces, not one: 1. `packages/keybr-generators/layouts/puq30.json` — the character dict, umlauts at the AltGr level. Run `npm run generate-layouts`, commit the output. 2. Two `Layout` entries sharing it, `Language.EN` and `Language.DE`, both `emulate: true`, both listing the TOTEM geometry. 3. `packages/keybr-keyboard/lib/geometry/totem.ts` — 38 keys, positions from `totem.dtsi`, zones from `tools/finger_map.py`. Then in the UI: layout PUQ30, geometry TOTEM, **emulation "hardware emulates layout"**. ### Requirement noted The on-screen board with finger colouring and next-key pointers is a requirement, not polish — it is the main thing Monkeytype could not give. That rules out shipping against `Geometry.MATRIX` as a first cut: the board has to match the real TOTEM for the finger positions to be readable at a glance. `keyboard.colors` and `keyboard.pointers` both default to `true`.

ma3s7r0 commented 2026-08-29 18:31:51 +02:00

Author

Owner

Copy link

Packaging: plain nix, and upstream's Docker CMD must not be used as-is

The npm build is the same work either way

Docker does not avoid building the 70-package workspace — upstream's Dockerfile
is just npm ci then npm run compile && npm run build. The only thing a
container buys is that npm ci runs with network access, so there is no
npmDepsHash to maintain and the build is not sandboxed.

Tested that constraint rather than guessing it: npm run build completes
fully offline once npm ci has run — 16s, every asset local, webpack
"compiled successfully". So the sandbox is satisfiable and the one argument for
containers goes away. Plain nix it is, and the container drops to a fallback
that is not currently needed.

Two things buildNpmPackage has to handle, both small:

• postinstall is husky && patch-package. Ignoring scripts is right for
  husky (it only installs git hooks) but wrong for patch-package — there are
  three real patches under patches/ (debug, objection, ws) that have to
  be applied. Needs an explicit step, not the default.
• Runtime entrypoint is root/index.js, plain built JS. Fine.

npm run start-docker ships a login backdoor

This is the reason the deployment cannot just be upstream's image with our
layout dropped in.

"start-docker": "env NODE_ENV=production ./packages/devenv/lib/initdb.ts && … node ./root/index.js"

packages/devenv/lib/initdb.ts does two things, and only the first is wanted:

const email = "user@localhost";
const accessToken = "xyz";
…
await createSchema(knex);
await UserLoginRequest.query().delete().where({ email });
await UserLoginRequest.query().insert({ email, accessToken });

GET /login/{token} is a real route
(packages/server/lib/app/auth/controller.ts:105) and it starts a session for
whatever user the token resolves to. So every start of an upstream-Dockerfile
container re-creates a fixed, publicly known token — anyone who has read
the repo can hit /login/xyz and land in /account as user@localhost.

That is fine for the dev container it was written for, and not fine on
keybr.cl12.de.

Consequences:

1. The systemd unit runs root/index.js directly, never start-docker, and
   does schema creation as a separate one-shot that omits the token insert.
2. Worth reconsidering the public vhost from the issue body. Adding kanidm OIDC
   while leaving a fixed-token login route reachable would be self-defeating —
   at minimum the exposure decision should be made knowing this route exists.
   Tailscale-only remains a two-line change.

Status

ma3s7r0/keybr-totem, bookmark puq30-totem, two commits on upstream master,
nothing pushed yet:

• add PUQ30 layout and TOTEM geometry
• add kanidm oidc provider

Workspace compiles (70 packages, 0 failures). @keybr/keyboard is 436/436,
including six new tests pinning the behaviour this design depends on: the home
row resolving to hieaodtrns, a/t as the index home and homing keys, the
inner column sharing the index finger, and ü ä ö ß € resolving by character
onto their GER-layer keys rather than following the RA(Y)-style code the
firmware sends. The only failing test in the full suite is @keybr/database
wanting a live MySQL on 3306, which is unrelated — production config is
DATABASE_CLIENT=sqlite, so no database service is needed on errata either.

## Packaging: plain nix, and upstream's Docker CMD must not be used as-is ### The npm build is the same work either way Docker does not avoid building the 70-package workspace — upstream's Dockerfile is just `npm ci` then `npm run compile && npm run build`. The only thing a container buys is that `npm ci` runs with network access, so there is no `npmDepsHash` to maintain and the build is not sandboxed. Tested that constraint rather than guessing it: **`npm run build` completes fully offline** once `npm ci` has run — 16s, every asset local, webpack "compiled successfully". So the sandbox is satisfiable and the one argument for containers goes away. Plain nix it is, and the container drops to a fallback that is not currently needed. Two things `buildNpmPackage` has to handle, both small: - `postinstall` is `husky && patch-package`. Ignoring scripts is right for husky (it only installs git hooks) but wrong for `patch-package` — there are three real patches under `patches/` (`debug`, `objection`, `ws`) that have to be applied. Needs an explicit step, not the default. - Runtime entrypoint is `root/index.js`, plain built JS. Fine. ### `npm run start-docker` ships a login backdoor This is the reason the deployment cannot just be upstream's image with our layout dropped in. ```json "start-docker": "env NODE_ENV=production ./packages/devenv/lib/initdb.ts && … node ./root/index.js" ``` `packages/devenv/lib/initdb.ts` does two things, and only the first is wanted: ```ts const email = "user@localhost"; const accessToken = "xyz"; … await createSchema(knex); await UserLoginRequest.query().delete().where({ email }); await UserLoginRequest.query().insert({ email, accessToken }); ``` `GET /login/{token}` is a real route (`packages/server/lib/app/auth/controller.ts:105`) and it starts a session for whatever user the token resolves to. So every start of an upstream-Dockerfile container **re-creates a fixed, publicly known token** — anyone who has read the repo can hit `/login/xyz` and land in `/account` as `user@localhost`. That is fine for the dev container it was written for, and not fine on `keybr.cl12.de`. Consequences: 1. The systemd unit runs `root/index.js` directly, never `start-docker`, and does schema creation as a separate one-shot that omits the token insert. 2. Worth reconsidering the public vhost from the issue body. Adding kanidm OIDC while leaving a fixed-token login route reachable would be self-defeating — at minimum the exposure decision should be made knowing this route exists. Tailscale-only remains a two-line change. ### Status `ma3s7r0/keybr-totem`, bookmark `puq30-totem`, two commits on upstream master, nothing pushed yet: - `add PUQ30 layout and TOTEM geometry` - `add kanidm oidc provider` Workspace compiles (70 packages, 0 failures). `@keybr/keyboard` is 436/436, including six new tests pinning the behaviour this design depends on: the home row resolving to `hieaodtrns`, `a`/`t` as the index home and homing keys, the inner column sharing the index finger, and `ü ä ö ß €` resolving *by character* onto their GER-layer keys rather than following the `RA(Y)`-style code the firmware sends. The only failing test in the full suite is `@keybr/database` wanting a live MySQL on 3306, which is unrelated — production config is `DATABASE_CLIENT=sqlite`, so no database service is needed on errata either.

ma3s7r0 commented 2026-08-29 19:20:24 +02:00

Author

Owner

Copy link

Built and wired. One secret left to add.

Verified, not assumed

Ran against the store-built package, not the dev tree:

Check	Result
Workspace compile	70 packages, 0 failures
@keybr/keyboard tests	436/436
@keybr/page-account tests	6/6
Full suite	only @keybr/database fails, wanting MySQL on 3306 — pre-existing, unrelated
nix build .#keybr	ok, 81 MB closure
GET / and /account	200
GET /login/xyz	403 — the devenv backdoor account does not exist
GET /auth/oauth-init/kanidm	302 to kanidm
errata system.build.toplevel	builds, except the sops secret below

The kanidm redirect resolves to exactly the right place:

https://idm.cl12.de/ui/oauth2?response_type=code&client_id=keybr
  &scope=openid+email+profile
  &redirect_uri=…%2Fauth%2Foauth-callback%2Fkanidm&state=…

Three things the source made us change course on

The schema command's exit code is unusable on sqlite, non-deterministically.
The schema commits and then node aborts finalizing better-sqlite3 statements
during teardown (Assertion failed: (env) != nullptr). Neither
knex.destroy() nor process.exit(0) avoids it — the destructors run during
isolate disposal regardless. Measured across six runs it returned 0 or 134 at
random, on both fresh and repeat runs, so an exit-code-based ExecStartPre
would have failed randomly at boot. nix/module.nix checks the postcondition
instead: four expected tables present. The command is idempotent
(createSchema guards each table with hasTable), so retrying is free.

MAIL_* is not optional. The auth controller's DI graph constructs
MailgunConfig eagerly and Env.getString throws on a missing property, so
without it the entire /auth and /login subtree 500s — the OAuth routes
included. Found by getting a 500 on /auth/oauth-init/kanidm. The module sets
placeholders; only the e-mail login flow would use them and that is not the way
in here.

npm run compile cannot run in the nix sandbox and does not need to. It
goes through lage, which shells out to git to hash the manifest; there is
neither git nor a repo in the sandbox. webpack transpiles the TypeScript
itself, so npm run build alone produces both bundles. Typechecking stays a CI
concern.

Commits

ma3s7r0/keybr-totem, bookmark puq30-totem, pushed, four commits on upstream
master:

• add PUQ30 layout and TOTEM geometry
• add schema command, for init without the devenv login token
• add kanidm oidc provider
• package with nix: buildNpmPackage and a nixos module

ma3s7r0/nix-config, bookmark keybr, not pushed: flake input, host module,
Caddy vhost, ACME cert, kanidm OAuth2 client and a keybr-users group.

ma3s7r0/zmk-config-dongle-totem, bookmark keybr-puq30, not pushed:
tools/keybr_layout.py, which generates both keybr artifacts from
totem.dtsi and layout_score.LAYOUTS. Nothing transcribed by hand, and
--layout puo30 regenerates for the escape hatch the same way the stage lists
already do.

Blocked on

KEYBR_OIDC_CLIENT_SECRET is missing from hosts/errata/secrets/secrets.yml,
which is the only thing stopping the errata system from building. It wants a
fresh random value — kanidm provisioning sets the client secret from that
file, so anything random works.

Also still to do, deliberately deferred until the service is confirmed working
rather than written against an unverified deployment: a DNS record for
keybr.cl12.de, and the training/README.md rewrite of the daily loop.

## Built and wired. One secret left to add. ### Verified, not assumed Ran against the store-built package, not the dev tree: | Check | Result | |---|---| | Workspace compile | 70 packages, 0 failures | | `@keybr/keyboard` tests | 436/436 | | `@keybr/page-account` tests | 6/6 | | Full suite | only `@keybr/database` fails, wanting MySQL on 3306 — pre-existing, unrelated | | `nix build .#keybr` | ok, 81 MB closure | | `GET /` and `/account` | 200 | | `GET /login/xyz` | **403** — the devenv backdoor account does not exist | | `GET /auth/oauth-init/kanidm` | **302** to kanidm | | errata `system.build.toplevel` | builds, except the sops secret below | The kanidm redirect resolves to exactly the right place: ``` https://idm.cl12.de/ui/oauth2?response_type=code&client_id=keybr &scope=openid+email+profile &redirect_uri=…%2Fauth%2Foauth-callback%2Fkanidm&state=… ``` ### Three things the source made us change course on **The schema command's exit code is unusable on sqlite, non-deterministically.** The schema commits and then node aborts finalizing better-sqlite3 statements during teardown (`Assertion failed: (env) != nullptr`). Neither `knex.destroy()` nor `process.exit(0)` avoids it — the destructors run during isolate disposal regardless. Measured across six runs it returned 0 or 134 at random, on both fresh and repeat runs, so an exit-code-based `ExecStartPre` would have failed randomly at boot. `nix/module.nix` checks the postcondition instead: four expected tables present. The command is idempotent (`createSchema` guards each table with `hasTable`), so retrying is free. **`MAIL_*` is not optional.** The auth controller's DI graph constructs `MailgunConfig` eagerly and `Env.getString` throws on a missing property, so without it the entire `/auth` and `/login` subtree 500s — the OAuth routes included. Found by getting a 500 on `/auth/oauth-init/kanidm`. The module sets placeholders; only the e-mail login flow would use them and that is not the way in here. **`npm run compile` cannot run in the nix sandbox and does not need to.** It goes through lage, which shells out to `git` to hash the manifest; there is neither git nor a repo in the sandbox. webpack transpiles the TypeScript itself, so `npm run build` alone produces both bundles. Typechecking stays a CI concern. ### Commits `ma3s7r0/keybr-totem`, bookmark `puq30-totem`, pushed, four commits on upstream master: - `add PUQ30 layout and TOTEM geometry` - `add schema command, for init without the devenv login token` - `add kanidm oidc provider` - `package with nix: buildNpmPackage and a nixos module` `ma3s7r0/nix-config`, bookmark `keybr`, not pushed: flake input, host module, Caddy vhost, ACME cert, kanidm OAuth2 client and a `keybr-users` group. `ma3s7r0/zmk-config-dongle-totem`, bookmark `keybr-puq30`, not pushed: `tools/keybr_layout.py`, which generates both keybr artifacts from `totem.dtsi` and `layout_score.LAYOUTS`. Nothing transcribed by hand, and `--layout puo30` regenerates for the escape hatch the same way the stage lists already do. ### Blocked on `KEYBR_OIDC_CLIENT_SECRET` is missing from `hosts/errata/secrets/secrets.yml`, which is the only thing stopping the errata system from building. It wants a fresh random value — kanidm provisioning sets the client secret *from* that file, so anything random works. Also still to do, deliberately deferred until the service is confirmed working rather than written against an unverified deployment: a DNS record for `keybr.cl12.de`, and the `training/README.md` rewrite of the daily loop.

ma3s7r0 commented 2026-08-29 23:41:22 +02:00

Author

Owner

Copy link

Done: live at https://keybr.cl12.de, signed in, PUQ30 selectable

All the work items are complete. What follows is what the plan got wrong,
since that is the part worth keeping.

The hypothesis in the issue body was wrong

"This is probably not a keybr layout problem" — it was both, and the
mechanism was one upstream had already built.

Emulation.Reverse exists precisely for firmware-side layouts. Its own doc
comment describes this keyboard: "Keyboard layout switching is done in
hardware… we use a layout table and a character code as reported by the OS to
fix the physical key location." Surfaced in the UI as "Keyboard hardware
emulates layout". So emulation is not switched off as it is in Monkeytype —
it is switched to the third mode.

That requires a character dict after all, because reverse emulation looks the
typed character up to recover which key was pressed. And the geometry is still
required, because Keyboard.getCodePoints() weights letters by the geometry's
row zone — home 1, top 2, everything else 1000 — which is what makes lessons
start on the home row.

So: layout and geometry, not geometry alone.

What was actually built

Piece	Where
Generator for both artefacts, from totem.dtsi + layout_score.LAYOUTS	tools/keybr_layout.py, this repo
PUQ30 character dict, TOTEM geometry, EN_PUQ30/DE_PUQ30	ma3s7r0/keybr-totem
kanidm OIDC provider, PKCE, SERVER_HOST, schema subcommand	same
buildNpmPackage + NixOS module	same, nix/
errata host module, MariaDB, Caddy vhost, ACME, kanidm client	ma3s7r0/nix-config

Nothing is transcribed by hand: --layout puo30 regenerates for the escape
hatch exactly as the stage lists already do.

Four things only the deployment could teach

npm run start-docker ships a login backdoor. It runs devenv/initdb.ts,
which re-inserts a fixed token for user@localhost on every start, and
GET /login/{token} starts a session for whoever presents it. The unit runs
root/index.js directly and a schema subcommand that omits the token.

sqlite is unusable on this build. better-sqlite3's native Statement
finalizer aborts the process — Assertion failed: (env) != nullptr — landing
on either side of a commit. It returned 0 or 134 on identical runs, left 0/4
tables on a real first boot, and killed the worker mid-OAuth-callback, which is
why the sign-in failure appeared as a bare 502 with no logged cause: the abort
ate the log line before it flushed. Switched to MariaDB, which is keybr's own
default (DATABASE_CLIENT defaults to mysql; only .env.example says
otherwise) and what upstream tests against. Notably upstream's CI passes on
sqlite, so the fault is in this build environment, not their code.

kanidm enforces PKCE, even for confidential clients, and keybr's adapter
never sent a challenge. Implemented properly (S256, verifier in the session)
rather than disabling the requirement.

kanidm subjects are UUIDs. user_external_id.external_id was sized for
Google's 21-digit sub; 36 characters overflowed it. Widened to 255, the OIDC
maximum.

Corrections to the issue body's decisions

• Packaging: plain nix, as planned, but for a different reason than
  written. The container was never the cheaper option — upstream's Dockerfile
  runs the same npm ci && compile && build. What settled it was testing that
  the build works fully offline, so the nix sandbox was never the obstacle.
• ensureUsers socket auth does not work here. keybr uses the old mysql
  npm driver, which never implemented that handshake and fails with
  ER_NOT_SUPPORTED_AUTH_MODE. Still passwordless: the user is switched to
  native auth with an empty password, which is no weaker than the postgres
  beside it, and the listener is loopback-only.
• Exposure: public, as decided — but e-mail sign-in is now refused at
  Caddy. It could not complete a login without a mailer, yet it still inserted
  a UserLoginRequest row per request from anyone, and the only thing between
  a stranger and an account was that delivery happened to be unconfigured.
  kanidm is now structurally the only way in.

Beyond the plan

• Binds 127.0.0.1 only (SERVER_HOST, added upstream-side), so it adds
  nothing to ma3s7r0/nix-config#118.
• systemd exposure 3.9 → 1.2.
• Backups: services.mysqlBackup plus user_stats/. That second path is the
  one that matters — the typing history is plain files under
  user_stats/<000>/<000>/<userid>, not rows in the database, so a
  database-only backup would have preserved the login and lost the training.
• training/README.md now documents both routes. keybr cannot interleave
  languages — one phonetic model per lesson — so the -mix files and
  Monkeytype remain the way to drill the DE/EN switch.

Still unverified

Nobody has yet confirmed by eye that the board draws as a 38-key split with the
homing marks on a and t, or that lessons open on the home row. The tests in
keybr-keyboard/lib/totem.test.ts pin both, and the README states them as
fact. If the board looks wrong, that is what to correct.

## Done: live at https://keybr.cl12.de, signed in, PUQ30 selectable All the work items are complete. What follows is what the plan got wrong, since that is the part worth keeping. ### The hypothesis in the issue body was wrong "This is probably **not** a keybr *layout* problem" — it was both, and the mechanism was one upstream had already built. `Emulation.Reverse` exists precisely for firmware-side layouts. Its own doc comment describes this keyboard: *"Keyboard layout switching is done in hardware… we use a layout table and a character code as reported by the OS to fix the physical key location."* Surfaced in the UI as **"Keyboard hardware emulates layout"**. So emulation is not switched off as it is in Monkeytype — it is switched to the third mode. That requires a character dict after all, because reverse emulation looks the typed character up to recover which key was pressed. And the geometry is still required, because `Keyboard.getCodePoints()` weights letters by the geometry's row zone — home 1, top 2, everything else 1000 — which is what makes lessons start on the home row. So: layout **and** geometry, not geometry alone. ### What was actually built | Piece | Where | |---|---| | Generator for both artefacts, from `totem.dtsi` + `layout_score.LAYOUTS` | `tools/keybr_layout.py`, this repo | | PUQ30 character dict, TOTEM geometry, `EN_PUQ30`/`DE_PUQ30` | `ma3s7r0/keybr-totem` | | kanidm OIDC provider, PKCE, `SERVER_HOST`, `schema` subcommand | same | | `buildNpmPackage` + NixOS module | same, `nix/` | | errata host module, MariaDB, Caddy vhost, ACME, kanidm client | `ma3s7r0/nix-config` | Nothing is transcribed by hand: `--layout puo30` regenerates for the escape hatch exactly as the stage lists already do. ### Four things only the deployment could teach **`npm run start-docker` ships a login backdoor.** It runs `devenv/initdb.ts`, which re-inserts a fixed token for `user@localhost` on every start, and `GET /login/{token}` starts a session for whoever presents it. The unit runs `root/index.js` directly and a `schema` subcommand that omits the token. **sqlite is unusable on this build.** `better-sqlite3`'s native `Statement` finalizer aborts the process — `Assertion failed: (env) != nullptr` — landing on either side of a commit. It returned 0 or 134 on identical runs, left 0/4 tables on a real first boot, and killed the worker mid-OAuth-callback, which is why the sign-in failure appeared as a bare 502 with no logged cause: the abort ate the log line before it flushed. Switched to MariaDB, which is keybr's own default (`DATABASE_CLIENT` defaults to `mysql`; only `.env.example` says otherwise) and what upstream tests against. Notably upstream's CI passes on sqlite, so the fault is in this build environment, not their code. **kanidm enforces PKCE**, even for confidential clients, and keybr's adapter never sent a challenge. Implemented properly (S256, verifier in the session) rather than disabling the requirement. **kanidm subjects are UUIDs.** `user_external_id.external_id` was sized for Google's 21-digit `sub`; 36 characters overflowed it. Widened to 255, the OIDC maximum. ### Corrections to the issue body's decisions - **Packaging**: plain nix, as planned, but for a different reason than written. The container was never the cheaper option — upstream's Dockerfile runs the same `npm ci && compile && build`. What settled it was testing that the build works fully offline, so the nix sandbox was never the obstacle. - **`ensureUsers` socket auth does not work here.** keybr uses the old `mysql` npm driver, which never implemented that handshake and fails with `ER_NOT_SUPPORTED_AUTH_MODE`. Still passwordless: the user is switched to native auth with an empty password, which is no weaker than the postgres beside it, and the listener is loopback-only. - **Exposure**: public, as decided — but e-mail sign-in is now refused at Caddy. It could not complete a login without a mailer, yet it still inserted a `UserLoginRequest` row per request from anyone, and the only thing between a stranger and an account was that delivery happened to be unconfigured. kanidm is now structurally the only way in. ### Beyond the plan - Binds `127.0.0.1` only (`SERVER_HOST`, added upstream-side), so it adds nothing to `ma3s7r0/nix-config#118`. - systemd exposure 3.9 → **1.2**. - Backups: `services.mysqlBackup` plus `user_stats/`. That second path is the one that matters — the typing history is plain files under `user_stats/<000>/<000>/<userid>`, not rows in the database, so a database-only backup would have preserved the login and lost the training. - `training/README.md` now documents both routes. keybr cannot interleave languages — one phonetic model per lesson — so the `-mix` files and Monkeytype remain the way to drill the DE/EN switch. ### Still unverified Nobody has yet confirmed by eye that the board draws as a 38-key split with the homing marks on `a` and `t`, or that lessons open on the home row. The tests in `keybr-keyboard/lib/totem.test.ts` pin both, and the README states them as fact. If the board looks wrong, that is what to correct.

ma3s7r0 closed this issue 2026-08-29 23:41:30 +02:00

ma3s7r0 commented 2026-08-29 23:42:45 +02:00

Author

Owner

Copy link

Board confirmed in the browser: it renders correctly. That closes the one item
the summary above left open, so nothing in this issue is now asserted without
having been seen.

The geometry is generated from config/boards/shields/totem/totem.dtsi by
tools/keybr_layout.py, so it stays correct by construction — if the shield's
physical layout ever changes, regenerate rather than edit:

nix run nixpkgs#python3 -- tools/keybr_layout.py \
  config/boards/shields/totem/totem.dtsi ../keybr-totem --layout puq30

Same for a PUO30 pivot: --layout puo30, exactly as the stage lists and the
finger map already work.

Board confirmed in the browser: it renders correctly. That closes the one item the summary above left open, so nothing in this issue is now asserted without having been seen. The geometry is generated from `config/boards/shields/totem/totem.dtsi` by `tools/keybr_layout.py`, so it stays correct by construction — if the shield's physical layout ever changes, regenerate rather than edit: ```fish nix run nixpkgs#python3 -- tools/keybr_layout.py \ config/boards/shields/totem/totem.dtsi ../keybr-totem --layout puq30 ``` Same for a PUO30 pivot: `--layout puo30`, exactly as the stage lists and the finger map already work.

Sign in to join this conversation.

No Branch/Tag specified

Branches Tags

totem-dongle

keybr-puq30

master

No results found.

Labels

Clear labels

No items

No labels

Milestone

Clear milestone

No items

No milestone

Projects

Clear projects

No items

No project

Assignees

Clear assignees

No assignees

1 participant

Notifications

Subscribe

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference

ma3s7r0/zmk-config-dongle-totem#3

No description provided.