Posts in category “Tips”

Snapshot backups to Cloudflare R2 without the S3 SDK

For backing up small state files (a SQLite control.db, config, keys), Cloudflare R2's S3-compatible API drags in a client, signing and a separate pair of S3 credentials. The plain Cloudflare REST API can PUT and GET objects directly — one curl, one API token, no SDK:

curl -sS -X PUT "https://api.cloudflare.com/client/v4/accounts/$ACCOUNT_ID/r2/buckets/$BUCKET/objects/$KEY" \
  -H "Authorization: Bearer $TOKEN" \
  --data-binary @snapshot.tar.age -H "Content-Type: application/octet-stream"

GET is the same with -o snapshot.tar.age, DELETE with -X DELETE. No egress fees on R2, and the free tier (10 GB storage, 1M Class A writes/month) dwarfs an hourly-snapshot workload.

Minting the token needs no dashboard click either — the Global API Key can do it through the API. The shape that finally worked (after a few 400s worth of resources-expression spelunking):

curl -sS -X POST "https://api.cloudflare.com/client/v4/user/tokens" \
  -H "X-Auth-Key: $CF_API_KEY" -H "X-Auth-Email: $CF_API_EMAIL" \
  -H "Content-Type: application/json" --data '{
  "name": "snapshots-backup",
  "policies": [{"effect": "allow",
    "resources": {"com.cloudflare.api.account.<ACCOUNT_ID>": "*"},
    "permission_groups": [
      {"id": "b4992e1108244f5d8bfbd5744320c2e1"},
      {"id": "bf7481a1826f439697cb59a20b22293e"}
    ]}],
  "expires_on": "2027-10-17T00:00:00Z"
}'

Those two group ids are Workers R2 Storage Read and Write. The response carries the token value exactly once — pipe it straight to its destination file (umask 077, owner root, 0600) and never let it touch a terminal or a shell variable you might echo.

Two traps when verifying from the host that holds the credential:

  • ssh -n redirects stdin from /dev/null and silently wins over < file — the credential file lands 0 bytes. Drop -n when piping stdin.
  • If the credential file is root-only, curl -H "Bearer $(cat ...)" in your ssh shell reads nothing and the API answers 9106: Missing Authorization header. Run the whole probe under sudo sh -c so the expansion happens where the file is readable.

Encrypt before upload so the host leak doesn't matter: age with the public key on the host and the private key in your password manager. Restore = GET on the operator machine, decrypt, PRAGMA integrity_check.

Windows 11 eating 2 GB a day of paged pool under Git Bash? Set ForegroundLockTimeout to 0 (the live value)

My Windows 11 dev box running AI agents through MSYS2/Git Bash lost about 2 GB of RAM a day to kernel paged pool. A reboot fixed it, and then it started climbing again. PoolMon-style tag stats put almost all of it under Toke (security token objects): ~3 million live tokens.

The tell-tale sign: millions of token objects, but only ~1,400 token handles system-wide. No process is holding them, so you won't find the culprit in Task Manager or with handle.exe. Something in the kernel is keeping a reference.

The culprit turns out to be a Windows 11 25H2/26H2 bug in win32kfull.sys (seen on 10.0.26100.94xx). Every new process that attaches to a console leaks one reference to its primary token whenever ForegroundLockTimeout is non-zero. Shell-heavy workloads hit it hard: each bash → child → grandchild spawn leaves a token behind. My WPR pool trace showed ~50 process starts per second from agent polling loops, leaking ~11 tokens per second.

The workaround is to set the timeout to 0. The registry value alone does nothing, though: at every logon win32k overwrites the live value with 2147483647, whatever HKCU\Control Panel\Desktop says. You have to set the live value with SystemParametersInfo, from inside the interactive session:

Add-Type -Namespace W -Name U -MemberDefinition @'
[DllImport("user32.dll", SetLastError=true)] public static extern bool SystemParametersInfo(uint a, uint b, ref uint c, uint d);
[DllImport("user32.dll", SetLastError=true)] public static extern bool SystemParametersInfo(uint a, uint b, IntPtr c, uint d);
'@
$v = [uint32]0
[void][W.U]::SystemParametersInfo(0x2000, 0, [ref]$v, 0)      # SPI_GETFOREGROUNDLOCKTIMEOUT
"before=$v"
[void][W.U]::SystemParametersInfo(0x2001, 0, [IntPtr]0, 2)    # SPI_SETFOREGROUNDLOCKTIMEOUT, SPIF_SENDCHANGE
[void][W.U]::SystemParametersInfo(0x2000, 0, [ref]$v, 0)
"after=$v"

The value resets at every logon, so run that script from a scheduled task with a logon trigger in interactive mode. This even works from an elevated SSH session:

schtasks /create /tn token-leak-flt-fix /sc onlogon /delay 0001:00 /it /f ^
  /tr "powershell.exe -NoProfile -ExecutionPolicy Bypass -WindowStyle Hidden -File C:\ProgramData\flt-fix.ps1"
schtasks /run /tn token-leak-flt-fix

Running the script over plain SSH would change session 0's value, which doesn't help. /it makes the task run in the logged-on user's session, where the console processes are. Mine went from before=2147483647 to after=0, and the leak rate dropped from 11.5 tokens/s to 0.26 tokens/s within minutes.

There are two catches. First, at 0 any app can steal foreground focus. Second, tokens that already leaked are freed only by a real Restart: with Fast Startup on, "Shut down" saves and restores the kernel, leak included.

I spent an hour on pool tracing before searching for "windows toke leaking", which turned up the answer right away (microsoft/terminal#20781, git-for-windows/git#6471, and the logon-fix write-up at gunrgbcr/windows-token-leak-logon-fix). When a symptom is this specific, search for it first.

Pair review with an agent, one diff block at a time — and the browser follows along

Diffwalk's own agent skill is a one-shot author: the agent reads the whole change, writes an ordered explanation for every block, and hands you a finished walkthrough. That's great for documenting your own PR, but when you're reviewing a colleague's PR it anchors you to the agent's verdict before you've formed your own.

pair-review turns the same CLI into a dialogue. The agent slices the diff into review units (a signature change travels with its call sites), opens the diff in the browser, then takes one unit per round: says what it does, raises its own suspicions, and stops for your take. The agreed text — not the debate — goes into the walk's explanations.yaml, so an interrupted review resumes at the next pending unit instead of starting over. A GitHub comment is only posted after you confirm its exact draft.

The newest bit: when the agent talks about a unit, the browser scrolls there and highlights the exact lines. It works through the Chrome DevTools Protocol. split-view launches a dedicated Chrome/Edge (separate profile, CDP port 9333) in the top half of the screen with the terminal below. Then each round the agent runs something like:

node scripts/focus.mjs change-003 --at src/Orders/OrderService.cs:42-50 --at src/Orders/OrderService.cs:40:old

focus.mjs attaches to that browser with Playwright's connectOverCDP, expands the file, and marks rows in the diff's shadow DOM. This works in split or unified layout, on the new side or (:old) the deleted side. The first range is scrolled to the centre of the screen. Your terminal keeps keyboard focus throughout, unlike the old approach of typing an anchor URL into the omnibox.

It needs a patched Diffwalk build: shukebeta/diffwalk branch local-both adds a Wrap toggle for long lines and shows each block's change-0NN id on the page, so "change-012" is something you can actually see. Build it with the bun version CI pins. A different bun silently tree-shakes the report's web components, and every diff renders unstyled:

git clone -b local-both https://github.com/shukebeta/diffwalk.git
cd diffwalk
npx --yes [email protected] install
npx --yes [email protected] run build
alias diffwalk="node $PWD/dist/diffwalk.js"   # add to your shell profile

Then install the skill and its one dependency (playwright-core, no browser download — it drives the Chrome you already have):

npx skills add shuke-personal/pair-review-skill --skill pair-review -g
cd ~/.claude/skills/pair-review/scripts && npm i

Start a new agent session and say "pair review this PR". Split view plus highlighting is scripted for Windows and Linux/X11. On macOS, start the CDP browser by hand (the command is in SKILL.md §3) and focus.mjs works unchanged.

Your AI agents page you too often? Give them an oncall agent

When you run a fleet of coding agents, most of their "I'm stuck, human please" pages are not really for you. A runner is wedged, a lock is stale, a relay died, or the question is one you already answered in a runbook or on an old PR. We built an oncall agent that reads every escalation first, acts as you, and only pages you with what truly needs you. The whole thing is small, and it runs on stock my-ai-team (mat) features. It took two days to go from idea to running system.

The flow:

agent ── notify-user --action ──► Telegram (you, unchanged)
   └── ssh oncall-host oncall-inbox        (same message on stdin)
             │
        oncall agent: triage (env / decision / noise), investigate over ssh
             ├── ssh <agent-host> send-relay %<pane> reply.md   → answer the stuck agent
             ├── incident record, committed to a repo
             ├── notify-user                                    → FYI to you
             └── notify-user --action + a GitHub issue          → only when you must decide

1. Forward action messages. On every agent host, point mat at the oncall host (any ssh alias that logs in without a prompt):

git config --global mat.devopsHost oncall-host
…more

git can't pull private GitHub repos? gh was holding the token all along

You're logged into gh and can browse every private repo with it, but plain git pull over HTTPS still fails with fatal: could not read Username for 'https://github.com'. git doesn't know how to read gh's token store — nobody introduced them. (A GH_TOKEN env var makes every gh command work and can hide exactly this breakage, because plain git ignores it.)

You don't need Microsoft's Git Credential Manager (it drags in a .NET runtime). gh ships its own credential helper; one command wires it in:

gh auth setup-git

That adds a single stanza to your ~/.gitconfig:

[credential "https://github.com"]
    helper = !/usr/local/bin/gh auth git-credential

From then on, any HTTPS clone/pull/push of private repos authenticates through gh's stored token — global config, so it works for every repo on the machine. No SSH keys, no per-remote remote-url surgery, no token pasting.

Verify without letting an env token mask the result:

env -u GH_TOKEN git ls-remote https://github.com/<you>/<private-repo> HEAD

If that prints a ref, you're done.