Scribbles for my memory

An AI agent posting to my API through Python's urllib kept getting 403 Forbidden. The same request sent with curl succeeded. The request never reached the API at all: Cloudflare rejected it at the edge.

The difference is the User-Agent header. Compare the two:

curl -s -o /dev/null -w '%{http_code} %header{server}\n' https://api.example.com/health
# 200 cloudflare
curl -s -A 'Python-urllib/3.12' https://api.example.com/health
# error code: 1010

error code: 1010 together with server: cloudflare means Cloudflare refused the client's signature. On a zone with no Bot Fight Mode and no WAF rules, the usual cause is Browser Integrity Check (BIC). It is on by default, and it blocks requests whose headers look like those of abusive bots. Against my zone I tried six User-Agents. Python-urllib/3.12 got 403. python-requests, curl, Go-http-client, node, and no UA at all all got 200. So one Python script fails while its neighbour using requests works, and the failure looks random.

You can confirm BIC is the cause with the zone settings API:

curl -s -H "X-Auth-Key: $CF_API_KEY" -H "X-Auth-Email: $CF_API_EMAIL" \
  "https://api.cloudflare.com/client/v4/zones/$ZONE/settings/browser_check"
# "value": "on"

The quick fix is to make every client send a browser-ish UA. That fixes your own scripts, but every other agent and SDK still hits the wall, and an API that only answers browsers is an odd thing to publish. A better fix is to turn BIC off only for the API hosts, and keep it on for the rest of the zone. A Configuration Rule does exactly that:

curl -s -X PUT -H "X-Auth-Key: $CF_API_KEY" -H "X-Auth-Email: $CF_API_EMAIL" \
  -H "Content-Type: application/json" \
  "https://api.cloudflare.com/client/v4/zones/$ZONE/rulesets/phases/http_config_settings/entrypoint" \
  -d '{"rules":[{
        "description": "API hosts: no Browser Integrity Check",
        "expression": "(http.host in {\"api.example.com\" \"upload.example.com\"})",
        "action": "set_config",
        "action_parameters": {"bic": false}
      }]}'

A PUT on the phase entrypoint replaces the whole rule list. If the zone already has Configuration Rules, fetch the entrypoint first and send the existing rules back along with the new one. On a zone with none, the GET returns error 10003 ("could not find entrypoint ruleset") and the PUT creates the entrypoint. The change took effect within seconds: the Python-urllib request went from 403 to the origin's own response.

Don't forget sibling hosts. The API worked after the change, but the image upload host behind it still returned 403 to the same agent until I added it to the rule.

A travel router can hand out addresses and resolve DNS while every web connection times out. On LEDE/OpenWrt, a common cause is putting the Wi-Fi client (STA) interface in the LAN zone instead of the masquerading WAN zone: DNS terminates on the router, but forwarded packets never get NATed upstream.

The Wi-Fi uplink must be a DHCP client (often called wwan), a member of the WAN firewall zone, and covered by WAN masquerading. The access-point interface and its clients stay in the LAN zone.

Inspect the zones first; anonymous section indexes vary:

uci show firewall | grep -E "zone\[[0-9]+\].(name|network)"

In the layout used by this LEDE router, the fix was:

uci del_list firewall.@zone[0].network='wwan'
uci add_list firewall.@zone[1].network='wwan'
uci commit firewall
/etc/init.d/firewall reload

The final check should show wwan in the WAN zone and a MASQUERADE rule for the active uplink:

uci show firewall | grep -E "zone\[[01]\].(name|network)"
iptables -t nat -L POSTROUTING -v -n

For a hotel Ethernet socket, connect the cable to the router's WAN device (eth0 here), disable the STA uplink, and leave the AP enabled. For hotel Wi-Fi, enable STA/wwan and leave the same AP enabled for your devices. Use one upstream at a time; two active uplinks make routing and DHCP failures much harder to diagnose. Complete any hotel captive-portal login before testing the downstream clients.

DNS success alone proves only that the router answered DNS. Verify an actual TCP request after fixing the firewall path; only then can a transparent proxy or split-routing policy reliably handle the traffic.

My tweet-queue service listens only on its Tailscale IP. Great for safety, annoying when I'm off the tailnet and want to post. The fix is not to rebind the service — it's to put a public nginx box that is also on the tailnet in front of it, and expose just the one path behind basic auth.

On the edge host (already a tailnet member, with a wildcard *.dev DNS record pointing at it):

server {
    listen 80;
    server_name x.dev.example.com;
    client_max_body_size 32m;          # match the backend's own body limit

    location = /add {
        auth_basic           "x-auto-poster";
        auth_basic_user_file /etc/nginx/.htpasswd;   # reuse the existing user file
        proxy_pass http://100.91.4.56:8090;          # backend's Tailscale IP
        proxy_set_header Authorization "";           # don't leak Basic creds upstream
    }

    location / { return 404; }
}

Then sudo certbot --nginx -d x.dev.example.com --redirect adds TLS and the HTTP→HTTPS redirect. With wildcard DNS there's no DNS step at all.

Three details that matter:

• location = /add plus a catch-all 404 means only the endpoint you meant to share is reachable — not every route the backend happens to have.
• proxy_set_header Authorization "" stops nginx's Basic header from reaching the backend. If the backend ever grows its own auth (or uses Authorization for something else, like an API key selector), a forwarded Basic header produces confusing 401/403s.
• Reusing one .htpasswd across sites is convenient, but every user in it can now hit every site that points at it. Fine for a single-user box; split the file once that stops being true.

Test it from outside: no creds should give 401, other paths 404.

curl -s -o /dev/null -w "%{http_code}\n" https://x.dev.example.com/add   # 401
curl -u user:pass -X POST https://x.dev.example.com/add \
  -H 'Content-Type: application/json' -d '{"text":"hello from anywhere"}'

A Host block only tells SSH where to connect and which key to offer. A new VM can be reachable and still return Permission denied (publickey) until the guest account trusts that public key.

First confirm the client selected the intended host, user, and identity:

ssh -G vm-alias | grep -E '^(hostname|user|identityfile) '

Then, from the VM console or hypervisor, find the account's real home instead of assuming it is /home/$USER; some Lima guests use a suffixed path:

home=$(getent passwd "$(id -un)" | cut -d: -f6)
install -d -m 700 "$home/.ssh"
cat /path/to/client_key.pub >> "$home/.ssh/authorized_keys"
chmod 600 "$home/.ssh/authorized_keys"

Only the client's .pub key belongs on the guest; keep the private key on the client. Test the complete path with:

ssh -o BatchMode=yes vm-alias 'hostname; id -un'

ssh -G validates client configuration, not server-side authorization.

An SSH .pub file is only the public half of a key pair. If it is missing, or if it is an old file that no longer matches the private key, regenerate it from the private key:

cp ~/.ssh/id_rsa.pub ~/.ssh/id_rsa.pub.backup 2>/dev/null || true
ssh-keygen -y -f ~/.ssh/id_rsa > ~/.ssh/id_rsa.pub
chmod 644 ~/.ssh/id_rsa.pub

ssh-keygen -y derives and prints the public key without exposing the private key. This also fixes errors such as:

identity_sign: private key ... contents do not match public

Verify the resulting public-key fingerprint with:

ssh-keygen -lf ~/.ssh/id_rsa.pub