Posts tagged with “infra”

GitHub Pages cname says "must verify" but your domain is Verified? Check your other account

You're migrating a GitHub Pages site from your personal account to an org. You verify the custom domain for the org, the org's Pages settings show it green "Verified", and then binding it to the repo fails — over and over:

You must verify your domain app.example.com before being able to use it.

Every route is blocked. The API returns 400:

gh api -X PUT repos/<org>/<repo>/pages -f cname=app.example.com
# => "Invalid cname" / "You must verify your domain..."

The repo's Settings → Pages custom-domain box throws the same error. Switching to legacy "Deploy from a branch" with a CNAME file already in the branch? The cname stays null. You can't even unpublish to reset — DELETE /repos/<org>/<repo>/pages comes back 422 Deactivating GitHub Pages for this repository is not allowed.

The verified-domains list says green. The cname check says unverified. Both are telling the truth.

The domain is also verified under your personal account. It's a leftover from when the site lived there and pointed at <user>.github.io. That personal verified claim silently takes precedence and blocks the org repo from binding the cname. GitHub never says "this domain is claimed elsewhere" — just the generic must-verify error, which sends you down a rabbit hole of re-verifying on the org side that never helps.

Fix: remove the verified domain from the personal account (user Settings → Pages → verified domains → remove), then bind the cname on the org repo. It goes through immediately.

A tell that this is your problem: the custom domain used to resolve to <user>.github.io before the migration.

One related trap: apex-domain verification does not auto-cover subdomains for cname binding, despite the docs implying it does. app.example.com needs its own _gh-<org>-o.app.example.com TXT even when example.com is already verified — otherwise the same must-verify wall.

Migrating a vagrant-libvirt VM to a newer host: skip `vagrant package`, hand it to virsh

Moving a vagrant-libvirt VM between Linux hosts looks like a two-step (vagrant package then vagrant up on the other side), and that's exactly what I tried. Both steps died in the same place: fog-libvirt's stream upload/download of a large qcow2 from/to libvirt's storage pool reset mid-flight (Cannot recv data: Connection reset by peer, hung at 0%).

The streaming bug is in fog-libvirt's vol upload/download. The fix is to bypass vagrant-libvirt's vol-upload/vol-download entirely: flatten the overlay qcow2 against its backing file, drop the result in a libvirt storage pool by hand, then virsh define + virsh start. Treat vagrant-libvirt as the boot-time scaffolding only; the running VM is plain libvirt after that.

1. Make the box self-contained

vagrant package exists to bake the VM's disk + metadata into a .box file. With a libvirt provider the disk is usually a qcow2 with a backing file (qemu-img info ... | grep "backing file"), and vol-download only streams the overlay — you'd ship an incomplete box. Skip it and flatten manually:

…more

SSH Agent Forwarding: Stop Copying Your Private Key to Every Jump Host

You SSH through a bastion box. So you copied your private key onto it. Now that key lives on one more machine — one more place it can be stolen from, one more copy to rotate when something goes wrong.

SSH agent forwarding removes the need entirely.

What it actually does

This is the part most explanations hand-wave past. Your private key never needed to be on the bastion — that's not how SSH auth works.

  • The target machine needs your public key (in ~/.ssh/authorized_keys). It always did.
  • The client (you) holds the private key and signs a challenge the target sends.

Without forwarding, when you SSH from the bastion to a third machine, the bastion becomes the client — so it needs the private key to sign. That's the only reason you ever copied it there.

With forwarding, the bastion doesn't sign anything itself. It forwards the challenge back to your agent on your laptop, your agent signs it, and the signature travels back. The bastion never touches the private key.

No forwarding:   laptop (key) → bastion (key) → target (pubkey)
With forwarding: laptop (key+agent) → bastion (no key) → target (pubkey)

Setup

  1. Load your key into the agent on your laptop (the agent is usually already running):
ssh-add ~/.ssh/id_rsa
  1. Enable forwarding for the bastion in ~/.ssh/config:
Host bastion
    HostName bastion.example.com
    User myuser
    ForwardAgent yes
  1. Verify it works:
ssh bastion
ssh-add -l        # lists your keys → forwarding is live
ssh internal-vm   # connects, bastion never had your key

The catch

Only forward through machines you trust. Anyone with root on the bastion can request signatures from your agent while your session is open — effectively borrowing your identity. Never use -A on a shared or untrusted host.

Do you still need it with a passphrase-less key?

If your key has no passphrase and you've already copied it everywhere, forwarding isn't strictly required — direct auth from the bastion works fine. But keep ForwardAgent yes in the config anyway. It costs nothing, and the day you switch to a passphrase-protected key, you'll only ssh-add once on your laptop instead of typing the passphrase on every connection.

Two GitHub accounts on one Windows box, HTTPS only? Don't let `gh` be your git credential helper

If you run gh auth setup-git, the GitHub CLI becomes git's credential helper — and it only ever hands out the token for the active account. The moment you git pull a repo belonging to your other account, you get the wrong token (403, or commits land under the wrong identity). There's no per-repo routing; you'd have to gh auth switch every single time you cross accounts.

You can prove it to yourself — ask the helper for the non-active account and it returns nothing:

"protocol=https`nhost=github.com`nusername=second-account`n`n" |
  & gh auth git-credential get      # exit 1, no token

The fix is to hand git over to Git Credential Manager (it already ships with Git for Windows). GCM picks the token by the username embedded in the remote URL, so routing becomes automatic. gh keeps working for gh issue/gh pr — it uses its own keyring, separate from the git helper.

Switch the helper and point GCM at GitHub:

git config --global --unset-all credential.https://github.com.helper
git config --global credential.helper manager
git config --global credential.https://github.com.provider github
git config --global credential.guiPrompt false   # terminal prompts, no GUI popup on a server
…more

Short-lived tokens don't belong in the same file as long-lived credentials

You protect ~/.bashrc.secret with chmod 444 so nothing accidentally overwrites your API keys and passwords. Then your token-refresh script needs to write a new session cookie somewhere — and hits Permission denied.

The fix isn't adding a second writable secrets file. It's stop persisting short-lived tokens entirely.

Refactor the refresh script to output TOKEN HASH on stdout and let the caller capture it:

# refresh_token — reads credentials, prints session token to stdout
CREDENTIALS=$(bash ~/bin/refresh_token 2>/dev/null)
TOKEN=$(echo "$CREDENTIALS" | awk '{print $1}')
HASH=$(echo "$CREDENTIALS" | awk '{print $2}')

No file writes. No second secrets layer. The token lives in the process environment for the duration of the operation and disappears when it's done.

This works because session tokens are cheap to re-obtain — one HTTP call with stored credentials. The only reason to persist them is avoiding that call, which is almost never worth the complexity of managing a writable token store alongside a read-only credential store.

The rule: if you can regenerate it from long-lived credentials in under a second, don't persist it.