Host key verification failed: six causes, one test
Host key verification failed. is the last line of an ssh run, not the first clue. OpenSSH prints it whenever it cannot confirm the server's identity against a key it trusts, and that one sentence covers at least six unrelated faults: an empty known_hosts in a fresh container, a cron job with no terminal to answer the prompt, a config block that never applies because of its position, and a key file that silently refuses the write and reports success anyway. This post reproduces each case against a real server, quotes the diagnostic line that separates them, and ends with a provisioning step that survives a redeploy.
Prerequisites: an OpenSSH client, a reachable host and port, and write access to your own known_hosts. These runs used OpenSSH 10.0p2 on Debian 13, aarch64, talking to a server that presents a freshly generated ED25519 host key on port 2222 — option names and defaults differ on older releases, so re-run the one-liners below on your own client before quoting them to a colleague.
One disclosure about the rig: this host has no sshd, so the server end is AsyncSSH 2.24.1 with a key generated by ssh-keygen. The client is genuine OpenSSH, and every string quoted below is emitted by that client — which is the side the error lives on. Server-side policy errors would look different.
TL;DR — the verdict is generic, the line above it is not
- The failure is always
Host key verification failed.with exit status 255. The diagnosis sits one line higher and only appears with-v:No ED25519 host key is known for [127.0.0.1]:2222 and you have requested strict checking. StrictHostKeyCheckingdefaults toask, andaskneeds a terminal. UnderBatchMode=yes— cron, CI, every Ansible task — the prompt cannot happen, so the default behaves likeyesand fails.ssh_configis first-match-wins. AHost *block placed aboveHost lab-*makes the specific block dead code, with no warning anywhere.UserKnownHostsFile=/dev/nullplusStrictHostKeyChecking=yesdoes not skip the check, it guarantees the failure./dev/nullonly makes sense withaccept-new.- Two widely repeated explanations are false on a current client: an entry stored without brackets does match a non-standard port, and an unset
HOMEdoes not break the write. accept-newagainst an unwritable file printsFailed to add the host to the list of known hostsand still exits 0. The run passes, nothing is stored, the next run fails.
What the client is actually checking
Start with the least informative case, because it is also the most common: a client with no entry for the endpoint at all. An empty known_hosts, strict checking, and a batch-mode client reproduce the error in one command.
With an empty known_hosts, strict checking and a batch-mode client, the run ends in Host key verification failed. with status 255. The line above the verdict names the algorithm the client looked for and the exact lookup key it used, which is the only part of the message that is specific. The file is the same size afterwards, because strict mode trusts nothing and therefore writes nothing — which is why retrying the same deploy in a loop never converges.
Two facts hide in that output. The first is the key type: the client tells you which algorithm it looked for, which matters when a server offers only one host key family and your file holds another. The second is that the file was empty before and after the run — wc -c on it returned 0, because strict mode trusts nothing and therefore writes nothing. That is the difference between a trust store and a cache, and it explains why retrying the same deploy in a loop never converges.
Everything below hinges on the same habit: run with -v (or -vv when you need the full algorithm negotiation), read the line before the verdict, and only then choose a fix. Without -v all six faults are one identical sentence.
| Cause | Measured signature | Correct fix |
|---|---|---|
| No entry for the endpoint | No ED25519 host key is known for [host]:port … strict checking | Fetch and pin the key before the run |
No TTY for the ask prompt | Verdict with no preceding No … host key is known under BatchMode=yes | accept-new on that host only |
| Config block shadowed | ssh -G host shows the wrong effective value | Move the specific block above Host * |
/dev/null plus yes | Same as cause 1, permanently | Drop one of the two options |
Unwritable known_hosts | Failed to add the host to the list of known hosts (path). with exit 0 | Fix ownership/mode, or pin with an explicit file |
HostKeyAlias renamed the lookup | Entry exists under the real host, still fails | Store the entry under the alias |
Cause 1: the entry genuinely is missing — pin it, do not disable the check
The honest fix for a fresh container is to fetch the key and store it before the session starts. ssh-keyscan does exactly that, and on a non-standard port it writes the bracketed form automatically, which is what the client looks up.
The honest fix for a fresh container is to fetch the key and store it before the session starts, and to do that in an order that cannot fail open. Scan into a temporary file rather than into the trust store; treat an empty scan, or a non-zero scan status, as fatal; compare the scanned SHA-256 fingerprint against the one held out of band before anything is written; install atomically, through a same-directory temporary file and a rename, and only after that comparison succeeds; and refuse to replace an existing pin unless a rotation was requested explicitly. A sequence that appends first and validates afterwards leaves the offered key on disk for the next run to trust — measured directly, a deliberately wrong expected fingerprint left the store at zero bytes only because the comparison ran first.
Note the first line of the file: ssh-keyscan writes a comment recording the server banner. It is harmless in known_hosts, but it is also the reason naive grep checks report the endpoint as present when the key line itself was filtered out by a stricter pipeline. Scan with a specific -t so you get the algorithm you expect rather than whatever the server lists first.
Verification is a separate step, and this is where the two tools disagree in a way that wastes hours. ssh-keygen -F searches the file for a literal host field; the client, for a non-standard port, looks up the bracketed name and also accepts a plain entry.
$ ssh-keygen -F "[127.0.0.1]:2222" -f h/kh_scan
# Host [127.0.0.1]:2222 found: line 2
[127.0.0.1]:2222 ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAILaHIV+ufTLcyjn159PQoHPcm/ilAQWSCwMn0UFK9WK8
found-bracket exit=0
$ ssh-keygen -F "127.0.0.1" -f h/kh_scan
found-plain exit=1Exit 1 there means "no line whose host field is exactly 127.0.0.1", not "the client will fail". Both statements are true at once, and the second one is the one that matters: a file holding 127.0.0.1 ssh-ed25519 … connected to port 2222 with strict checking enabled and exited 0 on this client. If you are auditing a jump host, always query with the form the client uses — [host]:port — and treat a plain-name miss as inconclusive.
Cause 2: there is no terminal to answer the prompt
StrictHostKeyChecking=ask is the default, and ask is an interactive promise. In a cron job or a CI runner nothing can answer it, so the run fails instead of hanging. The tell is that the useful diagnostic line is missing entirely: with BatchMode=yes and no prompt possible, the client goes straight to the verdict.
Same endpoint, same client, default ask policy, fresh trust store, BatchMode=yes. The verdict arrives with no key-identification line above it at all: there is no terminal to answer the prompt, so the client fails instead of waiting. That missing line is the signature of this cause, and it is why a deploy script that works when pasted into a shell fails in the pipeline.
One file, one run, no prompt, no window to accept anything. This is why a deploy script that works when you paste it into a shell fails in the pipeline: your shell has a TTY, the runner does not.
The fix is accept-new, scoped to the host that needs it, never globally. accept-new trusts an unseen key once and records it, after which the session is pinned to what it saw. It is not no: if the key later changes, accept-new still refuses, which is the property you actually want from a CI runner.
Cause 3: the config block you wrote is dead code
ssh_config resolves each keyword at the first value it finds while walking the file top to bottom. This is the opposite of most config systems, and it makes the natural way of writing a config — general rules first, exceptions after — silently wrong. The client will never tell you; ssh -G will.
Two files, identical directives, different order. The first puts the wildcard block on top:
$ ssh -G -F h/sshcfg lab-box | grep -Ei "^(stricthostkeychecking|userknownhostsfile) "
stricthostkeychecking true
userknownhostsfile /opt/data/.ssh/known_hosts
$ ssh -G -F h/sshcfg prod-box | grep -Ei "^(stricthostkeychecking|userknownhostsfile) "
stricthostkeychecking true
userknownhostsfile /opt/data/.ssh/known_hostslab-box matched Host * first and took its values: strict checking, default file. The Host lab-* block two lines below, with accept-new and a dedicated file, was never consulted. Swap the order and the same query returns what the file was written to mean.
$ ssh -G -F h/sshcfg2 lab-box | grep -Ei "^(stricthostkeychecking|userknownhostsfile) "
stricthostkeychecking accept-new
userknownhostsfile h/kh_accept
$ ssh -G -F h/sshcfg2 prod-box | grep -Ei "^(stricthostkeychecking|userknownhostsfile) "
stricthostkeychecking true
userknownhostsfile h/kh_scanSo the working layout is specific blocks first, Host * last as the default, and -o on the command line above all of it, since a command-line option beats every block. If you have ever added UserKnownHostsFile to a host block and watched the client keep reading the default file, this is why — and ssh -G would have shown it before the deploy rather than after.
Cause 4: /dev/null plus strict checking cannot work
The most copied workaround on the internet pairs UserKnownHostsFile=/dev/null with StrictHostKeyChecking=yes, presumably in the belief that discarding the file disables the check. It does the opposite: strict mode compares against a file that can never hold anything, so every connection fails forever, including the one that just succeeded a minute ago.
Discarding the file while keeping strict checking reproduces the first cause permanently: the diagnostic names the lookup key and the algorithm, and the connection fails on every attempt, including the one that succeeded a minute earlier. The pair is self-defeating by construction, not a shortcut past verification.
Discarding the file only makes sense together with accept-new, where the point is "trust on first use, per run, remember nothing". That combination connects, and it is honest about its cost: the trust decision is made fresh on every invocation, so a man-in-the-middle has the same window every time.
Paired with accept-new instead, the same empty store connects and reports the temporary addition on stderr on every single invocation. It works, it stores nothing, and the trust decision is made again from scratch each time — a deliberate one-shot probe, never a control for a job that calls itself secure.
That warning on every single run is the whole story: a trust store that persists nothing is not a trust store. Use /dev/null for a throwaway probe, never for a job that runs unattended and calls itself secure — and if a probe has to look like a real one, follow it with ssh-keygen -F "[host]:port" -f FILE before believing anything.
Cause 5: the write failed and the exit code said 0
accept-new treats the append as best effort. When the file cannot be written, the client says so once, on stderr, and then connects anyway.
Against a file the process cannot write, accept-new prints Failed to add the host to the list of known hosts (path). on stderr and then connects, exiting 0. A follow-up lookup of the bracketed host-and-port with ssh-keygen -F finds nothing, and that lookup is the only check that separates "connected" from "remembered".
Exit 0. Nothing was persisted, so the next run repeats the failure, and the pipeline looks green the whole time. The usual triggers are a known_hosts owned by root in an image built with sudo, a read-only mount, or a home directory that belongs to another uid. Two commands separate those: check ownership of the file and its directory, and confirm the write landed by looking up the endpoint instead of trusting the exit code.
Hashing adds a second trap in the same area. accept-new honours HashKnownHosts, which is on by default, so the new line looks like this rather than like the keyscan output:
|1|4xgFXE0O0UWEYWZ8aCFoTKXGQRI=|S8kW0wRpQVe2u4owpG6uIRPt2Qk= ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAILaHIV+ufTLcyjn159PQoHPcm/ilAQWSCwMn0UFK9WK8The hostname is gone, replaced by a salted hash. The client still matches it, and ssh-keygen -F still finds it because it hashes the query too — but grep 127.0.0.1 known_hosts returns nothing, and people conclude the entry is missing and start "fixing" a file that was correct. Search with ssh-keygen -F "[host]:port" -f file, or read the file through ssh-keygen -l -f when you want the fingerprints.
Cause 6: something renamed the lookup
HostKeyAlias decouples the name used for the trust lookup from the name used to connect. It exists so that twenty containers behind one address can share a single pinned key, and it is the quietest way to get this error with a correct known_hosts: the entry is under the real host, the client looks under the alias.
A trust-store entry stored under bastion-alias, with HostKeyAlias set to that name and strict checking enabled, verifies a connection addressed to the loopback endpoint: the lookup name, not the destination address, is what the client matches. Twenty containers behind one address can therefore share a single pinned key, which is the point of the option and also the quietest way to get this error with a correct known_hosts.
The entry names bastion-alias, the connection goes to 127.0.0.1, and it verifies. Grep your effective config for HostKeyAlias before you touch the key file — ssh -G host prints it like any other resolved option.
Two explanations that are wrong
The brackets. The folklore says an entry stored as host instead of [host]:port cannot match a non-standard port, and every guide repeats it. Measured on this client, it can: a file containing 127.0.0.1 ssh-ed25519 … connected to port 2222 under StrictHostKeyChecking=yes and returned exit 0. Bracketed form is what tools write and what you should write, but its absence is not the cause of this error, and rewriting a correct file to add brackets fixes nothing.
The unset HOME. The second folklore claim is that cron jobs fail because $HOME is empty and ~/.ssh/known_hosts cannot expand. On this client the tilde resolved through the password database instead: with HOME removed from the environment entirely, the client still reported ~/.ssh/known_hosts as /opt/data/.ssh/known_hosts, wrote the new entry and exited 0. A missing home directory is a real problem; a missing HOME variable is not the same thing.
The fix that survives a redeploy
Disabling verification is the usual answer and the wrong one; what a fleet needs is a provisioning step that runs before the first connection and refuses to guess. Fetch the endpoint's key, compare it against a fingerprint recorded out of band, decline to replace an existing pin unless a rotation is asked for explicitly, then confirm the entry resolves before the session starts — each step a single ssh-keyscan or ssh-keygen invocation, each returning its own non-zero status so a pipeline can tell a malformed argument from a dead endpoint or a key that is not the one in inventory.
Order matters more than the tool. The comparison has to happen before the append, because a script that writes first and validates afterwards leaves the offered key on disk for the next run to trust — measured directly: with a deliberately wrong expected fingerprint the trust store stayed at zero bytes. The rotation run, by contrast, left the file at two lines rather than three, so a retried deploy does not accumulate duplicate entries for the same endpoint.
Two details from the sections above carry straight into that step. The lookup name is host on port 22 and [host]:port everywhere else, because that is the shape ssh-keyscan writes in each case — validating the bracketed form on port 22 reports "not found" for an entry that is sitting right there. And the fingerprint, not the hostname, is the thing to store in inventory: a hostname can be re-provisioned by anyone who controls the address, while a SHA256 digest either matches what you recorded or it does not.
Did it work
Four checks, in this order. First, the pin resolves: ssh-keygen -F "[host]:port" -f FILE exits 0 and prints the line. Second, the session itself is strict and still connects — the whole point of pinning is that you no longer need to relax anything:
Second, the session itself runs strict and still connects: pinning exists so that nothing has to be relaxed for the real run.
Third, confirm which key you trusted, because a pinned session that used an unexpected algorithm is its own incident. The client reports it during negotiation, and the fingerprint can be compared against the one your provisioning system recorded.
Third, confirm which key was trusted, because a pinned session that negotiated an unexpected algorithm is its own incident. The client reports the server key and its SHA-256 digest during negotiation, and that digest has to equal the one the inventory recorded — compare the two in the deploy log. One caveat costs people an afternoon: the client prints that line even when verification then fails, so a pipeline that greps for it without propagating the client's own exit status reports success on a session that trusted nothing.
The two digests match because both are computed over the same key blob. Compare them in the deploy log — that check turns "the job went green" into "the job trusted this key". set -o pipefail is not decoration: the client prints the server key line even when verification then fails, so without it grep returns 0 while the session exited 255.
Fourth, prove the entry persisted, because exit 0 does not prove it. /dev/null and an unwritable file both exit 0 after storing nothing, so the only honest check reads the file back and compares fingerprints. One detail about the isolation itself: when a test uses a private known_hosts, pass -o GlobalKnownHostsFile=/dev/null as every command above does — otherwise a key already sitting in the system-wide file can make an empty-store case connect and fake a passing result.
Rolling this out without opening a window
Order the steps so a bad pin can never be papered over. Pin first, from a job that has network access to the endpoint and write access to the file. Then run the real session with StrictHostKeyChecking=yes and a file path you control, so an unreachable trust store fails loudly instead of quietly rewriting itself. Set GlobalKnownHostsFile explicitly when the image ships a system-wide file, so the check runs against the store you pinned and audited rather than one nobody looked at.
For a fleet, put the endpoint, port and expected fingerprint in the same inventory entry that names the host, and fail the deploy when the fingerprint recorded by ssh-keyscan differs from the expected one. That converts the whole class of failures into a single check with one meaning: either the key is what the inventory says, or nothing connects. Third-party mirrors and re-imaged VMs are exactly where accept-new earns its keep, and exactly where an unconditional no costs you the machine's identity.
Related reading on this desk: the sibling failure where a key exists but has changed is covered in WARNING: REMOTE HOST IDENTIFICATION HAS CHANGED, and the permission side of the same trust chain is in UNPROTECTED PRIVATE KEY FILE. If the failure arrives before any host key is exchanged, the agent is the suspect — see Could not open a connection to your authentication agent.
None of this needs a new tool. It needs the habit of reading the line above the verdict, and a provisioning step that stores trust instead of disabling it.