Cannot assign requested address (EADDRNOTAVAIL) at bind()
Your service refuses to start with Cannot assign requested address and you go looking for a port conflict — there isn't one. That message is EADDRNOTAVAIL (errno 99), printed when a process calls bind() on an IP address the kernel does not consider local; the port-conflict error is a different errno entirely (EADDRINUSE, 98). Every tutorial that answers this with "kill whatever is on the port" is answering the wrong question.
The setup is always the same: a service configured with a specific address instead of the wildcard — --bind 100.64.1.4 for a Tailscale address, --bind 172.18.0.7 for a container IP, listen 10.10.0.2:8443; for an interface that is up on the primary node but not the replica. On a healthy host it binds fine. Where that address is not (yet) assigned — tunnel not up, interface DOWN, container network not created when the process starts — the kernel refuses before it inspects the port. Below: the measured behaviour of each case, the rule the kernel applies, and the two fixes — bind the wildcard, or wait for the address.
TL;DR — errno 99 is not errno 98
EADDRNOTAVAIL/ errno 99, "Cannot assign requested address" — the address is not local to this network namespace. Measured here for172.18.0.7,10.0.0.1,100.64.1.1and8.8.8.8. Nothing to kill; the port is irrelevant.EADDRINUSE/ errno 98, "Address already in use" — the address is local, but the port is taken. This is the one thatss -ltnpandSO_REUSEADDRare for.- Address locality is checked before the port. Measured: with
0.0.0.0:18080occupied by another process, a bind to172.18.0.7:18080still returned errno 99, not 98. A port-conflict hypothesis sends you hunting in the wrong place. SO_REUSEADDRdoes not help. Set it to 1 and bind a non-local address: still errno 99. It only reopensTIME_WAITports (measured below).- Loopback is a whole /8, not one address.
localhostis whatever you configure, and binding127.0.0.2succeeds on Linux because the kernel marks127.0.0.0/8local — a trap when a container or test harness assumes "only 127.0.0.1 is usable". - The fix is a decision, not a patch: (a) did you mean to bind a specific address at all? — usually no: bind
0.0.0.0and let the firewall decide. (b) Do you genuinely need the tunnel address? — then retry the bind after the interface exists, and treat errno 99 as transient, errno 98 as terminal. The retry helper is further down and was executed to prove both paths. net.ipv4.ip_nonlocal_bind=1is the documented escape hatch and the wrong default: it lets a process on one host silently complete for a VIP that is supposed to live on another. Value on the verification host:0(measured, not settable rootless here).
What the kernel is actually asking
bind() has to answer two independent questions before it can attach a socket to an address. First: does this address belong to this network namespace? Second: is this port free for this (address, port, protocol) triple? The first question is answered from the routing table's local address set and produces EADDRNOTAVAIL when the answer is no. The second produces EADDRINUSE. The two error numbers are adjacent in the errno space, which is exactly why they get conflated in bug reports.
You can read both strings straight out of the platform instead of trusting a blog post, including this one:
import errno, os
print("strerror(98) =", os.strerror(98))
print("strerror(99) =", os.strerror(99))
print("errno.EADDRINUSE =", errno.EADDRINUSE,
"| errno.EADDRNOTAVAIL =", errno.EADDRNOTAVAIL)
Run on the verification host (aarch64, kernel 6.17.0-1019-oracle, Python 3.13.5):
strerror(98) = Address already in use
strerror(99) = Cannot assign requested address
errno.EADDRINUSE = 98 | errno.EADDRNOTAVAIL = 99
Note what the message does not contain: the address, the port, or the interface. A daemon printing only bind: Cannot assign requested address has told you that something it was configured to bind is not local, and nothing about which configuration line is wrong. That is the real cost of this error: the message points at the kernel, and the cause is one line written for a different host.
The measured lab: which addresses bind, and why
I stood up the lab rootless, so the numbers below are the kernel's, not a quotation from a man page. The host has exactly one routable address, 172.18.0.6, plus loopback. The probe binds each address on port 18080 with nothing else listening:
import errno, socket
def probe(family, addr, port, tag):
s = socket.socket(family, socket.SOCK_STREAM)
try:
s.bind((addr, port))
print(f"{tag}: bind {addr}:{port} -- OK")
return s
except OSError as e:
print(f"{tag}: bind {addr}:{port} -- {type(e).__name__} "
f"errno={e.errno} ({errno.errorcode.get(e.errno)}): {e.strerror}")
s.close()
return None
Called for each candidate address, this is the verbatim output. Read the left column: every OK is an address the kernel believes it owns, every errno 99 is one it does not.
-- A. address locality (nothing else listening) --
wildcard IPv4 bind('0.0.0.0', 18080) -> OK
loopback bind('127.0.0.1', 18080) -> OK
loopback /8 (unassigned) bind('127.0.0.2', 18080) -> OK
container IP (assigned) bind('172.18.0.6', 18080) -> OK
same /24, not assigned bind('172.18.0.7', 18080) -> OSError errno=99 (EADDRNOTAVAIL): Cannot assign requested address
private, not assigned bind('10.0.0.1', 18080) -> OSError errno=99 (EADDRNOTAVAIL): Cannot assign requested address
CGNAT/Tailscale, not up bind('100.64.1.1', 18080) -> OSError errno=99 (EADDRNOTAVAIL): Cannot assign requested address
public, not local bind('8.8.8.8', 18080) -> OSError errno=99 (EADDRNOTAVAIL): Cannot assign requested address
Three results deserve to be in your head before the next outage.
Being on the same subnet is not being local. 172.18.0.7 is one address away from the host's own 172.18.0.6, and it fails. "Local" means the kernel has that exact address bound to an interface in this namespace, not that a route exists.
127.0.0.2 works. The loopback interface carries 127.0.0.0/8, so the entire /8 is local. This is why a service bound to 127.0.0.5 can be reachable and a service bound to 172.18.0.7 cannot, on the same machine, with the same config style.
100.64.1.1 fails exactly like 8.8.8.8. That is the shape a Tailscale node or a CGNAT tunnel address takes: when the tunnel is not established — tailscaled not authenticated, wg0 not up — the address does not exist locally and the service dies with errno 99. This is the most common real-world instance of the error: the address is not wrong, it is not there yet.
The same rule holds for IPv6, with one extra wrinkle worth knowing. On this host bindv6only=0, which means an IPv6 wildcard also covers IPv4-mapped addresses and therefore collides with an existing IPv4 wildcard listener:
-- E. IPv6 (with 0.0.0.0:18080 already held by another process) --
IPv6 wildcard bind('::', 18080) -> OSError errno=98 (EADDRINUSE): Address already in use
IPv6 loopback bind('::1', 18080) -> OK
documentation prefix bind('2001:db8::1', 18080) -> OSError errno=99 (EADDRNOTAVAIL): Cannot assign requested address
ULA, not assigned bind('fd00::1', 18080) -> OSError errno=99 (EADDRNOTAVAIL): Cannot assign requested address
So :: failed with 98 while ::1 succeeded, on a host whose IPv4 port was occupied — not because the address was foreign, but because with bindv6only=0 the dual-stack wildcard shares the port space. Two errors, two remedies, one line of config apart.
The order of checks: why "port already in use" advice fails here
This experiment settles most of the confused forum answers. Keep a listener on 0.0.0.0:18080, then attempt binds to addresses that are both non-local and on the occupied port:
-- B. port conflict (EADDRINUSE), same family --
holder listening on 0.0.0.0:18080
second bind, same wildcard bind('0.0.0.0', 18080) -> OSError errno=98 (EADDRINUSE): Address already in use
specific IP under wildcard bind('127.0.0.1', 18080) -> OSError errno=98 (EADDRINUSE): Address already in use
assigned IP, port taken bind('172.18.0.6', 18080) -> OSError errno=98 (EADDRINUSE): Address already in use
-- C. which check runs FIRST: locality or port? --
PORT TAKEN + address not local bind('172.18.0.7', 18080) -> OSError errno=99 (EADDRNOTAVAIL): Cannot assign requested address
PORT TAKEN + address not local bind('8.8.8.8', 18080) -> OSError errno=99 (EADDRNOTAVAIL): Cannot assign requested address
Section C is the point of this article: both conditions were true, and the kernel reported errno 99. Locality is evaluated first, so Cannot assign requested address teaches you nothing about the port. Chasing it with fuser -k or Restart=always cannot fix it — the configuration names an address this host does not have.
Section B is the control, and it shows a nuance that costs time on restart: an existing wildcard listener blocks a bind to a specific address on the same port, and vice versa. If your reverse proxy occupies 0.0.0.0:443, your application cannot take 172.18.0.6:443. That is the mirror image of the 127.0.0.1 bind trap — listening correctly on an address no client can reach: same layer, opposite failure.
What SO_REUSEADDR does and does not do
SO_REUSEADDR is the reflex answer to any bind error, so it is worth pinning down precisely. It does not touch locality, and it does reopen ports whose connections are in TIME_WAIT. Both rows were measured here, on the same port, with a real connection closed from the server side first so the TIME_WAIT landed locally:
-- D. SO_REUSEADDR against EADDRNOTAVAIL --
SO_REUSEADDR=1 + non-local address -> errno=99 (EADDRNOTAVAIL): Cannot assign requested address
-- restart handover: TIME_WAIT vs SO_REUSEADDR --
rebind with SO_REUSEADDR=0 -> errno=98 (EADDRINUSE): Address already in use
rebind with SO_REUSEADDR=1 -> OK
So: set it on every listening socket (it is what makes a restart after a handful of closed connections succeed instantly instead of failing for a minute), and expect nothing from it when the address is foreign. The retry helper below sets it too, for exactly that reason.
Fix 1: bind the wildcard and control exposure elsewhere
The default answer, and the one that survives every ordering race: bind 0.0.0.0 (and :: with IPV6_V6ONLY=1 for a separate IPv6 listener) and decide who may reach the port with the firewall, a reverse proxy, or Cloudflare Tunnel. A wildcard bind never fails with errno 99, and it survives a tunnel address change — the case that breaks hard-coded bind configs on every network change.
Concretely, that means a service configured for the tunnel address:
# brittle: dies with EADDRNOTAVAIL whenever 100.64.1.4 is not
# assigned at process start (tunnel down, wg0 recreated, node offline)
myservice --bind 100.64.1.4 --port 8080
becomes a wildcard bind plus an explicit allow rule, with the tunnel interface as the selector:
# robust: bind everything, let the packet filter decide
myservice --bind 0.0.0.0 --port 8080
# only the tunnel subnet may reach 8080 on a host-local listener
sudo nft add rule inet filter input tcp dport 8080 \
ip saddr 100.64.0.0/10 accept
sudo nft add rule inet filter input tcp dport 8080 drop
For a container, the same reasoning applies to the published port, with one important difference in how the packet filter sees it. Binding the host side to a specific address is a config-level choice, and the failure lands in the daemon log rather than your process log:
# host side pinned to an address that may not exist yet
docker run -d --name api -p 172.18.0.7:8080:8080 ghcr.io/example/api:2.4
# host side wildcard; reachability decided in the FORWARD path
docker run -d --name api -p 8080:8080 ghcr.io/example/api:2.4
Rule the second command out as "tunnel-only by itself": Docker publishes ports by DNAT in the nat PREROUTING chain, so external traffic is routed to the container bridge and traverses the forward hook — never input. The nft rules above therefore restrict myservice (a host-local listener) and do nothing for a -p-published container port; on the wildcard form that port is world-reachable. Filter it where the packets actually go, in Docker's own chain, which is evaluated first:
# published container ports: filter in the forwarding path
sudo iptables -I DOCKER-USER -p tcp --dport 8080 -s 100.64.0.0/10 -j RETURN
sudo iptables -A DOCKER-USER -p tcp --dport 8080 -j DROP
The Docker daemon is not reachable on this verification host (the lab above is kernel-level so the measurements did not depend on it), so the daemon's wording for the pinned-address case is documented behaviour rather than a transcript: it reports a failure programming the endpoint, and the underlying error is the same bind: Cannot assign requested address. Treat the -p commands as configuration guidance. The related capability failure is in Operation not permitted in Docker.
Fix 2: when you genuinely need the tunnel address, wait for it
Sometimes the address is the point: you want the listener reachable only over WireGuard, or the address is a documented contract for another team, or a VIP is expected but its assignment is asynchronous. Then the correct handling is to treat errno 99 as transient and errno 98 as terminal — and to retry with a deadline instead of crash-looping under systemd. That distinction is the entire content of this helper, which I ran against all three paths:
#!/usr/bin/env python3
"""bind_wait.py — production bind helper.
Distinguishes EADDRNOTAVAIL (address not local YET -> retry) from
EADDRINUSE (someone holds the port -> do not retry) and from EACCES."""
import errno, socket, sys, time
RETRYABLE = {errno.EADDRNOTAVAIL} # interface may still be coming up
TERMINAL = {errno.EADDRINUSE, errno.EACCES, errno.EPERM}
def bind_wait(addr, port, family=socket.AF_INET, timeout=30.0, interval=0.5):
"""Bind and listen on (addr, port). Retries while the address is not local.
Returns (listening socket, attempts made); re-raises on give-up."""
deadline = time.monotonic() + timeout
attempt = 0
while True:
attempt += 1
s = socket.socket(family, socket.SOCK_STREAM)
s.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
if family == socket.AF_INET6:
s.setsockopt(socket.IPPROTO_IPV6, socket.IPV6_V6ONLY, 1)
try:
s.bind((addr, port))
s.listen(128)
return s, attempt
except OSError as e:
s.close()
if e.errno in TERMINAL:
raise
if e.errno not in RETRYABLE:
raise
if time.monotonic() >= deadline:
raise
print(f" attempt {attempt}: errno={e.errno} "
f"({errno.errorcode.get(e.errno)}) {e.strerror} -- "
f"address {addr} not local yet, retrying in {interval}s",
flush=True)
time.sleep(interval)
Wire it into a service by replacing the raw listen call with bind_wait(...), or by using the CLI form as an ExecStartPre gate so the unit fails fast and legibly when the address never appears. The CLI exit code is deliberate: 2 for "could not obtain this address", which Restart=on-failure and your alerting can both act on.
# fail fast, one clean line, no traceback in the journal
/opt/api/venv/bin/python /opt/api/bind_wait.py 100.64.1.4 8080 30
# -> FATAL: cannot bind 100.64.1.4:8080 after 30.0s -- errno=99
# (EADDRNOTAVAIL): Cannot assign requested address [exit 2]
And the measured evidence that the two paths behave differently, from the test driver run on this host (verbatim, exit code 0):
== case 1: non-local address (retry, then give up) ==
attempt 1: errno=99 (EADDRNOTAVAIL) Cannot assign requested address -- address 172.18.0.7 not local yet, retrying in 0.5s
attempt 2: errno=99 (EADDRNOTAVAIL) Cannot assign requested address -- address 172.18.0.7 not local yet, retrying in 0.5s
attempt 3: errno=99 (EADDRNOTAVAIL) Cannot assign requested address -- address 172.18.0.7 not local yet, retrying in 0.5s
attempt 4: errno=99 (EADDRNOTAVAIL) Cannot assign requested address -- address 172.18.0.7 not local yet, retrying in 0.5s
gave up after 2.0s: errno=99 (EADDRNOTAVAIL): Cannot assign requested address
== case 2: port held by someone else (terminal, no retry) ==
raised after 0.00s (no retry loop): errno=98 (EADDRINUSE): Address already in use
Case 1 spent its whole 2-second budget on an absent address and still exited with a precise message instead of a stack trace. Case 2 returned in 0.00s — a held port is never retried, because retrying it only delays the news. If the address is merely wrong rather than late (a typo, a config from the wrong node), retrying converts a fast failure into a slow one, so cap the budget at something a human will wait for and name the address in the final line.
Wiring it to systemd without the ordering race
The classic version of this bug is a unit that starts before the interface it binds exists. After=network-online.target orders you after the network stack is configured; it says nothing about a WireGuard interface created by [email protected] or a Tailscale address that arrives after authentication. Name the actual dependency, and let the retry absorb the remainder:
sudo tee /etc/systemd/system/api.service >/dev/null <<'UNIT'
[Unit]
Description=API listener on the tunnel address
After=network-online.target [email protected]
Wants=network-online.target
[email protected]
StartLimitIntervalSec=60
StartLimitBurst=5
[Service]
Type=simple
ExecStartPre=/opt/api/venv/bin/python /opt/api/bind_wait.py 100.64.1.4 8080 30
ExecStart=/opt/api/venv/bin/python /opt/api/server.py
Restart=on-failure
RestartSec=2
[Install]
WantedBy=multi-user.target
UNIT
StartLimitBurst=5 with StartLimitIntervalSec=60 means five attempts inside a minute, then systemd stops restarting and reports start-limit-hit — a state you can alert on, unlike a service that has been crash-looping since Friday and shows a green unit file. Both are [Unit] directives, not [Service] ones (systemd v229 moved them, and in the wrong section they are ignored with Unknown lvalue in the journal). Check the outcome with systemctl show api -p NRestarts -p Result rather than trusting the unit's presence.
The escape hatch, and why it should stay off
net.ipv4.ip_nonlocal_bind exists for exactly this problem and is documented in the kernel's ip-sysctl documentation: when set to 1, a process may bind to a non-local address, which is how some VIP setups start a listener before the address is moved onto the host. It converts the failure above into a listener that accepts connections whenever the address does become local — and, on a host where the address never arrives, into a silent service that looks healthy and is unreachable. The measured value on this verification host is 0, which is the correct default; the value is not settable here (rootless, and /proc/sys is read-only for this uid), so the recommendation below is documentation-backed rather than an experiment I ran:
# read it (this part is measured on the verification host)
cat /proc/sys/net/ipv4/ip_nonlocal_bind
# 0
# do NOT set this to 1 to "fix" a bind error — it hides the
# ordering bug and produces an unreachable-but-running service
# echo 1 > /proc/sys/net/ipv4/ip_nonlocal_bind
Keep it off. Prefer the retry with a deadline, so the failure is bounded and named, over a kernel switch that makes the failure invisible.
Verification: prove the listener is on the address you think it is
The last part of the loop is confirming what actually got bound, since a wildcard bind and a specific bind look identical from inside the process. The kernel's own tables answer it without extra tooling; the hex columns are address and port in network byte order.
import socket, struct
def listeners():
"""Every TCP socket in LISTEN state, as (address, port), no root needed."""
out = []
for path, family in (("/proc/net/tcp", socket.AF_INET),
("/proc/net/tcp6", socket.AF_INET6)):
with open(path) as fh:
next(fh) # header
for line in fh:
f = line.split()
if f[3] != "0A": # 0A == TCP_LISTEN
continue
raw, port = f[1].split(":")
if family == socket.AF_INET:
addr = socket.inet_ntop(family,
int(raw, 16).to_bytes(4, "little"))
else:
# /proc stores each 32-bit word little-endian: reverse
# per word, not all 16 bytes at once.
addr = socket.inet_ntop(family, b"".join(
bytes.fromhex(raw)[i:i + 4][::-1]
for i in range(0, 16, 4)))
out.append((addr, int(port, 16)))
return sorted(out, key=lambda x: (x[1], x[0]))
Then confirm reachability from the right side. From another host on the tunnel, a wildcard listener that is firewalled off and a tunnel-address listener are equivalent — the reason Fix 1 is safe. From the host itself, the check is a plain request; if the port speaks TLS, a plaintext probe returns a record error that looks alarming and is expected:
GET /healthz HTTP/1.1
Host: 100.64.1.4:8080
Connection: close
HTTP/1.1 200 OK
Content-Type: application/json
Content-Length: 32
X-Content-Type-Options: nosniff
{"status":"ok","bind":"0.0.0.0"}
If instead the port is TLS, expect the plaintext failure mode described in SSL_ERROR_RX_RECORD_TOO_LONG — that error means you reached a TLS listener with a plaintext client, not that your bind failed. Knowing which of the three you are looking at (bind error, wrong-address listener, protocol mismatch) is most of the debugging.
Pitfalls, in the order they bite
| Symptom | Real cause | Action |
|---|---|---|
Cannot assign requested address at start | Address not local to this namespace (errno 99) | Wildcard bind + firewall, or retry until the interface exists |
Address already in use after a clean stop | TIME_WAIT on the port (errno 98) | Set SO_REUSEADDR; it fixes this and nothing else |
| Same error, different nodes, same config | Only one node has the interface or the tunnel up | Make the unit depend on the interface, not just the network |
| Works on the primary, fails on the replica | Replica has no address in that subnet | Bind the wildcard; the address is not a portable contract |
| Binds fine, unreachable from the tunnel | Listener on loopback or a stale address | Check /proc/net/tcp for the actual bound address |
| IPv6 wildcard collides with an IPv4 listener | bindv6only=0 (measured 0 here) | Use IPV6_V6ONLY=1, or bind one family only |
| Container exits, daemon log shows the bind error | Pinned host-side port on an absent address | Publish a wildcard port, restrict with the firewall |
Two of these are rules, not rows. Never let a service name an address that exists on only one node unless the service is genuinely node-specific: the config becomes a landmine for every replica, replacement and restore. And distinguish "absent" from "occupied" before touching anything — the remedies are mutually exclusive, and killing the process on the port destroys evidence on a host that was healthy.
Related reading on this desk
- ECONNREFUSED from another host: the 127.0.0.1 bind trap — the client-side twin of this problem: the listen succeeded, the address was wrong.
- SSL_ERROR_RX_RECORD_TOO_LONG: HTTPS on a plaintext port — what a port mismatch looks like once the bind is correct.
- Operation not permitted in Docker: the capability wall — the other half of container privilege failures, where the syscall is reachable but refused.
All errno values, bind outcomes, retry behaviour and sysctl values in this article were measured on 2026-10-16 on the verification host (aarch64, kernel 6.17.0-1019-oracle, Python 3.13.5, no root). The Docker daemon was not reachable there, and the nginx-free configuration examples are marked where they are guidance rather than transcripts. Lab scripts: /opt/data/c2cz/lab/2026-10-16/.