self-signed certificate in certificate chain: Fix curl 60
curl: (60) SSL certificate problem: self-signed certificate in certificate chain means the chain your client received ends in a self-signed root that is not in its trust store. Nothing is broken on the server side, and the certificate you fetched is almost certainly valid — what is missing is the one certificate that signs it, usually an interception proxy, a VPN gateway, or an internal PKI root. Disabling verification is not the fix; making the client trust that specific root is.
I run into this constantly on engagement workstations and build agents: the same request that works from a laptop at home dies in a corporate network, or the reverse — a service that talks happily to the internal API chokes on github.com the moment someone points an environment variable at the corporate CA. The failure is never one bug. It is four different trust stores silently disagreeing, and the error message only names the last client that noticed. Everything below was reproduced on a lab origin with a private root CA; the command outputs are the real ones.
TL;DR
- Error 19 (this article) is a missing root CA — the leaf is signed by a self-signed CA your client has never seen. That is what TLS interception and internal PKI look like. Error 18 is a self-signed leaf; error 20/21 is a server that forgot to send its intermediate. Different fixes, same exit code 60.
- Identify before fixing:
openssl s_client -connect host:443 -servername hostprintsverify error:num=19:self-signed certificate in certificate chain. The certificate you must trust is the self-signed one — the last entry when the root is sent, or the issuer named in the last entry when the chain stops at an intermediate. - Install the CA once, system-wide (Debian/Ubuntu:
/usr/local/share/ca-certificates/*.crt+sudo update-ca-certificates). That covers curl, OpenSSL, git and anything else on the OpenSSL default paths. - It does not cover everything. Python's
requestsverifies against certifi's bundle, not your OS store, andSSL_CERT_FILE,SSL_CERT_DIRandREQUESTS_CA_BUNDLEreplace the bundle they point at. Set one of them to a private CA alone and you break every public site for that process. - Node and git are the easy ones:
NODE_EXTRA_CA_CERTSis additive (verified), andhttp.sslCAInfo/GIT_SSL_CAINFOaccepts a private bundle. - Never "fix" this with
-k,--insecureorNODE_TLS_REJECT_UNAUTHORIZED=0. If you cannot account for the root you are being asked to trust, you are not looking at a configuration problem, you are looking at a man in the middle.
Which of the four failures do you actually have?
Exit code 60 is CURLE_PEER_FAILED_VERIFICATION: curl got a certificate and could not build a chain from it to a root it trusts. The text after SSL certificate problem: is OpenSSL's verification result, and it is the only part that matters for the diagnosis. I stood up three origins — a full chain ending in an untrusted private root, a leaf with no issuer, and a single self-signed certificate — and asked OpenSSL to verify each one:
# one origin, three different chain shapes
for p in 18443 18445 18444; do
echo "--- port $p ---"
openssl s_client -connect 127.0.0.1:$p -servername localhost </dev/null 2>&1 \
| grep -E "verify error|Verify return code"
done
Real output from that run:
--- port 18443 --- # leaf + untrusted root, full chain presented
verify error:num=19:self-signed certificate in certificate chain
Verify return code: 19 (self-signed certificate in certificate chain)
--- port 18445 --- # leaf only, issuer never sent
verify error:num=20:unable to get local issuer certificate
verify error:num=21:unable to verify the first certificate
Verify return code: 21 (unable to verify the first certificate)
--- port 18444 --- # the server certificate is its own root
verify error:num=18:self-signed certificate
Verify return code: 18 (self-signed certificate)
| Code | Message | What it means | Correct fix |
|---|---|---|---|
| 18 | self-signed certificate | The leaf itself is self-signed. There is no CA to trust. | Replace the certificate — unless you deliberately run an internal service, in which case the leaf is the root you install. |
| 19 | self-signed certificate in certificate chain | The leaf is fine; the root above it is self-signed and unknown to you. Interception proxy, VPN client, internal PKI. | Install that root on the client. This article. |
| 20 / 21 | unable to get local issuer certificate / unable to verify the first certificate | The server did not send the intermediate that connects its leaf to a public root. | Fix the server: serve the full chain (fullchain.pem, not cert.pem). The client's store is not at fault. |
The distinction saves hours. Code 20 is a server-side packaging bug and no amount of trust-store work on the client will fix it; code 19 is a client-side trust decision and no server change will fix it. Both surface as the same curl: (60) line, which is why the two get confused in issue trackers constantly. The reflex is the same one you already use for SSH host key changes: establish what the client is looking at before deciding who is wrong.
Pull the certificate you are missing off the wire
You cannot install a CA you have not got. If the root belongs to a proxy your organisation runs, get the file from the proxy's own documentation page — Burp Suite, ZAP and most commercial inspection appliances all publish their CA. If you are handed nothing, extract it from a live connection instead of guessing. openssl s_client -showcerts prints every certificate the server sent, in order, and the last one is the root that has to be trusted:
# 1. dump every certificate in the presented chain
# (s_client waits on stdin if it is a terminal — close it explicitly)
openssl s_client -connect internal.example.com:443 -servername internal.example.com \
-showcerts </dev/null 2>/dev/null \
| awk '/BEGIN CERTIFICATE/,/END CERTIFICATE/' > presented.pem
# 2. split the bundle into one file per certificate
csplit -s -z -f chainpart- -b '%d.pem' presented.pem '/BEGIN CERTIFICATE/' '{*}'
# 3. read the last one — subject == issuer means you found the root
for f in chainpart-*.pem; do
printf '%s ' "$f"; openssl x509 -in "$f" -noout -subject -issuer | tr '\n' ' '; echo
done
Output on the lab origin, where an interception root signs a leaf for localhost:
chainpart-0.pem subject=C=NL, O=C2CZ Lab, CN=localhost issuer=C=NL, O=C2CZ Lab, CN=C2CZ Lab Intercept Root CA
chainpart-1.pem subject=C=NL, O=C2CZ Lab, CN=C2CZ Lab Intercept Root CA issuer=C=NL, O=C2CZ Lab, CN=C2CZ Lab Intercept Root CA
One line tells you everything: chainpart-1.pem has itself as issuer, so it is the root. Before you copy anything into a trust store, confirm that root actually signs the leaf — openssl verify -CAfile chainpart-1.pem leaf.pem returned leaf.pem: OK here. And record the fingerprint, because that is the value you will compare against your PKI inventory later:
openssl x509 -in chainpart-1.pem -noout -fingerprint -sha256 -subject
# sha256 Fingerprint=05:83:DD:05:F9:D3:45:90:8A:32:92:6A:EB:3D:FA:BC:A0:B7:5F:24:31:43:8D:E8:92:30:89:49:54:91:92:46
# subject=C=NL, O=C2CZ Lab, CN=C2CZ Lab Intercept Root CA
When you have to do this on a fleet, script it. The tool below connects, walks the chain the server actually presents, and prints each certificate with its role and SHA-256 fingerprint — the same numbers openssl x509 prints, so a diff against the PKI inventory is mechanical. It needs Python 3.13 or newer, where SSLSocket.get_unverified_chain() is available, and it deliberately disables verification because it is inspecting, not trusting:
#!/usr/bin/env python3
"""trust_doctor.py — show the certificate chain a host actually presents, in trust
order, with the SHA-256 fingerprint of each certificate.
The certificate you have to install is the self-signed one — normally the last entry,
but a chain that omits its root ends at an intermediate instead, so the role is derived
from the certificate itself and not from its position.
Usage: trust_doctor.py <host> [port]
Requires Python 3.13+ (SSLSocket.get_unverified_chain) and the openssl CLI.
"""
from __future__ import annotations
import hashlib
import shutil
import socket
import ssl
import subprocess
import sys
DEFAULT_PORT = 443
OPENSSL_TIMEOUT = 10
class CertParseError(RuntimeError):
"""openssl could not decode a certificate we were handed."""
def fetch_chain(host: str, port: int, timeout: float = 10.0):
"""Return (list_of_DER_certs, tls_version). We inspect, we do not trust."""
ctx = ssl.SSLContext(ssl.PROTOCOL_TLS_CLIENT)
ctx.check_hostname = False
ctx.verify_mode = ssl.CERT_NONE
with socket.create_connection((host, port), timeout=timeout) as sock:
with ctx.wrap_socket(sock, server_hostname=host) as tls:
return tls.get_unverified_chain(), tls.version()
def describe(pem: str) -> tuple[str, str]:
"""subject, issuer — delegated to the openssl CLI so the output matches what
`openssl x509` prints everywhere else."""
try:
proc = subprocess.run(
["openssl", "x509", "-noout", "-subject", "-issuer"],
input=pem, capture_output=True, text=True,
timeout=OPENSSL_TIMEOUT, check=True,
)
except subprocess.CalledProcessError as exc:
detail = (exc.stderr or "").strip().splitlines()
raise CertParseError(detail[-1] if detail else "openssl rejected the input") from exc
except subprocess.TimeoutExpired as exc:
raise CertParseError(f"openssl timed out after {OPENSSL_TIMEOUT}s") from exc
except OSError as exc:
raise CertParseError(f"cannot execute openssl: {exc}") from exc
parts = dict(
line.split("=", 1) for line in proc.stdout.strip().splitlines() if "=" in line
)
return parts.get("subject", "?").strip(), parts.get("issuer", "?").strip()
def fingerprint(der: bytes) -> str:
digest = hashlib.sha256(der).hexdigest().upper()
return ":".join(digest[i:i + 2] for i in range(0, len(digest), 2))
def main(argv: list[str]) -> int:
if not argv:
print(__doc__)
return 2
host = argv[0]
try:
port = int(argv[1]) if len(argv) > 1 else DEFAULT_PORT
except ValueError:
print(f"invalid port: {argv[1]!r} — expected a number")
return 2
if not 1 <= port <= 65535:
print(f"port out of range: {port}")
return 2
if shutil.which("openssl") is None:
print("openssl not found in PATH")
return 1
try:
chain, tls_version = fetch_chain(host, port)
except (OSError, ssl.SSLError) as exc:
print(f"connect failed: {type(exc).__name__}: {exc}")
return 1
print(f"target : {host}:{port}")
print(f"negotiated : {tls_version}")
print(f"chain depth : {len(chain)}\n")
last_subject = last_issuer = ""
for depth, der in enumerate(chain):
try:
subject, issuer = describe(ssl.DER_cert_to_PEM_cert(der))
except CertParseError as exc:
print(f"[{depth}] unreadable certificate — {exc}")
print(f" sha256 : {fingerprint(der)}")
continue
self_issued = subject == issuer
if depth == 0:
role = "leaf"
else:
role = "root (self-signed)" if self_issued else "intermediate"
print(f"[{depth}] {role}")
print(f" subject : {subject}")
print(f" issuer : {issuer}")
print(f" sha256 : {fingerprint(der)}")
print(f" self-signed : {self_issued}")
last_subject, last_issuer = subject, issuer
if len(chain) > 1 and last_subject and last_subject != last_issuer:
print("\nnote: this chain stops at an intermediate — the root is not being sent.")
print(" Trust the issuer named above, not the last certificate itself.")
return 0
if __name__ == "__main__":
raise SystemExit(main(sys.argv[1:]))
Output from that script against the lab origin — the root fingerprint is identical to the openssl x509 -fingerprint -sha256 line above, which is the point: what you paste into an inventory note is what the tooling reports later.
$ python3 trust_doctor.py 127.0.0.1 18443
target : 127.0.0.1:18443
negotiated : TLSv1.3
chain depth : 2
[0] leaf
subject : C=NL, O=C2CZ Lab, CN=localhost
issuer : C=NL, O=C2CZ Lab, CN=C2CZ Lab Intercept Root CA
sha256 : 5F:B9:55:9A:44:B7:21:28:77:33:66:90:B1:61:E8:92:31:E1:DD:B6:09:BA:31:E5:E9:BB:6E:A0:BA:70:0E:84
self-signed : False
[1] root (self-signed)
subject : C=NL, O=C2CZ Lab, CN=C2CZ Lab Intercept Root CA
issuer : C=NL, O=C2CZ Lab, CN=C2CZ Lab Intercept Root CA
sha256 : 05:83:DD:05:F9:D3:45:90:8A:32:92:6A:EB:3D:FA:BC:A0:B7:5F:24:31:43:8D:E8:92:30:89:49:54:91:92:46
self-signed : True
The role label comes from the certificate, not from its position, because plenty of real chains do not end in a root at all. A host that sends only its leaf and an intermediate produces this — and the note at the bottom is the part that matters, since the file you need to trust is the issuer, not the certificate in your hand:
$ python3 trust_doctor.py 127.0.0.1 18446
target : 127.0.0.1:18446
negotiated : TLSv1.3
chain depth : 2
[0] leaf
subject : C=NL, O=C2CZ Lab, CN=api.internal.example
issuer : C=NL, O=C2CZ Lab, CN=C2CZ Lab Issuing CA
sha256 : E4:B6:E9:A3:0F:B6:9F:2F:E9:1B:80:8F:70:D8:7F:EE:4B:F0:39:56:2A:D9:76:11:E2:A1:3B:7F:21:DF:DA:9E
self-signed : False
[1] intermediate
subject : C=NL, O=C2CZ Lab, CN=C2CZ Lab Issuing CA
issuer : C=NL, O=C2CZ Lab, CN=C2CZ Lab Intercept Root CA
sha256 : AB:D3:76:18:45:4C:4C:F7:B7:90:37:DF:A6:9A:F7:18:ED:A5:08:45:69:75:A0:9E:ED:32:F2:FA:72:20:4B:86
self-signed : False
note: this chain stops at an intermediate — the root is not being sent.
Trust the issuer named above, not the last certificate itself.
Fix 1 — install the root in the system trust store
On Debian and Ubuntu the mechanism is a directory plus a hook: certificates dropped into /usr/local/share/ca-certificates with a .crt extension are trusted implicitly, and update-ca-certificates regenerates /etc/ssl/certs/ca-certificates.crt — the bundle every OpenSSL consumer on the box reads by default. Both facts come straight from the update-ca-certificates(8) manual page, including the detail that the extension must be .crt and that the tool then runs every hook in /etc/ca-certificates/update.d (that is how a Java keystore on the same machine gets updated for free).
# Debian / Ubuntu — the CA file must end in .crt
sudo install -m 0644 corporate-root.crt /usr/local/share/ca-certificates/corporate-root.crt
sudo update-ca-certificates -f
# Updating certificates in /etc/ssl/certs... 1 added, 0 removed; done.
# RHEL / Fedora / Rocky — documented path, different command
sudo install -m 0644 corporate-root.crt /etc/pki/ca-trust/source/anchors/corporate-root.crt
sudo update-ca-trust
That single step fixes curl, OpenSSL itself, git and wget on the machine. It is also the step people assume fixes everything, and it does not. Which brings us to the part that costs real debugging time.
Fix 2 — the runtimes that do not read the system store
Every runtime carries its own idea of what "trusted" means, and the environment variables that control it behave differently: some add to the default store, some replace it. I tested each one against the same private-root origin (self-signed certificate in certificate chain, exit 60) and against a public site, because the interesting failure is not the fix, it is the fix silently breaking everything else.
| Client | Knob | Behaviour | Evidence from the lab run |
|---|---|---|---|
| curl | --cacert file or CURL_CA_BUNDLE=file | Sets CAfile; the default CApath stays in play | private origin 200; public site still 200 — verbose output shows CAfile: ca.crt and CApath: /etc/ssl/certs |
| OpenSSL CLI | -CAfile | Explicit, no defaults | Verify return code: 0 (ok) |
Python ssl / urllib | SSL_CERT_FILE, SSL_CERT_DIR | Replaces the default file / directory respectively | with only SSL_CERT_FILE=ca.crt: private origin 200; pinning SSL_CERT_DIR to an empty dir as well made the public site fail with unable to get local issuer certificate |
| requests / urllib3 | REQUESTS_CA_BUNDLE, or per-call verify= | Replaces certifi's bundle | default verification loads .../certifi/cacert.pem; with REQUESTS_CA_BUNDLE=ca.crt the private origin turned 200 and the public site broke |
| Node.js | NODE_EXTRA_CA_CERTS=file | Additive — appended to the built-in roots | private origin 200 and public site 200 in the same process; Node 26 also accepts --use-system-ca / --use-openssl-ca |
| git | http.sslCAInfo / GIT_SSL_CAINFO | Replaces the CA bundle libcurl uses | without it: server verification failed: certificate signer not trusted; with it, the clone proceeded to the next stage |
The classic trap is Python. A service container that sets REQUESTS_CA_BUNDLE=/etc/ssl/certs/corporate.pem to reach an internal API will lose access to every public endpoint the same process talks to, because the variable does not merge anything — it is the whole trust set. It fails loudly, which is the good case, but only for the first request that happens to hit a public host.
# curl: one-off inspection against an internal service
curl --cacert corporate-root.crt https://internal.example.com/health
# CURL_CA_BUNDLE=corporate-root.crt works too, but pinning it system-wide
# leaves fewer surprises for the next engineer.
# Python stdlib: exported before the interpreter starts, never mid-process
export SSL_CERT_FILE=/etc/ssl/certs/ca-certificates.crt # Debian default bundle
python3 -m mypkg.worker
# requests: per-call is the safest form — one trust set per boundary
# requests.get(url, verify="/opt/pki/merged-ca.pem")
# Node.js: additive, so nothing else needs touching
NODE_EXTRA_CA_CERTS=/opt/pki/corporate-root.crt node app.js
# git: global config for the machine, env var for CI
git config --global http.sslCAInfo /opt/pki/merged-ca.pem
One note on the Python line: I export the variable for the process instead of reaching for an interpreter flag. On Debian the file to point at is /etc/ssl/certs/ca-certificates.crt, which update-ca-certificates keeps current — you rarely need to name your private CA there at all once Fix 1 has run on the image.
If your Python service legitimately needs both worlds, the requests documentation shows the per-session form (session.verify = "/path/to/certfile") and notes the caveat that a directory passed to verify must be c_rehash-processed. The truststore package is the other documented route: it moves verification onto the platform store instead of certifi. I injected truststore.inject_into_ssl() in the lab and the private-root origin still failed — consistent with verification now going to the OS store, where the lab CA is not installed. It is a way to reach the system store, not a substitute for putting the CA in it.
Fix 3 — build one merged bundle and hand it to everything
Because the most common variables replace rather than extend, the durable pattern in containers and CI is to stop choosing between "corporate roots" and "public roots" and concatenate them once. The system bundle on Debian is /etc/ssl/certs/ca-certificates.crt; append your CA, count what you got, and point every runtime at the merged file:
cat /etc/ssl/certs/ca-certificates.crt /opt/pki/corporate-root.crt > /opt/pki/merged-ca.pem
# merged-ca.pem is self-contained: 151 certificates on this host (150 system + 1 private)
grep -c 'BEGIN CERTIFICATE' /opt/pki/merged-ca.pem
# now one file can serve every runtime without disarming public trust
export SSL_CERT_FILE=/opt/pki/merged-ca.pem
export REQUESTS_CA_BUNDLE=/opt/pki/merged-ca.pem
export CURL_CA_BUNDLE=/opt/pki/merged-ca.pem
export NODE_EXTRA_CA_CERTS=/opt/pki/corporate-root.crt
Verified in the lab: with the merged bundle, the private-root origin returned 200 and the public site returned 200 from the same Python process — the exact pair of results that the private-only bundle could not produce. Node needs only the private CA because NODE_EXTRA_CA_CERTS appends to its built-in roots; everything else gets the merged file.
In an image build the same logic applies through the ca-certificates hook, which is the mechanism the manual page describes: drop the .crt file in and run the tool as root. I could not execute the Docker step on the writing host (no daemon available), so treat the build itself as the documented Debian path rather than a run I observed — but the assertion inside it is not guesswork. Counting BEGIN CERTIFICATE lines proves nothing, because the base image already ships a bundle; naming the CA you just installed does — and the match has to tolerate the local OpenSSL's DN formatting, since 3.5 prints CN=Corporate Root CA where older builds printed CN = Corporate Root CA. I ran that check locally on OpenSSL 3.5.6: the loose pattern found ISRG Root X1 in the untouched system bundle and found the lab CA exactly once in the merged file.
FROM python:3.13-slim
COPY pki/corporate-root.crt /usr/local/share/ca-certificates/corporate-root.crt
RUN update-ca-certificates \
&& openssl crl2pkcs7 -nocrl -certfile /etc/ssl/certs/ca-certificates.crt \
| openssl pkcs7 -print_certs -noout | grep -qE 'CN ?= ?Corporate Root CA'
ENV SSL_CERT_FILE=/etc/ssl/certs/ca-certificates.crt \
REQUESTS_CA_BUNDLE=/etc/ssl/certs/ca-certificates.crt
Three ways this fix goes wrong
You replaced the bundle instead of extending it. Most visible with REQUESTS_CA_BUNDLE and SSL_CERT_FILE: the private origin starts working and public endpoints stop, in the same process. If a service talks to both the corporate network and the Internet, it needs the merged file — there is no supported variable that merges one for you.
You assumed curl's defaults disappeared. They did not. Setting --cacert changes CAfile and leaves CApath at the system directory, which is why the private CA verifies without breaking public hosts in that tool. Useful, but do not generalise it to the others — Python and requests do not behave that way, and the difference is invisible until a request fails at runtime.
You turned verification off. curl -k, --insecure, verify=False and NODE_TLS_REJECT_UNAUTHORIZED=0 all make the error disappear by removing the check that produced it. There is a legitimate use for exactly one of these: a single manual request against a host you control, while you are building the CA file. In a Dockerfile, a CI variable, or a service config, they are a permanent hole.
That last point is not hypothetical. self-signed certificate in certificate chain is precisely what a man in the middle produces — this is the error a client throws when someone re-signs a connection with a certificate the client cannot chain to anything it knows. If the root you were handed is not in your PKI inventory, or nobody can tell you which appliance issued it, do not install it and do not click through it: capture the fingerprint from trust_doctor.py, keep the presented.pem bundle as evidence, and treat it as an incident. On an engagement, the same check in reverse is how you prove an inspection device is in the path — the chain is the evidence, and it is why the fingerprint matters more than the error text.
Verification checklist
- Identify the code:
verify error:num=19means a missing private root;num=20means the server omitted its intermediate. Fix the right side of the connection. - Prove the root signs the leaf:
openssl verify -CAfile root.crt leaf.crtmust printOK. - Record the fingerprint before installing anything, and keep it next to the file name.
- Install once, system-wide, with a
.crtfile in/usr/local/share/ca-certificatesandupdate-ca-certificates -fon Debian, or/etc/pki/ca-trust/source/anchors/andupdate-ca-truston RHEL-family hosts. - Test both directions afterwards: the private host and a public host, from every runtime the service actually uses. A fix that only breaks public access is harder to attribute than the error you started with.
- Write it down: which CA, which fingerprint, which store, which env vars. The next person to see this error on the fleet should find your note, not the certificate.
Related work on this blog
The same theme — a client refusing to proceed because it cannot verify who it is talking to — shows up in other layers we have taken apart here:
- WARNING: REMOTE HOST IDENTIFICATION HAS CHANGED — the SSH equivalent: the host key you knew is not the key you got, and the correct response is to establish what changed, not to delete the line blindly.
- Nmap "requires root privileges" — when a tool needs more authority, grant it narrowly with capabilities instead of running the whole thing as root. The same instinct applies here: install the one CA you can account for, not a blanket bypass.
- Tailscale SSL_ERROR_RX_RECORD_TOO_LONG — the browser-side sibling: TLS errors from a mismatch between what the client expected and what the endpoint served.