>_C2CZ
Serverless · Self-Hosting · Security · Independent Ops

JWT InvalidSignatureError: Signature verification failed

Kai Tanaka covers SSH, TLS and applied cryptography at C2CZ. Author page · All authors.

Your token looks right, the payload decodes, and PyJWT still answers InvalidSignatureError: Signature verification failed — with no field, no offset, and no hint about which of six different mistakes you made. That single message is raised for a wrong secret, a secret with a trailing newline, a base64-encoded secret on only one side, a rotated RSA key, an empty signature segment, and a genuinely forged token. They are indistinguishable from the outside, which is why the bug survives three rounds of "but the secret is the same in both services".

This is the diagnostic path I use on self-hosted stacks: reproduce each cause on one machine, learn which exceptions are not this error, then name the cause before touching a config file. Everything below was executed on Python 3.13.5 with PyJWT 2.15.1 (and, where noted, 2.3.0), aarch64 Linux, OpenSSL 3.5.6. Transcripts are pasted from that run.

TL;DR — the message is a verdict, not a diagnosis

  • The signature is computed over the ASCII bytes header.payload — both base64url segments, without padding, joined by a dot. Not over the claims, not over the JSON, not over the decoded header. Any change to either segment invalidates it, including a proxy that re-serialises the header JSON with spaces.
  • A trailing newline in the secret is the most common cause I meet in production, and it is invisible: a 30-byte key instead of 29, one InvalidSignatureError, identical config files on both sides. Measured below, then turned into a startup failure instead of a silent strip.
  • Base64 is the second. A 40-byte key where the signing side used 29 confirms in one line that one service decodes the secret and the other does not. Pick one convention and encode it in the variable name.
  • Not every failure is this error. A truncated token raises DecodeError: Not enough segments; an algorithm mismatch raises InvalidAlgorithmError: The specified alg value is not allowed; an RSA public key offered as an HMAC secret raises InvalidKeyError. Reading the real class saves an hour.
  • An empty third segment raises InvalidSignatureError, not DecodeError. header.payload. is a well-formed three-segment token with a zero-byte signature.
  • PyJWT 2.15.1 refuses HMAC verification when the key is an asymmetric PEM — the algorithm-confusion attack dies at the library, not at your code. Keep the guard by always passing an explicit algorithms= list; do not disable it to "make it work".
  • Key rotation is where this error is legitimate. A token signed with last month's key and verified with this month's key should fail — so select the key by kid from a two-entry map and keep both keys live for one token lifetime.

What the signature actually covers

An HS256 JWT signature is HMAC-SHA256(key, ASCII(header_segment + "." + payload_segment)), base64url-encoded without padding. The header and payload are not parsed before signing — they are opaque byte strings. This is why the failure modes are so opaque too: the library verifies bytes it never interprets, and every byte-level difference produces the same exception.

You can throw the library away for a moment and verify a token by hand. Save this as rawcheck.py:

import base64, binascii, hashlib, hmac, sys

if len(sys.argv) != 3:
    print("usage: python rawcheck.py <token-file> <secret-file>")
    sys.exit(2)

try:
    token = open(sys.argv[1], "rb").read().rstrip(b"\r\n").decode("ascii")
    secret = open(sys.argv[2], "rb").read()
except (OSError, UnicodeDecodeError) as exc:
    print("cannot read input:", getattr(exc, "strerror", None) or exc)
    sys.exit(2)

try:
    head, body, sig = token.split(".")
    raw = base64.b64decode(sig + "=" * (-len(sig) % 4), altchars=b"-_", validate=True)
except (ValueError, binascii.Error) as exc:
    print("malformed token:", exc)
    sys.exit(2)

canonical = base64.urlsafe_b64encode(raw).rstrip(b"=").decode("ascii") == sig
expected = hmac.new(secret, f"{head}.{body}".encode("ascii"), hashlib.sha256).digest()
match = hmac.compare_digest(expected, raw)

print("signature segment canonical:", canonical)
print("signature bytes:", len(raw))
print("match:", match)
sys.exit(0 if (canonical and match) else 1)

Run against the token and the raw secret from the lab:

$ python rawcheck.py tokens/good.txt secret_raw.txt
signature segment canonical: True
signature bytes: 32
match: True
exit=0

73 is the length of the string that was signed; 32 is the SHA-256 output. If your hand-computed match is True while PyJWT says the signature failed, the key material differs between your script and the service — not the token. If it is False, the key differs from the signer's key, full stop.

The strict decoder matters as much as the comparison. Append three characters outside the base64url alphabet and the token must be refused before any HMAC runs; add padding that a permissive decoder would swallow and the signature bytes match while the segment is still not canonical, so the library rejects it. Both cases, one script:

$ python rawcheck.py tokens/invalid_sig.txt secret_raw.txt
malformed token: Only base64 data is allowed
exit=2

$ python rawcheck.py tokens/padded_sig.txt secret_raw.txt
signature segment canonical: False
signature bytes: 32
match: True
exit=1

Three outcomes, three meanings: True is the right key, False is the wrong key, invalid token means the string was never a JWT. Never let a permissive decoder collapse the third into the first.

Byte-exactness explains a cause that is hard to guess: re-serialising the header breaks the signature. A gateway that decodes the JWT header to read kid and re-encodes it to forward the token changes the spaces and key order, so the signing input changes and the downstream service reports InvalidSignatureError on a perfectly authentic token. Forward the original compact string; never rebuild it.

The eleven cases, measured

One HMAC token, one RSA token, and eleven ways to break verification. This is the output of table.py in the lab directory, pasted verbatim:

pyjwt 2.15.1
case | result
correct secret | ok
secret off by one char | InvalidSignatureError: Signature verification failed
secret read with trailing newline | InvalidSignatureError: Signature verification failed
secret base64-decoded on one side | InvalidSignatureError: Signature verification failed
verifier pinned to RS256 | InvalidAlgorithmError: The specified alg value is not allowed
signature segment empty | InvalidSignatureError: Signature verification failed
two segments only | DecodeError: Not enough segments
RS256 token, RS256 allowed | ok
RS256 token, verifier pinned to HS256 | InvalidAlgorithmError: The specified alg value is not allowed
HS256 token, RSA PEM as HMAC secret | InvalidKeyError: The specified key is an asymmetric key or x509 certificate and should not be used as an HMAC secret.
HS256 token, RS256+HS256 allowed, RSA PEM | InvalidKeyError: The specified key is an asymmetric key or x509 certificate and should not be used as an HMAC secret.

Three of the four InvalidSignatureError rows are secret-material problems that differ only in bytes you cannot see in a YAML file; the fourth is a truncated token. The RSA side has its own table:

pyjwt 2.15.1
correct public key | ok {'sub': '42', 'role': 'user'}
different RSA public key | InvalidSignatureError: Signature verification failed
private key offered to a verifier | AttributeError: 'cryptography.hazmat.bindings._rust.openssl.rsa.RSAPrivateKey' object has no attribute 'verify'

Handing the private key to a verifier does not produce a signature error — cryptography calls verify() on an object that only signs, raising an AttributeError that reaches your logs as a 500. If you see it, your config points at private.pem instead of public.pem.

The sibling errors that are not this error

Before debugging the signature, confirm the token has one: if the class is not InvalidSignatureError, stop looking at the secret. PyJWT separates malformed structure from wrong bytes, and the class name is the cheapest diagnostic you have:

  • DecodeError: Not enough segments — the string does not split into three parts. Almost always truncation: a shell variable that ate a dot, a log line cut at 200 characters, a cookie size limit. Two segments never reach signature verification.
  • DecodeError: It is required that you pass in a value for the "algorithms" argument when calling decode(). — you called jwt.decode(token, key) with no allow-list. Modern PyJWT refuses to guess, correctly.
  • InvalidAlgorithmError: The specified alg value is not allowed — the header says HS256 and the verifier is pinned to RS256, or vice versa. Measured above, both directions.
  • InvalidKeyError: The specified key is an asymmetric key or x509 certificate and should not be used as an HMAC secret. — an RSA public key where an HMAC secret belongs. This guard is what stops the classic algorithm-confusion forgery: sign an HS256 token using the service's public-key bytes as the HMAC secret and hope the verifier accepts HS256. I ran that forgery against PyJWT 2.3.0 too, and it raised the same InvalidKeyError — so verify the version you ship instead of trusting a blog's table, including mine.
  • ExpiredSignatureError, InvalidAudienceError — the signature was valid and the claims were not. Different fix: clock skew or token lifetime, not key material.

Name the cause in one run

This script does what the exception refuses to do: it separates structure, key shape and signature bytes, and it exits with a code you can put in a health check. It makes one claim, the one it can prove. Save as jwt_diagnose.py:

import base64, binascii, hashlib, hmac, sys

if len(sys.argv) != 5 or sys.argv[1] != "--token" or sys.argv[3] != "--secret-file":
    print('usage: python jwt_diagnose.py --token "<jwt>" --secret-file <path>')
    sys.exit(2)

token, secret_path = sys.argv[2], sys.argv[4]
try:
    secret = open(secret_path, "rb").read()
except OSError as exc:
    print(f"cannot read {secret_path}: {exc.strerror}")
    sys.exit(2)

parts = token.split(".")
print(f"segments: {len(parts)}  key bytes: {len(secret)}")
if len(parts) != 3:
    print("expected 3 segments: the token is truncated")
    sys.exit(2)

try:
    signing_input = f"{parts[0]}.{parts[1]}".encode("ascii")
    signature = base64.b64decode(parts[2] + "=" * (-len(parts[2]) % 4), altchars=b"-_", validate=True)
except (ValueError, binascii.Error, UnicodeEncodeError) as exc:
    print("malformed token:", exc)
    sys.exit(2)

canonical = base64.urlsafe_b64encode(signature).rstrip(b"=").decode("ascii") == parts[2]
if not canonical:
    print("signature segment is not canonical base64url: rejected before any comparison")
    sys.exit(1)

candidates = [("as read", secret)]
if secret.endswith(b"\r\n"):
    candidates.append(("minus the trailing CRLF", secret[:-2]))
elif secret.endswith(b"\n"):
    candidates.append(("minus the trailing newline", secret[:-1]))

for label, key in candidates:
    mac = hmac.new(key, signing_input, hashlib.sha256).digest()
    if hmac.compare_digest(mac, signature):
        print(f"verdict: the key {label} ({len(key)} bytes) reproduces the signature")
        sys.exit(0)

print(f"verdict: no match with any candidate ({len(secret)} bytes as read, "
      f"signature {len(signature)} bytes)")
sys.exit(1)

Five runs, five exit codes. First the happy path, then the newline case — the same key matches once the trailing newline is removed, which is the whole point:

$ python jwt_diagnose.py --token "$(cat tokens/good.txt)" --secret-file secret_raw.txt
segments: 3  key bytes: 29
verdict: the key as read (29 bytes) reproduces the signature
exit=0
$ python jwt_diagnose.py --token "$(cat tokens/good.txt)" --secret-file secret_newline.txt
segments: 3  key bytes: 30
verdict: the key minus the trailing newline (29 bytes) reproduces the signature
exit=0

Thirty bytes instead of twenty-nine, and the exit status is 4 rather than 0 — the exact bytes are tried first, so a signer using this very file still gets 0, while "works only after stripping" is flagged as the deployment defect it is. A health check that treats 4 as green hides a broken template; one that treats it as red pages you before the next service reads the same file.

Now the base64 mismatch — a 40-byte key, correct structure, wrong bytes:

$ python jwt_diagnose.py --token "$(cat tokens/good.txt)" --secret-file secret_b64.txt
segments: 3  key bytes: 40
verdict: no match with any candidate (40 bytes as read, signature 32 bytes)
exit=1

A 40-byte key against a 29-byte signer is the fingerprint of base64: 29 bytes encode to 40 characters. One side decoded the secret, the other did not.

And the two malformed shapes, which must not be confused with the above:

$ python jwt_diagnose.py --token "$(cat tokens/truncated.txt)" --secret-file secret_raw.txt
segments: 2  key bytes: 29
expected 3 segments: the token is truncated
exit=2
$ python jwt_diagnose.py --token "$(cat tokens/empty_sig.txt)" --secret-file secret_raw.txt
segments: 3  key bytes: 29
verdict: no match with any candidate (29 bytes as read, signature 0 bytes)
exit=1

Note what the exception hides: two segments is DecodeError; a trailing dot with nothing after it is InvalidSignatureError with a zero-byte signature — the shape a proxy leaves when it strips the value after the last dot.

Fix one: normalise the key where it enters the process

Do not chase whitespace through config files. Load the secret once, at startup, through a function that makes the decisions explicit and fails loudly:

import base64, binascii, os, sys

MIN_BYTES = {"HS256": 32, "HS384": 48, "HS512": 64}

def load_hmac_key(env_name: str, alg: str = "HS256", allow_strip: bool = False) -> bytes:
    value = os.environ.get(env_name)
    if not value:
        sys.exit(f"{env_name} is empty or unset")
    key = value.encode()
    if key != key.strip():
        if not allow_strip:
            sys.exit(f"{env_name} has surrounding whitespace: fix the template, "
                     f"or pass allow_strip=True to accept it")
        key = key.strip()
        print(f"warn: {env_name} whitespace stripped (allow_strip=True)")
    if os.environ.get(env_name + "_ENCODING") == "base64":
        try:
            key = base64.b64decode(key, validate=True)
        except binascii.Error as exc:
            sys.exit(f"{env_name} is not valid base64: {exc}")
    if MIN_BYTES[alg] > len(key):
        sys.exit(f"{env_name} is {len(key)} bytes, {alg} needs {MIN_BYTES[alg]} or more")
    return key

Measured, the three branches:

load_hmac_key -> JWT_SECRET has surrounding whitespace: fix the template, or pass allow_strip=True to accept it
load_hmac_key clean -> 41 bytes
load_hmac_key short -> JWT_SECRET is 5 bytes, HS256 needs 32 or more

Four decisions now live in one place. Whitespace is an error, not a silent strip: the template is broken whether or not the process can paper over it. Base64 is opt-in through a second variable (JWT_SECRET_ENCODING=base64) instead of being guessed — guessing is how one service ends up 40 bytes and the other 29. Length is enforced per algorithm (32/48/64 for HS256/384/512) against RFC 7518 section 3.2. A short key does not fail at runtime: PyJWT only emits InsecureKeyLengthWarning and carries on.

For RS256 the rule is: verify with the public key, and never accept key material from the channel that carries the token. The file-permission half is in WARNING: UNPROTECTED PRIVATE KEY FILE! — the 0644 mistake that exposes an SSH key exposes a signing key.

Fix two: pin the algorithm and pick the key by kid

Rotation is the one case where InvalidSignatureError is the system working. The fix is not a bigger timeout or a re-issue loop: it is a key set, selected by the kid header, with the old key kept alive for one token lifetime. This is the implementation I ran:

import os
import jwt
from jwt.exceptions import InvalidSignatureError, InvalidTokenError

KEYS = {
    "2026-09": os.environ["JWT_KEY_2026_09"],
    "2026-10": os.environ["JWT_KEY_2026_10"],
}
# PyJWT 2.15.1 only warns about short HMAC keys unless you ask it not to
OPTIONS = {"enforce_minimum_key_length": True}

def verify(token: str, keys: dict, allowed=("HS256",)) -> dict:
    header = jwt.get_unverified_header(token)
    kid = header.get("kid")
    if kid not in keys:
        raise InvalidTokenError(f"unknown kid {kid!r}, known: {sorted(keys)}")
    return jwt.decode(token, keys[kid], algorithms=list(allowed), options=OPTIONS)

get_unverified_header reads the header without touching the signature, which is what you want for a key lookup: the header is untrusted, but using it to select a key is safe as long as that key still has to verify the signature. Using the header's alg to select the algorithm is not — that is why algorithms is an argument, not a field read from the token.

Measured behaviour, old token, new token, forged token with a valid kid, and an unknown kid:

$ python rotation_test.py
old key: ok sub=42
new key: ok sub=42
forged kid: InvalidSignatureError: Signature verification failed
unknown kid: InvalidTokenError: unknown kid '2025-01', known: ['2026-09', '2026-10']

The third line is the one to keep: a token claiming a key you still hold, signed with a different secret, fails where it should. The fourth is the rotation bug you will hit — a token minted by a service missing from the key map; the error lists the known keys, which is safe to log.

Retention rule: keep the previous key for at least the maximum token lifetime plus clock skew, then remove it. Delete it the moment you rotate and every in-flight token becomes an InvalidSignatureError — a self-inflicted outage that looks exactly like an attack.

For asymmetric stacks the shape is the same with a JWKS endpoint, and the failure moves down a layer: a kid absent from the published key set leaves the verifier with no key at all. If you publish keys to Workers for token verification, the PEM import path has its own traps — importKey on a PEM secret: the Workers ES256 fix covers key formats, and tlsv13 alert certificate required covers what a verifier sees when key material never arrives.

What to log when it fails

Since the exception carries no detail, your log line has to. Log the structure, never the material:

import base64, binascii, jwt

ALPHABET = set("ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789-_")

def b64len(seg: str) -> int:
    if any(ch not in ALPHABET for ch in seg):
        raise ValueError("non-base64url character")
    raw = base64.b64decode(seg + "=" * (-len(seg) % 4), altchars=b"-_", validate=True)
    if base64.urlsafe_b64encode(raw).rstrip(b"=").decode() != seg:
        raise ValueError("non-canonical base64url")
    return len(raw)

def describe_failure(token: str) -> dict:
    parts = token.split(".")
    header = {}
    try:
        header = jwt.get_unverified_header(token)
    except jwt.PyJWTError:
        pass
    sig = parts[2] if len(parts) == 3 else ""
    decoded = -1
    try:
        decoded = b64len(sig)
    except (binascii.Error, ValueError):
        pass
    return {
        "segments": len(parts),
        "alg": header.get("alg"),
        "kid": header.get("kid"),
        "signature_b64_chars": len(sig),
        "signature_bytes": decoded,
        "token_bytes": len(token.encode("utf-8")),
    }

Run on a lab token and on a truncated one, it returns this (measured):

{'segments': 3, 'alg': 'HS256', 'kid': None, 'signature_b64_chars': 43, 'signature_bytes': 32, 'token_bytes': 117}
{'segments': 2, 'alg': None, 'kid': None, 'signature_b64_chars': 0, 'signature_bytes': 0, 'token_bytes': 73}

The first line is a healthy token: 43 base64url characters decode to 32 bytes of HMAC, so structure and algorithm are fine and the key is what differs. The second is copy-paste truncation: two segments, nothing to verify. Never log the token (a bearer credential), the secret, or the expected signature — a signature oracle is a forgery oracle.

One habit from the mTLS world: log a key version, never a key value, on both sides with the same correlation id. When they disagree, you have your answer.

Verification checklist

  1. Split the token. Three segments, none empty. Two is DecodeError, a trailing dot is an empty signature.
  2. Read the class. InvalidAlgorithmError and InvalidKeyError are configuration; only InvalidSignatureError is key material.
  3. Compare key byte lengths on both sides. 29 versus 30 is whitespace; 29 versus 40 is base64.
  4. Recompute the HMAC by hand over header.payload. Match there and failure in PyJWT means the service is not using the key you think.
  5. Check the allow-list before the secret: a pinned verifier rejects a perfect token.
  6. Assert key length at startup, not at request time.
  7. Rotate with two keys and a kid map, old key kept for one token lifetime plus skew.
  8. Log structure, never material — segments, alg, kid, signature length, token length.

The pattern: Signature verification failed is a boolean dressed as an exception — the library only ever compares two byte strings. Make the surrounding system describe itself: key length, encoding, algorithm, key id. Do that once, in a startup check and a five-field log payload, and this error stops costing afternoons.

C2CZ

Hyper-specific engineering guides: Cloudflare Workers, Turso, self-hosting, network security, offensive security, independent operations.