Drop unused test helper asn1MarshalOctetString (U1000) and the
vacuous uint64 >= 0 comparison in the readiness check (SA4003) —
readiness is simply that Total() answers without error.
Add LeafHash (SHA-256 of 0x00 || leaf per RFC 6962 §2), interior node
hashing, VerifyInclusion implementing the §2.1.2 audit-path algorithm,
and a ProofByHash client method for get-proof-by-hash (query properly
URL-escaped for base64 '+' and '/' bytes).
This is the machinery needed to bind fetched entries to a verified
signed tree head; the tailer wires it up in the next change.
Tests build a reference tree recursively (independent of the iterative
verifier) and cross-check every index for sizes 1..33, plus rejection
of wrong root, tampered/truncated paths, wrong index, foreign leaf.
The x509 path lowercased names, trimmed trailing dots and wildcard
prefixes, dropped control bytes, and deduplicated; the precert
(TBS) path returned raw SAN strings with none of that, leaving the
ingest filter single-layered for precerts and inconsistent casing in
stored names. Route both paths through the same normalization; test
covers mixed case, trailing dot, wildcard, and duplicate collapse.
VerifySTH accepted any root-hash length and ignored the DigitallySigned
hash/signature algorithm bytes, so a signed-but-degenerate STH (empty
root) or an algorithm-confused signature verified cleanly. Require a
32-byte root hash, a non-negative tree size, sha256 as the hash
algorithm, and a signature algorithm matching the pinned key type
(ecdsa=3, rsa=1).
Tests cover the valid case plus forged signature, short root, sha1
label, rsa label on an ecdsa key, and negative tree size — the
signature-verification failure path had no coverage before.
Endpoints come from externally fetched log lists and were used as-is:
any scheme was accepted and the default HTTP client followed up to 10
redirects, so a compromised list source could aim the tailer at
internal hosts (e.g. cloud metadata) even though STH signatures would
fail. Now NewClient requires https except for loopback (local test
servers), both clients refuse redirects outright, and a redirect
response surfaces as a normal HTTP error. Tests cover scheme
rejection and redirect refusal.
Log list entries joined as BaseURL+path break when the URL carries a
trailing slash: https://log.example.com/ + /ct/v1/get-sth becomes
...//ct/v1/get-sth and every request fails. Normalize once in
NewClient. Test covers the trailing-slash case end to end.
Certificate SANs went into the store after only lowercasing, so a
unicode name like 'buecher.example.com' was stored as raw UTF-8 and
could never be found by search, which runs IDNA. Ingest now maps
names through a permissive IDNA profile (keeping DKIM-style
underscores that the strict lookup profile rejects), enforces RFC
1035 label lengths, and stores the canonical form. Normalize also no
longer lowercases before IDNA mapping, which produced wrong punycode
for decomposed unicode input.
VerifySTH silently passed when no log key was pinned or when an STH
arrived without a signature, and a failed verification only produced a
log line while ingestion continued from the unauthenticated source.
NewClient now refuses logs without keys, the tailer skips the fetch
cycle on verification failure, and it rejects list entries whose
log_id does not match SHA-256 of the key.
Both call sites pass the extnValue OCTET STRING contents, which per
RFC 6962 section 3.3 are already the TLS-encoded SCT list. The extra
asn1.Unmarshal failed on virtually every real certificate (the first
byte is the high byte of the uint16 length), so embedded SCT
timestamps were silently dropped and first_seen fell back to the leaf
timestamp. The test masked this by double-wrapping the value.
Certificate SANs are attacker-controlled and were copied into store
keys unfiltered. Since 0x00 is the apex/sub key separator, a SAN like
'ab\0cd.qqq' corrupted per-apex counts during recount and made scan
bounds ambiguous. Names now must be printable ASCII at ingest, and
Normalize rejects them as defense in depth.
