fix(core): storage time ordering, watch state machine, replay safety, API pagination

Storage: fixed-width millisecond timestamp format with a one-time migration
so lexicographic comparisons stay chronological; snapshot auto-IDs get
nanosecond precision; Close is synchronized; commit failures roll back; DSN
paths are URI-escaped; the dedup index is no longer rebuilt every startup;
the in-memory store now dedups by event_id and honors Limit/Offset/SinceSeq.

Event/discovery: stream IDs URL-escape the selector so slash-containing label
selectors round-trip and cannot collide; alias resolution fills empty fields
only and no longer clobbers explicit group/version/kind; rc maps to
replicationcontrollers and rs to replicasets.

Watch: the label selector is now actually applied to List and Watch; the 410
relist path is backoff-throttled, backoff resets after a healthy watch, an
empty list resourceVersion backs off instead of looping forever, non-410
watch errors reconnect without writing a spurious permanent gap, an idle
watch timer forces reconnects, and Run cancels sibling streams and recovers
from panics. Synthetic relist events carry a distinct event_id so unchanged
objects re-listed after a gap survive dedup.

Materialize: a baseline now resets object state (objects deleted during a gap
disappear) and heals open gaps; diff ignores server metadata by path, not by
key name, so user fields named status/uid are kept; nested add/remove carry
Added/Removed flags; dotted paths are escaped; Diff validates the window and
treats a zero before as empty; Snapshot records per-stream watermarks and
writes a TypeSnapshot journal record.

Replay: apply requires a successful dry run; gvrFor covers every policy kind
so Include* toggles work; unsupported kinds are reported as skipped instead
of silently dropped; NodePort services lose clusterIP and nodePort while
headless keeps clusterIP: None; targetNS is always honored; namespace mapping
collisions are detected; plan IDs and field managers are collision-safe and
plans are mutex-guarded against concurrent dry-runs/applies.

API: cursor pagination pushes ingest_seq into the SQL filter (export --server
no longer truncates); gap records expose their payload; /v1/diff requires a
cluster_id; dry-run/apply no longer accept a client-supplied kubeconfig;
plans carry target_context; HTTP server gets timeouts.

Metrics: the watch collector and replay planner now bump the registered
counters, and the server refreshes store-derived gauges on a ticker.

CLI: replay apply works locally and refuses to apply after a failed dry run;
coverage --server follows cursors and shows gap details; timeline
auto-detects namespaces and resolves cluster-scoped objects; diff validates
the window; export writes 0600 files; PVCs are no longer treated as
cluster-scoped; krply-server reports the build version.
This commit is contained in:
lakshit verma 2026-08-06 06:50:30 +05:30
parent c6d43a9d15
commit 44fbd878a1
No known key found for this signature in database
GPG key ID: EB498AFC60A7A01A
24 changed files with 736 additions and 191 deletions

View file

@ -19,9 +19,10 @@ import (
"github.com/krply/krply/internal/metrics"
"github.com/krply/krply/internal/replay"
"github.com/krply/krply/internal/storage"
"github.com/krply/krply/internal/version"
)
const version = "0.1.0"
var buildVersion = version.Version
func main() {
if err := run(); err != nil {
@ -86,7 +87,7 @@ func run() error {
flag.Parse()
if *showVer {
fmt.Println(version)
fmt.Println(buildVersion)
return nil
}
@ -123,17 +124,40 @@ func run() error {
m := metrics.New()
m.RefreshFromStore(ctx, store)
planner.SetMetrics(m)
srv, err := api.NewServer(store, mat, planner, m, version)
// Store-derived gauges are refreshed on a ticker; nothing else in the
// process would update degraded streams, gap counts, or store size.
go func() {
ticker := time.NewTicker(30 * time.Second)
defer ticker.Stop()
for {
select {
case <-ctx.Done():
return
case <-ticker.C:
m.RefreshFromStore(ctx, store)
}
}
}()
srv, err := api.NewServer(store, mat, planner, m, buildVersion)
if err != nil {
return fmt.Errorf("build api server: %w", err)
}
httpSrv := &http.Server{Addr: listenAddr, Handler: srv.Handler()}
httpSrv := &http.Server{
Addr: listenAddr,
Handler: srv.Handler(),
ReadHeaderTimeout: 10 * time.Second,
ReadTimeout: 60 * time.Second,
WriteTimeout: 2 * time.Minute,
IdleTimeout: 2 * time.Minute,
}
errCh := make(chan error, 1)
go func() {
slog.Info("krply-server listening", "addr", listenAddr, "store", *storePath, "version", version)
slog.Info("krply-server listening", "addr", listenAddr, "store", *storePath, "version", buildVersion)
errCh <- httpSrv.ListenAndServe()
}()