Trellis.Status (Trellis v0.1.0-alpha)

Copy Markdown View Source

Where a transform is in its lifecycle, as Trellis.status/2 reports it.

A newly defined transform starts :waiting_to_backfill and reaches :live once its backfill has run and it has caught up. That takes a connection somewhere in the fleet running the staging worker and drain threads (see Trellis.connect/1); with none, a definition waits forever.

The other fields say what the transform is stuck on, and are nil when nothing holds it up:

  • backfill_failure is set while its build keeps failing (one of its build chunks), and names the cause.
  • capture_wait is set while installing or widening capture on a table it reads waits for a lock another session holds. It clears once that session lets go.
  • capture_failure is set while capture of a table it reads is broken, or the drain halted on it: a schema change paused it (resume it once fixed), installing capture keeps failing (it clears once the cause is fixed), with kind :halt a failure no retry gets past reached it, so the drain (or its build's merge) paused it and what depends on it (resume it once fixed), or, with kind :disruption, Trellis's capture of a table it reads was changed outside Trellis (a trigger disabled by a fixture or bulk load, a table swapped) or the table is gone, so changes may be missing: resume it to reinstall capture and rebuild (Trellis.TestSupport does this for a test suite). A transform an operator paused has none.
  • held_keys is set while it holds keys in quarantine: source keys whose changes kept failing in its apply, so it leaves them out and their target rows stay as they were, whatever its status, :live included. Trellis.sample_quarantined/3 lists them, and Trellis.release_key/4 releases one once its cause is fixed.
  • drain_failure is set while the drain keeps failing on a page holding changes to a table it reads, with nothing charged or paused (see Trellis.DrainFailure), so its target stops short of that page, whatever its status, :live included. Every drain pass retries the page; fix the cause its error names. A paused or quarantined transform has none.

source_index_gaps is a warning, never a hold-up: a Trellis.SourceIndexGap for each source index a GROUP BY or JOIN target needs and lacks (its role says which part of the definition), [] when it needs none or they exist. Define and a resume refuse a transform without them, so a gap is an index dropped since, or one the transform opted out of with WITHOUT SOURCE INDEX. Nothing is paused, but Trellis.self_check/3 refuses to walk the target unless it opted out, when the walk scans the source once per page.

correcting lists a Trellis.CorrectingNode for each partition of its partitioned source it is being corrected for, oldest first ([] when none): a partition was attached, detached, dropped or truncated, and the transform re-derives the keys it touched. It reads :backfilling while one is listed, and :live again once they are re-derived and folded into its target. A paused or quarantined transform lists none.

paused_upstream_columns names the paused columns of upstream 1-1 targets that make a :live transform report :catching_up, each as "target.column" ([] when none). A column is paused when its fuse tripped or PAUSE TRANSFORM t.col froze it; an aggregate or JOIN reading it keeps applying over its frozen values, so it is stale against the source. Nothing is paused or rebuilt: resuming the column writes through to the reader and it reads :live again.

Every process sees them, whichever one runs the staging worker.

Summary

Types

A transform's lifecycle status.

t()

Types

status()

@type status() ::
  :waiting_to_backfill
  | :backfilling
  | :catching_up
  | :live
  | :quarantined
  | :paused

A transform's lifecycle status.

t()

@type t() :: %Trellis.Status{
  backfill_failure: Trellis.BackfillFailure.t() | nil,
  capture_failure: Trellis.CaptureFailure.t() | nil,
  capture_wait: Trellis.CaptureWait.t() | nil,
  correcting: [Trellis.CorrectingNode.t()],
  drain_failure: Trellis.DrainFailure.t() | nil,
  held_keys: Trellis.HeldKeys.t() | nil,
  paused_upstream_columns: [String.t()],
  source_index_gaps: [Trellis.SourceIndexGap.t()],
  status: status()
}