Trellis.StructureDump (Trellis v0.1.0-alpha)

Copy Markdown View Source

Appends Trellis's definitions to the structure file mix ecto.dump writes, so mix ecto.load builds a database that has its transforms.

pg_dump --schema-only carries the Trellis schema, the target tables and the capture triggers, but no rows, so a database loaded from it has targets and no definitions. Ecto appends INSERTs for schema_migrations to the dump for the same reason; this appends Trellis.dump_definitions_sql/1 after them. mix trellis.dump runs dump/2 around mix ecto.dump, and the app's alias runs it in place of ecto.dump.

What the structure file needs

  • The Trellis schema in the dump. mix ecto.dump dumps every schema unless it is given --prefix (the repo's :dump_prefixes is read by the adapter, and as of ecto_sql 3.14 mix ecto.dump without --prefix leaves it out). When either names schemas, the list has to include the instance's :schema (default "trellis"), and this refuses to run without it.
  • The repo's :trellis configuration, Trellis.connect/1's options, as Trellis.Migration reads it. The handle starts no background work.

Loading

The file loads with mix ecto.load or plain psql, and nothing else runs: the worker that starts on the loaded database finds definitions that aren't built and builds them. Every definition loads :waiting_to_backfill, including one that was paused or quarantined where it was dumped, because a pause is runtime state.

Races

pg_dump and dump_definitions_sql/1 read separate snapshots. dump/2 reads the definitions before and after the pg_dump and redoes the dump when they differ, up to three times, so a define or DROP committing in between can't leave a file whose DDL and definitions disagree. append/2 reads once, after a dump someone else ran, and can't tell.

CREATE SCHEMA public

pg_dump 17 writes CREATE SCHEMA public; when it dumps public by name, which fails against a database that already has the schema, as every database mix ecto.create makes does, and mix ecto.load stops at the first error. This rewrites that line to CREATE SCHEMA IF NOT EXISTS public; so the file loads into a fresh database.

Summary

Functions

Appends the definitions of repo's Trellis to the structure file, which must exist: run it after mix ecto.dump. It reads the definitions once, so it can't tell a define that committed after the pg_dump (dump/2 can).

Runs dump, a function that writes the structure files (mix trellis.dump passes mix ecto.dump itself), between two reads of each repo's definitions, and appends them to each repo's file once the two agree.

Functions

append(repo, options \\ [])

@spec append(module(), keyword()) :: {:ok, Path.t()} | {:error, String.t()}

Appends the definitions of repo's Trellis to the structure file, which must exist: run it after mix ecto.dump. It reads the definitions once, so it can't tell a define that committed after the pg_dump (dump/2 can).

Options:

  • :dump_path: the file. Default: the repo's :dump_path, else :default_path.
  • :default_path: where the repo's structure.sql is when it has no :dump_path; mix trellis.dump passes Ecto's own default.
  • :dump_prefixes: the schemas the dump was restricted to, in place of the repo's :dump_prefixes (mix ecto.dump's --prefix).

Returns {:ok, path}, or {:error, message}.

dump(repos, dump)

@spec dump([{module(), keyword()}], (-> term())) ::
  {:ok, [Path.t()]} | {:error, String.t()}

Runs dump, a function that writes the structure files (mix trellis.dump passes mix ecto.dump itself), between two reads of each repo's definitions, and appends them to each repo's file once the two agree.

repos is a list of {repo, options}, with append/2's options.

Returns {:ok, paths}, or {:error, message}: a repo that can't be dumped, or definitions that changed during each of three dumps.