Skip excess keys on serialization load (#35748)
When loading from a backup, we should not fail if the previous version saved keys that are no longer needed in the
current version. This adds a step (after ingestion but before loading) that removes any "extra" keys from the ingested
entity.
How?
- make `load-xform-basics` remove extraneous keys from the ingested model.
- by default, it identifies the "extraneous keys" as keys *other than* keys corresponding to column names in the database.
- for *some* ingested models, there is not a 1:1 correlation between an ingested model and a `:metabase/model`. For example, when we ingest an `"Action"`, that corresponds to the `:model/Action` model. But that model has three "submodels" - `:model/QueryAction`, `:model/HttpAction`, and `:model/ImplicitAction` - and when we load the ingested map into the database, `action/insert!` will insert subsets of the map it receives into _two_ tables (`action` plus one of the submodel tables).
- to handle this case, `serdes/ingested-model-columns` is a multimethod, which we can override. For example, for `"Action", we override it to include the full set of columns that exist in any of the model/submodel tables.
Co-authored-by:
bryan <343288+escherize@users.noreply.github.com>
Showing
- enterprise/backend/test/metabase_enterprise/serialization/v2/load_test.clj 34 additions, 0 deletions...d/test/metabase_enterprise/serialization/v2/load_test.clj
- src/metabase/models/action.clj 3 additions, 0 deletionssrc/metabase/models/action.clj
- src/metabase/models/collection.clj 2 additions, 2 deletionssrc/metabase/models/collection.clj
- src/metabase/models/serialization.clj 42 additions, 3 deletionssrc/metabase/models/serialization.clj
- src/metabase/util/connection.clj 21 additions, 0 deletionssrc/metabase/util/connection.clj
- test/metabase/db/schema_migrations_test.clj 1 addition, 13 deletionstest/metabase/db/schema_migrations_test.clj
Loading
Please register or sign in to comment