Skip to content
Snippets Groups Projects
  • dpsutton's avatar
    2ada1fc4
    Fix errors when downgrading then upgrading to bigquery driver (#22121) · 2ada1fc4
    dpsutton authored
    This issue has a simple fix but a convoluted story. The new bigquery
    driver handles multiple schemas and puts that schema (dataset-id) in the
    normal spot on a table in our database. The old driver handled only a
    single schema by having that dataset-id hardcoded in the database
    details and leaving the schema slot nil on the table row.
    
    ```clojure
    ;; new driver describe database:
    [{:name "table-1" :schema "a"}
     {:name "table-2" :schema "b"}]
    
    ;; old driver describe database (with dataset-id "a" on the db):
    [{:name "table-1" :schema nil}]
    ```
    
    So if you started on the new driver and then downgraded for some reason,
    the table sync would see you had tables with schemas, but when it
    enumerated the tables in the database on the next sync, would see tables
    without schemas. It did not unify these two together, nor did it archive
    the tables with a schema. You ended up with both copies in the
    database, all active.
    
    ```clojure
    [{:name "table-1" :schema "a"}
     {:name "table-2" :schema "b"}
     {:name "table-1" :schema nil}]
    ```
    
    If you then tried to migrate back to the newer driver, we migrated them
    as normal: since the old driver only dealt with one schema but left it
    nil, put that dataset-id on all of the tables connected to this
    connection.
    
    But since the new driver and then the old driver created copies of the
    same tables, you would end up with a constraint violation: tables with
    the same name and, now after the migration, the same schema. Ignore this
    error and the sync in more recent versions will correctly inactivate the
    old tables with no schema.
    
    ```clojure
    [{:name "table-1" :schema "a"}  <-|
     {:name "table-2" :schema "b"}    | constraint violation
     {:name "table-1" :schema "a"}] <-|
    
    ;; preferrable:
    [{:name "table-1" :schema "a"}
     {:name "table-2" :schema "b"}
     {:name "table-1" :schema nil :active false}]
    ```
    Fix errors when downgrading then upgrading to bigquery driver (#22121)
    dpsutton authored
    This issue has a simple fix but a convoluted story. The new bigquery
    driver handles multiple schemas and puts that schema (dataset-id) in the
    normal spot on a table in our database. The old driver handled only a
    single schema by having that dataset-id hardcoded in the database
    details and leaving the schema slot nil on the table row.
    
    ```clojure
    ;; new driver describe database:
    [{:name "table-1" :schema "a"}
     {:name "table-2" :schema "b"}]
    
    ;; old driver describe database (with dataset-id "a" on the db):
    [{:name "table-1" :schema nil}]
    ```
    
    So if you started on the new driver and then downgraded for some reason,
    the table sync would see you had tables with schemas, but when it
    enumerated the tables in the database on the next sync, would see tables
    without schemas. It did not unify these two together, nor did it archive
    the tables with a schema. You ended up with both copies in the
    database, all active.
    
    ```clojure
    [{:name "table-1" :schema "a"}
     {:name "table-2" :schema "b"}
     {:name "table-1" :schema nil}]
    ```
    
    If you then tried to migrate back to the newer driver, we migrated them
    as normal: since the old driver only dealt with one schema but left it
    nil, put that dataset-id on all of the tables connected to this
    connection.
    
    But since the new driver and then the old driver created copies of the
    same tables, you would end up with a constraint violation: tables with
    the same name and, now after the migration, the same schema. Ignore this
    error and the sync in more recent versions will correctly inactivate the
    old tables with no schema.
    
    ```clojure
    [{:name "table-1" :schema "a"}  <-|
     {:name "table-2" :schema "b"}    | constraint violation
     {:name "table-1" :schema "a"}] <-|
    
    ;; preferrable:
    [{:name "table-1" :schema "a"}
     {:name "table-2" :schema "b"}
     {:name "table-1" :schema nil :active false}]
    ```
Code owners
Assign users and groups as approvers for specific file changes. Learn more.