-
dpsutton authored
https://github.com/camsaul/toucan2/pull/126 Editing the data model loads a _lot_ of data. It hits the following form from the api: ```clojure (hydrate [:tables [:fields [:target :has_field_values] :has_field_values] :segments :metrics]) ``` On my test instances, with 1500 tables each with 14 fields: 15mb and 359 tables each with 14 fields: 3.69mb. Not great. If you want to recreate, create an empty db, sync it, then ```clojure (time (let [conn (sql-jdbc.conn/db->pooled-connection-spec 3)] (dotimes [n 359] (jdbc/execute! conn [(format "create table if not exists foo_%d (id int, col_1 text, col_2 text, col_3 text, col_4 text, col_5 text, col_6 text, col_7 text, col_8 text, col_9 text, col_10 text, col_11 text, col_12 text, col_13 text, col_14 text)" n)])))) (sync/sync-database! (db/select-one 'Database :id 3) {:scan :schema}) (binding [api/*current-user-permissions-set* (delay #{"/"})] (count (#'metabase.api.database/db-metadata 3 true true))) ``` And the source of the bug was a classic recur with a lazy concat. ```diff diff --git a/src/toucan2/tools/hydrate.clj b/src/toucan2/tools/hydrate.clj index e1d0459..10c9c49 100644 --- a/src/toucan2/tools/hydrate.clj +++ b/src/toucan2/tools/hydrate.clj @@ -456,7 +456,7 @@ (if (empty? hydrated-instances) (concat acc (map :instance annotated-instances)) (let [[not-hydrated [_needed-hydration & more]] (split-with (complement :needs-hydration?) annotated-instances)] - (recur (concat acc (map :instance not-hydrated) [(first hydrated-instances)]) + (recur (vec (concat acc (map :instance not-hydrated) [(first hydrated-instances)])) more (rest hydrated-instances)))))) ```
dpsutton authoredhttps://github.com/camsaul/toucan2/pull/126 Editing the data model loads a _lot_ of data. It hits the following form from the api: ```clojure (hydrate [:tables [:fields [:target :has_field_values] :has_field_values] :segments :metrics]) ``` On my test instances, with 1500 tables each with 14 fields: 15mb and 359 tables each with 14 fields: 3.69mb. Not great. If you want to recreate, create an empty db, sync it, then ```clojure (time (let [conn (sql-jdbc.conn/db->pooled-connection-spec 3)] (dotimes [n 359] (jdbc/execute! conn [(format "create table if not exists foo_%d (id int, col_1 text, col_2 text, col_3 text, col_4 text, col_5 text, col_6 text, col_7 text, col_8 text, col_9 text, col_10 text, col_11 text, col_12 text, col_13 text, col_14 text)" n)])))) (sync/sync-database! (db/select-one 'Database :id 3) {:scan :schema}) (binding [api/*current-user-permissions-set* (delay #{"/"})] (count (#'metabase.api.database/db-metadata 3 true true))) ``` And the source of the bug was a classic recur with a lazy concat. ```diff diff --git a/src/toucan2/tools/hydrate.clj b/src/toucan2/tools/hydrate.clj index e1d0459..10c9c49 100644 --- a/src/toucan2/tools/hydrate.clj +++ b/src/toucan2/tools/hydrate.clj @@ -456,7 +456,7 @@ (if (empty? hydrated-instances) (concat acc (map :instance annotated-instances)) (let [[not-hydrated [_needed-hydration & more]] (split-with (complement :needs-hydration?) annotated-instances)] - (recur (concat acc (map :instance not-hydrated) [(first hydrated-instances)]) + (recur (vec (concat acc (map :instance not-hydrated) [(first hydrated-instances)])) more (rest hydrated-instances)))))) ```
Code owners
Assign users and groups as approvers for specific file changes. Learn more.