dpsutton
authored
* Create `cache_info` table in our persisted schemas ```sql -- postgres/redshift test-data=# select * from metabase_cache_424a9_379.cache_info ; key | value ------------------+-------------------------------------- settings-version | 1 created-at | 2023-03-28 instance-uuid | 407e4ba8-2bab-470f-aeb5-9fc63fd18c4e instance-name | Metabase Test (4 rows) ``` ```sql --mysql mysql> select * from metabase_cache_424a9_435.cache_info; +------------------+--------------------------------------+ | key | value | +------------------+--------------------------------------+ | settings-version | 1 | | created-at | 2023-03-28 | | instance-uuid | 407e4ba8-2bab-470f-aeb5-9fc63fd18c4e | | instance-name | Metabase Test | +------------------+--------------------------------------+ ``` our key values in v1: ```clojure (defn kv-table-values "Version 1 of the values to go in the key/value table `cache_info` table." [] [{:key "settings-version" :value "1"} {:key "created-at" ;; "2023-03-28" :value (.format (LocalDate/now) DateTimeFormatter/ISO_LOCAL_DATE)} {:key "instance-uuid" :value (public-settings/site-uuid)} {:key "instance-name" :value (public-settings/site-name)}]) ``` This will enable us to delete cached schemas in shared databases (cloud versions like redshift) without risking wiping out a cached schema during an active test run. The code to delete the schemas will be something like, ```clojure (def spec (sql-jdbc.conn/connection-details->spec :redshift {:host "xxx.us-east-1.reshift.amazonaws.com" :db "db" :port 5439 :user "user" :password "password"})) (let [days-prior 1 ;; probably 2 to handle crossing day time. Can also adjust to 6 hours, etc threshold (.minus (java.time.LocalDate/now) days-prior java.time.temporal.ChronoUnit/DAYS)] (doseq [schema (->> ["select nspname from pg_namespace where nspname like 'metabase_cache%'"] (jdbc/query spec) (map :nspname))] (let [[{created :value}] (jdbc/query spec (sql/format {:select [:value] :from [(keyword schema "cache_info")] :where [:= :key "created-at"]} {:dialect :ansi}))] (when created (let [creation-day (java.time.LocalDate/parse created)] (when (.isBefore creation-day threshold) (jdbc/execute! spec [(format "drop schema %s cascade" schema)]))))))) ``` or if we want to do it in a single query: ```clojure schemas=> (let [days-prior 1 threshold (.minus (java.time.LocalDate/now) days-prior java.time.temporal.ChronoUnit/DAYS)] (let [schemas (->> ["select nspname from pg_namespace where nspname like 'metabase_cache%'"] (jdbc/query spec) (map :nspname))] (jdbc/with-db-connection [conn spec] (jdbc/execute! conn [(format "set search_path= %s;" (str/join ", " schemas))]) (let [sql (sql/format {:select [:schema :created-at] :from {:union-all (for [schema schemas] {:select [[[:inline schema] :schema] [{:select [:value] :from [(keyword schema "cache_info")] :where [:= :key [:inline "created-at"]]} :created-at]]})}} {:dialect :ansi}) old? (fn [{create-str :created-at}] (let [created (java.time.LocalDate/parse create-str)] (.isBefore created threshold))) schemas-to-delete (->> (jdbc/query spec sql) (filter old?) (map :schema))] schemas-to-delete)))) ("metabase_cache_424a9_503") ;; when there are any, string join them and delete them as above ``` * Use Instants so we can compare by hours instead of just days * require toucan as db
Code owners
Assign users and groups as approvers for specific file changes. Learn more.