Use cached Malli validators/explainers; make `:fn` schemas cacheable (#39947)
This improves our Malli performance substantially! Many of our code paths were calling vanilla `malli.core/validate` or `malli.core/explain`; these redo the (possibly expensive) compilation of the schema into a validator or explainer for every call. We should use the caching versions in `metabase.util.malli.registry` everywhere, and our runtime performance will be much improved. However when I started using these cached versions, I found that the memory use was growing out of control. Eventually I tracked this down to `:fn` schemas. Functions are only comparable by pointer equality, so they make poor cache keys. `(fn ...)` or `comp` calls in a schema on a `mu/defn` function **get re-created for every call of the `mu/defn`'d function**! That's a big time sink recompiling the schemas if we're *not* caching, and a huge memory sink if we *are* caching! This PR pulls every such schema I could find out into a `def` so it uses the same closure and is cacheable. I'd like to automate that in some part of the Malli pipeline, or maybe a linter rule, but I haven't found a good way to do it yet. Part of #39946.
Showing
- enterprise/backend/src/metabase_enterprise/sandbox/api/table.clj 2 additions, 3 deletions...ise/backend/src/metabase_enterprise/sandbox/api/table.clj
- src/metabase/api/database.clj 1 addition, 1 deletionsrc/metabase/api/database.clj
- src/metabase/api/native_query_snippet.clj 1 addition, 1 deletionsrc/metabase/api/native_query_snippet.clj
- src/metabase/domain_entities/core.clj 2 additions, 2 deletionssrc/metabase/domain_entities/core.clj
- src/metabase/driver/common/parameters/dates.clj 2 additions, 2 deletionssrc/metabase/driver/common/parameters/dates.clj
- src/metabase/driver/sql_jdbc/execute.clj 11 additions, 8 deletionssrc/metabase/driver/sql_jdbc/execute.clj
- src/metabase/email.clj 1 addition, 1 deletionsrc/metabase/email.clj
- src/metabase/integrations/google.clj 2 additions, 2 deletionssrc/metabase/integrations/google.clj
- src/metabase/lib/convert.cljc 3 additions, 3 deletionssrc/metabase/lib/convert.cljc
- src/metabase/lib/metadata/calculation.cljc 8 additions, 5 deletionssrc/metabase/lib/metadata/calculation.cljc
- src/metabase/lib/query.cljc 3 additions, 3 deletionssrc/metabase/lib/query.cljc
- src/metabase/lib/remove_replace.cljc 4 additions, 2 deletionssrc/metabase/lib/remove_replace.cljc
- src/metabase/lib/schema/common.cljc 6 additions, 3 deletionssrc/metabase/lib/schema/common.cljc
- src/metabase/lib/schema/expression/arithmetic.cljc 1 addition, 2 deletionssrc/metabase/lib/schema/expression/arithmetic.cljc
- src/metabase/lib/schema/expression/temporal.cljc 1 addition, 2 deletionssrc/metabase/lib/schema/expression/temporal.cljc
- src/metabase/lib/schema/literal.cljc 1 addition, 2 deletionssrc/metabase/lib/schema/literal.cljc
- src/metabase/lib/schema/mbql_clause.cljc 4 additions, 1 deletionsrc/metabase/lib/schema/mbql_clause.cljc
- src/metabase/models/collection.clj 8 additions, 8 deletionssrc/metabase/models/collection.clj
- src/metabase/models/dispatch.clj 0 additions, 13 deletionssrc/metabase/models/dispatch.clj
- src/metabase/models/interface.clj 0 additions, 1 deletionsrc/metabase/models/interface.clj
Loading
Please register or sign in to comment