This project is mirrored from https://github.com/metabase/metabase.
Pull mirroring updated .
- Aug 06, 2023
-
-
Cam Saul authored
* QP: use Malli * Fix schema names * Fix missing namespaces * Fix busted stuff * Simplify the new schema * Test fix
* Test fixes * MongoDB fixes * FAST mu/defn * Malli registry * Misc fixes * Some cleanup * Misc cleanup and test fixes * mu/disable-enforcement * Test fix * Fix Cljs version of `disable-enforcement` * Misc improvements * Update tests * Fix Kondo warnings * Fix Kondo warnings * Fix busted Malli schema * PR feedback * Fix busted schema * Kondo fixx * Test fixes * Use Malli version of `describe` now that PR was merged * Work around https://github.com/metosin/malli/issues/924 * Test fixes * Improvements * Test fix
-
- Aug 02, 2023
-
-
Ngoc Khuat authored
-
- Jun 17, 2023
-
-
Cam Saul authored
* Squash * Replace connection-with-timezone usages * Add deprecation warning. * Fix docstring
-
- May 31, 2023
-
-
john-metabase authored
-
- May 15, 2023
-
-
Ngoc Khuat authored
* Database to toucan2 * Table to toucan2 * Field to toucan 2
-
- May 12, 2023
-
-
dpsutton authored
Before: ``` ❯ clj -M:"$ALIASES" WARNING: abs already refers to: #'clojure.core/abs in namespace: clojure.algo.generic.math-functions, being replaced by: #'clojure.algo.generic.math-functions/abs operator.clj:172 recur arg for primitive local: sum is not matching primitive, had: Object, needed: long Auto-boxing loop arg: sum Clojure 1.12.0-alpha2 user=> (do (dev) (start!)) Warning: environ value /Users/dan/.sdkman/candidates/java/current for key :java-home has been overwritten with /Users/dan/.sdkman/candidates/java/17.0.1-zulu/zulu-17.jdk/Contents/Home 2023-05-12 14:26:37,091 INFO metabase.util :: Maximum memory available to JVM: 8.0 GB 2023-05-12 14:26:47,248 INFO util.encryption :: Saved credentials encryption is DISABLED for this Metabase instance.
For more information, see https://metabase.com/docs/latest/operations-guide/encrypting-database-details-at-rest.html ..instrumented #'metabase.util.malli/with-api-error-message ..instrumented #'metabase.util.honey-sql-2/identifier ..instrumented #'metabase.util.honey-sql-2/normalize-type-info ..instrumented #'metabase.util.honey-sql-2/with-database-type-info ..instrumented #'metabase.util.honey-sql-2/cast ..instrumented #'metabase.util.honey-sql-2/quoted-cast ``` After: ``` ❯ clj -M:"$ALIASES" WARNING: abs already refers to: #'clojure.core/abs in namespace: clojure.algo.generic.math-functions, being replaced by: #'clojure.algo.generic.math-functions/abs operator.clj:172 recur arg for primitive local: sum is not matching primitive, had: Object, needed: long Auto-boxing loop arg: sum Clojure 1.12.0-alpha2 user=> (do (dev) (start!)) Warning: environ value /Users/dan/.sdkman/candidates/java/current for key :java-home has been overwritten with /Users/dan/.sdkman/candidates/java/17.0.1-zulu/zulu-17.jdk/Contents/Home 2023-05-12 15:11:21,678 INFO metabase.util :: Maximum memory available to JVM: 8.0 GB 2023-05-12 15:11:30,806 INFO util.encryption :: Saved credentials encryption is DISABLED for this Metabase instance. For more information, see https://metabase.com/docs/latest/operations-guide/encrypting-database-details-at-rest.html util.cljc:346 recur arg for primitive local: cumulative_byte_count is not matching primitive, had: Object, needed: long Auto-boxing loop arg: cumulative-byte-count WARNING: infinite? already refers to: #'kixi.stats.math/infinite? in namespace: kixi.stats.core, being replaced by: #'clojure.core/infinite? 2023-05-12 15:11:55,615 INFO driver.impl :: Registered abstract driver :sql ```
-
- May 05, 2023
-
-
Cal Herries authored
* Tidy database-supports? defmethods * Fix oracle build * Fix driver * Fix typo * prefer driver/database-supports? instead of driver/supports? * driver/supports? -> driver/database-supports? in tests * Fix foreign-key support for sqlite in tests * supports? -> database-supports? * `supports?` -> `database-supports?` * supports? -> database-supports? * Update supports? docstring * Fix tests * Fix test * Remove unused require * Add mt/with-everything-store in tests * fix test * Fix tests * Add note in driver changelog
-
- Apr 19, 2023
-
-
Cam Saul authored
* Don't load metabase.query-processor-test.test-mlv2 until we load QP * Add schema and tests for `:value` * Don't skip tests for #29895 * Conversion logic for `:value` * pMBQL normalization should handle `:effective-type` * Options should validate `:base-type`, `:effective-type`, etc. if present * Use `:effective-type` for type checking if present
-
- Apr 12, 2023
-
-
dpsutton authored
in query_processor.clj we were requiring ```clojure (when-not config/is-prod? (classloader/require 'metabase.query-processor-test.test-mlv2)) ``` And this was blowing up as it was not present in the aliases used for cypress and the FE crew. Now user.clj requires it in the same way that it requires `[metabase.test-runner.assert-exprs]` so we can run tests with `partial=` and their ilk. And it is required by the test entry namespace so CI and test runners find it as well. The `resolve` command in query_processor.clj now works in dev and CI ```clojure query-processor=> (resolve 'metabase.query-processor-test.test-mlv2/post-processing-middleware) ,#'metabase.query-processor-test.test-mlv2/post-processing-middleware ``` (comma to prevent commit message from eliding as a comment) so presumably tests that require the middleware tricks from this test namespace will continue to work.
-
- Apr 11, 2023
-
-
Cam Saul authored
* Add QP test middleware to validate converted pMBQL queries against MLv2 schema * Revert 'fix' for #29944 * Fix ups * Hack * Fix final test failure * Remove unused namespace * Fix Kondo warning * Skip #29958
-
- Mar 23, 2023
-
-
Ngoc Khuat authored
* convert db\/insert! to t2
-
- Mar 21, 2023
-
-
Ngoc Khuat authored
-
- Mar 20, 2023
-
-
Ngoc Khuat authored
* db/select-one-field -> t2/select-one-fn * `db/select-field` -> `t2/select-fn-set` * db/select-ids -> t2/select-pk-set * db/select-field->field -> t2/select-fn->fn * db/select-field->id -> t2/select-fn->pk * `db/select-id->field` -> `t2/select-pk->field` * add discourage var
-
- Mar 17, 2023
-
-
Ngoc Khuat authored
* db/select-one -> t2/select-one
-
Ngoc Khuat authored
* db/select-one-field -> t2/select-one-fn * add discourage var
-
- Mar 08, 2023
-
-
Ngoc Khuat authored
-
- Feb 10, 2023
-
-
Cam Saul authored
* Mind = blown
* Remove printls * Simplify things * Better exampls * Update dev.clj -
Mark Bastian authored
Adding a dev-level docker-compose file. Located in /dev and is self-documented.
-
- Feb 07, 2023
-
-
Tim Macdonald authored
* Log a warning when extraneous parameters are passed into a request * Massive Metabase logging performance improvement: use configured log levels in logs API endpoint --------- Co-authored-by:
Cam Saul <github@camsaul.com>
-
- Feb 02, 2023
-
-
Bryan Maass authored
insert next-gen permission paths alongside currently active permission paths + classification (#27911) * implement move, which returns v2 paths - TODO: insert these into the db (move v1-path) => [v2 paths] * cleanup + add some schemas * generative tests 4 permission path classification * whitespace lint * detect data, query, and paths for v2 * calling move on v2 paths is a no-op * differentiate between v1 and v2 permissions quickchecking for move, classify-path, and classify-data-path * fix tests + add idempotency test * add tests for classification of permission paths - rename move to ->v2-path - move some fxns around - ascii art in test * making the legos line up - need to insert both v1 and v2 versions of paths (of course) - valid-path? has to allow v2 paths to be inserted * replace mu/with-api-error-message * linter + code quality fixes * privatize rx->kind * remove some changes that should be 4 anotherbranch * revert ns * delete v2 permissions in - they aren't handled by the perm graph parser, so they don't get propagated into "old graph", so the diff between old and new indicates that they need to be rewritten. * only delete v2 paths for the current group_id - reorder declarations in models.permissions * remove extra line break * Update src/metabase/models/permissions.clj fix typo Co-authored-by:
metamben <103100869+metamben@users.noreply.github.com> --------- Co-authored-by:
metamben <103100869+metamben@users.noreply.github.com>
-
- Feb 01, 2023
-
-
Cam Saul authored
* Switch to the new Hawk test runner * Fix some stuff * Fix everything * Commentz * Bump init timeout? * Fix Kondo config * Remove unused * Bump test runner version to get `:exclude-tags` support
-
- Jan 26, 2023
-
-
Mark Bastian authored
Replaced, as needed, instances of str/lower|upper-case with the -en version. Also updated rules in our kondo config.
-
- Jan 04, 2023
-
-
metamben authored
* Implicit action pre-fetch values (#25940) * [Apps] Implicit action pre-fetch values WIP To return a parameter-id: value map in order to prefill ui. * Use jsonstring encoded parameters * Change parameters exposed by implicit actions Previously, we would look at a model's source-table to determine which parameters were valid for implicit actions. So if an implicit action is on a User model that hides the user.password column, you could still update or insert the password. With this commit, if the model doesn't expose a source table column, it is not a valid parameter. In the example user.password could not be inserted/updated or fetched for prefilling. * Fixing tests * Show data app page ID params in editing mode (#25943) * Data apps layout updates (#25930) * play with grid sizes to make apps feel appier [ci skip] * move app nav to bar inside app * only hide other header buttons outside of edit mode * tweak overflow issue * fix header width on data apps * add control to see app contents * set data apps layout constants contextually [ci skip] * remove hardcoded value [ci skip] * extract conextual header styles [ci skip] * set max-width on paramaters for data apps [ci skip] * move data apps nav deeper, only show if not editing [ci skip] * add spacing to contents trigger, rm old code [ci skip] * rm table thing for now [ci skip] * Fixes for data app layout updates (#25962) * Reorder import * Fix type * Fix missing translation, move out URL * Remove not used import * Rework `getSelectedItems` utility for data apps * Fix selected data app page highlighting * Don't reload app navbar on page change * Change nav item links look * Remove `DataAppPageSidebarLink` * Turn nav into HTML list * Use styled components * Fix edit button covered by visualizations * Fix opening homepage * Remove redundant prop Co-authored-by:
Anton Kulyk <kuliks.anton@gmail.com> * First take at parameterized data app page titles (#25938) * Bring `DataAppContext` back * Pass `onFocus` and `onBlur` to `EditableText` * Export `EditableTextProps` * Add basic `DataAppPageTitle` component * Use `DataAppPageTitle` * Fix component name * Add primitive autocompletion to page title input * Add `title_template` to `DataAppNavItem` type * Tweak value management in `DataAppPageTitle` * Handle `null` values in titles more gracefully * Don't show suggestions without detail cards * Don't add whitespace when pasting a token * Don't update app's collection all the time * Add data app related selectors to dashboard selectors * Add redux state to keep title template changes * Update page title template on page save * Wire up `DataAppPageTitle` with dashboard code * Fix unit tests * Reset state on exiting editing mode * Add back button to template data picker * Fix test that didn't include pk on insert * Use the actual pk value Co-authored-by:
Anton Kulyk <kuliks.anton@gmail.com> Co-authored-by:
Kyle Doherty <5248953+kdoh@users.noreply.github.com> * Revert "Remove data app permission code (#26016)" This reverts commit b8c92cf1. * Remove unused imports * Don't require the table_id on result_metadata elements (#26073) In some cases (which ones still has to be clarified) the result metadata doesn't contain the table_id and so no parameters are provided. In fact the table ID is probably not needed anyway because fields are already table specific. * Support filtering cards by used model (#26088) This change adds a new filter option for the GET /card endpoint. * Support filtering apps using a given model (#26124) * Handle type errors more cleanly (#26120) * [Apps] Handle type errors more cleanly Adds a number of tests for different db datatypes, these can be expanded on as we solve. Catches most exceptions during execution and logs then throws a generic message that leaks less of the details. * Make sure we order-by expected results * Fix tests, allow purposeful exception through * Accept model-ids for scaffolding apps (#26139) * [Apps] Accept model-ids for scaffolding apps WIP - accept model-ids but there's still some issues, might be able to start building UI from this point though. * Maybe working model * Work with card__n table-ids passed in. Fix problems with detail page * Dynamically add actions that exist on models * Update tests for existing custom actions * Can't def tru * Address review comments * Add required to action parameters (#26296) Looks at `database_required` field values for implicit actions, and `required` template-tags for custom query actions. Bubble those values up onto `action.parameters`. Finally, look at `requires_pk` to see if pk fields should be marked as required regardless. * Cleanup scaffolded back button (#26317) * Forward action execution error messages to the frontend (#26469) This is a crude temporary solution necessary to be able to debug problems related to executing actions. Although error parsing has been implemented for Postgres (only) with the goal to provide information for the FE like `{"errors":{"id":"violates foreign key constraint note_subject_id_fkey"}}`. This code doesn't care about the error message itself, it just forwards the error message from the DB to the FE. * Actions for public pages (#26554) * [Apps] Testing out public dashboards * Add WIP endpoints for public execution * Add throttle check to public execution route * Bump attempts-threshold to 1000 * Bump threshold to 5000 * Add test * Fix linter * Review changes * Fix lint * Run publicly shared actions with full permissions (#26610) * Support actions with MySQL and MariaDB (#26573) Support actions with MySQL/MariaDB * Check permissions for implicit actions (#26652) * Check permissions when executing implicit actions * Fix some action tests and reduce permissions boilerplate * Revert "[Apps] Disable data-apps prior to release (#25982)" This reverts commit 51147027. * WIP: Unify action representation (#26717) (#27349) * [Apps] WIP initial db migrations for giving implicit actions an action_id and removing card from query_action * Make crud work for http actions * Make implicit crud work * Rename implicit_action.namespace to kind. Rewrite merged-model-actions to actions-with-implicit-parameters * Fix query action creation * Make actions.test-util/with-action create a holding model * Fix unified-action-create-test * Fix get-action-test * Add tests for creating implicit and http dashcard actions * Enable query action in dashcard-action-create-update-test * Fix metabase.api.dashboard-test/dashcard-query-action-execution-test * Fix metabase.api.dashboard-test/dashcard-http-action-execution-test * Fix metabase.api.dashboard-test/dashcard-action-execution-auth-test * Fix metabase.api.dashboard-test/dashcard-action-execution-granular-auth-test * Make implicit action execution work * Scaffolding implicit actions from table * Fix metabase.api.action-test * Make scaffolding work with existing models. Update with-action macro to allow multiple actions * Fix metabase.api.public-test * Remove unused names from metabase.models.action * Remove some unused imports * Remove an unused import * Remove model_action and fix a bunch of tests * Fix dashcard-implicit-action-only-expose-and-allow-model-fields * Fix action type inheritance * Fill in remarks and add description to action * Add forward migration for existing actions and rollbacks that remove existing actions * Prevent overwriting parameters of custom actions * Support setting description on actions * Update migration to work with mysql - rollback is still not quite working in mysql due to database_id fk * Rename with-action macro to with-actions and document it * Remove report_card.is_write * Fix some more tests * Fix rollback across all appdb drivers, add not null constraints, use action.name to hold implicit slugs * Pull name and description from is_write card * Add name to scaffolded implicit actions * Remove ModelAction * Remove is_write from comments too * Different databases reference capture groups in regexp_replace differently. Also drop model_action table * Update migrations to pass linter. Introduce ${capturegroup.ref} to handle the different ways to reference capture groups within a regexp_replace. This allows collapsing large data migrations into a single `sql` element that the linter requires. * Add kind to action-mapper so we can sort as we wish * Add name to with-temp Action * Adjust FE to query actions migration (#26764) * Remove `is_write` card notion * Remove action `slug` from action types * Fix typo in `estimateCardSize` name * Remove hanging card's `is_write` prop * Remove not used writeback actions * Update endpoints * Remove not used prop * Remove action `slug` usage * Update actions entity * Tweak action creator to handle new actions shape * Fix saving action dashcards * Fix actions API usage How did that work before?
* Pass `database_id` when creating query actions * Remove custom list API implementation for actions * Clean up types Co-authored-by:Tamás Benkő <tamas@metabase.com> * Change regexp_replace to substring to support mysql - this is ok because we have consistent json serialization. * Check permissions on the model in the action API * Remove unused required namespace * Throw an exception when demoting a model with actions to question * Allow saved parameters on implicit actions to add to implicit parameters Co-authored-by:
Case Nelson <case@metabase.com> Co-authored-by:
Anton Kulyk <kuliks.anton@gmail.com> Co-authored-by:
metamben <103100869+metamben@users.noreply.github.com> Co-authored-by:
Case Nelson <case@metabase.com> * empty commit to kick CI * bump CI * Fix down migrations change v45.00-002 sets up a fk fk_query_action_ref_action_id. h2 v2 for some strange reason requires us to remove the constraint before we can remove the primary key. But then we have to restore the state that v45.00-002 creates manualy * Allow for version for `(migrate! :down 45)` * Remove data apps parts not planned for 0.46 Removed app container, app scaffolding and pages (dashboards with is_app_page) * Address linter issues * Make some cosmetic changes to reduce the diff * Make new permission API endpoints use defendpoint-schema * Fix bad automerge * Skip Mongo 4.2 instead of 4.0 Co-authored-by:
Case Nelson <case@metabase.com> Co-authored-by:
Anton Kulyk <kuliks.anton@gmail.com> Co-authored-by:
Kyle Doherty <5248953+kdoh@users.noreply.github.com> Co-authored-by:
dan sutton <dan@dpsutton.com>
-
- Jan 03, 2023
-
-
Bryan Maass authored
- add the rest of malli reserved aliases - malli.util :as [mu -> mut] (had collision with metabase.util.malli)
-
- Dec 20, 2022
-
-
Tim Macdonald authored
-
- Dec 09, 2022
-
-
Cam Saul authored
* Make Dimension unique on `field_id` and deduplicate * Fix rollback for new migrations * Don't require migration comments to contain 'added' anymore since it's part of the version ID now * Don't require 'Added <version>' in migration comments anymore * Fix SerDes test that created duplicate dimensions for one Field * Silly fix to fix MySQL 5.7
-
Bryan Maass authored
* add the library, and dev/start-malli! * Update deps.edn Co-authored-by:
metamben <103100869+metamben@users.noreply.github.com> * enable the use of regexes as malli schemas Co-authored-by:
metamben <103100869+metamben@users.noreply.github.com>
-
Cam Saul authored
-
- Nov 18, 2022
-
-
adam-james authored
* First pass at grouping axes This PR uses the results_metadata key to look at fingerprints for numerical axes to try determine if each series on the Y axis can be sanely represented on the same axis. This is done by calculating an overlap (some value between 0 and 1) and grouping all axes on the LEFT if they pass the threshold, which is some value between 0 and 1. The overlap is always calculated when there is SOME overlap between the ranges of each axis, and is calculated as: ```clojure (/ (- (max maximums) (min minimums)) (- (min maxiumums) (max minimums))) ``` This is done to try catch situations where one column's range is entirely inside the other, but is much smaller (- max min); such a case would have a small percent overlap by the above calculation, and implies that it might be better to split the axes. * Address feedback. * Fix shape of data in tests of 2 private fns * Add test for split axes * Fixed error in test util * render-utils does with-redef, disallows parallel tests. Makes sense
-
- Oct 21, 2022
-
-
adam-james authored
* Column settings correct keys. Show Axes labels logic defaults 'true' These are two small changes related to static viz column formatting. The show axes labels logic change fixes a faulty assumption from an earlier PR where it was assumed that if :labels_enabled key was NOT present, then it was 'false'. The default however, is 'true', so the check had to change to reflect the frontend. Column settings have to be formatted so that the js code running in the backend looks in the right places. For example, `:number-format` must become `:number_format`. I move the function that does that change up so that it can be rolled up into `settings-from-column` to reduce the number of call sites for `update-col-for-js`. * Adjust number format fn to consider :decimal_digits of currencies * Simpler date-separator fix * Prevent non-temporal labels from being passed to datetime formatter * Table Temporal format columns are improved The column headers will now be properly looked up, the key format was incorrect before, causing the `:column_title` lookup to return nil, which ignores when a user changes the column name. Table columns with temporal formatting settings come in with `:unit` `:default`, so that case was added to the datetime formatter. Additionally, a helper function was added to pull the correct column viz settings to pass to the datetime. * Change some table rendering to look for '_' keys instead of '-' And also allow the field-id to take the form `[:field-id 10 nil]` OR `[:field-id 10]` * Added test utils for static-viz rendering. Docstrings for details. A big set of functions to enable cleaner static-viz rendering tests. Next push will be thorough table tests, and some other static viz cleanup * Add some viz-scenarios to help with table testing * Lint * Wrong map here * Currently unused bindings * Writing tests with render-test utils now, and catching new bugs * Use column's field ref to maybe get the column settings * Adjustments so that column-settings are properly grabbed * Number formatter no longer assumes decimal places The frontend will only display decimal places when strictly necessary OR when the column formatting dictates the number of decimal places to show. The number formatter on the backend made different assumptions about decimal places, so precision may be lost OR needlessly displayed. The formatter now checks the actual value passed in and adjusts the format string according to the decimal places present in the value, not just via column-settings, though the column settings will still take priority. * Shame on me. inline def is not needed here :/ * Adjust number format tests to assert that now we match FE behavior * Really trying to get formatting buttoned up. * Matching frontend's default format behavior for numbers * Some tests failed because they assumed two decimal places always * Values less than one still round after 2 sig-figs * Column formatting on tables follows expected default behavior * Getting public-settings included in number and datetime formatters * Test for custom formatting public settings * Removed unused require. * address some PR feedback * Try to clarify the logic in the number formatter * Add some clarifying comments * Use the existing viz tools to help create field-refs properly * Cleaned up column-name impl. to use normed field-refs. Much nicer! * Remove some unnecessary tests, and add a negative scientific num * Simplify table columns test * Simplify and add more format settings to column-formatting test * Unused require * Another one * Fix format of column refs in test * Remove unused thread
-
Braden Shepherdson authored
Previously, some automatic behavior was causing problems in serdes. (The worst example is generating an `entity_id` on insert while deserializing an entity we don't own - if deserialized again it would be duplicated.) A whole cascade of design choices fell out of this problem: - `insert!` would generate `entity_id`s, so use `simple-insert!` - But `simple-insert!` doesn't convert eg. MBQL `:definition` maps back into JSON strings - We got the maps as Clojure data and not JSON strings because `select` and `simple-select` both run `post-select` and parse the JSON. - So we ended up with a raw query on the select side and `simple-insert!` on the storage side. This change unwinds that whole stack, and instead uses a dynamic var to suppress the few pieces of `pre-insert` and `pre-update` logic that causes problems. The end result is much cleaner, and much more consistent with the rest of Metabase's backend logic.
-
- Sep 15, 2022
-
-
metamben authored
* Add helper function sending debug-qp events to portal * Include the intended value in the transformed-result event
-
- Sep 14, 2022
-
-
metamben authored
* Refactor debugger output
-
- Sep 13, 2022
-
-
metamben authored
-
- Sep 07, 2022
-
-
Bryan Maass authored
* adds ddiff2 as a dev dependency - used in process-query-debug * Update dev/src/dev/debug_qp.clj use 0 arg version of println Co-authored-by:
metamben <103100869+metamben@users.noreply.github.com> Co-authored-by:
metamben <103100869+metamben@users.noreply.github.com>
-
- Aug 05, 2022
-
-
dpsutton authored
* Fix in-memory logger Our Admin > Troubleshooting > Logs page broke, just showing a spinner and never showing the logs. Don't quite understand why this fixes it. Javadocs are https://logging.apache.org/log4j/2.x/log4j-api/apidocs/org/apache/logging/log4j/LogManager.html#getContext-boolean- ```clojure logger=> (log/warn "test") nil logger=> (count @messages*) 0 ;; no in-memory logs so page is empty ;; change `(LogManager/getContext true)` in the momoized ns-logger fn ;; and then observe: logger=> (log/warn "test") nil logger=> (count @messages*) 4 ``` Some explorations that might shine some light: ```clojure logger=> (into {} (.getAppenders (.getLogger (LogManager/getContext false) (str *ns*)))) {} logger=> (into {} (.getAppenders (.getLogger (LogManager/getContext true) (str *ns*)))) {"metabase-appender" #object[metabase.logger.proxy$org.apache.logging.log4j.core.appender.AbstractAppender$ff19274a "0x4d680247" "metabase-appender"]} ``` So something is not hooked up quite right. * Add tests for metabase.logger * Ensure `api/util/logs` returns logs * tests * Check for presence of `MetabaseLoggerFactory` rather than whole str When in a namespace with a record, `Foo resolves to ns.Foo. But outside it resolves to ns/Foo. When running tests from the command line *ns* is user so it gets more complicated. * Kinda playing whackamole™ with `(LogManager/getContext true)` * Remove custom memoizing logger History: 39.2 we set `Multi-Release: true` in our manifest file and query speed drops like a stone. Jeff was able to track this to our logger calls in tight loops. We revert the multi-release and keep seeing the warning on startup > WARNING: sun.reflect.Reflection.getCallerClass is not supported. This will impact performance. Benchmarking on 39.2 (bench (log/trace "hi")) -> 15383 ns So we freaked out and set up a memoizing logger factory (bench (log/trace "hi")) -> 141 ns What a win. But we never noticed that the default *logger-factory* being picked up was slf4j ( `(log.impl/name log/*logger-factory*)` -> "org.slf4j" ). On 39.2 if you set the factory to the log4j2 version you get back to a great number: `(bench (log/trace "hi"))` -> 25 ns And thus ensuring that our logger is the default log4j2 version is even faster than our memoizing logger-factory. Memoizing factory: 141 ns slf4j factory: 2269 ns log4j2 factory: 31 ns What does `(LogManager/getContext false)` mean versus using `true`? We only need and want a single context. But log4j2 by default uses a context selector called `ClassLoaderContextSelector`. We could put all of this behind us if we used a context selector type `BasicContextSelector` but this is surprisingly hard to do: you have to set a system property. And since all of this stuff gets initialized in static initializers, we don't have an opportunity to do this programmatically. The way around this is to require people to pass this system property on startup which is not acceptable. So getContext true checks for a threadlocal context in a specific static variable and falls back to a Default context. getContext false looks at classloaders and ends up at a different context. BUT: the log.impl version uses a closure over getContext false instead of getting it each time. And I suspect that when it does this there is only one so it is the default and continues to use this one. In our LoggerFactory implementation we were looking up the context each time. This still seems to work and everything is playing nice in our application classloader but its totally possible that our drivers are not hitting this. I'll have to investigate this. Verification: - build the uberjar locally (`bin/build`) - copy to some temp directory and also copy criterium.jar ```shell MB_JETTY_PORT=4000 java "$(socket-repl 4001)" -cp locally-built.jar:criterium.jar metabase.core ``` ```clojure /tmp/locally-built via
v17.30 on metabase-query ❯ nc localhost 4001 user=> (doto 'metabase.logger require in-ns) metabase.logger metabase.logger=> (require '[criterium.core :refer [bench]]) nil metabase.logger=> (bench (log/trace "hi")) Evaluation count : 1686535500 in 60 samples of 28108925 calls. Execution time mean : 22.487972 ns Execution time std-deviation : 0.101004 ns Execution time lower quantile : 22.326806 ns ( 2.5%) Execution time upper quantile : 22.648368 ns (97.5%) Overhead used : 6.924761 ns nil metabase.logger=> (count (messages)) 358 metabase.logger=> ``` Verifies that we are on the order of 22 nanoseconds and the in-memory logger has messages in it. * Appease our namespace linters * I'll unplug you ns linter * Better tests and ns docstring * Bootstrap to set system properties New entrypoint for the application: metabase.bootstrap sets two properties for logging (context selector, log4j2 factory) and ensures those properties are set before any logging code is loaded. * docstrings and clean ns * metabase.logger ns docstring cleanup * docstring * rename a test now that there's no memoization * add logger properties to :dev profile * Revert "add logger properties to :dev profile" This reverts commit 4f09fa3b631f882a3c5edcab4508769ffb20d4fa. * deps
-
- May 26, 2022
-
-
Noah Moss authored
-
- May 13, 2022
-
-
dpsutton authored
* Record startup times updated `dev/start!`. `mb.core/init!` itself sets up the db, loads plugins, and sets the init status so those were superfluous. * clean up dev namespace * testing doesn't call init so set a value
-
- Apr 19, 2022
-
-
Braden Shepherdson authored
* Make namespace aliasing consistent everywhere; enforce with clj-kondo See the table of aliases in .clj-kondo/config.edn Notable patterns: - `[metabase.api.foo :as api.foo]` - `[metabase.models.foo :as foo]` - `[metabase.query-processor.foo :as qp.foo]` - `[metabase.server.middleware.foo :as mw.foo]` - `[metabase.util.foo :as u.foo]` - `[clj-http.client :as http]` and `[metabase.http-client :as client]` Fixes #19930.
-
- Feb 15, 2022
-
-
Cam Saul authored
-