This project is mirrored from https://github.com/metabase/metabase.
Pull mirroring updated .
- Mar 01, 2023
-
-
Cam Saul authored
* Metabase lib PoC RFC [ci skip] * Cleanup [ci skip] * `:lib/uuid` overhaul [ci skip] * More tests [ci skip] * Basic join support [ci skip] * Simplify a few things, more tests, etc. [ci skip] * Pipeline queries FTW [ci skip] * Metabase® Lib™ v2 * Fix cljs test assertions * No bees please * Update docstring * Mega simplification * Fix Kondo warnings * Address PR feedback * Fix Malli util for Cljc * Oops, `instrument!` should be public * Cloverage do not instrument JS util namespace * Fix Cloverage again * Don't count MySQL or Postgres namespaces for Coverage * Address PR feedback * Move dispatch value keyword function to util * Revert acccidental require * Test fix * Fix Cljs =? * Fix totally unrelated broken links
-
Ngoc Khuat authored
* expose number of active users as a setting
-
Ngoc Khuat authored
* Fix static-viz render with black background when values > 1000 * fix to work with any separator settings
-
- Feb 28, 2023
-
-
Bryan Maass authored
* Adds tests for chain filters security issue * add another test * Users with collection permissions can view chained filters for dashboard - this obeys sandboxing * Users with collection access should be allowed to view chain filters * add test + fix case for block filters * check with premium features * check premium features + revert with-premium-features in tests * remove block check * respond to review comments * ensure data-perms are there + check blocked * rewrite e2e test so that it actually tests chain filters * properly toggle linked filters on - don't use random classes, etc * rename *param-values-query* dynamic var * update chain-filter docstring * token-features should still be private
-
Noah Moss authored
* initial implementation of query_execution retention policy * fix logic and add tests for setting * refactor and more tests * set default on cloud to 0 * try to fix test by loading ee ns if available * address feedback and small refactors * interface file for truncate-audit-log * extract common definitions into an interface to avoid cyclic dependency * fix indentation
-
dpsutton authored
* Ensure query action's db matches model id This is only a half-way measure. Needs some FE loving as well. The +New -> Action workflow lets you create an action. This starts with the user choosing the database, writing the action. On save it prompts you to choose a model it should live under. But this model's db and the db chosen originally are not checked in the FE. Ideally we only show models on that existing db, but it kinda seems weird to have an action do things to a random table and it live under a model that just needs to be on the db. * reword error message Co-authored-by:
Maz Ameli <maz@metabase.com> * Check both databases for enabled actions when they disagree a custom action can have its own query which brings its own database. When creating a new custom action from the models detail page, these go hand in hand. When using the workflow from the "+New" button, the model is chosen at the last step, well after the action's db and query is set. And there's nothing holding these two together. Asked if this was a bug and was told no. But we want to make sure we check both for actions enabled. * fix lint --------- Co-authored-by:
-
dpsutton authored
* Delay substituting persisted native query until after analysis We were substituting persisted queries into the query at too early a stage. Card 729 is a persisted model of `select * from categories` ```clojure {:type :query, :query {:source-table "card__729", :expressions {:adjective ["case" [[[">" ["field" 100 nil] 30] "expensive"] [[">" ["field" 100 nil] 20] "not too bad"]] {:default "its ok"}]}, :aggregation [["count"]], :breakout [["expression" "adjective" nil] ["field" 101 nil]]}, :database 3, :parameters []} ``` This would immediately get turned into an equivalent ```clojure {:type :query, :query {:source-table 12 :persisted-info/native "select \"id\", \"ean\", \"title\", \"category\", \"vendor\", \"price\", \"rating\", \"created_at\" from \"metabase_cache_424a9_3\".\"model_729_products_r\"" :expressions {:adjective ["case" [[[">" ["field" 100 nil] 30] "expensive"] [[">" ["field" 100 nil] 20] "not too bad"]] {:default "its ok"}]}, :aggregation [["count"]], :breakout [["expression" "adjective" nil] ["field" 101 nil]]}, :database 3, :parameters []} ``` And the obvious failure here is the use of `[:field 100 nil]` (price) against an inner query of `select ... price ... from cache_table`. This could break many queries and caused a flood of ``` WARN middleware.fix-bad-references :: Bad :field clause [:field 100 #:metabase.query-processor.util.add-alias-info{:source-table :metabase.query-processor.util.add-alias-info/source, :source-alias "price"}] for field "products.price" at [:expressions "adjective" :case :>]: clause should have a :join-alias. Unable to infer an appropriate join. Query may not work as expected. WARN middleware.fix-bad-references :: Bad :field clause [:field 100 #:metabase.query-processor.util.add-alias-info{:source-table :metabase.query-processor.util.add-alias-info/source, :source-alias "price"}] for field "products.price" at [:expressions "adjective" :case :>]: clause should have a :join-alias. Unable to infer an appropriate join. Query may not work as expected. ``` * forgot to commit new test file * Handle persisted before native * Add test for native models - have to populate the metadata. You can do this by running the query, but there are some requirements around the `:info` object to actually save it. So just take the more mechanical route - `wait-for-result` returns a time out value, but it's namespaced autoresolved to a namespace inside of the metabase.test forest and not mt itself. So easy to mess up and watch for the wrong keyword. Want an arity so i can check my own timeout value and throw a helpful error message. I checked around and noone seems to care. So not great, but at least people aren't checking `::mt/timed-out` when it should be `::tu.async/timed-out` or `:metabase.test.util.async/timed-out`. Possibly best scenario is we remove the autoresolved bit and have it return `:async/timed-out` or something stable. * properly compile the native query * bump timeout to set redshift metadata -
john-metabase authored
Cleans/fixes serdes v2 export :collections handling in cmd and API
-
john-metabase authored
-
Ngoc Khuat authored
-
- Feb 27, 2023
-
-
Braden Shepherdson authored
Dev experience: - CLJS now has source maps - Devtools extension renders CLJS data in Chrome (et al) Devtools - Use a debug build of CLJS in the tests - Fixed line numbers and breakpoints being off-by-one in Devtools CLJC infra: - More robust direct approach to JS<->CLJS conversions, powered by Malli transformers. - Powerful `define-getters-and-setters` macro to generate the accessors.
-
Mark Bastian authored
Added post-collection retrieval function `remove-other-users-personal-collections` that will filter out any personal collection that doesn't belong to the current user when the `exclude-other-user-collections=true` path param is used. Also added unit tests for both the `api/collection/` and `api/collection/tree` endpoint.
-
Cal Herries authored
* Add comment for database-enable-actions * Enforce actions not supported implies actions are disabled * Remove test and update check * Fix typos * Revert "Remove test and update check" This reverts commit 8b1f14f44bd97514cd8ce0c9b0a5df048d713b16. * Fix test
-
- Feb 24, 2023
-
-
dpsutton authored
* Remove json fields from action parameters ```sql json-test=# select * from json_table; json_val | description ------------------+--------------- {"a": 1, "b": 2} | a description (1 row) ``` This ends up in our db as: ```clojure ({:database_type "json", :name "json_val", :effective_type :type/Structured, :nfc_path nil, :id 73, :display_name "Json Val", :database_required true} {:database_type "bigint", :name "json_val → a", :effective_type :type/Integer, :nfc_path ("json_val" "a"), :id 72, :display_name "Json Val → A", :database_required false} {:database_type "bigint", :name "json_val → b", :effective_type :type/Integer, :nfc_path ("json_val" "b"), :id 74, :display_name "Json Val → B", :database_required false} {:database_type "text", :name "description", :effective_type :type/Text, :nfc_path nil, :id 13670, :display_name "Description", :database_required false}) ``` AKA, exploded. But we cannot enter values for these fields for implicit actions. Before this change, the action thought its parameters were: ```clojure :parameters ({:id "json_val", :target [:variable [:template-tag "json_val"]], :type :type/Structured, :required true} {:id "json_val_%E2%86%92_a", :target [:variable [:template-tag "json_val_%E2%86%92_a"]], :type :type/Integer, :required false} {:id "json_val_%E2%86%92_b", :target [:variable [:template-tag "json_val_%E2%86%92_b"]], :type :type/Integer, :required false} {:id "description", :target [:variable [:template-tag "description"]], :type :type/Text, :required false}) ``` And we just don't know how to handle the exploded json values in the insert statement to create a nested json structure. So lets omit them. After this change: ```clojure action=> (:parameters (select-action :id 14)) ({:id "json_val", :target [:variable [:template-tag "json_val"]], :type :type/Structured, :required true} {:id "description", :target [:variable [:template-tag "description"]], :type :type/Text, :required false}) ``` * Translate `:type/Structured` to JSON for pg Fields have effective_type, and base_type `:type/Structured`. But several other types get marked as this as well: ```clojure actions=> (descendants :type/Structured) -> #{:type/SerializedJSON :type/XML :type/JSON} ``` So for the purposes of this, we _assume_ that structured is synonymous with json. Certainly will error if its xml. This makes the implicit action cast the text input for a json field to json. If the underlying field in pg is jsonb the driver cleans that up for us with no error. * Remove json fields, exploded json fields, unrecognized type fields In order to do this, had to change how we load json fields in test datasets for postgres. It just stuffed them in as text. Mysql would correctly make them json fields but postgres just bailed and left them as text strings. This changes `load-data/load-data!` to look for json columns, and if so, cast them to json when loading. * Remove json and xml checks from dashboard execution tests these are not doing what you think they are doing. they inspire confidence that json works but the underlying fields are actually always text. So it's unfounded confidence; misleading confidence. If you fix this, you quickly run into issues with expressivity in parameters and differences in dbs. A few: - no parameter on a dashboard can express that its type is json - mysql doesn't have an xml type - postgres does have an xml type, and you get back a native PGXML type - since you can't have an implicit type in the parameter, each sql has to cast in its query. And then you start getting differences in return types, queries, and it stops making sense for this all to be in the same query. And at the end of the day, of course you can write a query that casts a string to some native type. That's not really action related. * Let's not do this yet i think this is the correct call but want more tests on it when we flip the switch * Test pg enums in implicit actions * alignment and unused binding * Exclude in maria maria doesn't see the json_bit field as a json column. It's just a text column. So it is not excluded and all of the exploded fields like "json_bit → 1234" are not extracted. So this test is not applicable because it's just a generic implicit action on three fields. * accept suggestions from tamas -
metamben authored
-
Bryan Maass authored
* fixes nil schema being passed into ->malli-io-link - happens when function args aren't annotated. * (u/pprint-to-str nil) should be "nil\n", not nil
-
Cam Saul authored
* Mark `hx/` and `h1x/` namespaces and their vars as deprecated * Fix snowflake failures * Remove unneeded honey-sql-version bindings
-
Cal Herries authored
* Decouple checking ddl from classifying h2 stmts - should enable followup for easily blocking more kinds of queries - check all statements to make sure they aren't "ddl". * fix classify-query * linter fixes + get-field refactor * return the CommandInterface values as ints Reach into the CommandList when needed * docstring wording * catch invalid queries -- they can't be classified * Remove action subtypes from inlined-models * Add ddl check for `execute-write-query!` * Remove ACCESS_MODE_DATA * Check queries are single select statements * Add test for sample database privileges * Fix single-select check * Add single-select test * Rename and add more tests for checking read only commands * commands -> statements * Fix check-disallow-ddl-commands * new line * Add more read-only statements to the tests * Update error text * Use are * Add integration test for executing actions with disallowed commands * Add test before inserting row * Run GRANT ALL ON SCHEMA "PUBLIC" TO GUEST * Restore classify-query * whitespace * Whitelist command types for actions * Add comment * Rename * Remove dupes * Add truncate test * Add DDL commands * Rename to query-classification * Update error message * Fix test --------- Co-authored-by:Bryan Maass <bryan.maass@gmail.com>
-
- Feb 23, 2023
-
-
Case Nelson authored
Fixes #22872 In #21839, all order-bys were removed from pivot queries because columns in `ORDER BY` had to appear in `GROUP BY` clauses. Unfortunately that removed legal order by columns (e.g. count) from being applied. This now keeps aggregation columns in field defs to restore the sorting behaviour.
-
Cam Saul authored
* Snowflake Honey SQL 2 * Test fix
🔧 * Use Honey SQL 2 for GTAP compilation in sandboxing tests * Test fix🔧 -
Noah Moss authored
* new clojure migration for v1->v2 permissions * updates * fix rebase issue * fix reflection warning * fix final test and address comment about macro name * fix indentation * revert changes to defmigration macro to try to fix final test
-
Cal Herries authored
* Decouple checking ddl from classifying h2 stmts - should enable followup for easily blocking more kinds of queries - check all statements to make sure they aren't "ddl". * fix classify-query * linter fixes + get-field refactor * return the CommandInterface values as ints Reach into the CommandList when needed * docstring wording * catch invalid queries -- they can't be classified * Remove action subtypes from inlined-models * Add ddl check for `execute-write-query!` * Remove ACCESS_MODE_DATA * Check queries are single select statements * Add test for sample database privileges * Fix single-select check * Add single-select test * Rename and add more tests for checking read only commands * commands -> statements * Fix check-disallow-ddl-commands * new line * Add more read-only statements to the tests * Update error text * Use are * Add integration test for executing actions with disallowed commands * Add test before inserting row * Run GRANT ALL ON SCHEMA "PUBLIC" TO GUEST * Restore classify-query * whitespace --------- Co-authored-by: -
Ngoc Khuat authored
* add logics to skip generate subscription contents for link cards and actions
-
Tim Macdonald authored
* Add `model_id` to action search results so that URLs can be made [Fixes #28565] * Fix typo in Cypress filename
-
Ngoc Khuat authored
* set action.model_id to cascade on delete * add rollback
-
Case Nelson authored
* Strip semicolons from card refs in native queries Fixes #28218 While source-query substitution handled semicolons in native questions within outer mbql queries, we used a different code path when splicing into native queries. This brings the two paths inline. * Reference issue in testing * Add docstring
-
- Feb 22, 2023
-
-
metamben authored
* Resolve nested questions in native queries Fixes #25988.
-
Bryan Maass authored
* Decouple checking ddl from classifying h2 stmts - should enable followup for easily blocking more kinds of queries - check all statements to make sure they aren't "ddl". * fix classify-query * linter fixes + get-field refactor * return the CommandInterface values as ints Reach into the CommandList when needed * docstring wording * catch invalid queries -- they can't be classified * respond to review comments - fix see: comment - remove CommandList typehint - rename command-list -> command - skip checking disallowed ddl commands when query is `nil` then also: - fix a linter error - add a test for `nil` query ddl command checking * rename command-list-type -> first-command-type * fix schema + handle h2-parser build failure
-
Cal Herries authored
* Fix action loading and extraction of action type * Remove comment * Fix handling required parameters for public basic actions (#28415) * Fix presenting required parameters * Add tests * Add explanation text to the action editor (#28406) * Add explanation text to the action editor * Replace link * Add missing `nock.cleanAll` calls (#28441) * [CI] Improve `codenotify` workflow (#28432) * Add path filtering to codenotify workflow * Upgrade action to its latest version (v0.6.4) * [E2E] Prevent app permission flakes (#28445) * Ensure dashcards are deleted after actions are deleted (#28446) (c.f. discussion at https://metaboat.slack.com/archives/C04FG88HL95/p1676649795131089 ) * Show not found view when opening archived model (#28443) * Add `isArchived` method to `Question` class * Show nice not found view when opening archived model * [CI] Unify E2E tests workflows (#28431) * [Ci] Fix wildcard glob pattern in path filters (#28447) [ci skip] * Remove dupe tests * Add database_id to implicit actions * Tidy * Update select-actions docstring * Add test for database_id in action dashcards * Remove database_id before insert and update * whitespace * Refactor select-actions * Add database_id to action-create-test * Add test for create+update+select for implicit actions * Fix test * Remove conditional * style --------- Co-authored-by:
Anton Kulyk <kuliks.anton@gmail.com> Co-authored-by:
Nemanja Glumac <31325167+nemanjaglumac@users.noreply.github.com> Co-authored-by:
Tim Macdonald <tim@metabase.com>
-
Tim Macdonald authored
* If a model is converted to a card, archive the associated actions * If an action is unarchived and its model _is_ a card, throw an error
-
- Feb 21, 2023
-
-
Cam Saul authored
* MISC improvements * Update test/metabase/models/secret_test.clj Co-authored-by:
metamben <103100869+metamben@users.noreply.github.com> * Update .clj-kondo/hooks/clojure/test.clj Co-authored-by:
-
john-metabase authored
* Improves consistency of substring in MBQL Standard SQL SUBSTRING() uses a 1-based start index, with various drivers handling a 0 start index in different ways. To make the behavior of substring consistent across drivers, we will always convert a 0 start index in MBQL to a 1 in SQL. The ->honeysql implementation of substring has been generalized and drivers can now override just the name of the substring function, or the entire expression if necessary. SQL Server now calls the parent implementation. Resolves #12445 * Fixes substring from 0 in Mongo * Adds arg to substring parent call * Fix docs and helper text for substring() * Adds schema to require positive integer index for substring * WIP, adds frontend validation for substring index arg * Adds frontend test for substring arg validation * Updates substring test to expect exception on index 0 * Update src/metabase/driver/sql/query_processor.clj Co-authored-by:
-
Cam Saul authored
* Misc QP test improvements * Add test
-
Tim Macdonald authored
This'll be revisited as part of v47 [Fixes #28425]
-
- Feb 20, 2023
-
-
Case Nelson authored
* Fix linter * Make sure we're inlining num in substitution and sum aggregations * Set hsql version * Fixing tests * Fixing tests after the h2 conversion * Use h2x/extract in h2, fix test differences * Fixing tests for honeysql2 * Only numbers get inlined * More test fixes * Fixing tests * Undo bad commit * Inline Ratio must produce doubles or h2 will do integer division where 1/2 = 0 * Fix bad case statement * We can't run native queries with AS for oracle --------- Co-authored-by:
Cam Saul <github@camsaul.com> Co-authored-by:
Cam Saul <1455846+camsaul@users.noreply.github.com>
-
Cam Saul authored
-
metamben authored
* Make joins work * Run fuzzy-field-info-test only against H2 * Set driver/*driver* when performing actions * Restrict expressions+joins-test to DBs supporting date arithmetics * Fix ordering of (top level) fields in mongo tables * Explicitly exclude mongo from tests requiring foreign key support * Optimize away $sort stage for implicit order-by.
-
Ryan Laurie authored
* add dashcard testing mocks * improve virtual card overlays * update types * add search entity types * add metabase entity support to link cards * simplify link displays * update types and support collections and tables * use edit button to replace entity * test entity links * fix rebase * update types * address review comments * handle restricted link entities * Hydrate link cards info (#28297) * hydrate link cards info * we need to hydrate collections too * Address Cal's comments: - model use keywrod -> string - use model hydration - comments here and there * Link cards serdes (#28321) --------- Co-authored-by:Ngoc Khuat <qn.khuat@gmail.com>
-
Tim Macdonald authored
(c.f. discussion at https://metaboat.slack.com/archives/C04FG88HL95/p1676649795131089)
-
- Feb 17, 2023
-
-
john-metabase authored
* Removes :presto driver and tests * Merges :presto-common into :presto-jdbc * Adds migration to update presto databases to presto-jdbc --------- Co-authored-by:
-
