This project is mirrored from https://github.com/metabase/metabase.
Pull mirroring updated .
- Oct 30, 2024
-
-
Noah Moss authored
-
Case Nelson authored
* perf: Implement faster sync methods for postgres Fixes #48575 Pulls work from redshift into the common postgres driver. * Fix tests and formatting * Move nested-field-column sync to sync functions so describe-fields will also get them * Fix test * Fix test * Remove fixed safety test * Add test specific database-supports feature for pk metadata * Fix test * Adrress PR feedback * Fix test * Don't use subselect for field-comment * Fix quoting weird identifiers * Make format string inline
-
Oleksandr Yakushev authored
-
Noah Moss authored
-
- Oct 29, 2024
-
-
adam-james authored
* Mongo objects should download as JSON, not EDN Fixes #48198 Prior to this change, object columns (base or effective type of :type/Dictionary) were just formatted with `(str value)` which results in a csv or json download containing EDN formatted objects. This is a bug because we present object column values as json in the app, so the expected formatting of the download should match this. The formatter function now takes this type into account. As well, since this is a type of formatting that should be always applied (even when format_rows is false), the function is modified to unconditionally apply the json/encode formatting to dictionary types when encountered. * add a test * add proper condition to test * card-download should be public * uncomment json encoding formatter * set-cell! should keep encoded json string for Objects I think this is the correct change; I don't really understand the reason for wrapping, encoding, decoding, and then string-ing that value. Maybe I'm missing something. * Adjusted xlsx Object set-cell! implementation * forgot the not... inverted
-
Noah Moss authored
-
- Oct 25, 2024
-
-
Noah Moss authored
Co-authored-by:
bryan <bryan.maass@gmail.com>
-
dpsutton authored
* Fix csp directives for embed previews We set content security directives to allow for iframes on dashboards. This list did not include 'self' so we can't actually host an iframe pointing at our, well, self. Embed previews work by just embedding an iframe with the dashboard and this breaks if we don't allow iframes from our self. * e2e test --------- Co-authored-by:
Aleksandr Lesnenko <alxnddr@gmail.com>
-
Chris Truter authored
This standards on a "just maps" approach to returning disabled, disallowed, or failed query analysis. This is as opposed to the hodgepodge approach of significant `nil`, untagged unions with keywords, and exceptions. Essentially we have a classic "either" style tagged union, where `:error` is the tag.
-
- Oct 24, 2024
-
-
lbrdnk authored
* Format databricks as spark in prettify-native-form * Add test to be generalized in follow up
-
Chris Truter authored
-
- Oct 23, 2024
-
-
Noah Moss authored
Co-authored-by:
Ryan Laurie <iethree@gmail.com>
-
Aleksandr Lesnenko authored
* add allowed iframe host setting wip * use allowed-iframe-hosts setting in the CSP header * add a test for the frame-src csp directive * Update allowed-iframe-hosts setting definition * Add error state for forbidden iframe url domains * Move out iframe e2e test suite * Add e2e test * Update error message in view mode * Fix unit tests * Update setting on the admin page * Update error state * Add links to error states * Update docs links * Update link anchors * Add default allowed hosts to public setting * Update allowed domain check logic * Fix default value display in admin page * Don't update setting without changes * Update error message spacing * correct the parsing of allowed-hosts string for CSP header entries * fix test * fix not handling wildcard ports * Fix failing e2e test * Fix subdomain test * address review - the parse-allowed-iframe-hosts fn is now memoized - a * entry is handled and doesn't produce a weird *:* entry - no more try/catch, errors in parsing will be logged but the list returns all valid entries - when www. is encountered, an entry including www. is added - trailing / is 'cleaned' and the entry is used as if there was no trailing / * Fixup test for expecting a few more frame sources * indentation fix for linter
🥲 * Fix type error --------- Co-authored-by:Adam James <adam.vermeer2@gmail.com> Co-authored-by:
Anton Kulyk <kuliks.anton@gmail.com> Co-authored-by:
dan sutton <dan@dpsutton.com>
-
John Swanson authored
* Do not create personal collections for API keys Co-authored-by:
Noah Moss <32746338+noahmoss@users.noreply.github.com>
-
appleby authored
This is to allow for the possibiliy of adding more metricsv2 usage and error metrics in the future while still distinguishing the source of the errors.
-
Chris Truter authored
### Description This is a low level step towards applying table-level permissions to native queries. It replaces the more generic and powerful `:ast-walker-1` analyzer with the more conservative `:basic-select` version we recently landed. The advantage of this version is that it protects against dynamic table references and false negatives due to masking.
-
Chris Truter authored
-
Ngoc Khuat authored
-
- Oct 22, 2024
-
-
Ngoc Khuat authored
-
Oisin Coveney authored
-
- Oct 21, 2024
-
-
adam-james authored
* Fix a Couple bugs related to pivot exports - pivot-grouping value, when not an int (in the case of Oracle dbs), would cause exports to contain zero rows. This fixes that - json formatted/unformatted for the dataset API endpoints (unsaved questions) was not being used, so all exports were formatted * json and csv use (int group) to check row inclusion * grouping change fix tests * json formatted/unformatted test * xlsx formatters are correct when pivot-grouping col is removed * cljfmt * fix up failing tests * fix test * Update src/metabase/api/dataset.clj
-
- Oct 18, 2024
-
-
Cam Saul authored
* Modularize metabase.pulse; add metabase.pulse.core API namespace * Rename metabase.pulse.send-test * Split email result attachment stuff into its own namespace * Update dashboard-subscription-test * Allow metabase.email.result-attachment externally... for now * More code cleanup * Decouple models.card and email.messages * Decouple models.collection and metabase.email.messages * Hopefully the last fix
🔧 * Fix #48647🙁 * Fix event handler * :render/text, not :pulse/text * Test fixes🔧 * Test fix🔧 * RESPECT the modules * Remove unused namespace -
bryan authored
* Adds query_executions_by_source_24h and entity_id_translations_last_24h - add docstring * clear eid translation count in stats-post-cleanup * remove ->> with after? it's confusing * dissoc the right path
-
lbrdnk authored
* Use type date for case expression when there are Date and DateTime args * Use logic for getting case expr type from annotate middleware * Add e2e test * Add bigquery test * Update test * cljfmt and comment * Adjust e2e test
-
- Oct 17, 2024
-
-
lbrdnk authored
* Fix expression stage * Add test * Parallel test * Adjust test * Adjust test
-
lbrdnk authored
* Use metric's aggregation :name * Add test
-
Case Nelson authored
* fix: Always cast json number types as decimal Fixes #48507 * Fix tests * Fix tests * Fix test
-
metamben authored
Fixes #48639
-
Alexander Solovyov authored
-
- Oct 16, 2024
-
-
dpsutton authored
* Let non-admins see tables in recents ```shell ❯ http get "localhost:3000/api/activity/recents?context=views" Cookie:"metabase.SESSION=ba..3d" -pb { "recents": [ { "can_write": false, "database": { "id": 6, "initial_sync_status": "complete", "name": "pg restaurants" }, "description": null, "display_name": "Restaurants", "id": 112, "model": "table", "name": "restaurants", "table_schema": "public", "timestamp": "2024-10-15T22:11:39.412100Z" }, { "can_write": false, "database": { "id": 6, "initial_sync_status": "complete", "name": "pg restaurants" }, "description": null, "display_name": "Reviews", "id": 110, "model": "table", "name": "reviews", "table_schema": "public", "timestamp": "2024-10-15T19:53:19.999445Z" } ] } ``` We were getting permissions of a "fake" table. Instead, let the db select it and get the proper stuff. Worked for admins because mi/can-read? is presumably always true for tables, and worked in tests because we mocked mi/can-read? * bump ci
-
appleby authored
* Add new promethues counter for :metabase-metrics/adjust-errors To record errors in metabase.query-processor.middleware.metrics/adjust * Add prometheus metrics test to metrics middleware tests * Also inc :metabase-metrics/adjust-errors when throwing an exception * Make splice-compatible-metrics private It doesn't seem to be used outside of metabase.query-processor.middleware.metrics. * Add test for :metabase-metrics/adjust-errors when exception is thrown * Move inc-and-throw! helper into metabase.analytics.prometheus * Add inc-and-throw! test to prometheus tests * Rename inc-test to inc!-test and use approx= and metric-value helpers * Add try/catch to metrics adjust and record metric errors in a single place Record promethues metrics in a single place in metabase.query-processor.middleware.metrics and ensure we record failures for any exception throw. * Improve testing description string * Rename :metabase-metrics/adjust-errors to :metabase-query-processor/metrics-errors * Move metrics adjustment try/catch into a new helper This ensures we count all exceptions generated by adjust-metric-stages. * Add new prometheus metric to count number of MB metrics seen by the QP This new metric is intended to count the number of metrics that the query processor metrics middleware sees and attempts to adjust. * Add more metrics metrics tests - Ensure that queries with no metrics do not incremement either counter - Ensure that successful calls to adjust do not increment the error counter - Ensure that exceptions throw from other libs also increment the error counter * Rename test to adjust-prometheus-metrics-test and add helper to reduce code dup * If no metrics are found immediately return query in metrics middleware adjust Don't attempt to adjust query if we don't find any :metric clauses. * Appease cljfmt * Improve metric description Co-authored-by:
metamben <103100869+metamben@users.noreply.github.com> * Improve epsilon test in approx= Co-authored-by:
metamben <103100869+metamben@users.noreply.github.com> * Remove adjust-metric-stages-counting-errors This function was an attempt to avoid double-counting errors, but probably not worth the effort. See https://github.com/metabase/metabase/pull/48598#discussion_r1800197964 https://github.com/metabase/metabase/pull/48598#discussion_r1800200421 * Count metrics directly when mocking prometheus/inc! Based on PR feedback here: https://github.com/metabase/metabase/pull/48598#discussion_r1800207674 * Remove prometheus/inc-and-throw! Value prop was always questionable, but now that it's only called from a single place, just remove it and inline the call to prometheus/inc!. Easy to add back later if it proves generally useful. PR discussion thread https://github.com/metabase/metabase/pull/48598#discussion_r1799728596 * Rename match-one-metric to find-first-metric https://github.com/metabase/metabase/pull/48598#discussion_r1803321025 * Rename :metabase-query-processor/metrics-errors to metric-errors https://github.com/metabase/metabase/pull/48598#discussion_r1803313818 * Throw an exception rather than logging a warning if we fail to adjust a metric https://github.com/metabase/metabase/pull/48598#discussion_r1803328482 --------- Co-authored-by:
metamben <103100869+metamben@users.noreply.github.com>
-
bryan authored
* add prometheus tracking for sdk-embedding * indentation * improve track-sdk-response api * register :metabase-sdk/response-{ok,error} + test * avoid cyclic dependency whe nusing prometheus fixes: [ /metabase/api/common ] <- /metabase/models/setting <- /metabase/analytics/settings <- /metabase/analytics/prometheus <- /metabase/analytics/sdk <- /metabase/models/view_log <- /metabase/events/schema <- /metabase/events <- [ /metabase/api/common ] <- /metabase/public_settings/premium_features <- /metabase/auth_provider <- /metabase/driver <- /dev/debug_qp <- /dev * nix dependency: events.schema --> models.view-log - make them both read context from view-log-impl instead * cljfmt
-
Chris Truter authored
-
Noah Moss authored
-
Alexander Solovyov authored
-
Ngoc Khuat authored
-
- Oct 15, 2024
-
-
bryan authored
* adds data + schema for metrics stats ping * remove comment * annotate todos * fill in the rest of the metrics values - and add defaults * fix some definitions + use a single timestamp * shuffle stuff around to appease the namespace linter - we aren't reaching into the api API, so the linter is more of a formality here. * update docstring * fix jsonschema, maybe * review responses - revert changes to 1-0-0 - add metrics section to new file: 1-0-1 - bump ::instance_stats to "1-0-1" - add tags into 1-0-1 - make the code return tags (empty for now until we know what to tag things.) - also fix test that broke from shuffling settings around * remove a footgun * add tags to metrics, add grouped_metrics to jsonschema 1-0-1 format 1-0-1 * indent * cljfmt * version should match filename * update instance stats 1-0-1 schema * require `tags` in grouped_metric + snowcat comment * fix formatting noise * update schema to make it valid * remove grouped_metrics from instance_stats schema * cljfmt appeasement * unbin cache_num_queries_cached value - alphabetize metrics generation * we can now guarantee metric values will be ints * jsonschema for instance_uuid, settings, and grouped_metrics * add analytics_uuid and make it required * lint + alphabetize instance stats json schema * update setting key type - add maxLength to some strings * lint jsonschema - Add description to setting.items.tags - add maxLength, and {min,max}imum to setting.items.value * Bump instance stats to 2-0-0 - remove analytics_uuid string length == 36 check - adds assertion to ensure required fields are set (and it passes) - adds info for embedding settings * adds a grouped-metric to stats ping - adds length info to the schema to pass jsonschema linting * cljfmt
-
adam-james authored
* Incremental Pivot Processing for Exports WIP Fixes pivot exports for CSV and xlsx. The CSV export should use less memory by incrementally building up the data structure and aggregating necessary row data right away, so the memory overhead becomes only as large as the total pivot result. In cases where the pivot rows/cols do combine into many many columns and rows, this can still be a large set of data, but it should behave much better now in most cases. The Excel export is a little more straightforward: create the export rows in the same fashion, streaming one row at a time, and just post-process the sheet to add the pivot table in one shot at the end. * WIP adding row totals. * aggregate totals as rows are added Row, column, section, and grand totals are all aggregated as each row is added. This means the final step of building pivot output becomes just an exercise of lookups/arrangement, no further aggregation is needed. * CSV pivot works per-row, export respects formatting This is a big step forward; we don't need to hold the entire dataset in memory, we instead aggregate a row's data into the pivot datastructure, which only holds onto: - unique values for each pivot-row in a sorted set - unique values for each pivot-col in a sorted set - grand total for each measure N values, where N is number of measures, ususally 1 or 2 - row totals for each combination of each pivot-row * N measures - col totals for each combination of each pivot-col * N measures - totals for each 'section', determined by unique values of first pivot-row * N measures - values for each measure in every 'cell'; Row Combos * Col Combos * N Measures So, there can still be a decent amount of data to store; but it will never hold onto all of the 'raw rows' from the dataset. We can never completely guarantee that Row Combos * Col Combos * N Measures remains small, but two things let us move forward anyway: - there's now visible feedback in the app that the download is running (or if it's failed) - Pivot table utility diminishes rapidly with huge output anyway; users still need to curate/set up their data - effectively to improve the table's utility, so we can assume that a slow-to-download pivot table is also slow to - use/less effective, and will likely be something the user doesn't want (as often). * some test fixes * now, if we export 'raw pivot rows', they don't show pivot-grouping and they also don't include the 'extra' rows for totals/subtotals/grand totals (any row with pivot-grouping > 0). This means that now the non-pivot version of a pivot table export will match what a user sees if they change the viz to a regular table. * remove old test * re-incorporate some changes from master * fix csv for non-pivots due to oversight in my changes This is just a temporary change, I think I should clean up this bit of the code a little, I can probably make it a little more readable and use some cleaner logic regarding if the rows are 'raw pivot rows' or not. * start moving format_rows to POST bod, add pivot_results too There's still wiring work to do, but this starts to add format_rows and pivot_results to POST body for the various API endpoints. Also modify tests to improve coverage/consistency across downloads and alerts/subscriptions. The tests will not pass on this commit, but fixes will be incoming * native pivot tables in xlsx * add precondition to pass migration linter * try to get migrations fixed * pasing pivot-results through api and attachments * fix tests for format_rows in BODY vs query param * tests! * might have the tests all fixed now * the pivoted export now respects col/row totals settings * add test coverage for public questions and dashboards * col and row totals work as expected * build-pivot refactor for clarity * docstring change + tiny refactor in helper fn * see if dashcard download works with format_rows * csv pivot handles nil values * pass format_rows and pivot_results in :params not :body * fix some other tests * pivot-grouping col filtered out of xlsx * pivot-grouping-col removed for all rows * configurable pivot exports and attachments (#47880) * exports fe * specs * ui * specs * format/unformatted now works for xlsx * format test changes for xlsx formatting * embedding endpoints accept pivot_results * cljfmt and eslint fix * empty * embedding test should have formatting defaulted to true * embed test fixes * Use `Chip` for export settings widget * downloads e2e test fix * fix public download limit test * public card download defaults * fix public download defaults in some tests * Fix visual test --------- Co-authored-by:
Aleksandr Lesnenko <alxnddr@users.noreply.github.com> Co-authored-by:
Noah Moss <32746338+noahmoss@users.noreply.github.com> Co-authored-by:
Anton Kulyk <kuliks.anton@gmail.com>
-
Aleksandr Lesnenko authored
* fix iframe dashcards crash subscriptions * add a test to ensure iframes are filtered out of subscriptions --------- Co-authored-by:
Adam James <adam.vermeer2@gmail.com> Co-authored-by:
adam-james <21064735+adam-james-v@users.noreply.github.com>
-
Ngoc Khuat authored
-