Skip to content
Snippets Groups Projects
This project is mirrored from https://github.com/metabase/metabase. Pull mirroring updated .
  1. Sep 13, 2024
  2. Sep 12, 2024
  3. Sep 11, 2024
  4. Sep 10, 2024
  5. Sep 09, 2024
    • github-automation-metabase's avatar
      [QP] Correctly handle `:lib/internal-remap` columns in `:result_metadata` (#47587) (#47788) · 5588e939
      github-automation-metabase authored
      
      Fixes #45938.
      
      Remapping is handled at the top level of the query, and should be
      ignored for inner queries. This PR hides it from `results_metadata` on
      source queries.
      
      It also fixes the interaction with `large-int-id` logic to make big
      integers JS-safe.
      
      Co-authored-by: default avatarBraden Shepherdson <braden@metabase.com>
      2 tags
      5588e939
    • github-automation-metabase's avatar
    • github-automation-metabase's avatar
    • github-automation-metabase's avatar
      :robot: backported "Entity id translation analytics" (#47640) · c882eef8
      github-automation-metabase authored
      
      * fixup: Static embed dashboard and question endpoints (#47270)
      
      * entity id translation + tests
      
      * add api level test
      
      * simplify definition of eid-table->model + add test
      
      * update tests to take keywords
      
      * improve comment
      
      * generate the eid-table->model map
      
      * delete now-obsolete test
      
      * make it work in oss
      
      * put the resulting response into a key, so we can add more information later
      
      * formatting
      
      * use model names without the model/ prefix as keys
      
      * Creates list of `api/model->db-model`
      
      - update keys for util/entity_id request
      - update shape of util/entity_id response
      - add test for not-found eids
      
      * formatting
      
      * Respond to code review feedback
      
      * entity-ids work on /embed/{card,dashboard}* routes
      
      * make entity id translation work for all tokens
      
      * Merge branch 'static-embed-dashboard-and-question-endpoints' of github.com:metabase/metabase into static-embed-dashboard-and-question-endpoints
      
      * massage schema. add tons of tests
      
      * decode more eids
      
      * test alignment
      
      * don't require ns inside of itself :think:
      
      * make the test functions to create tokens better
      
      * bring back `dashboard-url`
      
      ---------
      
      Co-authored-by: default avatarOisin Coveney <oisin@metabase.com>
      
      * fixup more tests
      
      * fix a test in api.embed-test
      
      * fix typo
      
      * fix dox typo
      
      * api.embed-test is passing
      
      * Add backport resolution script
      
      * perf: Miscellaneous card rendering improvements (#47679) (#47691)
      
      * perf: Optimize reading rows into vectors in JDBC driver
      
      * perf: Use direct interop when computing with-time-zone-same-instant
      
      Co-authored-by: default avatarOleksandr Yakushev <alex@bytopia.org>
      
      * Catch Errors in Download process and only write error message (#47482) (#47696)
      
      * walk the error to remove any visualization settings
      
      Thanks Dan for the patch, this approach is nice because it doesn't require changing all 3 streaming-response-writer
      implementations (json csv and xlsx). It keeps behaviour basically the same, but just cleans up the output to not
      include potentially sensitive column/dataset details.
      
      * add a test that confirms json/csv errors are cleaned
      
      * walk the error to remove any visualization settings
      
      Thanks Dan for the patch, this approach is nice because it doesn't require changing all 3 streaming-response-writer
      implementations (json csv and xlsx). It keeps behaviour basically the same, but just cleans up the output to not
      include potentially sensitive column/dataset details.
      
      * add a test that confirms json/csv errors are cleaned
      
      * fix dataset error test
      
      * cljfmt
      
      * try to be a little more surgical with what we take out of the error
      
      * formatting again, oops
      
      * pass export format, only modify the obj for file exports
      
      In the case of errors being saved to files, the obj is a map with a :status :failed key assoc'd by the qp somewhere
      along the way. The format-exception cond branch is not used in this case, so we can't rely on modifying
      `format-exception` to just dissoc ex-data.
      
      Instead, I've kept the obj modification, but it excludes the query and the preprocessed keys, which are the only 2
      keys where we see :viz-settings. This doesn't eliminate the problem of needing to exclude other keys in the future,
      but it does improve the surface area - the query and viz settings are the most likely candidates for holding
      potentially sensitive info.
      
      I'd prefer to stick with this solution, as we can get a fix for the security concern in quicker, and perhaps a
      non-security bug can be opened regarding the most correct way to indicate errors to users whose downloads have failed.
      
      * keep format-exception the same
      
      * cleaner passing of export-format to write-error!
      
      Co-authored-by: default avataradam-james <21064735+adam-james-v@users.noreply.github.com>
      
      * Revert "Hide upload config when data warehouse is attached" (#47611) (#47666)
      
      The way this was implemented, it also hid the section for management
      of uploaded CSV files.  Further, admins might want to intentionally
      switch from uploading to the data warehouse provided by Metabase to
      one of their own.
      
      This reverts commit fbaf58ad.
      
      References: https://github.com/metabase/harbormaster/issues/5121
      References: https://metaboat.slack.com/archives/C032LFJFANL/p1725296255637989
      
      
      Co-authored-by: default avatarDennis Schridde <63082+devurandom@users.noreply.github.com>
      
      * Hide attached DWH database details (#47419)
      
      == Goal ==
      
      Hide attached DWH database details from anyone incl. admins:
      * Do not show them in the UI
      * Do not permit to change them
      * Do not serialize them
      
      The aim is that customers cannot gain access to (parts of) credentials,
      and they cannot break a feature they are paying for by changing
      connection details.
      
      Database details are already generally excluded from H2 dump snapshots
      (see `metabase.cmd.copy/*copy-h2-database-details*`), thus nothing
      changes there.
      
      == Implementation ==
      
      The Metabase backend already contains provisions in the implementation
      of `metabase.models.interface/to-json` for `:model/Database` to hide the
      `details` of the database in HTTP responses, if the user lacks write
      permission on the database.  We utilize this by adding an
      `is_attached_dwh` column to the `database` table and rejecting
      `metabase.models.interface/can-write?` when this flag is enabled.  In
      the "admin" UI, we show a replacement text instead of the edit form when
      the flag is set.  (It might be correct to show this whenever `details`
      is absent.  See below for possible follow-up work.)
      
      However, several sections of the frontend code expected the `details`
      field to always be present.  In order to make `details` optional, as the
      backend seems to handle it, we fix the respective code to treat this
      case in the way that appears appropriate in the context.
      
      == How to test ==
      
      === New behaviour ===
      
      Setting the `is_attached_dwh` field hides the database details:
      
      1. Configure a database as described in https://www.metabase.com/docs/latest/configuring-metabase/config-file#databases.
         - In addition to the fields you would normally set, also set
           `is_attached_dwh: true`.
         - This also works when adding this flag to a database that previously
           did not have this flag set.
      2. Start your Metabase instance.
      3. Verify the database shows up in the "admin" section
         (`/admin/databases`).
      4. Verify that clicking the database to see its details only reveals
         "This database cannot be modified."
      5. Verify that responses from the backend do not include a `details`
         field for this database.
      
      === Original behaviour ===
      
      Behaviour without setting the `is_attached_dwh` field is unchanged:
      
      1. Configure a database as described in https://www.metabase.com/docs/latest/configuring-metabase/config-file#databases.
         - Only set the fields you would normally set.  Do not set
           `is_attached_dwh` (or set it to `false`).
      2. Start your Metabase instance.
      3. Verify the database shows up in the "admin" section
         (`/admin/databases`).
      4. Verify that clicking the database to see its details only reveal the
         regular edit form, showing connection fields like `host`, `user`,
         etc. with the values you configured.
      
      == How this will be rolled out ==
      
      1. Upgrade existing Metabase Cloud instances with data warehouse to a
         Metabase version that supports `is_attached_dwh`.
      2. Set `is_attached_dwh` in the database section of the config file for
         Metabase Cloud instances with a data warehouse.
      
      == Possible follow-up work ==
      
      In https://github.com/metabase/metabase/issues/25715, absent
      `database.details` was identified as a bug.  Since then, `details` was
      made `NOT NULL` in the application database, so this bug can no longer
      occur.  However, today backend responses can be missing the `details`
      field, if the current user lacks write permission to the database
      setting (see above).  Fully re-evaluating #25715 in this context is
      outside the scope of this PR.
      
      Closes: https://github.com/metabase/harbormaster/issues/5051
      Backports: https://github.com/metabase/metabase/pull/47247
      Backports: https://github.com/metabase/metabase/pull/47517
      Backports: https://github.com/metabase/metabase/pull/47469
      Backports: https://github.com/metabase/metabase/pull/47419
      
      
      Co-authored-by: default avatarDennis Schridde <63082+devurandom@users.noreply.github.com>
      
      * Fix Transparent disabled state on mantine switch label (#47677) (#47718)
      
      * make custom disabled state more specific than transparent style
      
      * make it less nesty
      
      Co-authored-by: default avatarRyan Laurie <30528226+iethree@users.noreply.github.com>
      
      * update docs (#47725)
      
      * Support field remapping in pivot queries (#47592) (#47693)
      
      * Support field remapping in pivot queries
      
      * Generate breakouts without inversions
      
      Co-authored-by: default avatarmetamben <103100869+metamben@users.noreply.github.com>
      
      * perf: Optimize some query processor functions (#47658) (#47716)
      
      * perf: [metabase.util.performance] Faster mapv for small collections
      
      * perf: Optimize some query processor functions
      
      Co-authored-by: default avatarOleksandr Yakushev <alex@bytopia.org>
      
      * :robot: backported "recents updates to better support metrics" (#47141)
      
      * recents updates to better support metrics (#47032)
      
      * recents updates to better support metrics
      
      allows updating metric views through the API
      improves parent_collection calculation
      
      * add 2 tests for metrics
      
      * :robot:
      
       backported "Entity id translation service"
      
      ---------
      
      Co-authored-by: default avatarBryan Maass <bryan.maass@gmail.com>
      
      * fix: sqlserver handle uniqueidentifier uuids (#47544) (#47638)
      
      * fix: sqlserver handle uniqueidentifier uuids
      
      Fixes #46148
      
      Include sqlserver in `uuid-type` handling as its `uniqueidentifier` type
      stores uuids.
      
      * Don't be so precise with varchar size
      
      * Add seam for drivers to cast to text type
      
      * Fix arg order
      
      Co-authored-by: default avatarCase Nelson <case@metabase.com>
      
      * Always allow localhost:* on CORS (#47663) (#47710)
      
      * Always allow localhost:* on CORS
      
      * Use `embedding-app-origin-sdk` method for the headers
      
      * Fix BE code formatting
      
      * Fix BE formatting
      
      Co-authored-by: default avatarMahatthana (Kelvin) Nomsawadi <kelvin@metabase.com>
      
      * :robot:
      
       backported "Revert multiple inheritance with SnowflakeVariant" (#47695)
      
      * Revert multiple inheritance with SnowflakeVariant (#47680)
      
      * Revert multiple inheritance with SnowflakeVariant (#47680)
      
      * CI
      
      ---------
      
      Co-authored-by: default avatarAlexander Polyankin <alexander.polyankin@metabase.com>
      
      * Entity id translation analytics (#47432)
      
      * tracking entity id translation
      
      * add total and more tests
      
      * indentation
      
      * realign the last test
      
      * Backport fixups
      
      * :robot:
      
       backported "upgrade webpack to address security issue" (#47604)
      
      Co-authored-by: default avatarUladzimir Havenchyk <125459446+uladzimirdev@users.noreply.github.com>
      
      * Hide attached DWH database details (#47419)
      
      == Goal ==
      
      Hide attached DWH database details from anyone incl. admins:
      * Do not show them in the UI
      * Do not permit to change them
      * Do not serialize them
      
      The aim is that customers cannot gain access to (parts of) credentials,
      and they cannot break a feature they are paying for by changing
      connection details.
      
      Database details are already generally excluded from H2 dump snapshots
      (see `metabase.cmd.copy/*copy-h2-database-details*`), thus nothing
      changes there.
      
      == Implementation ==
      
      The Metabase backend already contains provisions in the implementation
      of `metabase.models.interface/to-json` for `:model/Database` to hide the
      `details` of the database in HTTP responses, if the user lacks write
      permission on the database.  We utilize this by adding an
      `is_attached_dwh` column to the `database` table and rejecting
      `metabase.models.interface/can-write?` when this flag is enabled.  In
      the "admin" UI, we show a replacement text instead of the edit form when
      the flag is set.  (It might be correct to show this whenever `details`
      is absent.  See below for possible follow-up work.)
      
      However, several sections of the frontend code expected the `details`
      field to always be present.  In order to make `details` optional, as the
      backend seems to handle it, we fix the respective code to treat this
      case in the way that appears appropriate in the context.
      
      == How to test ==
      
      === New behaviour ===
      
      Setting the `is_attached_dwh` field hides the database details:
      
      1. Configure a database as described in https://www.metabase.com/docs/latest/configuring-metabase/config-file#databases.
         - In addition to the fields you would normally set, also set
           `is_attached_dwh: true`.
         - This also works when adding this flag to a database that previously
           did not have this flag set.
      2. Start your Metabase instance.
      3. Verify the database shows up in the "admin" section
         (`/admin/databases`).
      4. Verify that clicking the database to see its details only reveals
         "This database cannot be modified."
      5. Verify that responses from the backend do not include a `details`
         field for this database.
      
      === Original behaviour ===
      
      Behaviour without setting the `is_attached_dwh` field is unchanged:
      
      1. Configure a database as described in https://www.metabase.com/docs/latest/configuring-metabase/config-file#databases.
         - Only set the fields you would normally set.  Do not set
           `is_attached_dwh` (or set it to `false`).
      2. Start your Metabase instance.
      3. Verify the database shows up in the "admin" section
         (`/admin/databases`).
      4. Verify that clicking the database to see its details only reveal the
         regular edit form, showing connection fields like `host`, `user`,
         etc. with the values you configured.
      
      == How this will be rolled out ==
      
      1. Upgrade existing Metabase Cloud instances with data warehouse to a
         Metabase version that supports `is_attached_dwh`.
      2. Set `is_attached_dwh` in the database section of the config file for
         Metabase Cloud instances with a data warehouse.
      
      == Possible follow-up work ==
      
      In https://github.com/metabase/metabase/issues/25715, absent
      `database.details` was identified as a bug.  Since then, `details` was
      made `NOT NULL` in the application database, so this bug can no longer
      occur.  However, today backend responses can be missing the `details`
      field, if the current user lacks write permission to the database
      setting (see above).  Fully re-evaluating #25715 in this context is
      outside the scope of this PR.
      
      Closes: https://github.com/metabase/harbormaster/issues/5051
      Backports: https://github.com/metabase/metabase/pull/47247
      Backports: https://github.com/metabase/metabase/pull/47517
      Backports: https://github.com/metabase/metabase/pull/47469
      Backports: https://github.com/metabase/metabase/pull/47419
      
      
      Co-authored-by: default avatarDennis Schridde <63082+devurandom@users.noreply.github.com>
      
      * update docs (#47725)
      
      * Support field remapping in pivot queries (#47592) (#47693)
      
      * Support field remapping in pivot queries
      
      * Generate breakouts without inversions
      
      Co-authored-by: default avatarmetamben <103100869+metamben@users.noreply.github.com>
      
      * perf: Optimize some query processor functions (#47658) (#47716)
      
      * perf: [metabase.util.performance] Faster mapv for small collections
      
      * perf: Optimize some query processor functions
      
      Co-authored-by: default avatarOleksandr Yakushev <alex@bytopia.org>
      
      * fix: sqlserver handle uniqueidentifier uuids (#47544) (#47638)
      
      * fix: sqlserver handle uniqueidentifier uuids
      
      Fixes #46148
      
      Include sqlserver in `uuid-type` handling as its `uniqueidentifier` type
      stores uuids.
      
      * Don't be so precise with varchar size
      
      * Add seam for drivers to cast to text type
      
      * Fix arg order
      
      Co-authored-by: default avatarCase Nelson <case@metabase.com>
      
      * Always allow localhost:* on CORS (#47663) (#47710)
      
      * Always allow localhost:* on CORS
      
      * Use `embedding-app-origin-sdk` method for the headers
      
      * Fix BE code formatting
      
      * Fix BE formatting
      
      Co-authored-by: default avatarMahatthana (Kelvin) Nomsawadi <kelvin@metabase.com>
      
      * :robot:
      
       backported "Revert multiple inheritance with SnowflakeVariant" (#47695)
      
      * Revert multiple inheritance with SnowflakeVariant (#47680)
      
      * Revert multiple inheritance with SnowflakeVariant (#47680)
      
      * CI
      
      ---------
      
      Co-authored-by: default avatarAlexander Polyankin <alexander.polyankin@metabase.com>
      
      * :robot:
      
       backported "upgrade webpack to address security issue" (#47604)
      
      Co-authored-by: default avatarUladzimir Havenchyk <125459446+uladzimirdev@users.noreply.github.com>
      
      ---------
      
      Co-authored-by: default avatarBryan Maass <bryan.maass@gmail.com>
      Co-authored-by: default avatarOisin Coveney <oisin@metabase.com>
      Co-authored-by: default avatarMetabase bot <metabase-bot@metabase.com>
      Co-authored-by: default avatarOleksandr Yakushev <alex@bytopia.org>
      Co-authored-by: default avataradam-james <21064735+adam-james-v@users.noreply.github.com>
      Co-authored-by: default avatarDennis Schridde <63082+devurandom@users.noreply.github.com>
      Co-authored-by: default avatarRyan Laurie <30528226+iethree@users.noreply.github.com>
      Co-authored-by: default avatarJeff Bruemmer <jeff.bruemmer@gmail.com>
      Co-authored-by: default avatarmetamben <103100869+metamben@users.noreply.github.com>
      Co-authored-by: default avatarCase Nelson <case@metabase.com>
      Co-authored-by: default avatarBryan <bryanl@metabase.com>
      Co-authored-by: default avatarMahatthana (Kelvin) Nomsawadi <kelvin@metabase.com>
      Co-authored-by: default avatarAlexander Polyankin <alexander.polyankin@metabase.com>
      Co-authored-by: default avatarUladzimir Havenchyk <125459446+uladzimirdev@users.noreply.github.com>
      c882eef8
    • github-automation-metabase's avatar
    • github-automation-metabase's avatar
  6. Sep 06, 2024
Loading