== Goal ==

Hide attached DWH database details from anyone incl. admins:
* Do not show them in the UI
* Do not permit to change them
* Do not serialize them

The aim is that customers cannot gain access to (parts of) credentials,
and they cannot break a feature they are paying for by changing
connection details.

== Implementation ==

The Metabase backend already contains provisions in the implementation
of `metabase.models.interface/to-json` for `:model/Database` to hide the
`details` of the database in HTTP responses, if the user lacks write
permission on the database.  We utilize this by adding an
`is_attached_dwh` column to the `database` table and rejecting
`metabase.models.interface/can-write?` when this flag is enabled.  In
the "admin" UI, we show a replacement text instead of the edit form when
the flag is set.  (It might be correct to show this whenever `details`
is absent.  See below for possible follow-up work.)

However, several sections of the frontend code expected the `details`
field to always be present.  In order to make `details` optional, as the
backend seems to handle it, we fix the respective code to treat this
case in the way that appears appropriate in the context.

Database details are already generally excluded from H2 dump snapshots
(see `metabase.cmd.copy/*copy-h2-database-details*`), thus nothing
changes there.

== How to test ==

=== New behaviour ===

Setting the `is_attached_dwh` field hides the database details:

1. Configure a database as described in https://www.metabase.com/docs/latest/configuring-metabase/config-file#databases.
   - In addition to the fields you would normally set, also set
     `is_attached_dwh: true`.
   - This also works when adding this flag to a database that previously
     did not have this flag set.
2. Start your Metabase instance.
3. Verify the database shows up in the "admin" section
   (`/admin/databases`).
4. Verify that clicking the database to see its details only reveals
   "This database cannot be modified."
5. Verify that responses from the backend do not include a `details`
   field for this database.

=== Original behaviour ===

Behaviour without setting the `is_attached_dwh` field is unchanged:

1. Configure a database as described in https://www.metabase.com/docs/latest/configuring-metabase/config-file#databases.
   - Only set the fields you would normally set.  Do not set
     `is_attached_dwh` (or set it to `false`).
2. Start your Metabase instance.
3. Verify the database shows up in the "admin" section
   (`/admin/databases`).
4. Verify that clicking the database to see its details only reveal the
   regular edit form, showing connection fields like `host`, `user`,
   etc. with the values you configured.

== How this will be rolled out ==

1. Upgrade existing Metabase Cloud instances with data warehouse to a
   Metabase version that supports `is_attached_dwh`.
2. Set `is_attached_dwh` in the database section of the config file for
   Metabase Cloud instances with a data warehouse.

== Possible follow-up work ==

In https://github.com/metabase/metabase/issues/25715, absent
`database.details` was identified as a bug.  Since then, `details` was
made `NOT NULL` in the application database, so this bug can no longer
occur.  However, today backend responses can be missing the `details`
field, if the current user lacks write permission to the database
setting (see above).  Fully re-evaluating the fix to #25715 in this
context is outside the scope of this PR.

Closes: https://github.com/metabase/harbormaster/issues/5051

* ask for tenancy isolation columns

* deny all permissions for all users group

* create new collections

* add jwt group mappings

* add the permissions step

* add multi-tenancy message in helper text format

* add permission graph

* wire together permissions

* use schema permissions

* use fields from table metadata from query_metadata

* add tenancy field reference

* remove log messages

* deny access to unsandboxed tables

* make permission graph more explicit

* deny access to sample database for customer groups

* add unit test for permission graph

* split permission groups and sandboxes

* jwt settings and hard-coded user attributes

* handle errors when updating sso mappings

* add express api and user switcher

* only fallback to api keys when license is invalid

* add util to sample tenancy column values

* conditional BASE_SSO_API imports

* improve embedding error message

* setup jwt configuration after license step

* setup permissions at the last step

* add missing import

* update steps that requires license

* fix incorrect imports

* add missing useContext

* handle permission update error

* remove tenancyIsolationEnabled field

* add tenancy column sampling

* differentiate tenancy column query error

* rename tenancyColumnValues to tenantIds

* assign sampled tenant ids to user attributes

* add tenant ids

* define collection permissions

* reference sandboxing group by name

* update snippet to be same as the README

* extract ask for tenancy columns to a separate step

* use the customer_id attribute

* query the table query metadata at origin

* append tables correctly

* improve error handling in table scanning

* add retry logic to metadata fetching

* only query metadata for selected fields

* fix race condition with retry

* update loading state and retries

* update comments on jwt license

Co-authored-by: Mahatthana (Kelvin) Nomsawadi <me@bboykelvin.dev>

* filter the target table by id

* highlight last selected tenant column

* use breakout to get list of ids

* temporary workaround to reload the whole page

* update row value types

* update row value types

* block non-selected tables

* remove the source-field from sandboxing

* use the fk_target_field_id as instead of target.id

* update unit test

* remove source-field as we only reference our own column

* make native permission types more strict

---------

Co-authored-by: Mahatthana (Kelvin) Nomsawadi <me@bboykelvin.dev>
Co-authored-by: Oisin Coveney <oisin@metabase.com>

* ask for tenancy isolation columns

* deny all permissions for all users group

* create new collections

* add jwt group mappings

* add the permissions step

* add multi-tenancy message in helper text format

* add permission graph

* wire together permissions

* use schema permissions

* use fields from table metadata from query_metadata

* add tenancy field reference

* remove log messages

* deny access to unsandboxed tables

* make permission graph more explicit

* deny access to sample database for customer groups

* add unit test for permission graph

* split permission groups and sandboxes

* jwt settings and hard-coded user attributes

* handle errors when updating sso mappings

* add util to sample tenancy column values

* improve embedding error message

* setup jwt configuration after license step

* setup permissions at the last step

* handle permission update error

* add tenancy column sampling

* differentiate tenancy column query error

* rename tenancyColumnValues to tenantIds

* define collection permissions

* reference sandboxing group by name

* extract ask for tenancy columns to a separate step

* query the table query metadata at origin

* append tables correctly

* improve error handling in table scanning

* add retry logic to metadata fetching

* only query metadata for selected fields

* fix race condition with retry

* update loading state and retries

* filter the target table by id

* highlight last selected tenant column

* use breakout to get list of ids

* update row value types

* block non-selected tables

* remove the source-field from sandboxing

* use the fk_target_field_id as instead of target.id

* update unit test

* remove source-field as we only reference our own column

* make native permission types more strict

---------

Co-authored-by: Oisin Coveney <oisin@metabase.com>

fix(questions + admin/performance): Improve Clear cache button text and question refresh button tooltip (#46791)

Co-authored-by: Mahatthana (Kelvin) Nomsawadi <me@bboykelvin.dev>

feat(admin/performance): In Admin / Performance, add a tab where you can manage the caching policies of dashboards and questions (#42990)

Closes #42567

* move default modal padding to general modal component

* small adjustments. Using base modal where apropriate

* shame

* Add subfolders in querying

* Add subfolders in querying

* Add subfolders in querying

* Remove FilterContent

* Remove FilterContent

* Remove FilterContent

* Remove FilterContent

* Remove FilterContent

* Update interactive embedding CTA UTM tags

* Make the UTM tags more readable

* Update static embed doc UTM tags

* Update static embed appearance customization doc UTM tags

* Update font upsell UTM tags

* Update tests

* Fix wrong upgrade URL

* Update tests

* only use viz height for visualization view

* move the default viz height fallback

* default to full-height if height is zero or unavailable

* remove test wrapper

* extending table component features

* adding unit tests

* PR Feedback

* PR Feedback

* unit test adjustment

this retains previous behavior

* Cljfmt config part 2

* Part 3 WIP

* Part 3 WIP

* Part 3 WIP

* Part 3 WIP

* Part 3 WIP

* Use fork with https://github.com/weavejester/cljfmt/pull/348 and https://github.com/weavejester/cljfmt/pull/350

* Backport updated config and linter fork from part 3

* Update formatting

* Reformat

* Fix bad indentation from #47064

* QuestionAlertWidget to FC + TS

* update e2e tests

* use regular alert for unsubscribes

* Create Question and Dashboard Sharing Menu

* Handle some edge cases

* top notch unit tests

* more testing and tweaking

* fix imports

* more tests

* add alerts to question sharing menu

* add tests for alerts

* update e2e tests

* use Oisin's toolbarbutton

* fix rebase

* prompt to save before sharing questions

* show prompt to set up notification channels

* fix import

* update tests and hide on notebook screen

* fixes

* clean up pulse types

* update e2e tests

* fix a bunch more tests

* more e2e test fixes

* maybe green now? 

* last one 

* design updates

* make channel setup modal nicer

* simpler component props

* lint fix

* lint fixes

* Cljfmt config part 2

* Backport updated config and linter fork from part 3

* Update formatting

feat(sdk): embedding cli opens the metabase store to get trial token and applies the license (#46810)

* open metabase store to get trial token

* remove license env from instance setup as we provide the license key later

* activate metabase license

* fix password field missing

* fix formatting for generated component files message

* activate license key

* add missing auth options for postgres

* add function to print with padding

* prevent infinite loop and update section representation

* show the activate license error in red

* rename variable

* Cljfmt

* Fix new GH action

* refactor: sort import members inside destructuring

* Show an error message when the temporal unit is not applicable to a dashboard card (#46983)

* Fix tests

* Add tests

* Add tests

* Fix types

* Fix types

---------

Co-authored-by: Alexander Polyankin <alexander.polyankin@metabase.com>

* Kondo Config cleanup

* Restore warnings for with-log-messages-for-level until #28827 is merged

* Updated clojure.test hooks

* Fix kondo warning

* Test defn/defmacro exclamation point linter should also use :parallel/disallowed (part 1)

* WIP

* Time for me to learn to spell

* Finish the cleanup

* Remove code I was typing as an example

* Fix renamed var

* Fix the SAML tests

* Fix Kondo warning

* allow `:blocked` to be saved for table level perms

* Adds 2 tests for table level blocked permission settings

- N.B. these are NOT ENFORCED YET

* update test that asserted we cannot set block on tables (we can)

* WIP: Pairing on making perms checking less wild

* cleanup, update docs, and add a test for view-data perm only

- Added a test where we have data permissions, but not create query, and
  I think it is failing when we have create query and blocked data
  permissions.
- renamed some functions from check-x -> has-x? since they return a
  value instead of throwing now

* Revert "WIP: Pairing on making perms checking less wild"

Keep the same behavior, but stick with the saner flow control

This reverts commit 63bcb5b4.

* update docs

* update test to be passing

- TODO: make sure it's correct w.r.t. perm settings

* Allow schema level blocked setting in permgraph

* remove invalid test cases

- continue to have a forcing function to test newly added perms

* conform function output

* ensure a single blocked table blocks native queries to its DB

* update error message

- we now catch this error in `metabase.models.query.permissions/has-perm-for-query?`

* we now check for data permissions to process query for card

* add more explanation to what we are testing

- to help see why it fails on CI and passes locally

* remove excess `def`

* Add test for table-level data X collection perms

- update test found to be in-error

* update param values qp permission check style

* set view-data and create-query explicitly

* set viewdata and createquery explicitly in qp test

* Respond to review comments (which fixes a case)

* setting a table to blocked: leave other tables the same

* [Permissions] Add "No access" schema/table permission (#46509)

* first pass

* refactors downgrading native permission logic and updates calculation so that "No access" downgrades native permissions to "No"

* stub for permissions help info on table block

* modal changes wip, updates downgrading create queries permissions to all happen at a single call site

* clean up, sandboxing modal copy changes, removes rekoke/limit access modal changes to make the diff smaller and move code to a seperate PR

* updates permissions help section to contain the final copy

* sandboxing copy fix and remove modal that was dropped from requirements

* adds blocked at the schema level, updates no access copy to blocked, updates permissions help section to contain new blocked and schema level changes

* fixes failed unit and e2e tests after sandboxing copy changes

* improve the block e2e test to include table blocking

* fixes failing blocked test, fixes other schemas create queries permissions getting correct with one schema was droped to blocked view data access, fixes a bug that prevents the save bar from going away when all permissions for group are set to the default values

* clean up

* remove color changes

* prevents parent being set to blocked preventing edits for children entities

* add new hasPermissionValueInSubgraph fn, adds modal to warn users we have to upgrade the view data permissions when they upgrade create queries permissions when a child entity is set to blocked

* adds test coverage for new modal

* removes unused function, adds new updateEntityPermission fn to help consolidate some logic elsewhere

* unit test fix and type fix

* most pr feedback

* updates the confirmation modal copy when changing a parent entity that contains a child with blocked permissions and/or sandboxed children, adds test coverage for that, adds test coverage for permission view data column not appearing in oss

* type fix

* [Permissions] Add e2e test coverage for blocked permissions enforcements (#46663)

* adds test coverage for enforcement of blocked permissions

* moves tests around based on pr feedback

* copy changes

* adds fix to make sure that blocked permissions are not removed from sibling tables that have the create queries permissions upgraded (#46854)

* Fix table name lookup for dbs w/ 1 schema per db

* add test for blank schema identifiers

* Refine sandboxed user perms for query builder access (#46939)

* Refine sandboxed user perms for query builder access

- Limit create-queries permissions to unblocked tables only
- Check user permissions for each table before granting query builder access
- Prevent querying of blocked joined tables from query builder for sandboxed users

* Adjust permissions for sandboxed users

- Grant view-data permissions only for unblocked tables
- Revert create-queries permissions to all tables in sandbox
- Remove unnecessary intermediate variable

* when sandboxing we no longer grant unrestricted view perms for blocked tables

* Update enterprise/backend/src/metabase_enterprise/sandbox/query_processor/middleware/row_level_restrictions.clj

remove blank line

Co-authored-by: Noah Moss <32746338+noahmoss@users.noreply.github.com>

* - make coalesce-test exhaustive (except for sandbox)

* Update enterprise/backend/src/metabase_enterprise/sandbox/query_processor/middleware/row_level_restrictions.clj

Co-authored-by: Noah Moss <32746338+noahmoss@users.noreply.github.com>

* t2/select ... -> database/table-id->database-id

* update comment

* [Permissions] Prevent "Granular" option in DB View Data options from changing permissions to unrestricted (#46976)

* fix

* adds back most of the code and limits it to only happen with impersonations, updates test to handle differing logic between the two flows

* removes test that is not longer needed

* more sandbox join table perms tests

---------

Co-authored-by: John Swanson <john.swanson@metabase.com>
Co-authored-by: Sloan Sparger <sloansparger@users.noreply.github.com>
Co-authored-by: Sloan Sparger <sloansparger@gmail.com>
Co-authored-by: Noah Moss <32746338+noahmoss@users.noreply.github.com>

* Remove simple references to `trackStructEvent`

* Remove `withAnalytics` entities helper

* Remove `trackLoginSSO`

* Remove auth trackers

* Remove permissions trackers

* Remove performance trackers

* Remove settings trackers

* Remove pulse trackers

* Remove undo trackers

* Remove click actions trackers

* Remove `trackStructEvent`

* Remove GoogleAnalytics from the server security middleware

* Remove GoogleAnalytics from Kondo config

* Fix type failures

* Revert "Remove GoogleAnalytics from Kondo config"

This reverts commit 363557c4.

* use ee api to find custom reports collection

* remove deprecated hook

* fix race condition

* add unit tests

* fix circular dependency

* fix unit tests

* another plugin strategy

* Kondo Config cleanup

* Restore warnings for with-log-messages-for-level until #28827 is merged

* Updated clojure.test hooks

* Fix kondo warning

* Time for me to learn to spell

was missing from the call.

Also had to disable hooks to commit:

```
 clj-kondo --config ./.clj-kondo/config.edn --config-dir ./.clj-kondo --parallel --lint:
metabase/enterprise/backend/test/metabase_enterprise/serialization/v2/extract_test.clj:458:20: warning: Unresolved var: ts/extract-one
metabase/enterprise/backend/test/metabase_enterprise/serialization/v2/extract_test.clj:1691:8: warning: Unresolved var: mbc/ensure-audit-db-installed!
metabase/enterprise/backend/test/metabase_enterprise/serialization/v2/extract_test.clj:1757:18: warning: Unresolved var: ts/create!
linting took 247ms, errors: 0, warnings: 3
error Command failed with exit code 1.
```

* Beautiful log message capturing  

* Don't have log.cljs try to load log.capture, don't know how to make that work

* Remove restriction against use in parallel tests

* Fix tests using invalid syntax

* Port legacy tests

* Make this stuff work with Cljs

* Fix bad syntax

* Convert usages of old version of with-log-messages-for-level to new version

* Update other stuff to use the updated macro

* Fix stuff

* Fix Cljs tests

* Fix world's largest test

* Appease Kondo

* Fix comment

* ClojureScript: only emit capture code in dev builds