* Fix some minor collections issues

- explicitly provide access to the trash to all users, in the same way
we provide access to personal collections. Due to migration order, we
don't necessarily have a permissions row with the correct `perm_type`,
`perm_value`, and `collection_id` for the Trash collection. That's ok -
we don't actually move things to the Trash, so there isn't any
item where `collection_id=$trash.id` - but this may affect things like
effective ancestors for children of the trash. Let's be explicit about
the permissions that users have.

- replace a case where we manually calculated effective location and
then got the parent_id with just hydrating `:effective_parent`. This is
more efficient.

- replace the simple hydration method for `effective-location-path` with
a batched hydration method that fetches `visible-collection-ids` *once*
and then uses it to figure out the effective location path for each
collection passed.

* Add custom image for empty metrics

* Add useFetchMetrics helper

* Copy browse models page to metrics page

* Add route for browse metrics page

* Add useHasMetrics hook

* Add link to browse metrics page to the sidebar

* Simplify MetricsTable component

* Render description markdown in table too

* Add unit tests for browse metrics page

* Remove custom padding

* Fix line height of markdown cell

* Add e2e tests for browse page

* Fix metrics sorting

* Fix types

* Fix column widths and cell padding

* Typo

Co-authored-by: Kamil Mielnik <kamil@kamilmielnik.com>

* Remove name check

* Simplify alphabet

* Remove resetSnowplow

* Use helper to get metric in table

* Use MouseEvent from React directly

* Use --mb-color-icon-primary

* Test ellipsification of description

* Simplify useHasMetrics

* Move useHasMetrics MainNavBar

* Merge sortMetrics and sortModels into sortCards

* Fix metric default description

* Extract SidebarSkeleton to a separate component

* Test sorting via Collection too

* Use ModelTableRow to fix cursor

* Share more components between ModelsTable and MetricsTable

* Use the same simplified structure in ModelsTable as for MetricsTable

* Render block-level elements as inline so ellipsis detection works

* Pick better testid for metric name

* Remove delay

* Avoid type specificty error in sortCards

* Add test for not opening in new tab

* Remove useFetchMetrics defaults

* Fix unit test import

* Test meta-click with cypress

* Add comment on inline markdown elements

* Use model-name test id

* Use correct colors for icon

* Remove delay from browse models page too

* Make meta key in test platform-dependent

* Fix hover color on metrics table

* Add focus styling for celss in the models and metrics table

* Use MarkdownPreview for table cells

* Define --mb-color-icon-primary

* Make the collection link take up the whole height of the cell

* Remove tabIndex from table rows

* Rename ModelsTable.module.css to BrowseTable.module.css

* Rename CardsTable to BrowseTable

* Remove single module.css class in favor of styled components

* Allow customising allowed elements

* Allow forcing all markdown on one line

* Render bold and italics in markdown descriptions and move content onto one line

* Fix legacy tooltip in test

* Make sure getValueForSorting returns a string

* Remove empty file

* Remove icon for metrics in the table

* Remove else statement.

Co-authored-by: Uladzimir Havenchyk <125459446+uladzimirdev@users.noreply.github.com>

* Remove async findBy where possible

* Add github issue link for TODO item

* Disambiguate between getMetricDescription type

* Be explicit with sortModelOrMetric

---------

Co-authored-by: Kamil Mielnik <kamil@kamilmielnik.com>
Co-authored-by: Uladzimir Havenchyk <125459446+uladzimirdev@users.noreply.github.com>

This migration is supposed to be backported to v50 and thus needs a v50
migration ID.  Since v51.2024-08-27T00:00:00 was already applied to dev
instances, we set `preConditions[].onFail = "MARK_RAN"`, so that if the
table is already present, the backdated migration does not fail.  See
the discussion on PR #47419 for more details.

This will not need an (automatic) backport, since I will include it in
PR #47419.

Fixes: 592360c9
References: https://github.com/metabase/metabase/pull/47419#discussion_r1738929516

* [notification] New method: `channel/can-connect?` (#44955)

* [notification] Channel APIs (#45207)

* [notification] namespaced channel type (#45527)

* [Notification] Render alert for http channel (#45545)

* [notification] Add channel description (#45840)

* [notification] update API to enable http channels for alert (#45839)

* [Notification] Remove channel details for users without write perms (#46034)

* [Notification] Serdes channel (#46031)

* [Notification] Update http details schema (#45960)

* [Notification] Deactivate channels will delete PulseChannel (#46115)

* [Notification] audit log for channel create and update (#46113)

* [Notification] Disallow undefined key for http channel details (#46712)

* [Notification] Handle channel name conflicts (#46818)

* Webhooks Admin Section (#46194)

* [notification] Fix test pulse endpoint does not work properly for http channels (#46474) (#47050)

* [Notification] Fix unable to update multiple channels per type (#47111)

* [Notification] Record Task History when pulse sends channel message (#46218)

* Enabling Webhook Alerts (#47022)

* [Notification] fix cyclic deps (#47379)

* [notification] channel serdes spec (#47386)

Co-authored-by: Nick Fitzpatrick <nick@metabase.com>

* move migration in release to match master

* add column if not exists

== Goal ==

Hide attached DWH database details from anyone incl. admins:
* Do not show them in the UI
* Do not permit to change them
* Do not serialize them

The aim is that customers cannot gain access to (parts of) credentials,
and they cannot break a feature they are paying for by changing
connection details.

== Implementation ==

The Metabase backend already contains provisions in the implementation
of `metabase.models.interface/to-json` for `:model/Database` to hide the
`details` of the database in HTTP responses, if the user lacks write
permission on the database.  We utilize this by adding an
`is_attached_dwh` column to the `database` table and rejecting
`metabase.models.interface/can-write?` when this flag is enabled.  In
the "admin" UI, we show a replacement text instead of the edit form when
the flag is set.  (It might be correct to show this whenever `details`
is absent.  See below for possible follow-up work.)

However, several sections of the frontend code expected the `details`
field to always be present.  In order to make `details` optional, as the
backend seems to handle it, we fix the respective code to treat this
case in the way that appears appropriate in the context.

Database details are already generally excluded from H2 dump snapshots
(see `metabase.cmd.copy/*copy-h2-database-details*`), thus nothing
changes there.

== How to test ==

=== New behaviour ===

Setting the `is_attached_dwh` field hides the database details:

1. Configure a database as described in https://www.metabase.com/docs/latest/configuring-metabase/config-file#databases.
   - In addition to the fields you would normally set, also set
     `is_attached_dwh: true`.
   - This also works when adding this flag to a database that previously
     did not have this flag set.
2. Start your Metabase instance.
3. Verify the database shows up in the "admin" section
   (`/admin/databases`).
4. Verify that clicking the database to see its details only reveals
   "This database cannot be modified."
5. Verify that responses from the backend do not include a `details`
   field for this database.

=== Original behaviour ===

Behaviour without setting the `is_attached_dwh` field is unchanged:

1. Configure a database as described in https://www.metabase.com/docs/latest/configuring-metabase/config-file#databases.
   - Only set the fields you would normally set.  Do not set
     `is_attached_dwh` (or set it to `false`).
2. Start your Metabase instance.
3. Verify the database shows up in the "admin" section
   (`/admin/databases`).
4. Verify that clicking the database to see its details only reveal the
   regular edit form, showing connection fields like `host`, `user`,
   etc. with the values you configured.

== How this will be rolled out ==

1. Upgrade existing Metabase Cloud instances with data warehouse to a
   Metabase version that supports `is_attached_dwh`.
2. Set `is_attached_dwh` in the database section of the config file for
   Metabase Cloud instances with a data warehouse.

== Possible follow-up work ==

In https://github.com/metabase/metabase/issues/25715, absent
`database.details` was identified as a bug.  Since then, `details` was
made `NOT NULL` in the application database, so this bug can no longer
occur.  However, today backend responses can be missing the `details`
field, if the current user lacks write permission to the database
setting (see above).  Fully re-evaluating the fix to #25715 in this
context is outside the scope of this PR.

Closes: https://github.com/metabase/harbormaster/issues/5051

This updates the instance analytics YAMLs:
- v50.22
- fixes several fk-target-field-id references

These migrations will be backported to v49.

Three changes:

- update IDs/locations of the migrations to v49 vs v51

- add one preCondition (when adding the foreign key constraint)

- add `onFail: MARK_RAN` to the index preconditions. I forgot this
before, so it blows up when the precondition doesn't hold.

* Increase log level to INFO from error

Closes #47396

logs before:
```
2024-08-29 08:37:03,836 INFO db.liquibase :: Checking if Database has unrun migrations...
2024-08-29 08:37:04,045 INFO db.liquibase :: Database has unrun migrations. Checking if migration lock is taken...
2024-08-29 08:37:04,047 INFO db.liquibase :: No migration lock found.
2024-08-29 08:37:04,047 INFO db.liquibase :: Migration lock acquired.
2024-08-29 08:37:04,167 INFO db.liquibase :: Running 83 migrations ...
```

logs after:

```
2024-08-29 08:39:45,019 INFO db.liquibase :: Running 83 migrations ...
...
2024-08-29 08:39:45,166 INFO liquibase.changelog :: ChangeSet migrations/001_update_migrations.yaml::v49.2024-06-27T00:00:00::calherries ran successfully in 0ms
2024-08-29 08:39:45,182 INFO liquibase.changelog :: Columns is_defective_duplicate(boolean) added to metabase_field
2024-08-29 08:39:45,182 INFO liquibase.changelog :: ChangeSet migrations/001_update_migrations.yaml::v49.2024-06-27T00:00:01::calherries ran successfully in 14ms
2024-08-29 08:39:45,199 INFO liquibase.changelog :: Custom SQL executed
2024-08-29 08:39:45,200 INFO liquibase.changelog :: ChangeSet migrations/001_update_migrations.yaml::v49.2024-06-27T00:00:02::calherries ran successfully in 16ms
2024-08-29 08:39:45,202 INFO liquibase.changelog :: Foreign key fk_field_parent_ref_field_id dropped
2024-08-29 08:39:45,202 INFO liquibase.changelog :: ChangeSet migrations/001_update_migrations.yaml::v49.2024-06-27T00:00:03::calherries ran successfully in 1ms
2024-08-29 08:39:45,204 INFO liquibase.changelog :: Foreign key constraint added to metabase_field (parent_id)
2024-08-29 08:39:45,205 INFO liquibase.changelog :: ChangeSet migrations/001_update_migrations.yaml::v49.2024-06-27T00:00:04::calherries ran successfully in 2ms
...
```

* empty commit to bump CI

* handle cache config overflowed

* fix root too

* do not comment out test

* make cljfmt happy

---------

Co-authored-by: Alexander Solovyov <alexander@solovyov.net>

There are a few separate changes here:

- Migrations: add and populate indexed columns perm_value, perm_type, and collection_id to permissions

These fields allows us to efficiently run queries based on collection permissions in the DB without string manipulation. Keeping the table as permissions allows us to do this migration in-place. Note that in some cases collections may have been deleted from the database without deleting the associated permissions row (since there was no foreign key before). We need to be defensive here: if we have a permissions row without a corresponding collection, delete the row before running the rest of the migration.

- Write perm_value, perm_type, and collection_id for new collection permissions

A very simple before-insert method sets these fields before a collection permission is written to the DB.

- Replace collection/permissions-set->visible-collection-ids and collection/visible-collection-ids->honeysql-filter-clause with collection/honeysql-filter-clause

Previously, just about everywhere we used permissions-set->visible-collection-ids, what we were essentially doing was an in-app join: select all the collection IDs you have permission on, then convert it to a SQL clause like WHERE collection_id IN ( all of those collection IDs).

Replace both of these with honeysql-filter-clause, which uses the new fields we added to permissions above to construct a honeysql filter representing "all the collections I have permissions on", without needing to round-trip them to the application and back to the DB.

Of course, we can then write a function visible-collection-ids, which uses honeysql-filter-clause, for those cases where we do actually need the whole bunch in the application (we use this, for example, when constructing the effective-location for a collection).

I also added one more toggle to the VisibilityConfig that's passed into the honeysql-filter-clause (and used to be passed to permissions-set->visible-collection-ids), allowing you to select only the effective children of some collection.

* tests ensure blocked tables downgrade > blocked db

* implement the blocked table -> blocked db rollback

- move/add tests for it

Make sure we insert special segmented permission paths for sandboxed tables on downgrade to 49 (#47085)

I didn't consider that we might not have permissions necessary to
disable triggers.

* new version of data_access migration + run ANALYZE beforehand

* relax constraints on migration timestamps

* more migration rewrites

* fix bug in migration

* fix another bug and remove extraneous logging

* reorder migrations to fix ID validation

* start adapting the changes to mysql as well

* wip

* add more ANALYZE calls

* revert logging changes

* finish duplicating work for mysql + a bit of clean up

* create H2 versions of migrations without the ANALYZE calls

* remove ANALYZE call from view_data.sql (probably not necessary)

* fix mysql syntax

* fix another mysql bug

* another mysql fix

* Don't exclude inactive tables for data_access

* Don't write rows for Administrators group

* Don't write rows for Admins (H2/MySQL)

* disable triggers before perms migration

---------

Co-authored-by: John Swanson <john.swanson@metabase.com>

* Add column_settings migration

* Remove old getColumnKey

* Remove old getColumnKey

* Fix export

* Change comment

* Change comment

* Change comment

* Change comment

* Add RemoveFieldRefsFromDashboardCardColumnSettings

* fix

* chore

* chore

* chore

* chore

* chore

* chore

* chore

* chore

* chore

* chore

* chore

* Fix

* Fix

* Fix tests

* Fix tests

* Fix tests

* Fix tests

* Fix tests

* Fix tests

* Update loki

* Update loki

* Fix the homepage dashboard being loaded everywhere

* Fix the homepage dashboard being loaded everywhere

* Fix tests

* Fix tests

* Fix tests

* Fixes

* Fix tests

* Fix tests

* Fix tests

* Fix

* Fix

* Fixes

* Fixes

* Fixes

* Fixes

* Fixes

* Fixes

* Fixes

* Fixes

* Fixes

* Fixes

* Fixes

* Fixes

* Fixes

* Fixes

* Fixes

* Fixes

* Fixes

* Fixes

* Fixes

* Fixes

* Fixes

* Fixes

* Fixes

* Fixes

* Fixes

* Fixes

* Fixes

* Fixes

* Fixes

* Fixes

* Fix typo

* Fix types

* Improve migration performance

* Fixes

* Revert changes

* Revert changes

* Revert changes

* Revert changes

* Remove migration

### Description

This change should fix some explosions in stats due to phantom fields, while also making things a bit more rigorous.

- Record the table schema in query_field - especially useful for unknown tables
- Allow null table names in query_field - they might be phantoms, or they might not
  - We still have the option to filter them out in tools we build on top
- Tighten up matching logic to consider schema

* Update popover dark theme on part of ParameterFieldWidget (#45813)

* Add filter visual tests

* Simplify embed CSS colors

* [WIP] theme filter popover

* Move theme to global node, so it affects portal too

* Fix embed colors

* Theme part of ParameterFieldWidget

* Add story for parameter with dropdown

* Update loki snapshots

* Update more popover style

* Update loki snapshots

* Remove outdated snapshots

* Update loki snapshots with production build

* Fix loki not working with `userEvent`

* Theme mantine popover

* Improve dark theme text-brand visibility

* Update loki snapshots from master change

* Fix new colors not working in SDK

* Fix flaky Loki tests

* Filter popover dark theme part 2 (#46284)

* Theme Date all options

* wip add month and year filter

* Fix wrong CSS custom property declaration

* Test hover state

* Update Loki snapshots

* Theme date month and year

* Add quarter and year tests

* Theme date single and date range

* Style relative date

* Update frontend/src/metabase/admin/datamodel/components/filters/pickers/DatePicker/DatePickerHeader.styled.tsx

Co-authored-by: Denis Berezin <denis.berezin@metabase.com>

---------

Co-authored-by: Denis Berezin <denis.berezin@metabase.com>

* Complete theming filter popover night mode (#46365)

* Style unit of time and number widget

* Attempt to fix flaky Loki snapshots

* Address white background checkbox on dark theme

* Style search filter single value

* Fix mis color

* Update snapshots

* Remove duplication on play functions since they're long

* Milestone 2 polishing (#46382)

* Matches the footer padding with header padding

* Update snapshots

* Add E2E to prevent further problem

* Try to reduce flaky Loki tests

* Remove unused color shades (#46421)

* Update no results illustration to work better on dark theme

* Update loki snapshots

---------

Co-authored-by: Denis Berezin <denis.berezin@metabase.com>

* Squash

* Clean up query modifier for validating card references (#46283)

* Fixup bad git merge on test expectation

* Tweak docstrings and de-densify logic

* Adds a middleware that sets 2 dynamic vars

- These vars will be used to insert context for embedding analytics

* migration adding embedding_client and embedding_version to qe and vl tables

* clarify function api

* headers come in -> dynamic vars -> written to db

When the vars are set we insert their values into query execution and
query_log under embedding_client and embedding_version.

* update update migrations

* add predconditions to migrations

* fix tablenames + dataset_test

* Code Review responses

- remove commented out requires
- clarify test for merge-count-maps

* Add `report_dashboard.last_viewed_at`

- defaults to `now()` (it's intended to be used as a timestamp after which
we're *sure* the item hasn't been used, not a definitive timestamp we're
sure the dashboard was actually used at)

- updated whenever the QP processes a query for a dashcard

---------

Co-authored-by: Ngoc Khuat <qn.khuat@gmail.com>

* refactor: Simplify code (#44275)

* Remove dead code

* Remove values same as default options

* Simplify hook API: Removing unused argument

* review: Remove redundant code

* Update look and feel copy (#44277)

* Update static embed tab name

* Update copies + rearrange settings

* Use Mantine Select

* Fix E2E tests by applying new copies

* Fix unit tests

* Fix more E2E tests

* review: Add a comment on Mantine `value` quirk

* Adjust spacing and relevant tests

* Move `transparent` theme to its own toggle option (#44362)

* Reorder the function to match the visual order

* Make function name consistent

* Introduce the new background option (do nothing yet)

* Remove redundant exports

* Tighten up values to reduce confusion

* Make the parameters order consistent

* Only show background option on dashboads (spec)

* Allow dashboard embed background to be transparent

* Make filter background and header/footer transparent

* Fix type error

* Fix unit tests

* Fix E2E tests

* Fix checkerboard pattern not showing when no background

* Add unit tests

* Add missing copy update

* Remove remnants from outdated solutions within the PR

* Improve code comment references

* Fix E2E tests since the last copy change

* Fix type errors from merging master

* Update light theme style (#44488)

* Add background to dashboard filter on light theme

* Make position sticky works with transparent background

* Fix sticky filters not working on theme=transparent

* Fix unit tests

* Simplify IntersectionObserver components + fix E2E tests

* Add storybook for embedded dashboard

* Clarify the use of ExplicitSize context

* Remove unused code

* Formatting

* Review: Better mock `isWithinIframe`

* Update dark theme to match the new design (#45134)

* Update light theme checkerboard colors

* Update checkerboard colors

* Consolidate dashboard light/dark border style

* Use `--mb-color-text-primary` for light theme dashboard

* Update embed dashboard dark theme primary color

* Migrate dashboard text-secondary color

* Migrate text-tertiary color

* Migrate gold, axis, and split line to CSS custom properties

* Fix failed e2e tests

* Fix static viz not working

* Update loki snapshots

* Remove unnecessary CSS custom property

* Make diff more consistent

* Address review: reusing type

* Support backward compatibility when passing `theme=transparent` (#45214)

* Match preview query with the actual code query

* Handle backward compatibility when `theme=transparent` on dashboard

* Add unit tests

* Address review: Make tests less repetitive

* Update Metabase banner upsell UI (#45424)

* Prepare folder for nested components

* Update upsell UI

- add `fullWidth` variant
- align title to the left

* Fix Upsell banner having horizontal scroll on full width

* Add new Metabase banner upsell component

* Fix unit tests

* Address review: Fix typo

Co-authored-by: Ryan Laurie <30528226+iethree@users.noreply.github.com>

* Address review: Move upsel UI to `admin/upsells`

Co-authored-by: Ryan Laurie <30528226+iethree@users.noreply.github.com>

* Remove unnecessary ESLint disable comment

We can remove it since the file is in admin folder, and is already excluded from the rule.

* Address review: remove default value property

Co-authored-by: Denis Berezin <denis.berezin@metabase.com>

* Address review: Extract magic number

---------

Co-authored-by: Ryan Laurie <30528226+iethree@users.noreply.github.com>
Co-authored-by: Denis Berezin <denis.berezin@metabase.com>

* More UI updates (#45450)

* Update dark theme text colors to match the new design

* Update powered by metabase logo

* Update loki snapshots

* Fix unit tests

* Fix formatting

* Fix wrong dark theme metabase logo badge color

* Address review: Compress SVG with svgo

Co-authored-by: Denis Berezin <denis.berezin@metabase.com>

* Fix E2E tests

* Update loki snapshots

* Clarify Metabase base colors with comments

---------

Co-authored-by: Denis Berezin <denis.berezin@metabase.com>

* Add more tests (#45628)

* Add static embed question stories

* Add loki snapshots

* Add transparent theme for static dashboard tests

* Add transparent theme tests for static questions

* Assert `background` option in analytics tests

* Remove tests for logic that has been moved

* Update loki snapshots

* Fix failed E2E tests from merging master

---------

Co-authored-by: Ryan Laurie <30528226+iethree@users.noreply.github.com>
Co-authored-by: Denis Berezin <denis.berezin@metabase.com>

* Alert on asset download error

* moving function to metabase global

* removing alert, adjusting e2e test

* update CSP