Fixes #4755

* Always allow localhost:* on CORS

* Use `embedding-app-origin-sdk` method for the headers

* Fix BE code formatting

* Fix BE formatting

We had a security issue where it was possible to query timelines without
having the requisite permissions on the collection. This test checks to
make sure this doesn't happen again.

* walk the error to remove any visualization settings

Thanks Dan for the patch, this approach is nice because it doesn't require changing all 3 streaming-response-writer
implementations (json csv and xlsx). It keeps behaviour basically the same, but just cleans up the output to not
include potentially sensitive column/dataset details.

* add a test that confirms json/csv errors are cleaned

* walk the error to remove any visualization settings

Thanks Dan for the patch, this approach is nice because it doesn't require changing all 3 streaming-response-writer
implementations (json csv and xlsx). It keeps behaviour basically the same, but just cleans up the output to not
include potentially sensitive column/dataset details.

* add a test that confirms json/csv errors are cleaned

* fix dataset error test

* cljfmt

* try to be a little more surgical with what we take out of the error

* formatting again, oops

* pass export format, only modify the obj for file exports

In the case of errors being saved to files, the obj is a map with a :status :failed key assoc'd by the qp somewhere
along the way. The format-exception cond branch is not used in this case, so we can't rely on modifying
`format-exception` to just dissoc ex-data.

Instead, I've kept the obj modification, but it excludes the query and the preprocessed keys, which are the only 2
keys where we see :viz-settings. This doesn't eliminate the problem of needing to exclude other keys in the future,
but it does improve the surface area - the query and viz settings are the most likely candidates for holding
potentially sensitive info.

I'd prefer to stick with this solution, as we can get a fix for the security concern in quicker, and perhaps a
non-security bug can be opened regarding the most correct way to indicate errors to users whose downloads have failed.

* keep format-exception the same

* cleaner passing of export-format to write-error!

* Support field remapping in pivot queries

* Generate breakouts without inversions

The way this was implemented, it also hid the section for management
of uploaded CSV files.  Further, admins might want to intentionally
switch from uploading to the data warehouse provided by Metabase to
one of their own.

This reverts commit fbaf58ad.

References: https://github.com/metabase/harbormaster/issues/5121
References: https://metaboat.slack.com/archives/C032LFJFANL/p1725296255637989

* API endpoint dox generation should resolve schemas from the registry #46799

* Update Kondo config

* Fix formatting

* Fix typo in MetadataProvider docstring

* Fix typo in caching-test: s/Second/Third/

* Store ::nil markers in CachedMetadataProvider for failed lookups

Store ::nil markers in CachedMetadataProvider for any ids for which the wrapped/uncached upstream provider fails to return
metadatas. This prevents repeatedly querying the uncached-provider for ids that don't exist.

The downside is that if the uncached-provider suddenly starts returning metadata for an id that previously did not
exist, we won't pick up on it, but the assumption here is that this is no different / worse than cache invalidation
for existing ids that happen to change after we cache them.

* PR suggestion: remove unnecessary test assertion

* tracking entity id translation

* add total and more tests

* indentation

* realign the last test

* fix: sqlserver handle uniqueidentifier uuids

Fixes #46148

Include sqlserver in `uuid-type` handling as its `uniqueidentifier` type
stores uuids.

* Don't be so precise with varchar size

* Add seam for drivers to cast to text type

* Fix arg order

Fixes #47341.

### Description

The QP now supports temporal bucketing of custom expressions, but that has
not been surfaced in the UI until now.

### How to verify

Describe the steps to verify that the changes are working as expected.

1. Use a database other than H2, since it doesn't support time zones.
1. New question -> Anything with a DateTime column
1. Create a custom expression like `convertTimezone([Date Column], "America/New_York", "UTC")`
1. Break out by that expression

The temporal bucketing should work in the UI and in the query results, save properly, etc.

* Implement a simple greedy approximation for dedupe-joins

Co-authored-by: Bryan Maass <bryan.maass@gmail.com>
Co-authored-by: Nicolò Pretto <info@npretto.com>

* Move query-processor.util/remove-lib-uuids to lib.schema.util

This function will soon be needed in lib.schema.util in order to implement distinct order-by clause schema checking.

This function feels like it should live in lib.util instead of lib.schema.util, but that would create the following
circular import dependency since lib.util indirectly requires stuff from lib.schema.util:

  lib.schema.util -> lib.util -> lib.schema.util

* Don't create queries with duplicate order-bys in remove_replace_test.cljc

Otherwise, theses tests would fail soon when we add update the order-by schema to reject duplicates.

* Declare lib.order-by/order-bys to return ::lib.schema.order-by/order-bys

Previously it inlined the schema instead.

* Disallow duplicate order-by clauses in ::lib.schema.order-by/order-bys

Fixes: 39384

Use consistent approach for disabling permission checks in QP to fix sandboxing error in downloads (#47481)

* Snowflake Variant cols should also work in attachments

A previous PR fixed the snowflake variant bug by using a try/catch aroung the global-type-settings function.
I realized there was a second codepath using the same function and it gets used in xlsx attachment code, so I've made
the change there as well to prevent the same error in attachments.

* add a test to confirm the viz-settings-for-col function doesn't throw

* Snowflake Variant Type should not prevent downloads or attachments

Fixes #46981

The :type/SnowflakeVariant key matches 2 methods in `metabase.query-processor.streaming.common/global-type-settings`.

In this case, it seems that the variant type can be any type, so we shouldn't try to guess anything here, except if
the user has provided a :semantic_type, which we can use.

Otherwise, we'll keep going without formatting details about the variant column, which is likely fine already, as it
should result in strings in the export/attachments.

* add a test

* [notification] New method: `channel/can-connect?` (#44955)

* [notification] Channel APIs (#45207)

* [notification] namespaced channel type (#45527)

* [Notification] Render alert for http channel (#45545)

* [notification] Add channel description (#45840)

* [notification] update API to enable http channels for alert (#45839)

* [Notification] Remove channel details for users without write perms (#46034)

* [Notification] Serdes channel (#46031)

* [Notification] Update http details schema (#45960)

* [Notification] Deactivate channels will delete PulseChannel (#46115)

* [Notification] audit log for channel create and update (#46113)

* [Notification] Disallow undefined key for http channel details (#46712)

* [Notification] Handle channel name conflicts (#46818)

* Webhooks Admin Section (#46194)

* [notification] Fix test pulse endpoint does not work properly for http channels (#46474) (#47050)

* [Notification] Fix unable to update multiple channels per type (#47111)

* [Notification] Record Task History when pulse sends channel message (#46218)

* Enabling Webhook Alerts (#47022)

* [Notification] fix cyclic deps (#47379)

* [notification] channel serdes spec (#47386)

Co-authored-by: Nick Fitzpatrick <nick@metabase.com>

* entity id translation + tests

* add api level test

* simplify definition of eid-table->model + add test

* update tests to take keywords

* improve comment

* generate the eid-table->model map

* delete now-obsolete test

* make it work in oss

* put the resulting response into a key, so we can add more information later

* formatting

* use model names without the model/ prefix as keys

* Creates list of `api/model->db-model`

- update keys for util/entity_id request
- update shape of util/entity_id response
- add test for not-found eids

* formatting

* Respond to code review feedback

* entity-ids work on /embed/{card,dashboard}* routes

* make entity id translation work for all tokens

* Merge branch 'static-embed-dashboard-and-question-endpoints' of github.com:metabase/metabase into static-embed-dashboard-and-question-endpoints

* massage schema. add tons of tests

* decode more eids

* test alignment

* don't require ns inside of itself :think:

* make the test functions to create tokens better

* bring back `dashboard-url`

---------

Co-authored-by: Oisin Coveney <oisin@metabase.com>

== Goal ==

Hide attached DWH database details from anyone incl. admins:
* Do not show them in the UI
* Do not permit to change them
* Do not serialize them

The aim is that customers cannot gain access to (parts of) credentials,
and they cannot break a feature they are paying for by changing
connection details.

== Implementation ==

The Metabase backend already contains provisions in the implementation
of `metabase.models.interface/to-json` for `:model/Database` to hide the
`details` of the database in HTTP responses, if the user lacks write
permission on the database.  We utilize this by adding an
`is_attached_dwh` column to the `database` table and rejecting
`metabase.models.interface/can-write?` when this flag is enabled.  In
the "admin" UI, we show a replacement text instead of the edit form when
the flag is set.  (It might be correct to show this whenever `details`
is absent.  See below for possible follow-up work.)

However, several sections of the frontend code expected the `details`
field to always be present.  In order to make `details` optional, as the
backend seems to handle it, we fix the respective code to treat this
case in the way that appears appropriate in the context.

Database details are already generally excluded from H2 dump snapshots
(see `metabase.cmd.copy/*copy-h2-database-details*`), thus nothing
changes there.

== How to test ==

=== New behaviour ===

Setting the `is_attached_dwh` field hides the database details:

1. Configure a database as described in https://www.metabase.com/docs/latest/configuring-metabase/config-file#databases.
   - In addition to the fields you would normally set, also set
     `is_attached_dwh: true`.
   - This also works when adding this flag to a database that previously
     did not have this flag set.
2. Start your Metabase instance.
3. Verify the database shows up in the "admin" section
   (`/admin/databases`).
4. Verify that clicking the database to see its details only reveals
   "This database cannot be modified."
5. Verify that responses from the backend do not include a `details`
   field for this database.

=== Original behaviour ===

Behaviour without setting the `is_attached_dwh` field is unchanged:

1. Configure a database as described in https://www.metabase.com/docs/latest/configuring-metabase/config-file#databases.
   - Only set the fields you would normally set.  Do not set
     `is_attached_dwh` (or set it to `false`).
2. Start your Metabase instance.
3. Verify the database shows up in the "admin" section
   (`/admin/databases`).
4. Verify that clicking the database to see its details only reveal the
   regular edit form, showing connection fields like `host`, `user`,
   etc. with the values you configured.

== How this will be rolled out ==

1. Upgrade existing Metabase Cloud instances with data warehouse to a
   Metabase version that supports `is_attached_dwh`.
2. Set `is_attached_dwh` in the database section of the config file for
   Metabase Cloud instances with a data warehouse.

== Possible follow-up work ==

In https://github.com/metabase/metabase/issues/25715, absent
`database.details` was identified as a bug.  Since then, `details` was
made `NOT NULL` in the application database, so this bug can no longer
occur.  However, today backend responses can be missing the `details`
field, if the current user lacks write permission to the database
setting (see above).  Fully re-evaluating the fix to #25715 in this
context is outside the scope of this PR.

Closes: https://github.com/metabase/harbormaster/issues/5051

These migrations will be backported to v49.

Three changes:

- update IDs/locations of the migrations to v49 vs v51

- add one preCondition (when adding the foreign key constraint)

- add `onFail: MARK_RAN` to the index preconditions. I forgot this
before, so it blows up when the precondition doesn't hold.

* handle cache config overflowed

* fix root too

* do not comment out test

* make cljfmt happy

---------

Co-authored-by: Alexander Solovyov <alexander@solovyov.net>

Exclude inactive tables from the perms cache, and block queries over inactive tables in the QP (#47363)

* Make metric metadata keep the metric's name

This is a fix for the problem reported in #40355, which is fixed separately for v50 and backported to v49.

* Patch broken test

* Normalize dataset-query

* Normalize legacy queries only

* Support multiple bucketings of a column in nested queries

Fixes #46644

* Fix e2e test