* perf: Implement faster sync methods for postgres

Fixes #48575

Pulls work from redshift into the common postgres driver.

* Fix tests and formatting

* Move nested-field-column sync to sync functions so describe-fields will also get them

* Fix test

* Fix test

* Remove fixed safety test

* Add test specific database-supports feature for pk metadata

* Fix test

* perf: faster mysql sync with describe-fields

Fixes: #49010

* Adrress PR feedback

* Fix tests

* Fix test

* Add nil table-schema

* Don't use subselect for field-comment

* Fix quoting weird identifiers

* Make format string inline

* Update src/metabase/driver/mysql.clj

Co-authored-by: metamben <103100869+metamben@users.noreply.github.com>

* Update src/metabase/driver/mysql.clj

Co-authored-by: metamben <103100869+metamben@users.noreply.github.com>

* Fix tests

* Fix database-type

* Fix tests

* Fix test

* Fix tests

* Exclude mysql table_schema

* Handle tinyInt1IsBit

* Fix test

* Only get one db at a time

---------

Co-authored-by: metamben <103100869+metamben@users.noreply.github.com>

These are helpful for creating literal dates and times in a
cross-platform way. Useful for generating queries, in particular for
expression and filter values.

The ref (usually) contains enough information to unambiguously find the
matching implictly-joined column, even if there are several FKs that
could join the target table.

The remaining ambiguous case is if the *same* FK column exists
multiple times - current MBQL uses raw field IDs for the FK field, so it
can't distinguish between eg. an FK in the main table and the one from a
self-join.

* [streaming-response] Use HttpEndpoint.isOpen() instead of reading a byte from TCP socket

* Comment out cancellation test

* Pivot Measures Order Used in Pivot Exports

Fixes #48442

A pivot table can have any number of measures, and the user can order these by dragging in the UI. Before this PR,
that order was ignored and measures would alway be in index order, which is confusing for any user who needs the
measures to be displayed in a particular order, especially if they've re-ordered them in the pivot viz settings UI.

A test has been added to check that measure order is used.

A few minor changes to the pivot qp and post-processor
 - measure indices are looked up in the pivot qp and added in viz-settings order
 - the pivot measures are only added if the qp has not already added them.
 - pivot-opts Malli spec has been made in the namespace, adjusted to allow `nil` as a valid pivot-opts output, and
 used in relevant functions

* address review points.

* add a rows order test

Also fix a bug in `lib.breakout/existing-breakouts` that treated
breakouts on duplicate columns from different joins as existing, even
though they're not.

* Adding moderation status to Dashboards

* react sins

* verification coll: subselect on card + dashboard

* fix the 3 failing BE tests

* add an api test for dashboard verification

* verified dashboards are never stale

* e2e tests

* code cleanup

* unit test and e2e test cleanup

* PR Feedback

* backend test fixes

* Nick shouldn't touch clojure

* type fix

* indent

* fix last 2 BE tests

---------

Co-authored-by: Bryan Maass <bryan.maass@gmail.com>

In particular, this allows for a nearly-blank query without its
`:database` set.

Previously these were omitted, and it caused invalid SQL to be
generated, since the ORDER BY (time unit not updated) and GROUP BY
(time unit updated) did not line up.

Fixes #49263.

* [WIP] Adjust dependent dashboard cards

* Use transducer to make it less hairy (?)

* Looks better now

* Exception handling and log

* Throw away indices

* Add mapping deletion

* Avoid redundant db roundtrips

* Add test

* Update update-mapping

* Docstring

* Docstrings

* Format

* Comment

* comment

* Shutdown deletion

* Adjust update generationcode

* Add tests

* Comment

* Remove deletion completely for now

* Use transaction

* Update test

* Comment

* Update src/metabase/models/card.clj

Co-authored-by: Braden Shepherdson <braden@metabase.com>

* Update src/metabase/models/card.clj

Co-authored-by: Braden Shepherdson <braden@metabase.com>

* Update src/metabase/models/card.clj

Co-authored-by: Braden Shepherdson <braden@metabase.com>

* Format and parens

* Address remaining remarks

* cljfmt

* Update src/metabase/models/card.clj

Co-authored-by: Braden Shepherdson <braden@metabase.com>

* Address review remarks

---------

Co-authored-by: Braden Shepherdson <braden@metabase.com>

Fixes https://github.com/metabase/metabase/issues/14437

* perf: Implement faster sync methods for postgres

Fixes #48575

Pulls work from redshift into the common postgres driver.

* Fix tests and formatting

* Move nested-field-column sync to sync functions so describe-fields will also get them

* Fix test

* Fix test

* Remove fixed safety test

* Add test specific database-supports feature for pk metadata

* Fix test

* Adrress PR feedback

* Fix test

* Don't use subselect for field-comment

* Fix quoting weird identifiers

* Make format string inline

* Mongo objects should download as JSON, not EDN

Fixes #48198

Prior to this change, object columns (base or effective type of :type/Dictionary) were just formatted with `(str
value)` which results in a csv or json download containing EDN formatted objects.

This is a bug because we present object column values as json in the app, so the expected formatting of the download
should match this.

The formatter function now takes this type into account. As well, since this is a type of formatting that should be
always applied (even when format_rows is false), the function is modified to unconditionally apply the json/encode
formatting to dictionary types when encountered.

* add a test

* add proper condition to test

* card-download should be public

* uncomment json encoding formatter

* set-cell! should keep encoded json string for Objects

I think this is the correct change; I don't really understand the reason for wrapping, encoding, decoding, and then
string-ing that value. Maybe I'm missing something.

* Adjusted xlsx Object set-cell! implementation

* forgot the not... inverted

Co-authored-by: bryan <bryan.maass@gmail.com>

* Fix csp directives for embed previews

We set content security directives to allow for iframes on
dashboards. This list did not include 'self' so we can't actually host
an iframe pointing at our, well, self.

Embed previews work by just embedding an iframe with the dashboard and
this breaks if we don't allow iframes from our self.

* e2e test

---------

Co-authored-by: Aleksandr Lesnenko <alxnddr@gmail.com>

### Description

This PR updates Macaw to the latest version, which exposes a new analyzer which handles compound queries.

It also finishes the migration to using tagged-union result maps consistently across all the interfaces.

This should also remove the atrociously verbose analysis error messages in CI.

This standards on a "just maps" approach to returning disabled, disallowed, or failed query analysis. This is as opposed to the hodgepodge approach of significant `nil`, untagged unions with keywords, and exceptions.

Essentially we have a classic "either" style tagged union, where `:error` is the tag.

* upgrades is-airgapped? defn to defsetting airgap-enabled

- uses the airgap-enabled settings value to remove Cloud section in
  SettingsEditor
- "Metabase Enterprise Airgap" -> "Enterprise Airgap"

* hide still need help section in airgap mode

* move flag checking into selectors

* fe lint

* ts type check

* use optional chaining and nullish coalescing

* hide StillNeedHelp in ag mode

* adding airgap-enabled to createMockSettings

---------

Co-authored-by: Nick Fitzpatrick <nickfitz.582@gmail.com>

* Format databricks as spark in prettify-native-form

* Add test to be generalized in follow up

Co-authored-by: Ryan Laurie <iethree@gmail.com>

* add allowed iframe host setting wip

* use allowed-iframe-hosts setting in the CSP header

* add a test for the frame-src csp directive

* Update allowed-iframe-hosts setting definition

* Add error state for forbidden iframe url domains

* Move out iframe e2e test suite

* Add e2e test

* Update error message in view mode

* Fix unit tests

* Update setting on the admin page

* Update error state

* Add links to error states

* Update docs links

* Update link anchors

* Add default allowed hosts to public setting

* Update allowed domain check logic

* Fix default value display in admin page

* Don't update setting without changes

* Update error message spacing

* correct the parsing of allowed-hosts string for CSP header entries

* fix test

* fix not handling wildcard ports

* Fix failing e2e test

* Fix subdomain test

* address review

 - the parse-allowed-iframe-hosts fn is now memoized
 - a * entry is handled and doesn't produce a weird *:* entry
 - no more try/catch, errors in parsing will be logged but the list returns all valid entries
 - when www. is encountered, an entry including www. is added
 - trailing / is 'cleaned' and the entry is used as if there was no trailing /

* Fixup test for expecting a few more frame sources

* indentation fix for linter 



* Fix type error

---------

Co-authored-by: Adam James <adam.vermeer2@gmail.com>
Co-authored-by: Anton Kulyk <kuliks.anton@gmail.com>
Co-authored-by: dan sutton <dan@dpsutton.com>

* Do not create personal collections for API keys

Co-authored-by: Noah Moss <32746338+noahmoss@users.noreply.github.com>

This is to allow for the possibiliy of adding more metricsv2 usage and error metrics in the future while still
distinguishing the source of the errors.

### Description

This is a low level step towards applying table-level permissions to native queries.

It replaces the more generic and powerful `:ast-walker-1` analyzer with the more conservative `:basic-select` version we recently landed. The advantage of this version is that it protects against dynamic table references and false negatives due to masking.

This was throwing errors trying to `nippy/freeze!` an Atom.

That was because in certain circumstances a pMBQL query can end up in
`(:json_query metadata)` for a query, and those can contain an atom
due to `metabase.lib.cache` and both `CachedMetadataProvider` and
`InvocationTracker`.

I added `lib.query/serializable` as a single location to strip out
anything that isn't serializable, such as atoms. (And the entire
`:lib/metadata`, which is a wasted effort to serialize.)