* Fix visible-columns :selected? flag

* Add comment

* Add comment

* Add comment

* Add comment

* Update src/metabase/lib/js.cljs

Co-authored-by: Braden Shepherdson <braden@metabase.com>

---------

Co-authored-by: Braden Shepherdson <braden@metabase.com>

Fixes #47341.

### Description

The QP now supports temporal bucketing of custom expressions, but that has
not been surfaced in the UI until now.

### How to verify

Describe the steps to verify that the changes are working as expected.

1. Use a database other than H2, since it doesn't support time zones.
1. New question -> Anything with a DateTime column
1. Create a custom expression like `convertTimezone([Date Column], "America/New_York", "UTC")`
1. Break out by that expression

The temporal bucketing should work in the UI and in the query results, save properly, etc.

I still don't want users' eyes to bleed looking at (unnecessary?) stacktraces, but we need to be able to see exception
causes to determine what is going wrong

references #47520

* Implement a simple greedy approximation for dedupe-joins

Co-authored-by: Bryan Maass <bryan.maass@gmail.com>
Co-authored-by: Nicolò Pretto <info@npretto.com>

* Move query-processor.util/remove-lib-uuids to lib.schema.util

This function will soon be needed in lib.schema.util in order to implement distinct order-by clause schema checking.

This function feels like it should live in lib.util instead of lib.schema.util, but that would create the following
circular import dependency since lib.util indirectly requires stuff from lib.schema.util:

  lib.schema.util -> lib.util -> lib.schema.util

* Don't create queries with duplicate order-bys in remove_replace_test.cljc

Otherwise, theses tests would fail soon when we add update the order-by schema to reject duplicates.

* Declare lib.order-by/order-bys to return ::lib.schema.order-by/order-bys

Previously it inlined the schema instead.

* Disallow duplicate order-by clauses in ::lib.schema.order-by/order-bys

Fixes: 39384

Use consistent approach for disabling permission checks in QP to fix sandboxing error in downloads (#47481)

fixes #46039

* perf: Use optimized stats functions for computing insights

* perf: Rewrite fingerprinters/with-error-handling to not generate closures

* fix: Don't reuse global-fingerprinter object

* Snowflake Variant cols should also work in attachments

A previous PR fixed the snowflake variant bug by using a try/catch aroung the global-type-settings function.
I realized there was a second codepath using the same function and it gets used in xlsx attachment code, so I've made
the change there as well to prevent the same error in attachments.

* add a test to confirm the viz-settings-for-col function doesn't throw

* Snowflake Variant Type should not prevent downloads or attachments

Fixes #46981

The :type/SnowflakeVariant key matches 2 methods in `metabase.query-processor.streaming.common/global-type-settings`.

In this case, it seems that the variant type can be any type, so we shouldn't try to guess anything here, except if
the user has provided a :semantic_type, which we can use.

Otherwise, we'll keep going without formatting details about the variant column, which is likely fine already, as it
should result in strings in the export/attachments.

* add a test

The Portal diff view wants `[before after]` pairs of the original data,
not the three-part output of `clojure.data/diff`. This fixes the
`qp.debug/debug>` output from the preprocessing middleware to emit
correct diffs.

Also, since the diffs are not useful when the query is converted between
legacy and pMBQL, this also attaches the *converted* input query to the
output query's Clojure metadata, and this converted value is used as the
basis for the diff.

* [notification] New method: `channel/can-connect?` (#44955)

* [notification] Channel APIs (#45207)

* [notification] namespaced channel type (#45527)

* [Notification] Render alert for http channel (#45545)

* [notification] Add channel description (#45840)

* [notification] update API to enable http channels for alert (#45839)

* [Notification] Remove channel details for users without write perms (#46034)

* [Notification] Serdes channel (#46031)

* [Notification] Update http details schema (#45960)

* [Notification] Deactivate channels will delete PulseChannel (#46115)

* [Notification] audit log for channel create and update (#46113)

* [Notification] Disallow undefined key for http channel details (#46712)

* [Notification] Handle channel name conflicts (#46818)

* Webhooks Admin Section (#46194)

* [notification] Fix test pulse endpoint does not work properly for http channels (#46474) (#47050)

* [Notification] Fix unable to update multiple channels per type (#47111)

* [Notification] Record Task History when pulse sends channel message (#46218)

* Enabling Webhook Alerts (#47022)

* [Notification] fix cyclic deps (#47379)

* [notification] channel serdes spec (#47386)

Co-authored-by: Nick Fitzpatrick <nick@metabase.com>

* entity id translation + tests

* add api level test

* simplify definition of eid-table->model + add test

* update tests to take keywords

* improve comment

* generate the eid-table->model map

* delete now-obsolete test

* make it work in oss

* put the resulting response into a key, so we can add more information later

* formatting

* use model names without the model/ prefix as keys

* Creates list of `api/model->db-model`

- update keys for util/entity_id request
- update shape of util/entity_id response
- add test for not-found eids

* formatting

* Respond to code review feedback

* entity-ids work on /embed/{card,dashboard}* routes

* make entity id translation work for all tokens

* Merge branch 'static-embed-dashboard-and-question-endpoints' of github.com:metabase/metabase into static-embed-dashboard-and-question-endpoints

* massage schema. add tons of tests

* decode more eids

* test alignment

* don't require ns inside of itself :think:

* make the test functions to create tokens better

* bring back `dashboard-url`

---------

Co-authored-by: Oisin Coveney <oisin@metabase.com>

* Query Validator FE

* collection path, unit tests

* wrapping feature with token flag

* updating util function, adding row type

* updating session_test.clj

* type adjustment

* fixing other table sorting

* Empty state, clean up utils

* unit test adjustment

* e2e adjustment

== Goal ==

Hide attached DWH database details from anyone incl. admins:
* Do not show them in the UI
* Do not permit to change them
* Do not serialize them

The aim is that customers cannot gain access to (parts of) credentials,
and they cannot break a feature they are paying for by changing
connection details.

== Implementation ==

The Metabase backend already contains provisions in the implementation
of `metabase.models.interface/to-json` for `:model/Database` to hide the
`details` of the database in HTTP responses, if the user lacks write
permission on the database.  We utilize this by adding an
`is_attached_dwh` column to the `database` table and rejecting
`metabase.models.interface/can-write?` when this flag is enabled.  In
the "admin" UI, we show a replacement text instead of the edit form when
the flag is set.  (It might be correct to show this whenever `details`
is absent.  See below for possible follow-up work.)

However, several sections of the frontend code expected the `details`
field to always be present.  In order to make `details` optional, as the
backend seems to handle it, we fix the respective code to treat this
case in the way that appears appropriate in the context.

Database details are already generally excluded from H2 dump snapshots
(see `metabase.cmd.copy/*copy-h2-database-details*`), thus nothing
changes there.

== How to test ==

=== New behaviour ===

Setting the `is_attached_dwh` field hides the database details:

1. Configure a database as described in https://www.metabase.com/docs/latest/configuring-metabase/config-file#databases.
   - In addition to the fields you would normally set, also set
     `is_attached_dwh: true`.
   - This also works when adding this flag to a database that previously
     did not have this flag set.
2. Start your Metabase instance.
3. Verify the database shows up in the "admin" section
   (`/admin/databases`).
4. Verify that clicking the database to see its details only reveals
   "This database cannot be modified."
5. Verify that responses from the backend do not include a `details`
   field for this database.

=== Original behaviour ===

Behaviour without setting the `is_attached_dwh` field is unchanged:

1. Configure a database as described in https://www.metabase.com/docs/latest/configuring-metabase/config-file#databases.
   - Only set the fields you would normally set.  Do not set
     `is_attached_dwh` (or set it to `false`).
2. Start your Metabase instance.
3. Verify the database shows up in the "admin" section
   (`/admin/databases`).
4. Verify that clicking the database to see its details only reveal the
   regular edit form, showing connection fields like `host`, `user`,
   etc. with the values you configured.

== How this will be rolled out ==

1. Upgrade existing Metabase Cloud instances with data warehouse to a
   Metabase version that supports `is_attached_dwh`.
2. Set `is_attached_dwh` in the database section of the config file for
   Metabase Cloud instances with a data warehouse.

== Possible follow-up work ==

In https://github.com/metabase/metabase/issues/25715, absent
`database.details` was identified as a bug.  Since then, `details` was
made `NOT NULL` in the application database, so this bug can no longer
occur.  However, today backend responses can be missing the `details`
field, if the current user lacks write permission to the database
setting (see above).  Fully re-evaluating the fix to #25715 in this
context is outside the scope of this PR.

Closes: https://github.com/metabase/harbormaster/issues/5051

* Add `clojure -M:kondo` and `clojure -M:kondo:kondo/all` and bump version

* Fix Kondo errors

* Fix Kondo+LSP issues with `defendpoint`, `defenterprise`, etc.

* Use replace-deps instead of deps for speed

* Ok apparently maybe we do need to copy configs when we run Kondo on CI

* Oops `./bin/kondo.sh` should not try to use `clj-kondo`

* Remove references to GA driver folders

Exclude inactive tables from the perms cache, and block queries over inactive tables in the QP (#47363)

* Make metric metadata keep the metric's name

This is a fix for the problem reported in #40355, which is fixed separately for v50 and backported to v49.

* Patch broken test

* Normalize dataset-query

* Normalize legacy queries only

* Support multiple bucketings of a column in nested queries

Fixes #46644

* Fix e2e test

* Adding schema name to recents, search results in CP

* adding unit test

* fixup tests + clarify recent item schema

---------

Co-authored-by: Bryan Maass <bryan.maass@gmail.com>

Previously, `visible-collection-ids` was effectively "free" in that we'd
cached `collection-id->collection` for *all* collections, within a
single request, and then locally filtered it for permissions without
needing to hit the database again.

To be honest, there is probably a better fix for this - we're repeatedly
calling `visible-collection-ids` when we probably could just save a
single copy of it and use it when calculating the effective location of
every collection.

However, this is a very quick and low-risk fix, and I want to prioritize
getting this done, and then we can improve it later.

Locally, I copied down the database from stats and timed the
`/api/search?model_ancestors=true` endpoint.

Before my "speedup" PR (https://github.com/metabase/metabase/pull/46942)
it took ~7 seconds to return results.

After my "speedup" PR, it took ~15s to return results. :grimace:

With this change, it takes 818ms to return results. 

* zoom-in-timeseries-drill: do not include :hour and :minute for DATE columns

When the user attempts to drill through a DATE column, we should not return :hour or :minute drill throughs from
available-drill-thrus. Previously we did, which resulted in an error being displayed when the user attempted to select
them.

Modify next-breakout-unit in zoom_in_timeseries.cljc to not return :hour or :minute units for :type/Date columns,
which prevents these from being returned by available-drill-thrus, which prevents them appearing in UI context menus.

* Use lib.temporal-bucket/available-temporal-buckets

Per review feedback, use lib.temporal-bucket/available-temporal-buckets to get the valid-current-units in
metabase.lib.drill-thru.zoom-in-timeseries.

* Do not construct zipmap just to lookup the next unit

Rather than constructing a zipmap from `valid-current-units', which requires iterating over the seq twice, just
traverse the seq once and return the next item in the seq (if any).

Constructing a zipmap made sense in the old code because it was constructed once from static data and used many times
for lookup. In the new version, the vec of `valid-current-units` is dynamic and dependent on the type of the breakout.

* PR suggestion: use threading macro rather than nested sexps

Co-authored-by: Braden Shepherdson <braden@metabase.com>

* PR suggestion: use threading macro rather than transducer / eduction

* PR suggestion: reduce boilerplate in zoom-in-timeseries tests

Reduce code dup by looping over aggregation and bucketing options rather than having a separate deftest for each.

---------

Co-authored-by: Braden Shepherdson <braden@metabase.com>
Fixes #39366

Analyze queries synchronously on save

* entity id translation + tests

* add api level test

* simplify definition of eid-table->model + add test

* update tests to take keywords

* improve comment

* generate the eid-table->model map

* delete now-obsolete test

* make it work in oss

* put the resulting response into a key, so we can add more information later

* formatting

* use model names without the model/ prefix as keys

* Creates list of `api/model->db-model`

- update keys for util/entity_id request
- update shape of util/entity_id response
- add test for not-found eids

* formatting

* Respond to code review feedback

---------

Co-authored-by: Oisin Coveney <oisin@metabase.com>