Fixes #4755

* perf: [metabase.util.performance] Faster mapv for small collections

* perf: Optimize some query processor functions

* Fix miss

* Another spot, plus clean up

* Always allow localhost:* on CORS

* Use `embedding-app-origin-sdk` method for the headers

* Fix BE code formatting

* Fix BE formatting

* walk the error to remove any visualization settings

Thanks Dan for the patch, this approach is nice because it doesn't require changing all 3 streaming-response-writer
implementations (json csv and xlsx). It keeps behaviour basically the same, but just cleans up the output to not
include potentially sensitive column/dataset details.

* add a test that confirms json/csv errors are cleaned

* walk the error to remove any visualization settings

Thanks Dan for the patch, this approach is nice because it doesn't require changing all 3 streaming-response-writer
implementations (json csv and xlsx). It keeps behaviour basically the same, but just cleans up the output to not
include potentially sensitive column/dataset details.

* add a test that confirms json/csv errors are cleaned

* fix dataset error test

* cljfmt

* try to be a little more surgical with what we take out of the error

* formatting again, oops

* pass export format, only modify the obj for file exports

In the case of errors being saved to files, the obj is a map with a :status :failed key assoc'd by the qp somewhere
along the way. The format-exception cond branch is not used in this case, so we can't rely on modifying
`format-exception` to just dissoc ex-data.

Instead, I've kept the obj modification, but it excludes the query and the preprocessed keys, which are the only 2
keys where we see :viz-settings. This doesn't eliminate the problem of needing to exclude other keys in the future,
but it does improve the surface area - the query and viz settings are the most likely candidates for holding
potentially sensitive info.

I'd prefer to stick with this solution, as we can get a fix for the security concern in quicker, and perhaps a
non-security bug can be opened regarding the most correct way to indicate errors to users whose downloads have failed.

* keep format-exception the same

* cleaner passing of export-format to write-error!

* Support field remapping in pivot queries

* Generate breakouts without inversions

* perf: Optimize reading rows into vectors in JDBC driver

* perf: Use direct interop when computing with-time-zone-same-instant

The way this was implemented, it also hid the section for management
of uploaded CSV files.  Further, admins might want to intentionally
switch from uploading to the data warehouse provided by Metabase to
one of their own.

This reverts commit fbaf58ad.

References: https://github.com/metabase/harbormaster/issues/5121
References: https://metaboat.slack.com/archives/C032LFJFANL/p1725296255637989

* perf: [insights] Optimize last-2 allocations

* perf: [insights] Optimize best-fit allocations when creating validation sample

* perf: [insights] Optimize simple-linear-regression

Come to find out languages like Turkish put the `X` first in "Copy of
`X`"

* API endpoint dox generation should resolve schemas from the registry #46799

* Update Kondo config

* Fix formatting

* Fix typo in MetadataProvider docstring

* Fix typo in caching-test: s/Second/Third/

* Store ::nil markers in CachedMetadataProvider for failed lookups

Store ::nil markers in CachedMetadataProvider for any ids for which the wrapped/uncached upstream provider fails to return
metadatas. This prevents repeatedly querying the uncached-provider for ids that don't exist.

The downside is that if the uncached-provider suddenly starts returning metadata for an id that previously did not
exist, we won't pick up on it, but the assumption here is that this is no different / worse than cache invalidation
for existing ids that happen to change after we cache them.

* PR suggestion: remove unnecessary test assertion

* Bump Kondo version to 2024.08.29

* Fix Kondo warnings from new version of Kondo

* Fix failing tests because schema was broken

* tracking entity id translation

* add total and more tests

* indentation

* realign the last test

* fix: sqlserver handle uniqueidentifier uuids

Fixes #46148

Include sqlserver in `uuid-type` handling as its `uniqueidentifier` type
stores uuids.

* Don't be so precise with varchar size

* Add seam for drivers to cast to text type

* Fix arg order

* Fix visible-columns :selected? flag

* Add comment

* Add comment

* Add comment

* Add comment

* Update src/metabase/lib/js.cljs

Co-authored-by: Braden Shepherdson <braden@metabase.com>

---------

Co-authored-by: Braden Shepherdson <braden@metabase.com>

Fixes #47341.

### Description

The QP now supports temporal bucketing of custom expressions, but that has
not been surfaced in the UI until now.

### How to verify

Describe the steps to verify that the changes are working as expected.

1. Use a database other than H2, since it doesn't support time zones.
1. New question -> Anything with a DateTime column
1. Create a custom expression like `convertTimezone([Date Column], "America/New_York", "UTC")`
1. Break out by that expression

The temporal bucketing should work in the UI and in the query results, save properly, etc.

I still don't want users' eyes to bleed looking at (unnecessary?) stacktraces, but we need to be able to see exception
causes to determine what is going wrong

references #47520

* Implement a simple greedy approximation for dedupe-joins

Co-authored-by: Bryan Maass <bryan.maass@gmail.com>
Co-authored-by: Nicolò Pretto <info@npretto.com>

* Move query-processor.util/remove-lib-uuids to lib.schema.util

This function will soon be needed in lib.schema.util in order to implement distinct order-by clause schema checking.

This function feels like it should live in lib.util instead of lib.schema.util, but that would create the following
circular import dependency since lib.util indirectly requires stuff from lib.schema.util:

  lib.schema.util -> lib.util -> lib.schema.util

* Don't create queries with duplicate order-bys in remove_replace_test.cljc

Otherwise, theses tests would fail soon when we add update the order-by schema to reject duplicates.

* Declare lib.order-by/order-bys to return ::lib.schema.order-by/order-bys

Previously it inlined the schema instead.

* Disallow duplicate order-by clauses in ::lib.schema.order-by/order-bys

Fixes: 39384

Use consistent approach for disabling permission checks in QP to fix sandboxing error in downloads (#47481)