QP: use MLv2 metadata provider. Eliminate 50% of app DB calls and improve performance by 10%+ (#33221)

* QP use MLv2 metadata provider (34/2)

* (22/2)

* (21 / 0)

* (20 / 4)

* (7 / 2)

* (2/0)

* Cleanup; (3/0)

* Last two test fixes

* (36 / 2)

* (8 / 1)

* Reorder stuff

* Test fixes 

* Test fixes

* Some test fixes

* More test fixes 

* Test fix 

* MongoDB test fix

* B I G  cached metadata provider performance improvements

* Revert breaking change

* Fix Kondo

* Make sure application database metadata provider returns Database :features

* Test fix 

* Parallel tests for QP macroexpansion middleware

* `with-current-user` is fine in parallel tests.

* Add test util remap metadata providers and rework remap middleware to use mock MLv2 data

* Address PR feedback

* Improve unrelated flaky test

* Some test improvements

* Fix lint error

* MLv2-ize `nest-query-test`

* Fix typo in PR feedback changes

* More parallelization and test fixes 

* Convert lots more tests to mock metadata

* Test fix? 

* Fix Card update logic

* Another Card update logic fix 

* Another round of fixes 

* Hopefully no more test fixes 

* Evil test fixes 

* Test fix 1

* Test fix

* Add new setting reset-token-ttl-h. Expose in selectors (#12829)

* Add reset-token-ttl-h setter, fix call in reset-token-ttl-ms. Add additional reset-token-ttl-h tests. (#12829)

* Revert previous changes. Define reset-token-ttl-hours and reset-token-ttl-hours-test (#12829)

* Fix formatting errors (#12829)

* Parallel QP tests; linter improvements; move QP test fns

* Cleanup

* Move query->expected-cols

* Fix QP test namespace

* Test fix 

* Fix merge issues

* Revert unrelated changes

* Test fix 

* Fix some things that are actually ^:parallel safe

* Remove Oracle logic for destroying DBs from different sessions since we don't test on AWS anymore

* Use ReentrantReadWriteLock for getting + creating test datasets to avoid race conditions

* Add non-`metabase.test` versions of things to the disallowed list for `^:parallel` tests

* Add recursive capabilities to segment expansion

* Add test

* Add error type

* Update `nested-segments-test`

* Remove redundant `Database` and `Table`

* Convert use of `with-temp*` to `t2.with-temp`

Hide columns from SQL Query such that they will take effect in dashboard subscription email/slack (#32991)

* initial columns

* fix tests

* only filter when exists

* add test

* add whitespace

* add noah code

* fix test

* add column reordering

* fix issues with column ordering

* rename test

* Update src/metabase/pulse/render/body.clj

Co-authored-by: Noah Moss <32746338+noahmoss@users.noreply.github.com>

---------

Co-authored-by: Noah Moss <32746338+noahmoss@users.noreply.github.com>

* [MLv2] Include temporal-bucketing on selected join condition columns

* Update test/metabase/lib/join_test.cljc

Co-authored-by: metamben <103100869+metamben@users.noreply.github.com>

---------

Co-authored-by: metamben <103100869+metamben@users.noreply.github.com>

* MLv2 cleanup: import more lib.metadata.calculation stuff; more test helpers

* Update QP code

* Test fix 

* Fix bad :require

* Fix bad :require

[Fixes #33193]

* Remove obsolete CLI flags for `export`; fix collection specification

[Fixes #33168]

* Allow comma-separated IDs

* Improve sql parameter mismatch error message

It is possible to write native sql queries that try to hide parameters
within comments or identifiers. Unfortunately PreparedStatement will
almost always explode in these cases since you can only parameterize
values in sql caluses. The error message of various drivers is quite
poor, so we add a heuristic to look for these poor messages and replace
it with a message prompting the user to look for a misuse of parameters.

* Address review comments

* Exclude drivers that do not support getParameterCount

* Remove `:metabase.lib.*` keys from field refs when converting refs to legacy

* Fix hardcoded database ID

* Normalize Formatted strings on \space char.

Some JVMs use nbsp chars in their formatted strings. I noticed weird test failures that printed the same because of
this and it was confusing. So I propose a simple string replace to normalize on one type of space char only.

* Change use of strings in str/replace to chars to pass the linter

Porting fix alias of joined column LHS of standard join conditions (#32987)

* Check if the LHS has the alias of the current join before stripping its alias.
* Remove dependent joins when removing a join

* Simplified impl.

* Fix kondo errors

* Cljs test fix 

* Search by table description

[Fixes #25792]

Fixes #33080

* Test metadata helper function consolidation

* More code consolidation

* Unify semicolon and comment handling in nested native queries

* Use different comment and semicolon removal approach

* Apply suggestions from code review

Co-authored-by: metamben <103100869+metamben@users.noreply.github.com>

* Update base and effective type verification for oracle

* Avoid running test with snowflake and bigquery

* Update `breakout-results` and test

1. In `breakout-results` case driver statement was redundant.
Original code had field name and types hardcoded. During further
examination, I've realized name and types are available thanks
to the use of `qp.test/col`, so are just "unhardcoded" :)

2. `card-id-native-source-queries-test` is altered, so instead of
hardcoding native sub-queries, roughly equivalent* mbql query is
compiled and its native form is used. This way I can hopefully avoid
problems with extraction of cloud databases' table names for testing
purposes.

* By roughly equivalent I mean instead of using * for selecting
fields, mbql is compiled selecting fields explicitly. Also limit is added
in some cases, but local testing showed no problems with that.

---------

Co-authored-by: metamben <103100869+metamben@users.noreply.github.com>

Modify `:contains`, `:starts-with` and `:ends-with` so field or expression can be used as the second argument (#32446)

* Update `LIKE` to handle fields and expressions

Previously only string could be used with like clause, ie. as second
argument of `:starts-with`, `:ends-with` or `:contains`. This commit modifies
translation of those functions, so fields or expressions can be
used in place of second argument.

* Make only expression or field args use `concat`

Using concat with values unnecessarily complects generated sql.

* Update tests

* Extend mongo string matching capabilities

Make mongo filter compilation use `$regexMatch` for `:contains`,
`:starts-with` and `:ends-with` so expressions and fields
can be matched against.

* Adjust mongo specific tests

* Update druid

Druid's filters work with `:contains`, `:starts-with` and `:ends-with`,
but only if second argument is literal pattern. This commit updates
existing code, so expection is thrown if `pattern` has different form
than `[:value ...]`.

* Address review comments - sql

* Address review comments - mongo

* Use mbql `:concat` instead of `hx/call :concat`

Vertica, orcacle and sqlite handle concat differently than other
drivers. To overcome this limitation, `pre`, `post` and `arg` are
wrapped into mbql's concat expression which is then translated
with regards to the specifics of particular driver.

* Update according to the review feedback

---------

Co-authored-by: metamben <103100869+metamben@users.noreply.github.com>

* Support segments

Fixes #32409.

* Fix fallback long display name for Cards

* Test updates

* Fix #32920

* Remove stray println

* Make `:branches` realize during `:case` compilation

Previously `:branches` were generated with use of `for`,
yeilding lazy sequence. It caused problems, if the sequence
got realized after processing of query ie. without initialized
qp store. That was the case with /dataset/native api
call containing `:case` with `:now`.

* Update /dataset/native test

---------

Co-authored-by: metamben <103100869+metamben@users.noreply.github.com>

* Fix for login history, the fix was written by Cam

* kondo

* Fully hydrate PUT-ed collections, which the FE depends on

[Fixes #30235]

* Add repro test for #30235

* Set token features properly

---------

Co-authored-by: Denis Berezin <denis.berezin@metabase.com>

* QP: use Malli

* Fix schema names

* Fix missing namespaces

* Fix busted stuff

* Simplify the new schema

* Test fix 

* Test fixes 

* MongoDB fixes 

* FAST mu/defn

* Malli registry

* Misc fixes

* Some cleanup

* Misc cleanup and test fixes 

* mu/disable-enforcement

* Test fix

* Fix Cljs version of `disable-enforcement`

* Misc improvements

* Update tests

* Fix Kondo warnings

* Fix Kondo warnings

* Fix busted Malli schema

* PR feedback

* Fix busted schema

* Kondo fixx

* Test fixes 

* Use Malli version of `describe` now that PR was merged

* Work around https://github.com/metosin/malli/issues/924

* Test fixes 

* Improvements

* Test fix