* docstring to highlight partial graph acceptance

- updates how we decode the incoming graph

* sort ns

* Update src/metabase/api/collection.clj

Co-authored-by: Noah Moss <32746338+noahmoss@users.noreply.github.com>

* code review responses

Co-authored-by: Noah Moss <32746338+noahmoss@users.noreply.github.com>

The new macro is called defendpoint. The previous defendpoint is now defendpoint-schema, as per @camsaul's idea. It'll make it easier to realize that we should use the Malli version of defendpoint going forward.

* adds malli defendpoint + linting + tests

- I decided to add a new defendpoint macro -- it is nearly exactly the
  same as `defendpoint`. If the duplication is an issue I can handle that too.

* malli-defendpoint -> defendpoint-malli

* defendpoint -> defendpoint-schema

* defendpoint-malli -> defendpoint

* fine tuning error messages + comments

- fix kondo config

* put back dox-for-plumatic

* fix name collision

* adds local malli description ns + test

* update alias in test

* linting fix

* linting fix pt. 2

* hook up umd/describe

It seems the expectation was that the binding *execute-asynchronously* to false
in the test would affect the binding seen by the server. Instead of this a new
parameter is added to the request that makes the server return only when the
asynchronous operation has completed.

* adds mu/defn, which behaves like s/defn but for malli schemas

- includes link to malli.io with your schema and type.
- making room for some namespaces

* require and use malli.destructure in mu/defn

* fix require alias

* clean ns

* fix linter errror

* fix linter error again

* standardize ns formatting in all src/ namespaces

* small fixes

* test formatting

* enterprise files

* fix whitespace linter error

* fix kondo error

* bin files

* reformat shared files

* fix errors

* fix another error

* mostly fix import to use parenthesis and have it at a new line

* fix wrong spacing

Co-authored-by: Ngoc Khuat <qn.khuat@gmail.com>

* Use gvenzl/oracle-xe docker to test Oracle
* Configure SSL connection for Oracle 21.3
* Fix secret handling

* Switch back to main fork of `clojure.java-time` and bump version

* Run uberjar build steps in one command (faster CI)

* Fix `truncate-to` for new version of `clojure.java-time`

Co-authored-by: Alexander Polyankin <alexander.polyankin@metabase.com>

* Merge follow-up and abandonment emails

* Update the call to action

* Remove unused requires

* Remove all traces of abandonment emails

* Remove unused require

* source -> values_source for parameter

* Rename FE source type & options

* ParameterSourceOptions -> ValuesSourceConfig

Co-authored-by: Alexander Polyankin <alexander.polyankin@metabase.com>

* Handle abbreviation and date-separator in a fn, where defaults are passed in in any place where nil COULD exist

Moving abbreviation and date-separator string changes into a post-processing fn allows each unit case to handle the
date-style default formatting, and eliminates the problem of date-style being `nil` in the let statement. It was this
scenario where `abbreviate` was `true`, but `date-style` was `nil`.

In many cases, if `date-style` is nil, then so is `abbreviate` and `date-separator`, in fact, the default from the
frontend sends nil for all of these keys. In such a case, none of the branches run and nil is safely passed
through. It is then handled appropriately with `(or ...)` expressions in each entry for the case statement.

But, it is indeed possible to have a nil `date-style` with non-nil `abbreviate` or `date-separator` keys. Now, the
post-process-date-style function is only ever called AFTER the default date-style is set in whatever case branch is needed.

* Turn frontend day format 'dddd' into backend format 'EEEE'

The frontend sends a datetime format string to the backend and some elements are not equivalent to the expectations
the java datetime formatter has.

* Unskip e2e reproductions

* Add a test for all 'written date' combos

* These tests fail b/c I fixed the 'default' from Abbreviated to non-abbreviated

* Add test 'matrix' for numerically formatted dates, and each separator

* Finally, add tests to confirm that custom-formatting is applied

* Add datetime-diff for h2

* Set supports? to true

* Refactor h2 datetime-diff

* Formatting

* Fix typo in log.clj

formating -> formatting

* Update src/metabase/server/middleware/log.clj

som -> some

Co-authored-by: flamber <1447303+flamber@users.noreply.github.com>

Co-authored-by: flamber <1447303+flamber@users.noreply.github.com>

Without this conversion the data ends up as an EDN string in the
usage_stats table and that's not easily accessible in PostgreSQL.

Refactor datetime-diff implementations: sqlserver, mysql, bigquery, redshift, snowflake, vertica, postgres (#27262)

* Refactor bigquery datetime-diff to use multimethod

* Refactor snowflake datetime-diff to use multimethod

* Refactor SQL server datetime-diff to use multimethod

* Refactor mysql datetime-diff to use multimethod

* Refactor redshift datetime-diff to use multimethod

* Simplify vertica datetime-diff

* Tidy postgres datetime-diff

* ; -> ;; in snowflake comment

* Add to bigquery comment

* Reduce the number of rows read when syncing mongo

Also prefer recent records to old ones, as these better represent
the current schema.

* Add database-type function

* Add datetime-diff multimethod and default impl

* Refactor postgres

* Refactor vertica

* Remove accidentally left comment

* Tidy datetime-diff-check-args

* Add to datetime-diff-check-args docstring

* Add multimethod to changelog

* Add to docstrings

* Add `:added` metadata to `datetime-diff`

Co-authored-by: Cam Saul <1455846+camsaul@users.noreply.github.com>

Co-authored-by: Cam Saul <1455846+camsaul@users.noreply.github.com>

* Replace str/lower-case with u/lower-case-en

* Replace str/upper-case with u/upper-case-en

* Replace more str/lower-case

* BE endpoint to clear group membership

* update ee group manager tests with clear-memberships

* make auth settings adminOnly on FE

* WIP backend changes

* Update relevant settings

* address lint warning

* new tests, and a bunch of reorganization

* remove broken EE settings test, and fix validation in google & ldap endpoints

* fix cypress tests

* add clarifying table to setting docs

* Relieve db pressure on api/health check

https://github.com/metabase/metabase/issues/26266

Servers under heavy load can be slow to respond to the api/health
check. This can lead to k8s killing healthy instances happily humming
along serving requests.

One idea floated was to use QoSFilters
https://www.eclipse.org/jetty/javadoc/jetty-9/org/eclipse/jetty/servlets/QoSFilter.html
to prioritize those requests in front of others. But I suspect this
might not be our bottleneck.

Our health endpoint was updated to see if it could acquire an endpoint
when we were dealing with connection pool issues. We were reporting the
instance was healthy once it has finished the init process, but would
report healthy if 60/15 app-db connections were used and no actual
queries could complete.

The remedy was adding
`(sql-jdbc.conn/can-connect-with-spec? {:datasource (mdb.connection/data-source)})`
to the endpoint. But now to get information about the health of the
system we have to wait in the queue to get a datasource.

The hope is that this change which monitors for recent db
checkins (query success) and checkouts (query begun) can be a proxy for
db activity without having to wait for a connection and hit the db ourselves.

Some simple and crude benchmarking:
- use `siege` to hit `api/database/<app-db>/sync_schema`
- in a separate tab, use `siege` to hit `api/health`

Three trials with unconditional db access and conditional db
access (look for recent activity set by the new `ConnectionCustomizer`).

One siege client is synching the app-db's schema with 80 clients each
sending 60 requests. the other has 1 client sending 60 requests to api/health.

Run             |  Elapsed Time | max tx  | tx rate
 before change  |    7.16s      |  0.79s  |  8.38 tx/s
 before change  |   23.91s      |  1.44s  |  2.51 tx/s
 before change  |   13.00s      |  0.50s  |  4.62 tx/s
----------------------------------------------------
 after change   |    4.46s      |  0.27s  |  13.45 tx/s
 after change   |    5.81s      |  0.61s  |  10.33 tx/s
 after change   |    4.54s      |  0.44s  |  13.22 tx/s

Full(er) results below:

```
Unconditional db access
=======================

siege -c80 -r 40 "http://localhost:3000/api/database/2/sync_schema POST" -H "Cookie: $SESSION"

siege -c 1 -r 60 "http://localhost:3000/api/health"

Elapsed time:		        7.16 secs
Response time:		        0.12 secs
Transaction rate:	        8.38 trans/sec
Longest transaction:	        0.79
Shortest transaction:	        0.01

Elapsed time:		       23.91 secs
Response time:		        0.40 secs
Transaction rate:	        2.51 trans/sec
Longest transaction:	        1.44
Shortest transaction:	        0.02

Elapsed time:		       13.00 secs
Response time:		        0.22 secs
Transaction rate:	        4.62 trans/sec
Longest transaction:	        0.50
Shortest transaction:	        0.06

Conditional db access
==============================================================

Elapsed time:		        4.46 secs
Response time:		        0.07 secs
Transaction rate:	       13.45 trans/sec
Longest transaction:	        0.27
Shortest transaction:	        0.01

Elapsed time:		        5.81 secs
Response time:		        0.10 secs
Transaction rate:	       10.33 trans/sec
Longest transaction:	        0.61
Shortest transaction:	        0.00

Elapsed time:		        4.54 secs
Response time:		        0.08 secs
Transaction rate:	       13.22 trans/sec
Longest transaction:	        0.44
Shortest transaction:	        0.01
```

* Remove reflection in `.put` call (not the reflections trategy)

also remove the call to `classloader/the-classloader` as it did nothing

* Comment and settle on a single method

* tests

* select from db twice

had a failure in CI. give it time to do its thing with another db call

* block to wait for timestamp update?

* unflake the tests

tasks and events from outside the thread can hit the db. the
ConnectionCustomizer is also run from c3p0 controlled threads so we
can't easily isolate everything to our thread

Was running

```clojure
(comment
  (dotimes [n 5]
    (dotimes [_ 100]
      (recent-activity-test)
      (CheckinTracker-test))
    (println (* (inc n) 100)))
    )
```

to run the tests 500 times and would keep getting flakes at a rate
~1/100 to 1/500. Just frustration for the future.

* typehint

* Switch it up a bit

Tests were flaking in h2 and I don't know why. I'm switching to just
updating recent activity on most methods.

* Update test

* Update implementations

Updates embedded H2 driver to v2.1.212. When a local H2 v1 file-based application database is detected, it will be
automatically updated in-place at startup. Migrations and tests have been updated to account for changes in H2
behavior. Sample data and test harness database files in the repo have all been updated to H2 v2 file format.

Co-authored-by: Braden Shepherdson <braden@metabase.com>
Co-authored-by: Cam Saul <github@camsaul.com>
Co-authored-by: Cam Saul <1455846+camsaul@users.noreply.github.com>

* Remove more misc unused backend stuff

* Remove unused requires

* Remove more misc unused stuff

* Ok, name field is still used

* Remove `sql.qp` stuff deprecated in 41 and 42

* Remove old test

* We still need to import `FieldInstance`

* Remove unused `metabase.models.table` `metrics` and `segments` functions

* Remove more unused

* Add quarter for postgres

* Add quarter to schema

* Add quarter tests

* Add quarter for mysql

* Add quarter for bigquery

* Add quarter for snowflake

* Add quarter for redshift

* Add quarter for vertica

* Add quarter for sqlserver

* Add quarter to docs

* Fix snowflake implementation

* Make time zone dataset smaller

* Revert "Make time zone dataset smaller"

This reverts commit 4c97048d6c01e0f83b259a00c39d34169036fe77.

* Revert "Revert "Make time zone dataset smaller""

This reverts commit d1b8fc49c8bf86fecd9897a7c4f496dbdb1b9b06.

* Fix test for drivers not supporting set-timezone

This is to enable _ as field name.

* update token check URL, using the current one as the fallback

* update comment

* fix tests

* update docstring

* update doc string again

* Add import and export (SerDes v2) commands

* Allow use of deprecated dump/load in EE tests

* Store DBMS version in Database
* Add DBMS version to anonymous stats
* Add dbms_version sync tests
* Add test for DBMS versions in anonymous stats

* memoize snowflake describe-database w/ 5 min ttl

* pass in db-metadata where it is used,

- rather than querying for it in random functions

* update callsite of sync-tables-and-database!

* let sync-metabase-metadata! look up db-metadata

- only when needed

* pass catalog (db), and schema to .getPrimaryKeys

* let describe-table* add in the nil table-name

* cleanup

* use :snowflake instead of driver for inheritance

* concurrent-sync-test should call describe-database once

* cleaning up

* call add-table-pks with 3 args

* call add-table-pks with 3 args

* try escaping entity names to appease Oracle Driver

- Oracle wants the table-name to be escaped
- escape is usually a no-op

* pass driver to escape entity

* pull get-table-pks into a multimethod,

- handle the discrepancy directly in Oracle driver

* remove inline def

* fix colliding consistent-namespace linting

- metabase.driver.sql-jdbc.common and
  metabase.driver.sql-jdbc.sync.common were
  mapped to sql-jdbc.common
- metabase.driver.sql-jdbc.sync.common is now
  mapped to sql-jdbc.sync.common

* fix reflection warnings

* apply fix for getting oracle pks

* nix an unused arg

* Fingerprint bigquery by previewing tables
* Address review comments