* Relieve db pressure on api/health check

https://github.com/metabase/metabase/issues/26266

Servers under heavy load can be slow to respond to the api/health
check. This can lead to k8s killing healthy instances happily humming
along serving requests.

One idea floated was to use QoSFilters
https://www.eclipse.org/jetty/javadoc/jetty-9/org/eclipse/jetty/servlets/QoSFilter.html
to prioritize those requests in front of others. But I suspect this
might not be our bottleneck.

Our health endpoint was updated to see if it could acquire an endpoint
when we were dealing with connection pool issues. We were reporting the
instance was healthy once it has finished the init process, but would
report healthy if 60/15 app-db connections were used and no actual
queries could complete.

The remedy was adding
`(sql-jdbc.conn/can-connect-with-spec? {:datasource (mdb.connection/data-source)})`
to the endpoint. But now to get information about the health of the
system we have to wait in the queue to get a datasource.

The hope is that this change which monitors for recent db
checkins (query success) and checkouts (query begun) can be a proxy for
db activity without having to wait for a connection and hit the db ourselves.

Some simple and crude benchmarking:
- use `siege` to hit `api/database/<app-db>/sync_schema`
- in a separate tab, use `siege` to hit `api/health`

Three trials with unconditional db access and conditional db
access (look for recent activity set by the new `ConnectionCustomizer`).

One siege client is synching the app-db's schema with 80 clients each
sending 60 requests. the other has 1 client sending 60 requests to api/health.

Run             |  Elapsed Time | max tx  | tx rate
 before change  |    7.16s      |  0.79s  |  8.38 tx/s
 before change  |   23.91s      |  1.44s  |  2.51 tx/s
 before change  |   13.00s      |  0.50s  |  4.62 tx/s
----------------------------------------------------
 after change   |    4.46s      |  0.27s  |  13.45 tx/s
 after change   |    5.81s      |  0.61s  |  10.33 tx/s
 after change   |    4.54s      |  0.44s  |  13.22 tx/s

Full(er) results below:

```
Unconditional db access
=======================

siege -c80 -r 40 "http://localhost:3000/api/database/2/sync_schema POST" -H "Cookie: $SESSION"

siege -c 1 -r 60 "http://localhost:3000/api/health"

Elapsed time:		        7.16 secs
Response time:		        0.12 secs
Transaction rate:	        8.38 trans/sec
Longest transaction:	        0.79
Shortest transaction:	        0.01

Elapsed time:		       23.91 secs
Response time:		        0.40 secs
Transaction rate:	        2.51 trans/sec
Longest transaction:	        1.44
Shortest transaction:	        0.02

Elapsed time:		       13.00 secs
Response time:		        0.22 secs
Transaction rate:	        4.62 trans/sec
Longest transaction:	        0.50
Shortest transaction:	        0.06

Conditional db access
==============================================================

Elapsed time:		        4.46 secs
Response time:		        0.07 secs
Transaction rate:	       13.45 trans/sec
Longest transaction:	        0.27
Shortest transaction:	        0.01

Elapsed time:		        5.81 secs
Response time:		        0.10 secs
Transaction rate:	       10.33 trans/sec
Longest transaction:	        0.61
Shortest transaction:	        0.00

Elapsed time:		        4.54 secs
Response time:		        0.08 secs
Transaction rate:	       13.22 trans/sec
Longest transaction:	        0.44
Shortest transaction:	        0.01
```

* Remove reflection in `.put` call (not the reflections trategy)

also remove the call to `classloader/the-classloader` as it did nothing

* Comment and settle on a single method

* tests

* select from db twice

had a failure in CI. give it time to do its thing with another db call

* block to wait for timestamp update?

* unflake the tests

tasks and events from outside the thread can hit the db. the
ConnectionCustomizer is also run from c3p0 controlled threads so we
can't easily isolate everything to our thread

Was running

```clojure
(comment
  (dotimes [n 5]
    (dotimes [_ 100]
      (recent-activity-test)
      (CheckinTracker-test))
    (println (* (inc n) 100)))
    )
```

to run the tests 500 times and would keep getting flakes at a rate
~1/100 to 1/500. Just frustration for the future.

* typehint

* Switch it up a bit

Tests were flaking in h2 and I don't know why. I'm switching to just
updating recent activity on most methods.

* Update test

* Update implementations

* disable additivity when using "with-log-level"

* only set when parent is root

* wording

* remove a test that is no longer needed

* fix indents

* one missing indent fix

Updates embedded H2 driver to v2.1.212. When a local H2 v1 file-based application database is detected, it will be
automatically updated in-place at startup. Migrations and tests have been updated to account for changes in H2
behavior. Sample data and test harness database files in the repo have all been updated to H2 v2 file format.

Co-authored-by: Braden Shepherdson <braden@metabase.com>
Co-authored-by: Cam Saul <github@camsaul.com>
Co-authored-by: Cam Saul <1455846+camsaul@users.noreply.github.com>

* Remove more misc unused backend stuff

* Remove unused requires

* Remove more misc unused stuff

* Ok, name field is still used

* Remove `sql.qp` stuff deprecated in 41 and 42

* Remove old test

* We still need to import `FieldInstance`

* Remove unused `metabase.models.table` `metrics` and `segments` functions

* Remove more unused

* Add quarter for postgres

* Add quarter to schema

* Add quarter tests

* Add quarter for mysql

* Add quarter for bigquery

* Add quarter for snowflake

* Add quarter for redshift

* Add quarter for vertica

* Add quarter for sqlserver

* Add quarter to docs

* Fix snowflake implementation

* Make time zone dataset smaller

* Revert "Make time zone dataset smaller"

This reverts commit 4c97048d6c01e0f83b259a00c39d34169036fe77.

* Revert "Revert "Make time zone dataset smaller""

This reverts commit d1b8fc49c8bf86fecd9897a7c4f496dbdb1b9b06.

* Fix test for drivers not supporting set-timezone

This is to enable _ as field name.

* update token check URL, using the current one as the fallback

* update comment

* fix tests

* update docstring

* update doc string again

* Update normal-drivers-with-feature to use driver test db, not just the current test db

* Exclude failing mongo test

* Improve perf

* Make Dimension unique on `field_id` and deduplicate

* Fix rollback for new migrations

* Don't require migration comments to contain 'added' anymore since it's part of the version ID now

* Don't require 'Added <version>' in migration comments anymore

* Fix SerDes test that created duplicate dimensions for one Field

* Silly fix to fix MySQL 5.7

* ≈ [WIP] [ci skip]

* Minor cleanup [ci skip]

* Code cleanup. [ci skip]

* Kondo in CI should fetch library configs

* Bump Methodical version

* Change name to `=?`

* Add `#exactly` reader tag

* Add `#schema` data reader

* Fix dev deps indentation and add `algo.generic`

* Add `approx=`

* Improved version of `#approx`

* Just check in third party Kondo config for now instead of fighting CI

* Update test/metabase/test_runner/assert_exprs/approximately_equal_test.clj

Co-authored-by: Tim Macdonald <tim@metabase.com>

* Address PR feedback and fix sequence comparison

Co-authored-by: Tim Macdonald <tim@metabase.com>

* Add import and export (SerDes v2) commands

* Allow use of deprecated dump/load in EE tests

* Store DBMS version in Database
* Add DBMS version to anonymous stats
* Add dbms_version sync tests
* Add test for DBMS versions in anonymous stats

* memoize snowflake describe-database w/ 5 min ttl

* pass in db-metadata where it is used,

- rather than querying for it in random functions

* update callsite of sync-tables-and-database!

* let sync-metabase-metadata! look up db-metadata

- only when needed

* pass catalog (db), and schema to .getPrimaryKeys

* let describe-table* add in the nil table-name

* cleanup

* use :snowflake instead of driver for inheritance

* concurrent-sync-test should call describe-database once

* cleaning up

* call add-table-pks with 3 args

* call add-table-pks with 3 args

* try escaping entity names to appease Oracle Driver

- Oracle wants the table-name to be escaped
- escape is usually a no-op

* pass driver to escape entity

* pull get-table-pks into a multimethod,

- handle the discrepancy directly in Oracle driver

* remove inline def

* fix colliding consistent-namespace linting

- metabase.driver.sql-jdbc.common and
  metabase.driver.sql-jdbc.sync.common were
  mapped to sql-jdbc.common
- metabase.driver.sql-jdbc.sync.common is now
  mapped to sql-jdbc.sync.common

* fix reflection warnings

* apply fix for getting oracle pks

* nix an unused arg

* MongoDB test data config should not hardcode the user and password we use in CI

* Minor test data interface dox improvements

* Support standard deviation and variance for MongoDB

* Oops I got :expressions mixed up with :expression-aggregations

* Note about calculating variance

* `connection-type` instead of `db-or-server`

* Remove hard-coded drivers from datetime-diff-type-test

* Add requirement for set-timezone for datetime-time-zones-test

* Add sqlserver implementation

* Change datetime-diff-time-zones-test to test drivers that don't support set-timezone

* Refactor

* Temporarily remove snowflake from datetime-diff-type-test

* Fix numeric parameter values in jwt

Don't call `seq` on numbers. If its a string, assert it isn't blank,
otherwise check that it is non-null.

* bind (:value request-param)

* Fix flaky `remap-human-readable-string-column-test`

Quite annoying test that would flake quite a bit in CI.

the problem is that the mechanism for `mt/with-column-remappings` wasn't
aware of the case when there were pre-existing mappings. It just blindly
added a new mapping. But when the middleware fetching the remappings, it
found the preexisting ones.

For the venue.name field in this test, that was ("20th Century
Cafe","25°","33 Taps","800 Degrees Neapolitan Pizzeria", ...). We were
passing in fake names like "apple", "banana", "kiwi" and it found no
remappings for those in the db.

So the fix is to check for pre-existing ones, and if found, use
`tu/with-temp-vals-in-db` to temporarily set them to the desired values,
otherwise use some new ones with `tt/with-temp*` as it was doing
previously.

* log errors for CI

* Only use type :full field values when remapping

we store cached chained filter values in here (with type :linked-filter,
there are also type :sandbox in there) but the :full are the ones that
are used for remapping.

In the tests, if the chain link tests have already run, there can be 6
field values for the field here (`(mt/id :venues :name)`) and we want to
make sure we update the full one, not the chain filter ones

```clojure
[{:hash_key "576275248",
  :has_more_values false,
  :last_used_at #t "2022-12-07T21:08:35.592931Z",
  :type :linked-filter,
  :updated_at #t "2022-12-07T21:10:22.972907Z",
  :human_readable_values
  ("Appletini" "Bananasplit" "Kiwi-flavored Thing"), ;; the test updatedthe existing :linked-filter one
  :id 560,
  :values ("apple" "banana" "kiwi"),
  :field_id 15,
  :created_at #t "2022-12-07T21:08:35.592931Z"}
 {:hash_key "-897793959",
  :has_more_values false,
  :last_used_at #t "2022-12-07T21:08:35.695344Z",
  :type :linked-filter,
  :updated_at #t "2022-12-07T21:08:35.695344Z",
  :human_readable_values [],
  :id 564,
  :values ("Kinaree Thai Bistro" "Krua Siri"),
  :field_id 15,
  :created_at #t "2022-12-07T21:08:35.695344Z"}
 {:hash_key "388078290",
  :has_more_values false,
  :last_used_at #t "2022-12-07T21:08:35.742561Z",
  :type :linked-filter,
  :updated_at #t "2022-12-07T21:08:35.742561Z",
  :human_readable_values [],
  :id 566,
  :values ("Tacos Villa Corona" "Tito's Tacos"),
  :field_id 15,
  :created_at #t "2022-12-07T21:08:35.742561Z"}
 {:hash_key "-299897589",
  :has_more_values false,
  :last_used_at #t "2022-12-07T21:08:35.777517Z",
  :type :linked-filter,
  :updated_at #t "2022-12-07T21:08:35.777517Z",
  :human_readable_values [],
  :id 567,
  :values ("Festa" "Fred 62"),
  :field_id 15,
  :created_at #t "2022-12-07T21:08:35.777517Z"}
 {:hash_key "-391654650",
  :has_more_values false,
  :last_used_at #t "2022-12-07T21:08:36.681046Z",
  :type :linked-filter,
  :updated_at #t "2022-12-07T21:08:36.681046Z",
  :human_readable_values [],
  :id 569,
  :values ("Baby Blues BBQ" "Beachwood BBQ & Brewing" "Bludso's BBQ"),
  :field_id 15,
  :created_at #t "2022-12-07T21:08:36.681046Z"}
 {:hash_key "1784886858",
  :has_more_values false,
  :last_used_at #t "2022-12-07T21:08:36.860010Z",
  :type :linked-filter,
  :updated_at #t "2022-12-07T21:08:36.860010Z",
  :human_readable_values [],
  :id 572,
  :values ("Baby Blues BBQ" "Beachwood BBQ & Brewing" "Bludso's BBQ"),
  :field_id 15,
  :created_at #t "2022-12-07T21:08:36.860010Z"}
 {:hash_key nil,
  :has_more_values false,
  :last_used_at #t "2022-12-07T21:04:04.299251Z",
  :type :full, ;; but should have updated this one
  :updated_at #t "2022-12-07T21:04:04.299251Z",
  :human_readable_values [],
  :id 4,
  :values
  ("20th Century Cafe"
   "25°"
   "33 Taps"
   "800 Degrees Neapolitan Pizzeria"
   "BCD Tofu House"),
  :field_id 15,
  :created_at #t "2022-12-07T20:57:21.896298Z"}]
```

* remove CI logging

* Add snowflake datetime-diff implementation

* Get timestamps without timezones or offsets working

* Convert to timestamp_tz, not timestamp

* Format rows in tests

* Fix snowflake set-parameter

* Add failing tests

* Fix bigquery

* Formatting

* Replace literals for datetime-diff-time-zones-test

* Revert "Fix snowflake set-parameter"

This reverts commit c24348a436f3f0f2639a12d6ea368399d1549739.

* Actually use fields

* Fix typo

* Rename dataset

* Add type/DateTimeWithTZ column, since redshift doesn't like type/DateTimeWithZoneOffset

* Fix snowflake for time zones

* Undo changes to extract and date-trunc

* Explicitly set UTC timezone

* Fix mismatched types

* Remove redundant bindings

* Remove redundant comment

* Add failing tests

* Remove duplicate dataset

* Disable field value scanning for inactive fields

Fixes #26863

Adds `last-used-at` to `metabase_fieldvalues` table.

This is used to track when full FieldValues are fetched,
which should always happen through `get-or-create-full-field-values!`
If an inactive FieldValues is queried, they will be updated immediately
 to be returned to the user and a new last-used-at will be set so that
 sync will start picking it up.

During field values sync, if `last-used-at` occurred more than 14 days
ago, then we will no longer query and synchronize this field until it is
queried for again.

* Fix flaky test by waiting half a second for last-used-at to change

* Updates from review

* Remove flakiness from test by checking sync

* Add failing tests

* Fix bigquery

* Formatting

* Replace literals for datetime-diff-time-zones-test

* Actually use fields

* Fix typo

* Rename dataset

* Add type/DateTimeWithTZ column, since redshift doesn't like type/DateTimeWithZoneOffset

* Add now with extract test

* Test against UTC too

* Refactor based on Ngoc's suggestion

* Move the config-from-file code into the advanced-config directory since that's the feature we want to flag on

* Always apply the :settings section first in the config file

* config-from-file should check that we have a premium token with the :advanced-config feature

* Sort namespace

* Settings does not have to come first anymore

* Remove empty `binding` forms

* Test fix? 

* Test fix? 



* Fix failing tests, for real this time

* Revert "Fix failing tests, for real this time"

This reverts commit 0a57055b1be249d9ba5b6b6ebe4f163e1f21e2b2.

* Don't make the tests parallel since that breaks other stuff

Co-authored-by: Nemanja Glumac <31325167+nemanjaglumac@users.noreply.github.com>

* Fix `has-user-setup` Setting not being cached correctly

* Fix Kondo error

* pie percent switch

* review

* Migrate to card viz-settings v.2 (#26798)

* Migrate to card viz-settings v.2

This uses the new :pie.percent_visibility key (an enum) instead of a
collection of booleans

[#26776]

* Move to def-json-migration

* Don't migrate empty values

* fix specs

Co-authored-by: Tim Macdonald <tim@metabase.com>

* Refactor so static combo chart is ready for reordering

* Correct test names

* Clean up static viz combo chart BE API

* sort static combo chart

* Order static funnel chart

* Remove unused import and function

* Fix BE tests

* Remove custom column name tests since logic is moved to FE

* Remove graph.series_order when changing dimensions (#26842)

* review

Co-authored-by: Nick Fitzpatrick <nick@metabase.com>
Co-authored-by: Aleksandr Lesnenko <alxnddr@gmail.com>

* Implement now for sql drivers

* Remove unused import

* Add tests

* Add more tests

* Implement bigquery, mysql, postgres

* Test now as argument

* Fix FE type inferencer

* Change feature flag to date-arithmetics

* Implement h2 driver

* Fix tests for sqlite

* Add snowflake implementation

* Remove type test

* Fix test for sqlite

* Update how we handle autocomplete and format for zero-arity functions in ExpressionEditor (#26563)

* Update suggestionText function

* Suggest with closing parens if zero-arity

* Format zero-arity functions with ()

* Refactor formatFunction

* Remove unused imports

* Add presto implementation

* Add sqlserver implementation

* Test in multiple timezones

* Remove setting test driver

* Add sparksql implementation

* Fix count() fe unit test

* Add fe MBQL unit test for now()

* Fix sparksql implementation

* Add vertica implementation

* Add oracle implementation

* Add vertica implementation

* Remove accidental form

* Fix vertica implementation for report timezones

* Remove unused multimethod

* Remove unused feature

* Update sparksql

* Fix fe unit tests

* Fix fe unit tests

* Update hive implementation

* Revert "Fix fe unit tests"

This reverts commit fde4dd0d.

* Revert "Fix fe unit tests"

This reverts commit 7b7429ea.

* Revert "Fix count() fe unit test"

This reverts commit c5c6f6f9.

* Revert "Refactor formatFunction"

This reverts commit ac50e73e.

* Revert "Update how we handle autocomplete and format for zero-arity functions in ExpressionEditor (#26563)"

This reverts commit cf39634e.

* Update hive

* snowflake: convert type to timestamptz

* sqlserver: rename date-trunc to zeroed-date-part

* Wrap with database type info

* Change fe unit test back to now without parens

* Change MBQL clauses config for now to return type "datetime"

* Fix presto

* Add some fe unit tests for parsing and resolving

* Update helper-text-strings

* Change from second to millisecond precision

* Remove test for second precision

* presto: Change from second to millisecond precision

* vertica: Change from second to millisecond precision

* vertica: Change from second to millisecond precision

* Change feature flag from date-arithmetics to now

* Change feature flag from date-arithmetics to now, mbql schema

* Allow now to be used in filter clauses

* Add test for filter

* Implement driver/database-supports? for each driver that implements :now

* Tidy tests

* Add back default sql implementation

* Add sqlite feature flag

* Add mongo implementation

* Fix mongo

* Format rows

* Change vertica to return timestamp with time zone

* Fix presto tests

* Remove unused require

* Update h2 function to return timestamp with time zone

* mongo: calculate now during query execution, not processing, for versions >=4.2 (#26822)

* replace with $$NOW

* Remove unused import

* Add back previous now implementation for versions <4.2

* Throw an error if version is <4.2

* Add i18n

* Ignore patch version

* Clean up `SchedulePicker` component (#26839)

* Move `SchedulePicker` to its own directory

* Sort imports

* Do basic clean up

* Extract `DEFAULT_DAY` to a constant

* Extract styled components

* Convert `SchedulePicker` to TypeScript

* Add `SchedulePicker` container

* Export `SchedulePicker` options

* Add basic `SchedulePicker` story

* Fix `SchedulePicker` crashes Storybook

* Tweak Storybook file

* Build datetime filters as datetime before question is saved (#26679)

* Revert "Update h2 function to return timestamp with time zone"

This reverts commit f7cce3ea7ccbbfc886d0942d66b1f8f60feb6aba.

Co-authored-by: Gustavo Saiani <gus@metabase.com>
Co-authored-by: Anton Kulyk <kuliks.anton@gmail.com>

* initial implementation

* some refactor and improved error handling

* another refactor

* fix reflection warning

* add test and more refactor

* another test

* misc comment improvements and formatting

* fix tests

* small refactor

* fix function call

* address tamas's comments