Use the vertica/vertica-ce:12.0.2-0 image.

Also point out that PKCS12 (.p12) keystores are now supported.

* move malli schema from metabase.util.schema to a new ns metabase.util.malli.schema

* revert name changes for plumatic schema. e.g: NonBlankStringPlumatic -> NonBlankString

* add clj-kondo rule to enforce consistent alias for metabase.util.malli.schema

* Switch to Honey SQL 2, Part 1

* Test fix

* Fix Kondo error

* Always return fields selected by $project in native MongoDB queries
* Keep _id at the position specified in $project if it's included there or at first position if it's not included

* Fix mongo version checking for now

* Add minor versions for comparison

* Handle nil case

* newline

* Replace _

* Use dbms-version semantic-version instead

* Compare only the major and minor version

* Fix mongo database-supports?

* Fix duplicate require

* Fix typo

* Remove expressions from version check

* Fix various mongo expressions

`trim`, `rtrim`, `ltrim` needed to be wrapped in `{"input" expr}` object
`replace` needed to be wrapped in `{"input" expr "find" ...
"replacement" ...}' object
`substring` needed to fill in the 3rd argument, optional in mbql but
required in mongo. Also to use a 1 based index

* Generalize semantic-version-gte

* Used synced dbms_version for testing feature support

* Expressions are only supported by mongo 4.2+

* Disable some tests

* Fix mongo division

Handle nulls with an upfront condition check.
Handle multiple divisors.

* Remove unused namespace

* Throw exeception if using replace on mongo < 4.4

* Skip test because of #27249

* Move minimum Mongo CI version to 4.2

* Fix sorting by expressions

$sort needs to come before $project otherwise we can only see the
projected fields, however expressions are only added in $project.

So now if a sort includes an expression, we will use $addFields to be
able to sort by that expression.

* Disable tests for expressions inside aggregations

To be addressed by #27275

* Handle aggregation nested in an expression

* Remove :truncation-start workaround

* Enable tests for expressions inside aggregations

* Fix datetime-math-tests

* Make sure dbms_version is included when fetching database for store

* Update doc for replace to indicate it should replace all occurrences

* Add mongo datetime-diff

* Fix database-supports?

* Fix order-by-test

* Handle embedded special aggregations (#27315)

* Handle embedded special aggregations
* Preserve aggregation options for nested aggregations
* Use top-level aggregation name as group name

* Disable nil punning on semantic version check (#27317)

* Remove nil punning from database-supports?

* Formatting

* Refactor based on Tamas' suggestions

* Change datetime-diff from case statement to multimethod

Co-authored-by: Case Nelson <case@metabase.com>
Co-authored-by: Tamás Benkő <tamas@metabase.com>
Co-authored-by: metamben <103100869+metamben@users.noreply.github.com>

* Fix mongo version checking for now

* Add minor versions for comparison

* Handle nil case

* newline

* Replace _

* Use dbms-version semantic-version instead

* Compare only the major and minor version

* Fix mongo database-supports?

* Fix duplicate require

* Fix typo

* Remove expressions from version check

* Fix various mongo expressions

`trim`, `rtrim`, `ltrim` needed to be wrapped in `{"input" expr}` object
`replace` needed to be wrapped in `{"input" expr "find" ...
"replacement" ...}' object
`substring` needed to fill in the 3rd argument, optional in mbql but
required in mongo. Also to use a 1 based index

* Generalize semantic-version-gte

* Used synced dbms_version for testing feature support

* Expressions are only supported by mongo 4.2+

* Disable some tests

* Fix mongo division

Handle nulls with an upfront condition check.
Handle multiple divisors.

* Remove unused namespace

* Throw exeception if using replace on mongo < 4.4

* Skip test because of #27249

* Move minimum Mongo CI version to 4.2

* Fix sorting by expressions

$sort needs to come before $project otherwise we can only see the
projected fields, however expressions are only added in $project.

So now if a sort includes an expression, we will use $addFields to be
able to sort by that expression.

* Disable tests for expressions inside aggregations

To be addressed by #27275

* Handle aggregation nested in an expression

* Remove :truncation-start workaround

* Enable tests for expressions inside aggregations

* Fix datetime-math-tests

* Make sure dbms_version is included when fetching database for store

* Update doc for replace to indicate it should replace all occurrences

* Fix order-by-test

* Handle embedded special aggregations (#27315)

* Handle embedded special aggregations
* Preserve aggregation options for nested aggregations
* Use top-level aggregation name as group name

* Disable nil punning on semantic version check (#27317)

* Optimize produced query for division when dividing by literals

* Rename var

* Use reduce to build division

* Fix reduction

* Clean up formatting and document some details

Co-authored-by: Callum Herries <hi@callumherries.com>
Co-authored-by: Tamás Benkő <tamas@metabase.com>
Co-authored-by: metamben <103100869+metamben@users.noreply.github.com>

* Add datetime-diff implementation for athena

* Don't run some tests for drivers not supporting timestamptz

* Fix for date types

* Use date-trunc helper

* Add comment about Athena's timestamp with time zone type

* Add datetime-diff-time-zones-athena-test

* Fix typo

* Fix typo

* Rewrite comment

* Tidying

* Add athena-datetime-diff-mixed-types-test

* Fix typo

* Adjust comments + formatting

* Adjust comments

* Format SQL query

* Align requires

* Add sqlite datetime-diff

* Refactor sqlite

* Exclude sqlite from datetime-diff-type-test

* Fix sqlite temporal fields losing type info on sql.qp->honeysql

* Leave the default impl for sql.qp/date

* Add oracle datetime-diff

* Remove duplicate method impls

* Remove unused require

* adds mu/defn, which behaves like s/defn but for malli schemas

- includes link to malli.io with your schema and type.
- making room for some namespaces

* require and use malli.destructure in mu/defn

* fix require alias

* clean ns

* fix linter errror

* fix linter error again

* Use gvenzl/oracle-xe docker to test Oracle
* Configure SSL connection for Oracle 21.3
* Fix secret handling

* Fix BigQuery parsing of `bigdecimal` results

* Avoid big changes to `ns` form

* Update tests

* Start-of-cycle 46 basic dependency version bump

Some dependencies with available major version bumps are only updated
to the latest minor version of the current major version. Some
additional dependencies will be updated in future PRs.

* Removes commons-lang from sparksql deps exceptions

* Fixes MongoDB SRV test for monger 3.6.0

* fixes SQLite datetime tests for new driver version 3.40

* Add sparksql datetime-diff

* Set supports-timestamptz-type? to false for sparksql

* Refactor sparksql datetime-diff

* Change float to int

Refactor datetime-diff implementations: sqlserver, mysql, bigquery, redshift, snowflake, vertica, postgres (#27262)

* Refactor bigquery datetime-diff to use multimethod

* Refactor snowflake datetime-diff to use multimethod

* Refactor SQL server datetime-diff to use multimethod

* Refactor mysql datetime-diff to use multimethod

* Refactor redshift datetime-diff to use multimethod

* Simplify vertica datetime-diff

* Tidy postgres datetime-diff

* ; -> ;; in snowflake comment

* Add to bigquery comment

* Reduce the number of rows read when syncing mongo

Also prefer recent records to old ones, as these better represent
the current schema.

* Add database-type function

* Add datetime-diff multimethod and default impl

* Refactor postgres

* Refactor vertica

* Remove accidentally left comment

* Tidy datetime-diff-check-args

* Add to datetime-diff-check-args docstring

* Add multimethod to changelog

* Add to docstrings

* Add `:added` metadata to `datetime-diff`

Co-authored-by: Cam Saul <1455846+camsaul@users.noreply.github.com>

Co-authored-by: Cam Saul <1455846+camsaul@users.noreply.github.com>

* Add datetimeDiff tests for the same time

* Fix redshift for same time args

* Duplicate comment

* Replace str/lower-case with u/lower-case-en

* Replace str/upper-case with u/upper-case-en

* Replace more str/lower-case

* Update test

* Update implementations

* Add quarter for postgres

* Add quarter to schema

* Add quarter tests

* Add quarter for mysql

* Add quarter for bigquery

* Add quarter for snowflake

* Add quarter for redshift

* Add quarter for vertica

* Add quarter for sqlserver

* Add quarter to docs

* Fix snowflake implementation

* Make time zone dataset smaller

* Revert "Make time zone dataset smaller"

This reverts commit 4c97048d6c01e0f83b259a00c39d34169036fe77.

* Revert "Revert "Make time zone dataset smaller""

This reverts commit d1b8fc49c8bf86fecd9897a7c4f496dbdb1b9b06.

* Fix test for drivers not supporting set-timezone

This is to enable _ as field name.

Fixes #27014

Our cloud hosting requires using hard coded IAM auth for Athena. So make
sure that if the access_key is left blank, we only fall back to the
credential chain if we are not in a hosted environment.

* Store DBMS version in Database
* Add DBMS version to anonymous stats
* Add dbms_version sync tests
* Add test for DBMS versions in anonymous stats

* memoize snowflake describe-database w/ 5 min ttl

* pass in db-metadata where it is used,

- rather than querying for it in random functions

* update callsite of sync-tables-and-database!

* let sync-metabase-metadata! look up db-metadata

- only when needed

* pass catalog (db), and schema to .getPrimaryKeys

* let describe-table* add in the nil table-name

* cleanup

* use :snowflake instead of driver for inheritance

* concurrent-sync-test should call describe-database once

* cleaning up

* call add-table-pks with 3 args

* call add-table-pks with 3 args

* try escaping entity names to appease Oracle Driver

- Oracle wants the table-name to be escaped
- escape is usually a no-op

* pass driver to escape entity

* pull get-table-pks into a multimethod,

- handle the discrepancy directly in Oracle driver

* remove inline def

* fix colliding consistent-namespace linting

- metabase.driver.sql-jdbc.common and
  metabase.driver.sql-jdbc.sync.common were
  mapped to sql-jdbc.common
- metabase.driver.sql-jdbc.sync.common is now
  mapped to sql-jdbc.sync.common

* fix reflection warnings

* apply fix for getting oracle pks

* nix an unused arg

* Fingerprint bigquery by previewing tables
* Address review comments

* MongoDB test data config should not hardcode the user and password we use in CI

* Minor test data interface dox improvements

* Support standard deviation and variance for MongoDB

* Oops I got :expressions mixed up with :expression-aggregations

* Note about calculating variance

* `connection-type` instead of `db-or-server`

* Remove hard-coded drivers from datetime-diff-type-test

* Add requirement for set-timezone for datetime-time-zones-test

* Add sqlserver implementation

* Change datetime-diff-time-zones-test to test drivers that don't support set-timezone

* Refactor

* Temporarily remove snowflake from datetime-diff-type-test

* Add redshift implementation

* Add failing tests

* Fix bigquery

* Formatting

* fix? set-param for redshift

* Replace literals for datetime-diff-time-zones-test

* Actually use fields

* Fix typo

* Rename dataset

* Add type/DateTimeWithTZ column, since redshift doesn't like type/DateTimeWithZoneOffset

* Cast to timestamp for week diff

* Revert "fix? set-param for redshift"

This reverts commit 235f3aca2a4b3bf41469d7bb73266d3f5e54f5d2.

* Fix leap years

* Remove redundant date-trunc

* Formatting

* Fix mismatched types

* Fix extract

* Add comment for why redshift needs an implementation

Co-authored-by: Ngoc Khuat <qn.khuat@gmail.com>

* vertica datetime-diff implementation

* Don't need type info

* Refactor implementation

* Fix seconds

* Cast to timestamp before comparison

* Fix set-parameter for vertica

* formatting

* Sort ns

* Remove set-parameter changes

* Replace literals for datetime-diff-time-zones-test

* Fix requires

* Actually use fields

* Fix typo

* Rename dataset

* Add type/DateTimeWithTZ column, since redshift doesn't like type/DateTimeWithZoneOffset

* Add failing tests

* Fix bigquery

* Formatting

* Fix mismatched types

Resolves #16132

We have been simplifying $expr operations for
[awhile](https://github.com/metabase/metabase/blob/8e989499e936c04000b13a88f4934e96692083a2/modules/drivers/mongo/src/metabase/driver/mongo/query_processor.clj#L571-L588), since at least 4fba5cfe.

It's possible that there are some cases that we do not simplify that we
should, but from this investigation, they seem like legit uses
remaining.

* review athena manifest

* review case and add helper text

* add better helper text to AWS Credentials Provider Chain

* fix capitalization

* add helper text to password too

* Update modules/drivers/athena/resources/metabase-plugin.yaml

Co-authored-by: Jeff Bruemmer <jeff.bruemmer@gmail.com>

Co-authored-by: Jeff Bruemmer <jeff.bruemmer@gmail.com>

* Add snowflake datetime-diff implementation

* Get timestamps without timezones or offsets working

* Convert to timestamp_tz, not timestamp

* Format rows in tests

* Fix snowflake set-parameter

* Add failing tests

* Fix bigquery

* Formatting

* Replace literals for datetime-diff-time-zones-test

* Revert "Fix snowflake set-parameter"

This reverts commit c24348a436f3f0f2639a12d6ea368399d1549739.

* Actually use fields

* Fix typo

* Rename dataset

* Add type/DateTimeWithTZ column, since redshift doesn't like type/DateTimeWithZoneOffset

* Fix snowflake for time zones

* Undo changes to extract and date-trunc

* Explicitly set UTC timezone

* Fix mismatched types

* Remove redundant bindings

* Remove redundant comment

* Add failing tests

* Remove duplicate dataset