* Update test

* Update implementations

* Add quarter for postgres

* Add quarter to schema

* Add quarter tests

* Add quarter for mysql

* Add quarter for bigquery

* Add quarter for snowflake

* Add quarter for redshift

* Add quarter for vertica

* Add quarter for sqlserver

* Add quarter to docs

* Fix snowflake implementation

* Make time zone dataset smaller

* Revert "Make time zone dataset smaller"

This reverts commit 4c97048d6c01e0f83b259a00c39d34169036fe77.

* Revert "Revert "Make time zone dataset smaller""

This reverts commit d1b8fc49c8bf86fecd9897a7c4f496dbdb1b9b06.

* Fix test for drivers not supporting set-timezone

This is to enable _ as field name.

Fixes #27014

Our cloud hosting requires using hard coded IAM auth for Athena. So make
sure that if the access_key is left blank, we only fall back to the
credential chain if we are not in a hosted environment.

* Store DBMS version in Database
* Add DBMS version to anonymous stats
* Add dbms_version sync tests
* Add test for DBMS versions in anonymous stats

* memoize snowflake describe-database w/ 5 min ttl

* pass in db-metadata where it is used,

- rather than querying for it in random functions

* update callsite of sync-tables-and-database!

* let sync-metabase-metadata! look up db-metadata

- only when needed

* pass catalog (db), and schema to .getPrimaryKeys

* let describe-table* add in the nil table-name

* cleanup

* use :snowflake instead of driver for inheritance

* concurrent-sync-test should call describe-database once

* cleaning up

* call add-table-pks with 3 args

* call add-table-pks with 3 args

* try escaping entity names to appease Oracle Driver

- Oracle wants the table-name to be escaped
- escape is usually a no-op

* pass driver to escape entity

* pull get-table-pks into a multimethod,

- handle the discrepancy directly in Oracle driver

* remove inline def

* fix colliding consistent-namespace linting

- metabase.driver.sql-jdbc.common and
  metabase.driver.sql-jdbc.sync.common were
  mapped to sql-jdbc.common
- metabase.driver.sql-jdbc.sync.common is now
  mapped to sql-jdbc.sync.common

* fix reflection warnings

* apply fix for getting oracle pks

* nix an unused arg

* Fingerprint bigquery by previewing tables
* Address review comments

* MongoDB test data config should not hardcode the user and password we use in CI

* Minor test data interface dox improvements

* Support standard deviation and variance for MongoDB

* Oops I got :expressions mixed up with :expression-aggregations

* Note about calculating variance

* `connection-type` instead of `db-or-server`

* Remove hard-coded drivers from datetime-diff-type-test

* Add requirement for set-timezone for datetime-time-zones-test

* Add sqlserver implementation

* Change datetime-diff-time-zones-test to test drivers that don't support set-timezone

* Refactor

* Temporarily remove snowflake from datetime-diff-type-test

* Add redshift implementation

* Add failing tests

* Fix bigquery

* Formatting

* fix? set-param for redshift

* Replace literals for datetime-diff-time-zones-test

* Actually use fields

* Fix typo

* Rename dataset

* Add type/DateTimeWithTZ column, since redshift doesn't like type/DateTimeWithZoneOffset

* Cast to timestamp for week diff

* Revert "fix? set-param for redshift"

This reverts commit 235f3aca2a4b3bf41469d7bb73266d3f5e54f5d2.

* Fix leap years

* Remove redundant date-trunc

* Formatting

* Fix mismatched types

* Fix extract

* Add comment for why redshift needs an implementation

Co-authored-by: Ngoc Khuat <qn.khuat@gmail.com>

* vertica datetime-diff implementation

* Don't need type info

* Refactor implementation

* Fix seconds

* Cast to timestamp before comparison

* Fix set-parameter for vertica

* formatting

* Sort ns

* Remove set-parameter changes

* Replace literals for datetime-diff-time-zones-test

* Fix requires

* Actually use fields

* Fix typo

* Rename dataset

* Add type/DateTimeWithTZ column, since redshift doesn't like type/DateTimeWithZoneOffset

* Add failing tests

* Fix bigquery

* Formatting

* Fix mismatched types

Resolves #16132

We have been simplifying $expr operations for
[awhile](https://github.com/metabase/metabase/blob/8e989499e936c04000b13a88f4934e96692083a2/modules/drivers/mongo/src/metabase/driver/mongo/query_processor.clj#L571-L588), since at least 4fba5cfe.

It's possible that there are some cases that we do not simplify that we
should, but from this investigation, they seem like legit uses
remaining.

* review athena manifest

* review case and add helper text

* add better helper text to AWS Credentials Provider Chain

* fix capitalization

* add helper text to password too

* Update modules/drivers/athena/resources/metabase-plugin.yaml

Co-authored-by: Jeff Bruemmer <jeff.bruemmer@gmail.com>

Co-authored-by: Jeff Bruemmer <jeff.bruemmer@gmail.com>

* Add snowflake datetime-diff implementation

* Get timestamps without timezones or offsets working

* Convert to timestamp_tz, not timestamp

* Format rows in tests

* Fix snowflake set-parameter

* Add failing tests

* Fix bigquery

* Formatting

* Replace literals for datetime-diff-time-zones-test

* Revert "Fix snowflake set-parameter"

This reverts commit c24348a436f3f0f2639a12d6ea368399d1549739.

* Actually use fields

* Fix typo

* Rename dataset

* Add type/DateTimeWithTZ column, since redshift doesn't like type/DateTimeWithZoneOffset

* Fix snowflake for time zones

* Undo changes to extract and date-trunc

* Explicitly set UTC timezone

* Fix mismatched types

* Remove redundant bindings

* Remove redundant comment

* Add failing tests

* Remove duplicate dataset

* Add failing tests

* Fix bigquery

* Formatting

* Implement now for sql drivers

* Remove unused import

* Add tests

* Add more tests

* Implement bigquery, mysql, postgres

* Test now as argument

* Fix FE type inferencer

* Change feature flag to date-arithmetics

* Implement h2 driver

* Fix tests for sqlite

* Add snowflake implementation

* Remove type test

* Fix test for sqlite

* Update how we handle autocomplete and format for zero-arity functions in ExpressionEditor (#26563)

* Update suggestionText function

* Suggest with closing parens if zero-arity

* Format zero-arity functions with ()

* Refactor formatFunction

* Remove unused imports

* Add presto implementation

* Add sqlserver implementation

* Test in multiple timezones

* Remove setting test driver

* Add sparksql implementation

* Fix count() fe unit test

* Add fe MBQL unit test for now()

* Fix sparksql implementation

* Add vertica implementation

* Add oracle implementation

* Add vertica implementation

* Remove accidental form

* Fix vertica implementation for report timezones

* Remove unused multimethod

* Remove unused feature

* Update sparksql

* Fix fe unit tests

* Fix fe unit tests

* Update hive implementation

* Revert "Fix fe unit tests"

This reverts commit fde4dd0d.

* Revert "Fix fe unit tests"

This reverts commit 7b7429ea.

* Revert "Fix count() fe unit test"

This reverts commit c5c6f6f9.

* Revert "Refactor formatFunction"

This reverts commit ac50e73e.

* Revert "Update how we handle autocomplete and format for zero-arity functions in ExpressionEditor (#26563)"

This reverts commit cf39634e.

* Update hive

* snowflake: convert type to timestamptz

* sqlserver: rename date-trunc to zeroed-date-part

* Wrap with database type info

* Change fe unit test back to now without parens

* Change MBQL clauses config for now to return type "datetime"

* Fix presto

* Add some fe unit tests for parsing and resolving

* Update helper-text-strings

* Change from second to millisecond precision

* Remove test for second precision

* presto: Change from second to millisecond precision

* vertica: Change from second to millisecond precision

* vertica: Change from second to millisecond precision

* Change feature flag from date-arithmetics to now

* Change feature flag from date-arithmetics to now, mbql schema

* Allow now to be used in filter clauses

* Add test for filter

* Implement driver/database-supports? for each driver that implements :now

* Tidy tests

* Add back default sql implementation

* Add sqlite feature flag

* Add mongo implementation

* Fix mongo

* Format rows

* Change vertica to return timestamp with time zone

* Fix presto tests

* Remove unused require

* Update h2 function to return timestamp with time zone

* mongo: calculate now during query execution, not processing, for versions >=4.2 (#26822)

* replace with $$NOW

* Remove unused import

* Add back previous now implementation for versions <4.2

* Throw an error if version is <4.2

* Add i18n

* Ignore patch version

* Clean up `SchedulePicker` component (#26839)

* Move `SchedulePicker` to its own directory

* Sort imports

* Do basic clean up

* Extract `DEFAULT_DAY` to a constant

* Extract styled components

* Convert `SchedulePicker` to TypeScript

* Add `SchedulePicker` container

* Export `SchedulePicker` options

* Add basic `SchedulePicker` story

* Fix `SchedulePicker` crashes Storybook

* Tweak Storybook file

* Build datetime filters as datetime before question is saved (#26679)

* Revert "Update h2 function to return timestamp with time zone"

This reverts commit f7cce3ea7ccbbfc886d0942d66b1f8f60feb6aba.

Co-authored-by: Gustavo Saiani <gus@metabase.com>
Co-authored-by: Anton Kulyk <kuliks.anton@gmail.com>

* Bigquery

* Redshift

* Snowflake

* no convert-timezone for sparksql because it only has one timestamp data type

* redshift -- wrap type when adding interval

* don't override results-timezone-id in tests

* fix redshift ignore timezone when insert timestamp with time zone test data

* datetime-add, datetime-subtract, temporal-extract, convert-timezone now can takes datetime literal as argument

* fix vertica

* convert-timezone for mysql, mariadb, sqlserver, oracle

* sqlserver doesn't understand Asia/Ho_Chi_Minh

* vertica

* skip report-tz tests for sqlserver

* use the util fn to check args for convert-timezone

* fix for vertica not formatting in report-tz 

* Include Amazon Athena support for dacort/metabase-athena-driver

* Use Metabase Maven repo to fetch Athena driver

* Copy test extensions from Damon's Athena driver repo

* Some code cleanup.

* Move namespaces => metabase.driver.athena

* Clean up the test extensions namespace

* 42 failures, 16 errors

* Fix regex support; disable a few tests

* Minor tweaks

* Fix data-source-name

* Fix :week and :day-of-week impls (mostly): 2 failures, 3 errors

* Fix OFFSET, :week-of-year; skip test that has TIME column

* Add Athena to CircleCI config. Don't wait for Java 11 tests to finish before driver tests.

* ALL TESTS ARE PASSING! <3

* Copy fixes for https://github.com/dacort/metabase-athena-driver/issues/115;

 add test

* We don't need to prep source files or fetch dependencies before running backend tests.

* Fix Eastwood error.

* Tweak CircleCI config.

* Fix TIMESTAMP WITH TIME ZONE

* Include the Athena/Redshift repos in the build-drivers deps.edn

* Build and release scripts need to have the :mvn/repos as well

* Revert change that enabled test for Presto

* Un-enable failing test for Snowflake as well.

* moves all is clauses into the test

- previously only the first few tests were being run

* Sort ns in `metabase.driver.athena-test`

* Prevent athena's log4j2.properties file from becoming log config

athena includes log4j2.properties top-level with the properties:

```
status = debug
rootLogger.level=debug
```

And this kills our beautifully crafted logging. Set
"log4j2.configurationFile" in bootstrap to our own log4j2.xml.

Log4j2 looks in a few places for its logging config, the first of which
is the properties file. So when the jar is loaded, log4j2 considers this
a logging config change and we lose our logging.

Co-authored-by: Damon P. Cortesi <d.lifehacker@gmail.com>
Co-authored-by: Bryan Maass <bryan.maass@gmail.com>
Co-authored-by: dan sutton <dan@dpsutton.com>

* treat private-key conn props as base64 encoded

- fix typo

* handle connecting with local keypair

* adds test for snowflake keypair auth

* remove unused arg

* remove id, and only call byte-array->string once

* dont reflect for conversion from byte-array to str

* remove inline def

* more cleanup

- mess with tests to find the fabled metabase ci snowflake user

* remove un-needed test

* handle conn uri in connection-details->spec

* make sure tests are actually running

* encode properties in the connection uri

* typehint private keyfile

* respond to some review comments

- add a test for can-connect? => true, even when jdbc/query returns falsey

* remove unused logging require

* adds assoc-qp

- which will be used to non-destructively "assoc" onto the snowflake
connection query params

* hook in u.qp/assoc-qp to better update conn uri

* kill a newline

* reuse `conn-str-with-additional-opts` on snowflake conn uri

* update use utf-8 when encoding private-key-value

conveniently use `bytes` and `bytes?`

* Initial support for pg for `date-diff`

* Make the useful-dates closer to one day

* Add simple FE stuff

* shorter test bodies

* Ensure we can use datediff functions in arithmetic expressions

* Correctly disable datediff for redshift

* simplify var names

* Support week

* cleanup test

* :datediff -> :datetimediff

* ngoc's suggestions

* Better acceptance test for datetimediff

* sort ns

* embrace the different cases for results

* bigquery day month year

* Reverse args

* Update test

* Centralize tests

* Change postgres day, month, year behaviour

* Refactor keep identity

* Tidy tests

* Tidy

* Fix bigquery week

* Add week tests

* Fix bigquery week

* Change mysql day, month, year behaviour

* Add test for hour, minute, second

* Fix postgres hour minute second

* Formatting

* Fix bigquery hour, minute, second

* Formatting

* Fix postgres timestamptz

* WIP

* Allow literals in datediff clauses

* Uncomment tests

* Fix bigquery when reporting timezone is not UTC

* Linting

* Moving away from dataset based tests

* Add timezone tests for week and tidy

* Remove unused import

* Consolidate tests

* Remove with-time-column dataset

* Remove more-useful-dates defdataset

* Remove redshift driver WIP

* Typo

* Move DatetimeLiteral clause into DateTimeExpressionArg

* Try changing test order

* Remove mt/with-report-timezone-id nil

* Add year report timezone tests

* Rename

* Rename

* Remove unused tables from useful-dates

* Remove useful-dates

* Update helper-text-strings

* Tidy

* Swap order of mt/with-report-timezone-id

* Change with-report-timezone-id; notify databases after running test

* Use temp setting for report-timezone instead

* Update helper-test-strings

* Handle literals in `datetimediff-base-base`

* Update src/metabase/driver/postgres.clj

Co-authored-by: metamben <103100869+metamben@users.noreply.github.com>

* Simplify postgres second

* Tidy: prefer hx arithmetic functions

* Drop coercion for string timestamp args for now

* Extract helper

* Revert "Change with-report-timezone-id; notify databases after running test"

This reverts commit 7abb543bed13d9d13295d0e00ad1293a40e40d53.

* Use ->timestamptz

* Use hx/->timestamp

* Rename dataset

* Undo changes to datetime-arithmetics? as these will not match by default

* Use proper format for offset datetime literals

* Shorten datetime literals in tests

* Removing notify-all-databases-updated from report-timezone setting

* Fix datetime-arithmetics test

* Revert "Fix datetime-arithmetics test"

This reverts commit 9141582508170bdbe99c4aa74bc4444e41475be6.

* Revert "Undo changes to datetime-arithmetics? as these will not match by default"

This reverts commit 9cb05f5475b24a78cb89601c918f3440606cc525.

* Add UTC timezone to tests

* Coerce strings to datetimes for ISO formats

* Revert "Removing notify-all-databases-updated from report-timezone setting"

This reverts commit 37356435a2e3981deca9ae76118d857babb2aada.

* Add comment to postgres driver implementation

* Formatting

* Disable datediff from redshift for the moment (for ever?)

* Override redshift driver/database-supports?

* Fix comments mixed up by refactoring

* Fix comments mixed up by refactoring 2

* Tidy comment

* Rename datetimediff to datetime-diff/datetimeDiff

* date-add -> datetime-add

* Linting

* add datetime-subtract to `datetime-arithmetics?`

rework tests a bit as well.

* Update docstring

* Use ->temporal-type and trunc to handle report-timezone for bigquery

* Log errors caught during sync steps (#26306)

* log errors caught during sync steps

* remove accidental extra parens

* Errors combining datetime interval addition with datetime functions (#26279)

* Add failing tests

* Fix failing tests

* Update shared/src/metabase/mbql/util.cljc

Co-authored-by: Ngoc Khuat <qn.khuat@gmail.com>

* datetime-add and datetime subtract should annotate type by col type

* Fix infer-expression-type for datetime-add/subtract with second, minute, hour

* Undo last commit; they actually always return :type/DateTime

* Fix test based on last commit

* Undo unrelated refactor

* Only test drivers that support expressions

* Only test drivers that support expressions, again

* Update tests from legacy mbql

* Change infered-col-type to be a function again, not macro

* Fix test

Co-authored-by: Ngoc Khuat <qn.khuat@gmail.com>

* whitespace

* Remove comments

* Add explanation for datetime_diff

* Refactor: replace cast and add `mt/with-driver :bigquery-cloud-sdk` where report-timezone is relevant

* Add failing tests

* Fix failing tests

* Technically it should be bigquery-type

* Update modules/drivers/bigquery-cloud-sdk/src/metabase/driver/bigquery_cloud_sdk.clj

Co-authored-by: Cam Saul <1455846+camsaul@users.noreply.github.com>

* Move documentation to metabase.mbql.schema

* Update DatetimeDiffUnits

Co-authored-by: Cam Saul <1455846+camsaul@users.noreply.github.com>

* Add error type and optimize case expressions to driver implementations

* refactor for brevity

* Fix error with postgres

* Fix error with postgres

* Handle string literal parsing in wrap-value-literals

* Remove ->timestamptz

* Add comments + TODOs showing arithmetic expressions should return numeric values

* Add explanation of arithmetic expression as docstring

* Add test for normalize-mbql-clause-tokens

* Remove unused import

* Remove unused form

* Switch tests to use attempted-murders dataset

* Undo optimization that broke tests

* Validate non-temporal types for bigquery

* Add error handling for incorrect types

* Fix mysql type checking

* Fix mysql type checking

* invalid-parameter -> invalid-query

* Use date-trunc and extract

* Remove unit error handling

* DatetimeLiteral -> DateOrDatetimeLiteral

* Remove unused binding

* Fix mysql type checking

Co-authored-by: Callum Herries <hi@callumherries.com>
Co-authored-by: Cal Herries <39073188+calherries@users.noreply.github.com>
Co-authored-by: metamben <103100869+metamben@users.noreply.github.com>
Co-authored-by: Noah Moss <32746338+noahmoss@users.noreply.github.com>
Co-authored-by: Ngoc Khuat <qn.khuat@gmail.com>
Co-authored-by: Cam Saul <1455846+camsaul@users.noreply.github.com>

- Add tests to check when bucketing by week and week-of-year behave consistently, Closes #4910
- While doing the above, found that sqlserver does not respect `start-of-week` setting, so fix that. Fix #25356

* Adding parameters based on the comment

* add tests

Co-authored-by: Ngoc Khuat <qn.khuat@gmail.com>

* Refactor: replace cast and add `mt/with-driver :bigquery-cloud-sdk` where report-timezone is relevant

* Add failing tests

* Fix failing tests

* Technically it should be bigquery-type

* [:field-id id] => [:field id nil] or $name. Except places where tests
are meant to check normalization

* [:datetime-field [:field id] unit] -> [:field id {:temporal-unit unit}]
or !unit.name

* [:fk-> [:field 1] [:field 2]] => [:field 2 {:source-field 1}]

* rename dateAdd to datetimeAdd and dateSubtract to datetimeSubtrac

* Reset changes and go thru everything again to make sure it makes sense

* Test fixes 

* Implement advanced date/time/zone manipulation, part 1

Incorporate new functions into MBQL and add tests:
 - get-year
 - get-quarter
 - get-month
 - get-day
 - get-day-of-week
 - get-hour
 - get-minute
 - get-second

* Fix BigQuery implementations to call extract

Mark as not supported in legacy driver

* Add date extraction fns for Postgres

* Disable in MongoDB (for now at least)

Disable in BigQuery (legacy driver)

Add implementations for presto-jdbc

* Misc cleanup from Jeff's PR

* Update Jeff's implementation of bigquery-cloud-sqk

* Reorganized tests

* Mongo

* Oracle

* Sqlserver

* Sqlite

* Add casting supports for presto

* Remove Jeff's implementation of presto-jdbc because  its parent is
sql-jdbc

* Update presto-jdbc tests to use the same catalog for all datasets

* Add date extraction functions to the expression editor (#25382)

* make sure the semantic type of aggregated columns are integer

* no recursive call in annotate for date-extract func

* get-unit -> temporal-extract(column, unit)

* desguar nested datetime extraction too