* convert-timezone for mysql, mariadb, sqlserver, oracle

* sqlserver doesn't understand Asia/Ho_Chi_Minh

* vertica

* skip report-tz tests for sqlserver

* use the util fn to check args for convert-timezone

* fix for vertica not formatting in report-tz 

- Add tests to check when bucketing by week and week-of-year behave consistently, Closes #4910
- While doing the above, found that sqlserver does not respect `start-of-week` setting, so fix that. Fix #25356

* Implement advanced date/time/zone manipulation, part 1

Incorporate new functions into MBQL and add tests:
 - get-year
 - get-quarter
 - get-month
 - get-day
 - get-day-of-week
 - get-hour
 - get-minute
 - get-second

* Fix BigQuery implementations to call extract

Mark as not supported in legacy driver

* Add date extraction fns for Postgres

* Disable in MongoDB (for now at least)

Disable in BigQuery (legacy driver)

Add implementations for presto-jdbc

* Misc cleanup from Jeff's PR

* Update Jeff's implementation of bigquery-cloud-sqk

* Reorganized tests

* Mongo

* Oracle

* Sqlserver

* Sqlite

* Add casting supports for presto

* Remove Jeff's implementation of presto-jdbc because  its parent is
sql-jdbc

* Update presto-jdbc tests to use the same catalog for all datasets

* Add date extraction functions to the expression editor (#25382)

* make sure the semantic type of aggregated columns are integer

* no recursive call in annotate for date-extract func

* get-unit -> temporal-extract(column, unit)

* desguar nested datetime extraction too

* Enable clj-kondo for driver namespaces

* Fix CI command

* Fix CI config (?)

* Try hardcoding the driver directories in CI config

* Fix some busted stuff

* Enable the redundant fn wrapper linter

* Appease the namespace linter

Settings require db access so cannot be called at read/require time. Not
until we have initialized the db are setting values possible to be read.

* bumps outdated deps versions to be current

* un-upgrade h2 and jetty

* un-upgrade joda-time and kixi/stats

* drop Java 8 support in circle CI config

- things that used to rely on be-tests-java-8-ee now rely on be-tests-java-11-ee

* remove java 8 from github health check matrix

* revert toucan to 1.17.0

* revert mariadb java client to 2.7.5

* Back to 18, and handle new behavior

toucan used to just look in *.models.<model-name> for models and just
give up apparently. I made a feature that toucan will look in a model
registry to create models rather than using the convention
https://github.com/metabase/toucan/commit/762ad69defc1477423fa9423e9320ed318f7cfe7


but now we're getting errors in these tests about maps vs models.

```clojure
revision_test.clj:154
Check that revisions+details pulls in user info and adds description
expected: [#metabase.models.revision.RevisionInstance{:is_reversion false,
                                                      :is_creation false,
                                                      :message nil,
                                                      :user
                                                      {:id 1,
                                                       :common_name "Rasta Toucan",
                                                       :first_name "Rasta",
                                                       :last_name "Toucan"},
                                                      :diff
                                                      {:o1 nil, :o2 {:name "Tips Created by Day", :serialized true}},
                                                      :description nil}]
  actual: (#metabase.models.revision.RevisionInstance{:description nil,
                                                      :is_creation false,
                                                      :is_reversion false,
                                                      :user
                                                      {:id 1,
                                                       :first_name "Rasta",
                                                       :last_name "Toucan",
                                                       :common_name "Rasta Toucan"},
                                                      :message nil,
                                                      :diff
                                                      {:o1 nil,
                                                       :o2
                                                       #metabase.models.revision_test.FakedCardInstance{:name
                                                                                                        "Tips Created by Day",
                                                                                                        :serialized
                                                                                                        true}}})
```

The only difference here is `:o2` is a
`metabase.models.revision_test.FakedCardInstance` but still has the same
keys, `:name`, and `:serialized`. So all is well, we're just able to
make the model.

So a few different fixes. Some are use `partial=` which doesn't care
about record/map distinction. Some are just make the model, and some are
turning them into maps for revision strings (which more closely mimics
what the real revision stuff does):

```clojure
(defn default-diff-map
  "Default implementation of `diff-map` which simply uses clojures `data/diff` function and sets the keys `:before` and `:after`."
  [_ o1 o2]
  (when o1
    (let [[before after] (data/diff o1 o2)]
      {:before before
       :after  after})))

(defn default-diff-str
  "Default implementation of `diff-str` which simply uses clojures `data/diff` function and passes that on to `diff-string`."
  [entity o1 o2]
  (when-let [[before after] (data/diff o1 o2)]
    (diff-string (:name entity) before after)))
```

So all in all this change impacts nothing in the app itself, because
those models follow convention and are correct in
`metabase.models.<model-name>` and are thus "modelified":

```clojure
revision-test=> (revision/revisions Card 1)
[#metabase.models.revision.RevisionInstance{:is_creation true,
                                            :model_id 1,
                                            :id 1,
                                            :is_reversion false,
                                            :user_id 2,
                                            :timestamp #object[java.time.OffsetDateTime
                                                               "0x77e037f"
                                                               "2021-10-28T15:10:19.828539Z"],
                                            :object #metabase.models.card.CardInstance
                                            {:description nil,
                                             :archived false,
                                             :collection_position nil,
                                             :table_id 5,
                                             :database_id 2,
                                             :enable_embedding false,
                                             :collection_id nil,
                                             :query_type :query,
                                             :name "ECVYUHSWQJYMSOCIFHQC",
                                             :creator_id 2,
                                             :made_public_by_id nil,
                                             :embedding_params nil,
                                             :cache_ttl 1234,
                                             :dataset_query {:database 2,
                                                             :type :query,
                                                             :query {:source-table 5,
                                                                     :aggregation [[:count]]}},
                                             :id 1,
                                             :display :scalar,
                                             :visualization_settings {:global {:title nil}},
                                             :dataset false,
                                             :public_uuid nil},
                                            :message nil,
                                            :model "Card"}]
```

so the model/no-model is just arbitrary distinction in the test. All of
them in the actual app are turned into models:

```clojure
(defn- do-post-select-for-object
  "Call the appropriate `post-select` methods (including the type functions) on the `:object` this Revision recorded.
  This is important for things like Card revisions, where the `:dataset_query` property needs to be normalized when
  coming out of the DB."
  [{:keys [model], :as revision}]
  ;; in some cases (such as tests) we have 'fake' models that cannot be resolved normally; don't fail entirely in
  ;; those cases
  (let [model (u/ignore-exceptions (db/resolve-model (symbol model)))]
    (cond-> revision
    ;; this line would not find a model previously for FakedCard and
    ;; just return the map. But now the registry in toucan _finds_ the
    ;; model defintion and returns the model'd map
      model (update :object (partial models/do-post-select model)))))

(u/strict-extend (class Revision)
  models/IModel
  (merge models/IModelDefaults
         {:types       (constantly {:object :json})
          :pre-insert  pre-insert
          :pre-update  (fn [& _] (throw (Exception. (tru "You cannot update a Revision!"))))
          :post-select do-post-select-for-object}))
```

* try using mssql-jdbc 10.2.1.jre11

- Important that we get off the jre8 version

* various fixes that needn't be reverted

* Revert "various fixes that needn't be reverted"

This reverts commit 2a820db0743d0062eff63366ebe7bc78b852e81f.

* go back to using circle ci's java 11 docker image

* java-16 (?) -> java-17

* Revert "go back to using circle ci's java 11 docker image"

This reverts commit b9b14c535a689f701d7e2541081164288c988c4e.

Co-authored-by: dan sutton <dan@dpsutton.com>

* Make namespace aliasing consistent everywhere; enforce with clj-kondo

See the table of aliases in .clj-kondo/config.edn

Notable patterns:
- `[metabase.api.foo :as api.foo]`
- `[metabase.models.foo :as foo]`
- `[metabase.query-processor.foo :as qp.foo]`
- `[metabase.server.middleware.foo :as mw.foo]`
- `[metabase.util.foo :as u.foo]`
- `[clj-http.client :as http]` and `[metabase.http-client :as client]`

Fixes #19930.

* Remove references to qp/query->preprocessed

* Fix Oracle

* Support overriding ROWCOUNT for SQL Server

Add new "ROWCOUNT Override" connection property for `:sqlserver`, which will provide a DB-level mechanism to override the `ROWCOUNT` session level setting as needed for specific DBs

Change `max-results-bare-rows` from a hardcoded constant to a setting definition instead, which permits a DB level override, and move the former constant default to a new def instead (`default-max-results-bare-rows`)

For `:sqlserver`, set the DB-level setting override (if the connection property is set), via the `driver/normalize-db-details` impl

Add test to confirm the original scenario from #9940 works using this new override (set to `0`)

Move common computation function of overall row limit to the `metabase.query-processor.middleware.limit` namespace, and invoke it from execute now, called `determine-query-max-rows`

Add new clause to the `determine-query-max-rows` function that preferentially takes the value from `row-limit-override` (if defined)

Nested Queries Overhaul 2022: Split logic for determining appropriate table & column aliases out of SQL QP (#19384)

Co-authored-by: Alexander Polyankin <alexander.polyankin@metabase.com>

* Fix some Eastwood failures

* Fix a lot of Eastwood errors now that it runs against test namespaces [ci skip]

* Run Eastwood against test namespaces [WIP]

* Bump Eastwood version to fix some errors in Eastwood itself

* Fix another lint warning

* Fix test failure

* More test fixes 

* Fix all the warnings!

* Test fix 

* Bump Eastwood GH action timeout to 20 minutes (!)

* Add yyyymmddhhss coercions to type system

* Implementations for h2/mysql/postgres for yyyymmddhhss bytes and strings

* Mongo and oracle

* Adding checksum on migration it said was bad

* import OffsetDateTime

* Redshift and bigquery

* snowflake expectations

* sql server yyyymmddhhmmss. have to format the string then parse

sqlserver lacks a parse function that takes a format string, they just
take an integer that specifies a predefined format string. So we have
to make the string into the right format then parse.

* presto yyyymmddhhmmss

* Test byte conversions

* Remove errant `mt/set-test-drivers!`

* Remove sqlite, change keyword for multiple native types in def

the spec couldn't handle different shapes under the same keyword. so
just use :natives {:postgres "BYTEA"} :native "BYTEA"

* Make schema work with different shape maps

* hx/raw "'foo'" -> hx/literal "foo" and remove checksums

* _coercion_strategy -> _coercion-strategy

* Handle coercion hierarchy for :Coercion/YYYYMMDDHHMMSSBytes->Temporal

Similar to #15664, update the applicationName to include the local process identifier

Updating test accordingly

Bump mssql-jdbc version from 7.4.1.jre8 to 9.2.1.jre8

Bump plugin version accordingly

Override prepared-statement and statement multimethods for :sqlserver to not set holdability at the statement level

Fixing inaccurate log statements

* First pass using coercions

* Coercions

* Handle effective_type coercion_strategy in test data sets

* special-type -> semantic type in sample db

```clojure
user> (def config (metabase.db.spec/h2 {:db (str "/Users/dan/projects/clojure/metabase/resources/sample-dataset.db"
                                                 ";UNDO_LOG=0;CACHE_SIZE=131072;QUERY_CACHE_SIZE=128;COMPRESS=TRUE;"
                                                 "MULTI_THREADED=TRUE;MVCC=TRUE;DEFRAG_ALWAYS=TRUE;MAX_COMPACT_TIME=5000;"
                                                 "ANALYZE_AUTO=100")}))
user> (jdbc/execute! config ["UPDATE _metabase_metadata
                        SET keypath = 'PEOPLE.ZIP.semantic_type'
                        WHERE keypath = 'PEOPLE.ZIP.special_type'" ])
[1]
user> (jdbc/execute! config ["UPDATE _metabase_metadata
                        SET keypath = 'REVIEWS.BODY.semantic_type'
                        WHERE keypath = 'REVIEWS.BODY.special_type'" ])
[1]
```

* Correct mismatch in validation preventing sync

* fixing up alternative date tests

* More passing tests

* Tests for values, nested queries, fetch metadata

* tests

* tests passing

* Fixup mongo qp for coercions

locally i have some failing tests that are off by 1 errors:

Fail in compile-time-interval-test

[36m:mongo[0m Make sure time-intervals work the way they're supposed to. [:time-interval $date -4 :month] should give us something like Oct 01 2020 - Feb 01 2021 if today is Feb 17 2021

expected: [{$match {$and [{:$expr {$gte [$date {:$dateFromString {:dateString 2020-10-01T00:00Z}}]}} {:$expr {$lt [$date {:$dateFromString {:dateString 2021-02-01T00:00Z}}]}}]}} {$group {_id {date~~~day {:$let {:vars {:parts {:$dateToParts {:date $date}}}, :in {:$dateFromParts {:year $$parts.year, :month $$parts.month, :day $$parts.day}}}}}}} {$sort {_id 1}} {$project {_id false, date~~~day $_id.date~~~day}} {$sort {date~~~day 1}} {$limit 1048576}]

  actual: [{"$match"
            {"$and"
             [{:$expr {"$gte" ["$date" {:$dateFromString {:dateString "2020-11-01T00:00Z"}}]}}
              {:$expr {"$lt" ["$date" {:$dateFromString {:dateString "2021-03-01T00:00Z"}}]}}]}}
           {"$group"
            {"_id"
             {"date~~~day"
              {:$let
               {:vars {:parts {:$dateToParts {:date "$date"}}},
                :in {:$dateFromParts {:year "$$parts.year", :month "$$parts.month", :day "$$parts.day"}}}}}}}
           {"$sort" {"_id" 1}}
           {"$project" {"_id" false, "date~~~day" "$_id.date~~~day"}}
           {"$sort" {"date~~~day" 1}}
           {"$limit" 1048576}]
    diff: - [{"$match"
              {"$and"
               [{:$expr {"$gte" [nil {:$dateFromString {:dateString "2020-10-01T00:00Z"}}]}}
                {:$expr {"$lt" [nil {:$dateFromString {:dateString "2021-02-01T00:00Z"}}]}}]}}]
          + [{"$match"
              {"$and"
               [{:$expr {"$gte" [nil {:$dateFromString {:dateString "2020-11-01T00:00Z"}}]}}
                {:$expr {"$lt" [nil {:$dateFromString {:dateString "2021-03-01T00:00Z"}}]}}]}}]

* ee fixes

* UI to set coercion type

* Don't need to populate effective-type here

it actually has knock on effects:
- does more work now as almost every field has an update to do in
`add-extra-metadata`
- we have databases that have state that we don't create. druid for
example has stuff to mimic the dataset in tqpt/with-flattened-dbdef on
checkins but we don't actually create this. And our dbdef has a field
called "date" that is not present in the druid db, so if we attempt to
add metadata it fails and kills the rest of the metadata that we add.
- tests need this metadata to be present and the error causes field
visibilities (for example) to not be set

* Docstrings on shared lib

* Add effective and coercion to redshift expectations

* Fixup google analytics

* Derecordize instead of recordize the expectation

object details didn't work out well here. they added way more stuff
from the db than what is flowing through here.

```clojure
  actual: {:field
           {:name "DATE",
            :parent_id nil,
            :table_id 69,
            :base_type :type/Date,
            :effective_type :type/Date,
            :coercion_strategy nil,
            :semantic_type nil},
           :value {:type :date/all-options, :value "past5days"}}
    diff: - {:field
             {:description nil,
              :database_type "VARCHAR",
              :fingerprint_version 0,
              :has_field_values nil,
              :settings nil,
              :caveats nil,
              :fk_target_field_id nil,
              :custom_position 0,
              :active true,
              :last_analyzed nil,
              :position 1,
              :visibility_type :normal,
              :preview_display true,
              :database_position 0,
              :fingerprint nil,
              :points_of_interest nil}}
```

Object defaults adds quite a bit of stuff such that we'd be dissoc'ing
more than we are currently adding in

Co-authored-by: Cam Saul <1455846+camsaul@users.noreply.github.com>

* Optimize SQL Server GROUP BY and ORDER BY with :year, :month, or :day bucketing

* Avoid unneeded CAST() statements

* More tests 

* Revert unneeded changes

* Test fixes 

* Don't count MySQL and Postgres driver namespaces for test coverage

* Address feedback from @dpsutton; test fixes

* Test fixes 

Switch query execution to use `Statement` instead of `PreparedStatement` when there are no params

Add new multimethod to create a `Statement`, rather than `PreparedStatement`, which works similarly, called `statement`

Add new multimethod to run a SQL query against a Statement, similar to `execute-query!`, called `execute-statement!`

For consistency, rename `execute-query!` to `execute-prepared-statement!`

Change `execute-reducible-query` to capture whether the query has params, and if not, using the new multimethods instead

Update fetch-results-metadata-test so it replaces execute-select! instead of prepared-statement

Adding `statement-supported?` multimethod (defaults to true), to let drivers control whether statements are used at all

Updating Oracle driver to not override holdability for `statement`, similar to `prepared-statement`, and bumping module version

Updating SparkSQL driver to indicate that statements are not supported, and bumping module version

Fixing Redshift test to also override `execute-statement!` so it can capture the SQL for a statement

* Semantic types migration

* Fixup for basic querying

* Remove the relation type migration.

it makes the diff far too hard at the moment. need to be able to just
move everything from special_type -> semantic_type and then correct
the few that look at it to care about the effective type.

the PK/FK stuff gets really invasive and needs to be in its own much
smaller changeset

* Just rename special_type -> semantic_type for first change

* Special type -> semantic type

* special-type -> semantic-type

* SpecialType -> SemanticType

* special type -> semantic type

mostly in documentation, docstrings, etc

* Fix tests which relied on order of sets

database position was annotated by map-index'ing over the
set. changing `:special-type` to `:semantic-type` changed the order of
the seq produced from it.

* special -> semantic in schema_metadata

* Tim is awesome: Undo overeager special->semantic in docstrings

* Un-rename semantic_type in data_migrations

These migrations are run unless a migrations table marks them as
already having run. If they haven't run, then the db is older and the
column is special_type most likely. So we let them run as
`:special_type` and add some error handling to the migration runner
that is _opt in_.

```clojure
(defmigration ^{:author "camsaul", :added "0.20.0", :catch? true}
  migrate-field-types
  ...
  )

(try
  (@migration-var)
  (catch Exception e
    (if catch? ;; catch? from metadata
      (log/warn (format "Data migration %s failed: %s" migration-name (.getMessage e)))
      (throw e))))
```

* Fix merged master changes

just accepted their changes and fixed up rather than fix conflicts.

* Goodbye forever to expect-with-driver

* 

* Update expectations counts

* Convert metabase.mbql.util-test to the new style

* Convert some more test namespaces to the new style

* Convert four more namespaces to the new style

* Proposal: no more prefix namespaces

* Fix a few missing files

* group by "pivot-grouping" column

* update row order in tests

* fix tests

* fix assertion for successful response

* set up question without UI

* eslint

* random attempt to fix test before I set things up locally

* switch to other table 



* pr feedback [ci all]

* whoops! keyboard macro gone wrong [ci all]

* forgot to update a test [ci all]

* another test [ci all]

* put number in right place [ci all]

* Fixing up the errors on PostgreSQL (and other SQL databases)

Also ensuring tests run against drivers that support both expressions
and left-joins

[ci all]

* fixing test failures

* On Oracle and Snowflake, we need to add IDs *and* insert in smaller
batches

* Do not run these tests on Redshift

it takes more than 10 minutes just to insert the sample-dataset into
Redshift, so do not run these tests against Redshift. All other
databases in the `applicable-drivers` def are valid though.

Co-authored-by: Robert Roland <rob@metabase.com>

* Allow text strings to special type to date [ci drivers]

* Allow for timestamps/time/date for iso8601 strings [ci drivers]

* First pass at tests for :type/ISO8601DateTimeString [ci drivers]

* Only run ISO8601datetimestring tests on sql dbs

* Make the default a timestamp and mysql the exception [ci drivers]

* Fixes for sqlite and sqlserver

i'm not happy with sqlite yet. they are returning strings not
java.dates. But this is the same with a field who's base type is a
datetime. However, the UI allows for ordering on the date times but
not on the :type/ISO8601 fields and i'm not sure why

* Handle oracle [ci drivers]

oracle doesn't have a time type so use a similar edn dataset that
doesn't have the time column

* Move dataset definitions inline [ci drivers]

* correct oracle datetime format [ci drivers]

sparksql seems to have issues with time so just punt on that for now
by running in the test that doesn't have the time column

* wack-a-mole [ci drivers]

* Fix tests for redshift, oracle, and sparksql [ci drivers]

* Oracle is funky [ci drivers]

* need to investigate what's going on with oracle here [ci drivers]

text fields with special_type :type/ISO8601DateTimeString are queryable as dates a datetime field
:oracle
expected: (= 1 (count (mt/rows (mt/dataset string-times (mt/run-mbql-query times {:filter [:= [:datetime-field $ts :day] "2008-10-19"]})))))
  actual: java.lang.NullPointerException: null
 at oracle.jdbc.driver.T4CTTIdcb.fillupAccessors (T4CTTIdcb.java:1041)
    oracle.jdbc.driver.T4CTTIdcb.receiveCommon (T4CTTIdcb.java:221)
    oracle.jdbc.driver.T4CTTIdcb.receive (T4CTTIdcb.java:161)
    oracle.jdbc.driver.T4C8Oall.readDCB (T4C8Oall.java:991)
    oracle.jdbc.driver.T4CTTIfun.receive (T4CTTIfun.java:560)
    oracle.jdbc.driver.T4CTTIfun.doRPC (T4CTTIfun.java:252)
    oracle.jdbc.driver.T4C8Oall.doOALL (T4C8Oall.java:612)
    oracle.jdbc.driver.T4CPreparedStatement.doOall8 (T4CPreparedStatement.java:226)
    oracle.jdbc.driver.T4CPreparedStatement.doOall8 (T4CPreparedStatement.java:59)
    oracle.jdbc.driver.T4CPreparedStatement.executeForDescribe (T4CPreparedStatement.java:747)
    oracle.jdbc.driver.OracleStatement.executeMaybeDescribe (OracleStatement.java:904)
    oracle.jdbc.driver.OracleStatement.doExecuteWithTimeout (OracleStatement.java:1082)
    oracle.jdbc.driver.OraclePreparedStatement.executeInternal (OraclePreparedStatement.java:3780)
    oracle.jdbc.driver.T4CPreparedStatement.executeInternal (T4CPreparedStatement.java:1343)
    oracle.jdbc.driver.OraclePreparedStatement.executeQuery (OraclePreparedStatement.java:3822)
    oracle.jdbc.driver.OraclePreparedStatementWrapper.executeQuery (OraclePreparedStatementWrapper.java:1165)
    com.mchange.v2.c3p0.impl.NewProxyPreparedStatement.executeQuery (NewProxyPreparedStatement.java:431)
    metabase.driver.sql_jdbc.execute$eval71770$fn__71771.invoke (execute.clj:267)
    clojure.lang.MultiFn.invoke (MultiFn.java:234)
    metabase.driver.sql_jdbc.execute$execute_reducible_query.invokeStatic (execute.clj:392)
    metabase.driver.sql_jdbc.execute$execute_reducible_query.invoke (execute.clj:377)
    metabase.driver.sql_jdbc.execute$execute_reducible_query.invokeStatic (execute.clj:386)
    metabase.driver.sql_jdbc.execute$execute_reducible_query.invoke (execute.clj:377)
    metabase.driver.sql_jdbc$eval102870$fn__102871.invoke (sql_jdbc.clj:49)
    metabase.driver.oracle$eval187218$fn__187219.invoke (oracle.clj:268)
    clojure.lang.MultiFn.invoke (MultiFn.java:244)
    metabase.query_processor.context$executef.invokeStatic (context.clj:59)
    metabase.query_processor.context$executef.invoke (context.clj:48)
    metabase.query_processor.context.default$default_runf.invokeStatic (default.clj:69)
    metabase.query_processor.context.default$default_runf.invoke (default.clj:67)
    metabase.query_processor.context$runf.invokeStatic (context.clj:45)
    metabase.query_processor.context$runf.invoke (context.clj:39)
    metabase.query_processor.reducible$pivot.invokeStatic (reducible.clj:34)
    metabase.query_processor.reducible$pivot.invoke (reducible.clj:31)

* sparksql has some issues querying on datetime [ci drivers]

text fields with special_type :type/ISO8601DateTimeString are queryable as dates a datetime field
:sparksql: org.apache.spark.sql.catalyst.parser.ParseException:
DataType time() is not supported.(line 2, pos 178)

== SQL ==
-- Metabase
SELECT `t1`.`id` AS `id`, `t1`.`name` AS `name`, CAST(CAST(`t1`.`ts` AS timestamp) AS timestamp) AS `ts`, CAST(CAST(`t1`.`d` AS date) AS timestamp) AS `d`, CAST(CAST(`t1`.`t` AS time) AS timestamp) AS `t` FROM `string_times`.`times` `t1` WHERE (CAST(CAST(`t1`.`ts` AS timestamp) AS timestamp) >= CAST(to_utc_timestamp('2008-10-19 00:00:00', 'UTC') AS timestamp) AND CAST(CAST(`t1`.`ts` AS timestamp) AS timestamp) < CAST(to_utc_timestamp('2008-10-20 00:00:00', 'UTC') AS timestamp)) LIMIT 1048576
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------^^^

expected: (= 1 (count (mt/rows (mt/dataset string-times (mt/run-mbql-query times {:filter [:= [:datetime-field $ts :day] "2008-10-19"]})))))
  actual: java.sql.SQLException: org.apache.spark.sql.catalyst.parser.ParseException:
DataType time() is not supported.(line 2, pos 178)

== SQL ==
-- Metabase
SELECT `t1`.`id` AS `id`, `t1`.`name` AS `name`, CAST(CAST(`t1`.`ts` AS timestamp) AS timestamp) AS `ts`, CAST(CAST(`t1`.`d` AS date) AS timestamp) AS `d`, CAST(CAST(`t1`.`t` AS time) AS timestamp) AS `t` FROM `string_times`.`times` `t1` WHERE (CAST(CAST(`t1`.`ts` AS timestamp) AS timestamp) >= CAST(to_utc_timestamp('2008-10-19 00:00:00', 'UTC') AS timestamp) AND CAST(CAST(`t1`.`ts` AS timestamp) AS timestamp) < CAST(to_utc_timestamp('2008-10-20 00:00:00', 'UTC') AS timestamp)) LIMIT 1048576
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------^^^

 at org.apache.hive.jdbc.HiveStatement.execute (HiveStatement.java:297)
    org.apache.hive.jdbc.HiveStatement.executeQuery (HiveStatement.java:392)
    org.apache.hive.jdbc.HivePreparedStatement.executeQuery (HivePreparedStatement.java:109)
    com.mchange.v2.c3p0.impl.NewProxyPreparedStatement.executeQuery (NewProxyPreparedStatement.java:431)
    metabase.driver.sql_jdbc.execute$eval71770$fn__71771.invoke (execute.clj:267)
    clojure.lang.MultiFn.invoke (MultiFn.java:234)
    metabase.driver.sql_jdbc.execute$execute_reducible_query.invokeStatic (execute.clj:392)
    metabase.driver.sql_jdbc.execute$execute_reducible_query.invoke (execute.clj:377)
    metabase.driver.sql_jdbc.execute$execute_reducible_query.invokeStatic (execute.clj:386)
    metabase.driver.sql_jdbc.execute$execute_reducible_query.invoke (execute.clj:377)
    metabase.driver.sql_jdbc$eval102870$fn__102871.invoke (sql_jdbc.clj:49)
    metabase.driver.sparksql$eval187174$fn__187176.invoke (sparksql.clj:142)
    clojure.lang.MultiFn.invoke (MultiFn.java:244)
    metabase.query_processor.context$executef.invokeStatic (context.clj:59)
    metabase.query_processor.context$executef.invoke (context.clj:48)
    metabase.query_processor.context.default$default_runf.invokeStatic (default.clj:69)
    metabase.query_processor.context.default$default_runf.invoke (default.clj:67)
    metabase.query_processor.context$runf.invokeStatic (context.clj:45)
    metabase.query_processor.context$runf.invoke (context.clj:39)
    metabase.query_processor.reducible$pivot.invokeStatic (reducible.clj:34)
    metabase.query_processor.reducible$pivot.invoke (reducible.clj:31)
    metabase.query_processor.middleware.mbql_to_native$mbql__GT_native$fn__57878.invoke (mbql_to_native.clj:26)
    metabase.query_processor.middleware.check_features$check_features$fn__57065.invoke (check_features.clj:42)
    metabase.query_processor.middleware.optimize_datetime_filters$optimize_datetime_filters$fn__58077.invoke (optimize_datetime_filters.clj:133)
    metabase.query_processor.middleware.auto_parse_filter_values$auto_parse_filter_values$fn__55756.invoke (auto_parse_filter_values.clj:44)
    metabase.query_processor.middleware.wrap_value_literals$wrap_value_literals$fn__59813.invoke (wrap_value_literals.clj:149)
    metabase.query_processor.middleware.annotate$add_column_info$fn__55502.invoke (annotate.clj:575)
    metabase.query_processor.middleware.permissions$check_query_permissions$fn__56924.invoke (permissions.clj:70)
    metabase.query_processor.middleware.pre_alias_aggregations$pre_alias_aggregations$fn__58633.invoke (pre_alias_aggregations.clj:40)
    metabase.query_processor.middleware.cumulative_aggregations$handle_cumulative_aggregations$fn__57154.invoke (cumulative_aggregations.clj:61)
    metabase_enterprise.sandbox.query_processor.middleware.row_level_restrictions$apply_row_level_permissions$fn__60343.invoke (row_level_restrictions.clj:292)
    metabase.query_processor.middleware.resolve_joined_fields$resolve_joined_fields$fn__58912.invoke (resolve_joined_fields.clj:36)
    metabase.query_processor.middleware.resolve_joins$resolve_joins$fn__59239.invoke (resolve_joins.clj:183)
    metabase.query_processor.middleware.add_implicit_joins$add_implicit_joins$fn__50719.invoke (add_implicit_joins.clj:254)
    metabase.query_processor.middleware.large_int_id$convert_id_to_string$fn__57823.invoke (large_int_id.clj:44)
    metabase.query_processor.middleware.limit$limit$fn__57856.invoke (limit.clj:38)
    metabase.query_processor.middleware.format_rows$format_rows$fn__57795.invoke (format_rows.clj:84)
    metabase.query_processor.middleware.desugar$desugar$fn__57226.invoke (desugar.clj:22)
    metabase.query_processor.middleware.binning$update_binning_strategy$fn__56123.invoke (binning.clj:229)
    metabase.query_processor.middleware.resolve_fields$resolve_fields$fn__56714.invoke (resolve_fields.clj:24)
    metabase.query_processor.middleware.add_dimension_projections$add_remapping$fn__50239.invoke (add_dimension_projections.clj:318)
    metabase.query_processor.middleware.add_implicit_clauses$add_implicit_clauses$fn__50462.invoke (add_implicit_clauses.clj:141)
    metabase_enterprise.sandbox.query_processor.middleware.row_level_restrictions$apply_row_level_permissions$fn__60343.invoke (row_level_restrictions.clj:292)
    metabase.query_processor.middleware.add_source_metadata$add_source_metadata_for_source_queries$fn__50882.invoke (add_source_metadata.clj:105)
    metabase_enterprise.sandbox.query_processor.middleware.column_level_perms_check$maybe_apply_column_level_perms_check$fn__60025.invoke (column_level_perms_check.clj:24)
    metabase.query_processor.middleware.reconcile_breakout_and_order_by_bucketing$reconcile_breakout_and_order_by_bucketing$fn__58858.invoke (reconcile_breakout_and_order_by_bucketing.clj:98)

* Remove sparksql can't query on date [ci drivers]

* SQL: sync only tables for which we have SELECT privilege

* More elegant tests [ci drivers]

* Turn `get-tables` and `has-select-privilege?` into multimethods

* Rename `get-tables` → `db-tables`

* Remove debug import

* Align multimethod's signature with method's

* Very minor style changes

[ci drivers] - drivers will fail as of now, want CI to fail this PR

* Add postgres, vertica, oracle, redshift [ci drivers]

* Minor refactor [ci drivers]

* Fix tests: add grants to ad-hoc tables & views [ci drivers]

* We don't need DatabaseMetadata any more [ci drivers]

* Fix args [ci drivers]

* Fix redshift result transform function [ci drivers]

* Add debug data [ci all]

* Don't run tests for default (always true) implementation [ci drivers]

* Refactro [ci drivers]

* Pass driver explicitly [ci drivers]

* Pass db explicitly [ci drivers]

* Add fallback if no privileges are defined [ci drivers]

* Ignore errors when checking privileges [ci drivers]

* Fix format [ci drivers]

* Add logging [ci drivers]

* Remove cruft [ci drivers]

* Fix format [ci drivers]

* Use sql-jdbc.conn/db->pooled-connection-spec [ci drivers]

* We don't need full Database object [ci drivers]

* Refactor [ci drivers]

* cleanup [ci drivers]

* Batched metadata queries [ci drivers]

* Update tests [ci drivers]

* Make linter happy [ci drivers]

* Update test [ci drivers]

* Quote table name to make Oracle happy [ci drivers]

* Use sql.qp to generate probe query [ci drivers]

* make linter happy [ci drivers]

* Alias dummy in select [ci drivers]

* Add MySQL, Presto, MSSQL, Snowflake [ci drivers]

* Make linter happy [ci drivers]

* Don't use JDBC in Presto [ci drivers]

* Make probe select work in snowflake [ci drivers]

* Don't needlessly use `:default` [ci drivers]

* Nicer code grouping [ci drivers]

* Dispatch to correct (parent) method [ci drivers]

* Don't be too clever [ci drivers]

* Pass driver along [ci drivers]

* Better error msg [ci snowflake]

* snowflake weirdness [ci snowflake]

* Snowflake: pass along db name [ci snowflake]

* Snowflake: more debug data [ci snowflake]

* Cleanup [ci drivers]

* Implement code review suggestions [ci drivers]

* Test fallback path [ci drivers]

* Add missing refer [ci drivers]

* Move tests to individual drivers [ci drivers]

* Make linter happy [ci drivers]

* Fix test [ci drivers]

* Fix tests [ci drivers]

* Fix tests [ci drivers]

* Fix tests [ci drivers]

* Add vertica tests [ci drivers]

* Fix tests [ci drivers]

* Fix tests [ci drivers]

* Fix oracle tests [ci drivers]

* Fix tests [ci drivers]

* Fix Presto [ci drivers]

* snowflake: reuse db user [ci drivers]

* Fix tests [ci drivers]

* Add some doc strings [ci drivers]

* Redshift: don't mess around with passwords [ci drivers]

* Snowflake: reuse PUBLIC role [ci drivers]

* Fix tests [ci drivers]

* Further simplify [ci drivers]

* [ci drivers]

* Typo [ci drivers]

* Reuse user [ci drivers]

* Cleanup SQL [ci drivers]

* Use more sql utils [ci drivers]

* Presto is a lost cause [ci drivers]

* Oracle: drop semicolons [ci drivers]

* Oracle: use 2 users [ci oracle]

* Add debug data [ci oracle]

* add require [ci oracle]

* Oralce: get all perms [ci oracle]

* Oracle: don't setup nested users [ci oracle]

* Oracle: use system user [ci drivers]

* Oracle: proper cleanup [ci drivers]

* Oracle is case sensitive :( [ci drivers]

* snowflake: create a dummy user [ci snowflake]

* Test for privilege inheritance [ci drivers]

* Fix tests [ci drivers]

* Fix tests [ci drivers]

* Use the correct driver flag [ci drivers]

* Postgres: correctly handle foreign tables [ci drivers]

* Use probing [ci drivers]

* Fix presto

* Bring presto api in line with the rest [ci drivers]

* Make linter happy [ci drivers]

* Create user and give ownership [ci drivers]

* Don't pollute state [ci postgres]

* Remove db-specific tests [ci drivers]

* Fix requires [ci drivers]

* Fix indentation [ci drivers]

* Refactor [ci drivers]

* Fix test [ci drivers]

* Make linter happy [ci drivers]

* Fix presto tests [ci drivers]

* Snowflake: reorder fns [ci drivers]

* Typo [ci drivers]

* Cleanup [ci drivers]

* Account for excluded-schemas being nil [ci drivers]

* Refactor [ci drivers]

* Snowflake: grant select on view [ci drivers]

* Fix snowflake regression [ci drivers]

* Add test for `fast-active-tables` [ci drivers]

Co-authored-by: Walter Leibbrandt <23798+walterl@users.noreply.github.com>
Co-authored-by: Robert Roland <rob@metabase.com>

* Add `db-start-of-week` multimethod

* Update drivers & settings [ci drivers]

* Use tru in setting def [ci drivers]

* Remove reflections [ci drivers]

* Fix migration [ci drivers]

* Update adjust

* Punt on adjust for now

* Fix tests [ci drivers]

* Fix tests [ci drivers]

* Add missing args [ci drivers]

* Fix tests [ci all]

* Short-circuit if offset = 0 [ci drivers]

* Fix week of year test [ci drivers]

* h2: use iso-week [ci drivers]

* Typo [ci drivers]

* Add UI for setting

* fix FE assumptions

* Unified week-of-year [ci drivers]

* Make h2 work regardless of locale [ci drivers]

* h2: make `add-interval-honeysql-form` work with expression amounts [ci drivers]

* Fix h2 tests [ci all]

* Don't rely on int division [ci drivers]

* Better week-of-year [ci drivers]

* Make linter happy [ci drivers]

* More celanup [ci drivers]

* Typo [ci drivers]

* Fix hive & snowflake [ci drivers]

* Add missing require [ci drivers]

* Bump hive [ci drivers]

* Mongo: add week-of-year [ci all]

* Druid: week-of-year [ci drivers]

* Fix Mongo [ci drivers]

* More fixes [ci drivers]

* Fix oracle [ci drivers]

* tweak copy and move the setting up

* Oracle: use to_char [ci drivers]

* More tests [ci drivers]

* Remove unneeded migration [ci drivers]

* Druid: don't be fancy with woy [ci drivers]

* Remove references to `start_of_week` from tests [ci drivers]

* Druid: use correct values in test [ci drivers]

* More tests [ci drivers]

* Typo [ci drivers]

* Fix Google driver [ci drivers]

Co-authored-by: Paul Rosenzweig <paul.a.rosenzweig@gmail.com>
Co-authored-by: Maz Ameli <maz@metabase.com>
Co-authored-by: Cam Saul <github@camsaul.com>

* Add more math functions to MBQL [ci drivers]

* Add non-default implementations [ci drivers]

* Add non-default implementations [ci all]

* More lenient schema [ci drivers]

* Add type hints [ci drivers]

* Fix tests [ci all]

* Fix tests [ci all]

* Presto: stddev_samp -> stddev_pop [ci all]

* Fix indentation [ci all]

* Add tests [ci drivers]

* Don't support percentile in H2 [ci drivers]

* Fix tests [ci all]

* Fix mssql [ci drivers]

* FE support for new functions

* Fix mssql [ci drivers]

* Fix mssql [ci drivers]

* Fix unit test

* Implement code review suggestions [ci all]

* Fix vertica [ci drivers]

* FE: add length & rename expt -> exp [ci drivers]

* Don't i18n displayName [ci drivers]

* FE fixes

* Cleanup aggregation/filter type predicates, isStandard, isMetric/isSegment, and isCustom

* Test: alow expressions in filters

* Alow expressions in filters [ci drivers]

* Allow expressions in :between [ci drivers]

* Allow expressions in :inside [ci drivers]

* Fix log [ci drivers]

* h2: use log10 [ci drivers]

* Fix mssql & spark [ci drivers]

* fix hive [ci drivers]

* Don't reuse :percentile name [ci drivers]

* Fix filter methods

Co-authored-by: Tom Robinson <tlrobinson@gmail.com>

* Upgrade Chevrotain to v6.5.0

* Switch from embedded actions to visitor

* rename tokens to lexer, remove no-longer needed Chevrotain hack from webpack.config.js

* get tests passing again

* progress converting syntax parser

* got intermediate Sum(A) test passing

* more progress on parsing expressions

* get complex expression passing

* prettier

* linting

* Add TokenizedExpression snapshot test

* Add support to parser for recovering whitespace tokens

* Add case clause [ci all]

* fix test [ci all]

* Harden test [ci all]

* More tests [ci all]

* Correctly infer case return type [ci all]

* Fix test [ci all]

* Fix tests [ci all]

* Improved syntax parser

* More parser cleanup + start adding filters and functions

* Add CASE and partial filter support, cleanup tests

* Add string extracts [ci all]

* Typo [ci all]

* Make linter happy [ci all]

* Add regex-match-first [ci all]

* Fix arglist [ci all]

* Fix test [ci all]

* Add sum-where count-where and share support. Refactor aggregation tokens

* Fix tests [ci all]

* Add ns prefixes [ci all]

* Add ns qualifiers [ci all]

* Fix trim [ci all]

* Tests: ensure stable ordering of results [ci all]

* Fix trim [ci all]

* Support only 1-arg trim [ci all]

* Fix ns prefixes [ci all]

* Fix psql [ci all]

* Fix redshift [ci redshift]

* Refactor/simplify grammer to consolidate aggregation, expression, and filter functions

* Add isCase to isExpression

* Directly splice regex pattern for psql & redshift [ci all]

* Fix schema [ci all]

* Redshift: splice replace args [ci redshift]

* Add ns prefix [ci redshift]

* Typo [ci redshift]

* Filter expressions

* Add metrics and segments back in to expressions

* Disable tokenized editing

* Update tests to clojure.test [ci all]

* Add concat to sqlite [ci all]

* Add length [ci all]

* Add missing ns [ci all]

* Fix test [ci all]

* Fix messed up multimethods [ci all]

* Fix mssql [ci all]

* Fix sqlite concat [ci all]

* sqlite: correctly qoute literal strings [ci all]

* Misc improvements to syntax highlighting etc

* Cleanup tests [ci all]

* Reoder defs [ci all]

* Make linter happy [ci all]

* Mongo: add case [ci mongo]

* mongo: fix switch [ci mongo]

* typo [ci mongo]

* Improve syntax highlighter and suggestions

* Mongo: always have default fallback in case [ci mongo]

* Rearrange test to split out ones using expressions [ci mongo]

* Remove uneeded merge [ci all]

* misc

* Much improved syntax highlighting using recovery mode and partial CST

* Improved suggestions

* try monospace font for expressions

* Better typing in parser, various other fixes

* Avoid parsing multiple times

* Fix tests

* Fix tests

* More test fixes

* Special case for UnderlyingRecords drill with sum-where/count-where/share

* switch back to square brackets for identifiers

* change extract to regexextract

* Lift expressions to subselect [ci all]

* Commit on enter

* rename extract to substitute, fix tests

* Refactor [ci all]

* Fix normalization [ci all]

* Correctly handle joins [ci all]

Co-authored-by: Daniel Higginbotham <daniel@flyingmachinestudios.com>
Co-authored-by: Simon Belak <simon@metabase.com>
Co-authored-by: Maz Ameli <maz@metabase.com>

[ci drivers]