* Add failing tests

* Fix bigquery

* Formatting

* Replace literals for datetime-diff-time-zones-test

* Actually use fields

* Fix typo

* Rename dataset

* Add type/DateTimeWithTZ column, since redshift doesn't like type/DateTimeWithZoneOffset

* Add now with extract test

* Test against UTC too

* Refactor based on Ngoc's suggestion

* Move the config-from-file code into the advanced-config directory since that's the feature we want to flag on

* Always apply the :settings section first in the config file

* config-from-file should check that we have a premium token with the :advanced-config feature

* Sort namespace

* Settings does not have to come first anymore

* Remove empty `binding` forms

* Test fix? 

* Test fix? 



* Fix failing tests, for real this time

* Revert "Fix failing tests, for real this time"

This reverts commit 0a57055b1be249d9ba5b6b6ebe4f163e1f21e2b2.

* Don't make the tests parallel since that breaks other stuff

Co-authored-by: Nemanja Glumac <31325167+nemanjaglumac@users.noreply.github.com>

* Fix `has-user-setup` Setting not being cached correctly

* Fix Kondo error

* pie percent switch

* review

* Migrate to card viz-settings v.2 (#26798)

* Migrate to card viz-settings v.2

This uses the new :pie.percent_visibility key (an enum) instead of a
collection of booleans

[#26776]

* Move to def-json-migration

* Don't migrate empty values

* fix specs

Co-authored-by: Tim Macdonald <tim@metabase.com>

* Refactor so static combo chart is ready for reordering

* Correct test names

* Clean up static viz combo chart BE API

* sort static combo chart

* Order static funnel chart

* Remove unused import and function

* Fix BE tests

* Remove custom column name tests since logic is moved to FE

* Remove graph.series_order when changing dimensions (#26842)

* review

Co-authored-by: Nick Fitzpatrick <nick@metabase.com>
Co-authored-by: Aleksandr Lesnenko <alxnddr@gmail.com>

* Implement now for sql drivers

* Remove unused import

* Add tests

* Add more tests

* Implement bigquery, mysql, postgres

* Test now as argument

* Fix FE type inferencer

* Change feature flag to date-arithmetics

* Implement h2 driver

* Fix tests for sqlite

* Add snowflake implementation

* Remove type test

* Fix test for sqlite

* Update how we handle autocomplete and format for zero-arity functions in ExpressionEditor (#26563)

* Update suggestionText function

* Suggest with closing parens if zero-arity

* Format zero-arity functions with ()

* Refactor formatFunction

* Remove unused imports

* Add presto implementation

* Add sqlserver implementation

* Test in multiple timezones

* Remove setting test driver

* Add sparksql implementation

* Fix count() fe unit test

* Add fe MBQL unit test for now()

* Fix sparksql implementation

* Add vertica implementation

* Add oracle implementation

* Add vertica implementation

* Remove accidental form

* Fix vertica implementation for report timezones

* Remove unused multimethod

* Remove unused feature

* Update sparksql

* Fix fe unit tests

* Fix fe unit tests

* Update hive implementation

* Revert "Fix fe unit tests"

This reverts commit fde4dd0d.

* Revert "Fix fe unit tests"

This reverts commit 7b7429ea.

* Revert "Fix count() fe unit test"

This reverts commit c5c6f6f9.

* Revert "Refactor formatFunction"

This reverts commit ac50e73e.

* Revert "Update how we handle autocomplete and format for zero-arity functions in ExpressionEditor (#26563)"

This reverts commit cf39634e.

* Update hive

* snowflake: convert type to timestamptz

* sqlserver: rename date-trunc to zeroed-date-part

* Wrap with database type info

* Change fe unit test back to now without parens

* Change MBQL clauses config for now to return type "datetime"

* Fix presto

* Add some fe unit tests for parsing and resolving

* Update helper-text-strings

* Change from second to millisecond precision

* Remove test for second precision

* presto: Change from second to millisecond precision

* vertica: Change from second to millisecond precision

* vertica: Change from second to millisecond precision

* Change feature flag from date-arithmetics to now

* Change feature flag from date-arithmetics to now, mbql schema

* Allow now to be used in filter clauses

* Add test for filter

* Implement driver/database-supports? for each driver that implements :now

* Tidy tests

* Add back default sql implementation

* Add sqlite feature flag

* Add mongo implementation

* Fix mongo

* Format rows

* Change vertica to return timestamp with time zone

* Fix presto tests

* Remove unused require

* Update h2 function to return timestamp with time zone

* mongo: calculate now during query execution, not processing, for versions >=4.2 (#26822)

* replace with $$NOW

* Remove unused import

* Add back previous now implementation for versions <4.2

* Throw an error if version is <4.2

* Add i18n

* Ignore patch version

* Clean up `SchedulePicker` component (#26839)

* Move `SchedulePicker` to its own directory

* Sort imports

* Do basic clean up

* Extract `DEFAULT_DAY` to a constant

* Extract styled components

* Convert `SchedulePicker` to TypeScript

* Add `SchedulePicker` container

* Export `SchedulePicker` options

* Add basic `SchedulePicker` story

* Fix `SchedulePicker` crashes Storybook

* Tweak Storybook file

* Build datetime filters as datetime before question is saved (#26679)

* Revert "Update h2 function to return timestamp with time zone"

This reverts commit f7cce3ea7ccbbfc886d0942d66b1f8f60feb6aba.

Co-authored-by: Gustavo Saiani <gus@metabase.com>
Co-authored-by: Anton Kulyk <kuliks.anton@gmail.com>

* initial implementation

* some refactor and improved error handling

* another refactor

* fix reflection warning

* add test and more refactor

* another test

* misc comment improvements and formatting

* fix tests

* small refactor

* fix function call

* address tamas's comments

* Bigquery

* Redshift

* Snowflake

* no convert-timezone for sparksql because it only has one timestamp data type

* redshift -- wrap type when adding interval

* don't override results-timezone-id in tests

* fix redshift ignore timezone when insert timestamp with time zone test data

* datetime-add, datetime-subtract, temporal-extract, convert-timezone now can takes datetime literal as argument

* fix vertica

* Replace attempted-murders where possible

* Replace mt/rows with mt/formatted-rows

* Remove integer test and change comment

* convert-timezone for mysql, mariadb, sqlserver, oracle

* sqlserver doesn't understand Asia/Ho_Chi_Minh

* vertica

* skip report-tz tests for sqlserver

* use the util fn to check args for convert-timezone

* fix for vertica not formatting in report-tz 

* Include Amazon Athena support for dacort/metabase-athena-driver

* Use Metabase Maven repo to fetch Athena driver

* Copy test extensions from Damon's Athena driver repo

* Some code cleanup.

* Move namespaces => metabase.driver.athena

* Clean up the test extensions namespace

* 42 failures, 16 errors

* Fix regex support; disable a few tests

* Minor tweaks

* Fix data-source-name

* Fix :week and :day-of-week impls (mostly): 2 failures, 3 errors

* Fix OFFSET, :week-of-year; skip test that has TIME column

* Add Athena to CircleCI config. Don't wait for Java 11 tests to finish before driver tests.

* ALL TESTS ARE PASSING! <3

* Copy fixes for https://github.com/dacort/metabase-athena-driver/issues/115;

 add test

* We don't need to prep source files or fetch dependencies before running backend tests.

* Fix Eastwood error.

* Tweak CircleCI config.

* Fix TIMESTAMP WITH TIME ZONE

* Include the Athena/Redshift repos in the build-drivers deps.edn

* Build and release scripts need to have the :mvn/repos as well

* Revert change that enabled test for Presto

* Un-enable failing test for Snowflake as well.

* moves all is clauses into the test

- previously only the first few tests were being run

* Sort ns in `metabase.driver.athena-test`

* Prevent athena's log4j2.properties file from becoming log config

athena includes log4j2.properties top-level with the properties:

```
status = debug
rootLogger.level=debug
```

And this kills our beautifully crafted logging. Set
"log4j2.configurationFile" in bootstrap to our own log4j2.xml.

Log4j2 looks in a few places for its logging config, the first of which
is the properties file. So when the jar is loaded, log4j2 considers this
a logging config change and we lose our logging.

Co-authored-by: Damon P. Cortesi <d.lifehacker@gmail.com>
Co-authored-by: Bryan Maass <bryan.maass@gmail.com>
Co-authored-by: dan sutton <dan@dpsutton.com>

* Match static combo chart colors with app viz colors

* Match colors when individual series color is set

* Remove tests that test colors

Since colors is now determined in FE

* Fix static viz Series type

* Match static progress bar colors with app viz colors

* Simplify Clojure code, makes it more idiomatic

* Fix certain combo charts don't respect whitelabel colors

* Make Clojure code more idiomatic

* Fix Series type

* Add colors back to static internal page

* Simplify type predicate

* fix colors matching for multiple series with mulitple metrics or dimensions

* Handle legends not formatted in static viz in some cases

* Fix failed BE tests after BE API changed

* Fix failed BE tests after BE API changed

* Add multiple series dashcard test to `getSeriesWithColors` test

* Add `getSeriesWithLegends` tests

* Fix URL `/_internal/static-viz` not loading due to API change

* Correctly render legend when having dashcard title set

* Simplify combo-chart API

don't really need `settings-seqs` yet

* Make static combo chart component type easier to understand

* Address review: refactor `render-multiple-lab-chart`

* Make variable names easier to understand

* Simplify color API

* Address review on `body.clj`

* Address review, make parameter name more consistent

* Fix multiple scalars not rendering

* Remove unused imports

convert-timezone now requires source-timezone if the input column doesn't have timezone

* Adds rollback where necessary to v45 migrations
* Adds linter for migrations requiring rollback key
* Removes unused writeback table migrations
* Removes migrate down-one subcommand

* treat private-key conn props as base64 encoded

- fix typo

* handle connecting with local keypair

* adds test for snowflake keypair auth

* remove unused arg

* remove id, and only call byte-array->string once

* dont reflect for conversion from byte-array to str

* remove inline def

* more cleanup

- mess with tests to find the fabled metabase ci snowflake user

* remove un-needed test

* handle conn uri in connection-details->spec

* make sure tests are actually running

* encode properties in the connection uri

* typehint private keyfile

* respond to some review comments

- add a test for can-connect? => true, even when jdbc/query returns falsey

* remove unused logging require

* adds assoc-qp

- which will be used to non-destructively "assoc" onto the snowflake
connection query params

* hook in u.qp/assoc-qp to better update conn uri

* kill a newline

* reuse `conn-str-with-additional-opts` on snowflake conn uri

* update use utf-8 when encoding private-key-value

conveniently use `bytes` and `bytes?`

* Fix bug where grouping strategy would NPE when fingerprint was `nil`

The axes grouping relies on fingerprint data existing, AND for now only works on :type/Number columns, so this bugfix
guards against those situations where the logic would fail.

The bug that brought this to my attention came up when there was a Custom column. This set the col-meta count to be
one greater than the count of the 'real' columns, which threw off my old assumptions, where if the grouped-columns
count was less than the total col-meta count, you had a potential grouping. This broke in part because some custom
columns are essentially derived from a real column, and so will most likely be groupable with the column they're
derived from.

Anyway, this filters out columns that won't work, so we hopefully eliminate the problem.

* Update src/metabase/pulse/render/body.clj

Co-authored-by: Tim Macdonald <tim@metabase.com>

* Fix some broken logic

Co-authored-by: Tim Macdonald <tim@metabase.com>

* First pass at grouping axes

This PR uses the results_metadata key to look at fingerprints for numerical axes to try determine if each series on
the Y axis can be sanely represented on the same axis.

This is done by calculating an overlap (some value between 0 and 1) and grouping all axes on the LEFT if they pass the threshold, which is
some value between 0 and 1. The overlap is always calculated when there is SOME overlap between the ranges of each
axis, and is calculated as:

```clojure
(/ (- (max maximums) (min minimums))
   (- (min maxiumums) (max minimums)))
```

This is done to try catch situations where one column's range is entirely inside the other, but is much smaller (- max
min); such a case would have a small percent overlap by the above calculation, and implies that it might be better to
split the axes.

* Address feedback.

* Fix shape of data in tests of 2 private fns

* Add test for split axes

* Fixed error in test util

* render-utils does with-redef, disallows parallel tests. Makes sense

Card, Dashboard, Dimension, Metric, NativeQuerySnippet, Segment and
Timeline all now using their `:name` fields for the file label.

File names should now be more human-friendly.

* Initial support for pg for `date-diff`

* Make the useful-dates closer to one day

* Add simple FE stuff

* shorter test bodies

* Ensure we can use datediff functions in arithmetic expressions

* Correctly disable datediff for redshift

* simplify var names

* Support week

* cleanup test

* :datediff -> :datetimediff

* ngoc's suggestions

* Better acceptance test for datetimediff

* sort ns

* embrace the different cases for results

* bigquery day month year

* Reverse args

* Update test

* Centralize tests

* Change postgres day, month, year behaviour

* Refactor keep identity

* Tidy tests

* Tidy

* Fix bigquery week

* Add week tests

* Fix bigquery week

* Change mysql day, month, year behaviour

* Add test for hour, minute, second

* Fix postgres hour minute second

* Formatting

* Fix bigquery hour, minute, second

* Formatting

* Fix postgres timestamptz

* WIP

* Allow literals in datediff clauses

* Uncomment tests

* Fix bigquery when reporting timezone is not UTC

* Linting

* Moving away from dataset based tests

* Add timezone tests for week and tidy

* Remove unused import

* Consolidate tests

* Remove with-time-column dataset

* Remove more-useful-dates defdataset

* Remove redshift driver WIP

* Typo

* Move DatetimeLiteral clause into DateTimeExpressionArg

* Try changing test order

* Remove mt/with-report-timezone-id nil

* Add year report timezone tests

* Rename

* Rename

* Remove unused tables from useful-dates

* Remove useful-dates

* Update helper-text-strings

* Tidy

* Swap order of mt/with-report-timezone-id

* Change with-report-timezone-id; notify databases after running test

* Use temp setting for report-timezone instead

* Update helper-test-strings

* Handle literals in `datetimediff-base-base`

* Update src/metabase/driver/postgres.clj

Co-authored-by: metamben <103100869+metamben@users.noreply.github.com>

* Simplify postgres second

* Tidy: prefer hx arithmetic functions

* Drop coercion for string timestamp args for now

* Extract helper

* Revert "Change with-report-timezone-id; notify databases after running test"

This reverts commit 7abb543bed13d9d13295d0e00ad1293a40e40d53.

* Use ->timestamptz

* Use hx/->timestamp

* Rename dataset

* Undo changes to datetime-arithmetics? as these will not match by default

* Use proper format for offset datetime literals

* Shorten datetime literals in tests

* Removing notify-all-databases-updated from report-timezone setting

* Fix datetime-arithmetics test

* Revert "Fix datetime-arithmetics test"

This reverts commit 9141582508170bdbe99c4aa74bc4444e41475be6.

* Revert "Undo changes to datetime-arithmetics? as these will not match by default"

This reverts commit 9cb05f5475b24a78cb89601c918f3440606cc525.

* Add UTC timezone to tests

* Coerce strings to datetimes for ISO formats

* Revert "Removing notify-all-databases-updated from report-timezone setting"

This reverts commit 37356435a2e3981deca9ae76118d857babb2aada.

* Add comment to postgres driver implementation

* Formatting

* Disable datediff from redshift for the moment (for ever?)

* Override redshift driver/database-supports?

* Fix comments mixed up by refactoring

* Fix comments mixed up by refactoring 2

* Tidy comment

* Rename datetimediff to datetime-diff/datetimeDiff

* date-add -> datetime-add

* Linting

* add datetime-subtract to `datetime-arithmetics?`

rework tests a bit as well.

* Update docstring

* Use ->temporal-type and trunc to handle report-timezone for bigquery

* Log errors caught during sync steps (#26306)

* log errors caught during sync steps

* remove accidental extra parens

* Errors combining datetime interval addition with datetime functions (#26279)

* Add failing tests

* Fix failing tests

* Update shared/src/metabase/mbql/util.cljc

Co-authored-by: Ngoc Khuat <qn.khuat@gmail.com>

* datetime-add and datetime subtract should annotate type by col type

* Fix infer-expression-type for datetime-add/subtract with second, minute, hour

* Undo last commit; they actually always return :type/DateTime

* Fix test based on last commit

* Undo unrelated refactor

* Only test drivers that support expressions

* Only test drivers that support expressions, again

* Update tests from legacy mbql

* Change infered-col-type to be a function again, not macro

* Fix test

Co-authored-by: Ngoc Khuat <qn.khuat@gmail.com>

* whitespace

* Remove comments

* Add explanation for datetime_diff

* Refactor: replace cast and add `mt/with-driver :bigquery-cloud-sdk` where report-timezone is relevant

* Add failing tests

* Fix failing tests

* Technically it should be bigquery-type

* Update modules/drivers/bigquery-cloud-sdk/src/metabase/driver/bigquery_cloud_sdk.clj

Co-authored-by: Cam Saul <1455846+camsaul@users.noreply.github.com>

* Move documentation to metabase.mbql.schema

* Update DatetimeDiffUnits

Co-authored-by: Cam Saul <1455846+camsaul@users.noreply.github.com>

* Add error type and optimize case expressions to driver implementations

* refactor for brevity

* Fix error with postgres

* Fix error with postgres

* Handle string literal parsing in wrap-value-literals

* Remove ->timestamptz

* Add comments + TODOs showing arithmetic expressions should return numeric values

* Add explanation of arithmetic expression as docstring

* Add test for normalize-mbql-clause-tokens

* Remove unused import

* Remove unused form

* Switch tests to use attempted-murders dataset

* Undo optimization that broke tests

* Validate non-temporal types for bigquery

* Add error handling for incorrect types

* Fix mysql type checking

* Fix mysql type checking

* invalid-parameter -> invalid-query

* Use date-trunc and extract

* Remove unit error handling

* DatetimeLiteral -> DateOrDatetimeLiteral

* Remove unused binding

* Fix mysql type checking

Co-authored-by: Callum Herries <hi@callumherries.com>
Co-authored-by: Cal Herries <39073188+calherries@users.noreply.github.com>
Co-authored-by: metamben <103100869+metamben@users.noreply.github.com>
Co-authored-by: Noah Moss <32746338+noahmoss@users.noreply.github.com>
Co-authored-by: Ngoc Khuat <qn.khuat@gmail.com>
Co-authored-by: Cam Saul <1455846+camsaul@users.noreply.github.com>

* add app DB and version to common snowplow event context

* use .getDatabaseProductName for db type

* fix kondo error

* add new instance schema with app db type and version fields

* add maxLength to new fields

* address cam's feedback

- Add tests to check when bucketing by week and week-of-year behave consistently, Closes #4910
- While doing the above, found that sqlserver does not respect `start-of-week` setting, so fix that. Fix #25356

This reverts commit 2a857d7d.

The service provider's output has changed back again.

* realize lazy seqs during exports

* add test & helper

* update based on tamas's comment

* use mapv

* remove unused import

* try to fix test

* switch to using just a LocalDate for the second test since the ZonedDateTime test was failing for certain drivers and not others, and that isnt relevant to this test

* Add failing tests

* Fix failing tests

* Update shared/src/metabase/mbql/util.cljc

Co-authored-by: Ngoc Khuat <qn.khuat@gmail.com>

* datetime-add and datetime subtract should annotate type by col type

* Fix infer-expression-type for datetime-add/subtract with second, minute, hour

* Undo last commit; they actually always return :type/DateTime

* Fix test based on last commit

* Undo unrelated refactor

* Only test drivers that support expressions

* Only test drivers that support expressions, again

* Update tests from legacy mbql

* Change infered-col-type to be a function again, not macro

* Fix test

Co-authored-by: Ngoc Khuat <qn.khuat@gmail.com>

* [:field-id id] => [:field id nil] or $name. Except places where tests
are meant to check normalization

* [:datetime-field [:field id] unit] -> [:field id {:temporal-unit unit}]
or !unit.name

* [:fk-> [:field 1] [:field 2]] => [:field 2 {:source-field 1}]

* Uses all text scorers in the final result

- instead of just the maximum one
- add tests

* add prefix scorer test + fix text-score-with

* linter fixes

* pass in number of results to find

* refactor test function

* fix linter by removing unused namespace: metabase.util

* limit arity of serialize to 3

* make oss-score and ee-score different things

- They were defined to be exactly the same, but should be different!
- Update some tests that broke when a test function was fixed

* remove extra let

* move rseq back out of sorted-take

* improve test feedback

* force weight of text based scorers always weigh 10

* handle 0 score/weights when normalizing scores

* add nil check

* fix more subtle test differences

* more test fiddling

- still test that :offset and :limit respect limits

* reuse bit->boolean from api collection

* clean up some tests

- filter -> remove
- replace some magic numbers
- revert to testing entire maps instead of names of sorted items

* add test, docstring, and weight

* sort ns requires

* responding to most of the review comments

* start our zero-score sum check with 0

* do not tokenize / normalize nil raw-search-string

* force equality in basic search test

* modify test to work in dev and test environments

* use display_name in results when appropriate

- This was looking for the _first_ column that had a non-zero score, but
actually we need to consider all relevant columns.
- Uses them to figure out if there is a display name, and if there is,
to use it.
- Coppied over the logic about showing :context from the prior approach