Co-authored-by: Mark Bastian <markbastian@gmail.com>

* Fix flaky test and test for a new case

flaky test was annoying. It would timeout after 45 seconds. And
annoyingly, it times out on the _client_, not the server. So we don't
get the proper error message we expect.

And what's happening is that the backend just hooks up the pipe from
it's request to the response. And if no bytes ever come across, we just
twiddle our thumbs until the http request in the test gives up.

The only reliable way i could think of to fix this is to spin up a
webserver that just accepts and then ignores requests (which is what
github did to us in the flaky test).

Had to bump the ring adapter to get the newly exposed `:async-timeout
60000` parameter. By default the webserver in the test has a timeout of
30 seconds, which is shorter than the 45 second timeout for the test's
request to the geojson endpoint.

* add comment

* clean up test server logic

---------

Co-authored-by: dpsutton <dan@dpsutton.com>
Co-authored-by: Nemanja Glumac <31325167+nemanjaglumac@users.noreply.github.com>

* Build script overhaul 2023 (#28767)

* Build Script overhaul

* Remove stray deps.edn

* Include :ci profile for build and release tests

* Fix release script

* Update markdown dox

* Update another dox

* Tweakz

* Don't pin new version of data.xml to core project, just for build scripts

* Ignore unrecognized options

* Fix i18n/enumerate

* Fix broken link

---------

Co-authored-by: Cam Saul <1455846+camsaul@users.noreply.github.com>
Co-authored-by: Nemanja <31325167+nemanjaglumac@users.noreply.github.com>

Dev experience:
- CLJS now has source maps
- Devtools extension renders CLJS data in Chrome (et al) Devtools
- Use a debug build of CLJS in the tests
- Fixed line numbers and breakpoints being off-by-one in Devtools

CLJC infra:
- More robust direct approach to JS<->CLJS conversions, powered by
  Malli transformers.
- Powerful `define-getters-and-setters` macro to generate the accessors.

According to @dpsutton is not used anymore. I wasn’t able to find it on the code as well

* Removes :presto driver and tests
* Merges :presto-common into :presto-jdbc
* Adds migration to update presto databases to presto-jdbc

---------

Co-authored-by: Cam Saul <github@camsaul.com>

* Update H2 command

* Clarify documentation for token check

A `metabase.domain-entities.malli/defn` macro that (in CLJS) uses the
schemas for the arguments to drive wrapping of vanilla JS objects for
idiomatic use in CLJC code.

See the first two functions in `metabase.domain-entities.queries.util`
which get passed vanilla JS objects but can use them as CLJS maps.

https://github.com/camsaul/toucan2/pull/126

Editing the data model loads a _lot_ of data. It hits the following form
from the api:

```clojure
(hydrate [:tables [:fields [:target :has_field_values]
                   :has_field_values] :segments :metrics])
```

On my test instances, with 1500 tables each with 14 fields: 15mb and 359
tables each with 14 fields: 3.69mb. Not great.

If you want to recreate, create an empty db, sync it, then

```clojure
  (time
   (let [conn (sql-jdbc.conn/db->pooled-connection-spec 3)]
     (dotimes [n 359]
       (jdbc/execute! conn [(format "create table if not exists foo_%d (id int, col_1 text, col_2 text, col_3 text, col_4 text, col_5 text, col_6 text, col_7 text, col_8 text, col_9 text, col_10 text, col_11 text, col_12 text, col_13 text, col_14 text)" n)]))))

  (sync/sync-database! (db/select-one 'Database :id 3) {:scan :schema})

  (binding [api/*current-user-permissions-set* (delay #{"/"})]
    (count (#'metabase.api.database/db-metadata 3 true true)))
```

And the source of the bug was a classic recur with a lazy concat.

```diff
diff --git a/src/toucan2/tools/hydrate.clj b/src/toucan2/tools/hydrate.clj
index e1d0459..10c9c49 100644
--- a/src/toucan2/tools/hydrate.clj
+++ b/src/toucan2/tools/hydrate.clj
@@ -456,7 +456,7 @@
     (if (empty? hydrated-instances)
       (concat acc (map :instance annotated-instances))
       (let [[not-hydrated [_needed-hydration & more]] (split-with (complement :needs-hydration?) annotated-instances)]
-        (recur (concat acc (map :instance not-hydrated) [(first hydrated-instances)])
+        (recur (vec (concat acc (map :instance not-hydrated) [(first hydrated-instances)]))
                more
                (rest hydrated-instances))))))

```

Upgrade to Jetty 11 via https://github.com/sunng87/ring-jetty9-adapter. Required updating from javax servlet packages to jakarta, since jakarta is newer.

Includes much-improved testing for the logger in CLJ and CLJS.

**This moves us from explicit `shadow-cljs.edn` dependencies to relying
on deps.edn for everything. It seems to be working nicely.

This has the same interface as `clojure.tools.logging` but supports CLJS
as well. It was formerly known as `metabase.shared.util.log`.

* Switch to the new Hawk test runner

* Fix some stuff

* Fix everything

* Commentz

* Bump init timeout?

* Fix Kondo config

* Remove unused

* Bump test runner version to get `:exclude-tags` support

This work removes a spec and a p.schema for data graph permissions.

There are still some specs but they are for execution, and collection graph permissions. Those should be targeted next.

* decoded + verified FIXME

* still good

* some cleanup

* a little more cleaning up

* bit more cleaning

* converted update-group-permissions! -> mu/defn

* use the correct shape for new-group-perms

* remove bunch of pschemas

* linter appeasement

* update test for setting dissallowed schema

* respond to review comments

* fix some tests + improve mu/defn error messages

* a lint fix

* lint fix

* read and query are optional

* update expected error message to the improved one

* stop calling trs at the top level

* fix function signature

* improve test

* Bump clj-kondo to 2022.12.10

[Fixes #27512]

* Use Kondo's :config-in-call instead of grosser hacks

* Switch to Toucan 2 part 1

* Fix Kondo errors

* Switch to Honey SQL 2 for some application DB things

* Fix Chain Filter query

* More fixes

* Test fixes :wrench

* Fix Kondo lint errors

* Test fixes 

* Test and formatting fixes for MySQL

* Fix new code using `db/query`

* Fix circular references

* Switch to Honey SQL 2, Part 1

* Test fix

* Revert deletion of ratios-test

* Fix Kondo error

* Fix stack overflow

* Update code to use new names for schemas

* Switch to Honey SQL 2, Part 1

* Test fix

* Fix Kondo error

* [Toucan 2 Prep] Add helper for extending `IModel`; namespace Toucan properties

* Clean namespaces

* Test fixes 

* [Toucan 2 Prep] Add `define-*-hydration-method` macros

* Sort namespaces

* Disable Eastwood `:unused-private-vars` linter

* Switch back to main fork of `clojure.java-time` and bump version

* Run uberjar build steps in one command (faster CI)

* Fix `truncate-to` for new version of `clojure.java-time`

* Start-of-cycle 46 basic dependency version bump

Some dependencies with available major version bumps are only updated
to the latest minor version of the current major version. Some
additional dependencies will be updated in future PRs.

* Removes commons-lang from sparksql deps exceptions

* Fixes MongoDB SRV test for monger 3.6.0

* fixes SQLite datetime tests for new driver version 3.40

* bump apache poi

* bump docjure

* move test-chuck and sort dev deps

* Update deps.edn comment typo

Co-authored-by: Tim Macdonald <tim@metabase.com>

* wrangle some curly braces

Co-authored-by: Tim Macdonald <tim@metabase.com>

Updates embedded H2 driver to v2.1.212. When a local H2 v1 file-based application database is detected, it will be
automatically updated in-place at startup. Migrations and tests have been updated to account for changes in H2
behavior. Sample data and test harness database files in the repo have all been updated to H2 v2 file format.

Co-authored-by: Braden Shepherdson <braden@metabase.com>
Co-authored-by: Cam Saul <github@camsaul.com>
Co-authored-by: Cam Saul <1455846+camsaul@users.noreply.github.com>

* add the library, and dev/start-malli!

* Update deps.edn

Co-authored-by: metamben <103100869+metamben@users.noreply.github.com>

* enable the use of regexes as malli schemas

Co-authored-by: metamben <103100869+metamben@users.noreply.github.com>

* ≈ [WIP] [ci skip]

* Minor cleanup [ci skip]

* Code cleanup. [ci skip]

* Kondo in CI should fetch library configs

* Bump Methodical version

* Change name to `=?`

* Add `#exactly` reader tag

* Add `#schema` data reader

* Fix dev deps indentation and add `algo.generic`

* Add `approx=`

* Improved version of `#approx`

* Just check in third party Kondo config for now instead of fighting CI

* Update test/metabase/test_runner/assert_exprs/approximately_equal_test.clj

Co-authored-by: Tim Macdonald <tim@metabase.com>

* Address PR feedback and fix sequence comparison

Co-authored-by: Tim Macdonald <tim@metabase.com>

* Include Amazon Athena support for dacort/metabase-athena-driver

* Use Metabase Maven repo to fetch Athena driver

* Copy test extensions from Damon's Athena driver repo

* Some code cleanup.

* Move namespaces => metabase.driver.athena

* Clean up the test extensions namespace

* 42 failures, 16 errors

* Fix regex support; disable a few tests

* Minor tweaks

* Fix data-source-name

* Fix :week and :day-of-week impls (mostly): 2 failures, 3 errors

* Fix OFFSET, :week-of-year; skip test that has TIME column

* Add Athena to CircleCI config. Don't wait for Java 11 tests to finish before driver tests.

* ALL TESTS ARE PASSING! <3

* Copy fixes for https://github.com/dacort/metabase-athena-driver/issues/115;

 add test

* We don't need to prep source files or fetch dependencies before running backend tests.

* Fix Eastwood error.

* Tweak CircleCI config.

* Fix TIMESTAMP WITH TIME ZONE

* Include the Athena/Redshift repos in the build-drivers deps.edn

* Build and release scripts need to have the :mvn/repos as well

* Revert change that enabled test for Presto

* Un-enable failing test for Snowflake as well.

* moves all is clauses into the test

- previously only the first few tests were being run

* Sort ns in `metabase.driver.athena-test`

* Prevent athena's log4j2.properties file from becoming log config

athena includes log4j2.properties top-level with the properties:

```
status = debug
rootLogger.level=debug
```

And this kills our beautifully crafted logging. Set
"log4j2.configurationFile" in bootstrap to our own log4j2.xml.

Log4j2 looks in a few places for its logging config, the first of which
is the properties file. So when the jar is loaded, log4j2 considers this
a logging config change and we lose our logging.

Co-authored-by: Damon P. Cortesi <d.lifehacker@gmail.com>
Co-authored-by: Bryan Maass <bryan.maass@gmail.com>
Co-authored-by: dan sutton <dan@dpsutton.com>

* Initial support for pg for `date-diff`

* Make the useful-dates closer to one day

* Add simple FE stuff

* shorter test bodies

* Ensure we can use datediff functions in arithmetic expressions

* Correctly disable datediff for redshift

* simplify var names

* Support week

* cleanup test

* :datediff -> :datetimediff

* ngoc's suggestions

* Better acceptance test for datetimediff

* sort ns

* embrace the different cases for results

* bigquery day month year

* Reverse args

* Update test

* Centralize tests

* Change postgres day, month, year behaviour

* Refactor keep identity

* Tidy tests

* Tidy

* Fix bigquery week

* Add week tests

* Fix bigquery week

* Change mysql day, month, year behaviour

* Add test for hour, minute, second

* Fix postgres hour minute second

* Formatting

* Fix bigquery hour, minute, second

* Formatting

* Fix postgres timestamptz

* WIP

* Allow literals in datediff clauses

* Uncomment tests

* Fix bigquery when reporting timezone is not UTC

* Linting

* Moving away from dataset based tests

* Add timezone tests for week and tidy

* Remove unused import

* Consolidate tests

* Remove with-time-column dataset

* Remove more-useful-dates defdataset

* Remove redshift driver WIP

* Typo

* Move DatetimeLiteral clause into DateTimeExpressionArg

* Try changing test order

* Remove mt/with-report-timezone-id nil

* Add year report timezone tests

* Rename

* Rename

* Remove unused tables from useful-dates

* Remove useful-dates

* Update helper-text-strings

* Tidy

* Swap order of mt/with-report-timezone-id

* Change with-report-timezone-id; notify databases after running test

* Use temp setting for report-timezone instead

* Update helper-test-strings

* Handle literals in `datetimediff-base-base`

* Update src/metabase/driver/postgres.clj

Co-authored-by: metamben <103100869+metamben@users.noreply.github.com>

* Simplify postgres second

* Tidy: prefer hx arithmetic functions

* Drop coercion for string timestamp args for now

* Extract helper

* Revert "Change with-report-timezone-id; notify databases after running test"

This reverts commit 7abb543bed13d9d13295d0e00ad1293a40e40d53.

* Use ->timestamptz

* Use hx/->timestamp

* Rename dataset

* Undo changes to datetime-arithmetics? as these will not match by default

* Use proper format for offset datetime literals

* Shorten datetime literals in tests

* Removing notify-all-databases-updated from report-timezone setting

* Fix datetime-arithmetics test

* Revert "Fix datetime-arithmetics test"

This reverts commit 9141582508170bdbe99c4aa74bc4444e41475be6.

* Revert "Undo changes to datetime-arithmetics? as these will not match by default"

This reverts commit 9cb05f5475b24a78cb89601c918f3440606cc525.

* Add UTC timezone to tests

* Coerce strings to datetimes for ISO formats

* Revert "Removing notify-all-databases-updated from report-timezone setting"

This reverts commit 37356435a2e3981deca9ae76118d857babb2aada.

* Add comment to postgres driver implementation

* Formatting

* Disable datediff from redshift for the moment (for ever?)

* Override redshift driver/database-supports?

* Fix comments mixed up by refactoring

* Fix comments mixed up by refactoring 2

* Tidy comment

* Rename datetimediff to datetime-diff/datetimeDiff

* date-add -> datetime-add

* Linting

* add datetime-subtract to `datetime-arithmetics?`

rework tests a bit as well.

* Update docstring

* Use ->temporal-type and trunc to handle report-timezone for bigquery

* Log errors caught during sync steps (#26306)

* log errors caught during sync steps

* remove accidental extra parens

* Errors combining datetime interval addition with datetime functions (#26279)

* Add failing tests

* Fix failing tests

* Update shared/src/metabase/mbql/util.cljc

Co-authored-by: Ngoc Khuat <qn.khuat@gmail.com>

* datetime-add and datetime subtract should annotate type by col type

* Fix infer-expression-type for datetime-add/subtract with second, minute, hour

* Undo last commit; they actually always return :type/DateTime

* Fix test based on last commit

* Undo unrelated refactor

* Only test drivers that support expressions

* Only test drivers that support expressions, again

* Update tests from legacy mbql

* Change infered-col-type to be a function again, not macro

* Fix test

Co-authored-by: Ngoc Khuat <qn.khuat@gmail.com>

* whitespace

* Remove comments

* Add explanation for datetime_diff

* Refactor: replace cast and add `mt/with-driver :bigquery-cloud-sdk` where report-timezone is relevant

* Add failing tests

* Fix failing tests

* Technically it should be bigquery-type

* Update modules/drivers/bigquery-cloud-sdk/src/metabase/driver/bigquery_cloud_sdk.clj

Co-authored-by: Cam Saul <1455846+camsaul@users.noreply.github.com>

* Move documentation to metabase.mbql.schema

* Update DatetimeDiffUnits

Co-authored-by: Cam Saul <1455846+camsaul@users.noreply.github.com>

* Add error type and optimize case expressions to driver implementations

* refactor for brevity

* Fix error with postgres

* Fix error with postgres

* Handle string literal parsing in wrap-value-literals

* Remove ->timestamptz

* Add comments + TODOs showing arithmetic expressions should return numeric values

* Add explanation of arithmetic expression as docstring

* Add test for normalize-mbql-clause-tokens

* Remove unused import

* Remove unused form

* Switch tests to use attempted-murders dataset

* Undo optimization that broke tests

* Validate non-temporal types for bigquery

* Add error handling for incorrect types

* Fix mysql type checking

* Fix mysql type checking

* invalid-parameter -> invalid-query

* Use date-trunc and extract

* Remove unit error handling

* DatetimeLiteral -> DateOrDatetimeLiteral

* Remove unused binding

* Fix mysql type checking

Co-authored-by: Callum Herries <hi@callumherries.com>
Co-authored-by: Cal Herries <39073188+calherries@users.noreply.github.com>
Co-authored-by: metamben <103100869+metamben@users.noreply.github.com>
Co-authored-by: Noah Moss <32746338+noahmoss@users.noreply.github.com>
Co-authored-by: Ngoc Khuat <qn.khuat@gmail.com>
Co-authored-by: Cam Saul <1455846+camsaul@users.noreply.github.com>

Co-authored-by: Noah Moss <32746338+noahmoss@users.noreply.github.com>

First seen in trivy report:
https://github.com/metabase/metabase/pull/26161/checks?check_run_id=9326286850

CVE:
https://avd.aquasec.com/nvd/cve-2022-40151

xstream: Xstream to serialise XML data was vulnerable to Denial of
Service attacks High
Package: com.fasterxml.woodstox:woodstox-core
Installed Version: 6.2.6
Vulnerability CVE-2022-40151
Severity: HIGH
Fixed Version: 5.4.0, 6.4.0

Bumping deps and comparing `clj -X:deps tree` shows the change only adds
the new dep top level and no new deps are brought in by the change.

```
❯ diff --unified deps deps-updated
--- deps	2022-11-07 08:43:21.000000000 -0600
+++ deps-updated	2022-11-07 08:49:56.000000000 -0600
@@ -9,6 +9,8 @@
   X org.slf4j/slf4j-api 1.7.25 :use-top
   X org.apache.logging.log4j/log4j-api 2.18.0 :use-top
   X org.apache.logging.log4j/log4j-core 2.18.0 :use-top
+com.fasterxml.woodstox/woodstox-core 6.4.0
+  . org.codehaus.woodstox/stax2-api 4.2.1
 joda-time/joda-time 2.10.13
 commons-codec/commons-codec 1.15
 weavejester/dependency 0.2.1
@@ -285,8 +287,7 @@
   . org.apache.santuario/xmlsec 2.3.0
     X org.slf4j/slf4j-api 1.7.32 :use-top
     X commons-codec/commons-codec 1.15 :use-top
-    . com.fasterxml.woodstox/woodstox-core 6.2.6
-      . org.codehaus.woodstox/stax2-api 4.2.1
+    X com.fasterxml.woodstox/woodstox-core 6.2.6 :use-top
     . jakarta.xml.bind/jakarta.xml.bind-api 2.3.3
       . jakarta.activation/jakarta.activation-api 1.2.2
   . org.opensaml/opensaml-xmlsec-api 3.4.6
```

* Initialize Metabase from a file

* Code cleanup

* Support optional template tags; more dox

* Finish documentation for ns str

* Force the first User to be an admin

* Tests for DB creation

* Add test to ensure that you can set User password with an {{env ...}} template

* Remove data app permission code (#26016)

* Revert "Use "apps" collection namespace for app collections (#25963)"

This reverts commit a1894202.

* Revert "Implement EE/Pro app permissions (#25764)"

This reverts commit cf9b1bf5.

* Revert "Implement global app permissions for the "All Users" group (#25679)"

This reverts commit 63e950f2.

* Revert "Require data permissions for executing actions (#25784)"

This reverts commit 732bf6b1.

* Revert "Prevent modifying app permissions via the collection endpoint (#25684)"

This reverts commit 5d3f8279.

* Revert "Implement DB specific execution permissions (#25629)"

This reverts commit 141a13ce.

* Revert "Introduce /execution/ permission (#25529)"

This reverts commit 57cbb921.

* Keep the app permission revision table

* [Apps] Remove emitters (#25999)

* [Apps] Remove emitters

Emitters were created when we thought that we would be adding actions to
multiple places in metabase. Instead we've coalesced to actions being a
core feature of data-apps and are executed from within them. Attaching
actions to models, as a form of re-use rather than a form of execution,
means that actions can still be executed directly from a model, if we
want to allow that in the future.

* Fix lint errors

* Move perform-action! tests out of the api.

Since we want to keep the implicit bulk functionality but remove those
endpoints move the tests to the actions namespace.

* Fix linter

* Change driver test to use function instead of endpoint

Co-authored-by: metamben <103100869+metamben@users.noreply.github.com>
Co-authored-by: Case Nelson <case@metabase.com>

* Switch to humane-are

* Fix some busted `are`s

* Remove unused namespaces