* Unrelated fix: app DB data source should assume default host/port for MySQL/Postgres

* Remove stray println

* Add CLI SerDes command for seeding entity IDs (#25655)

* Remove stray comment

* Fix Kondo error

* Fix Eastwood error

* Update identity-hash-fields to multimethod for ModelAction

* Fix identity-hash-fields for Dimension and DashboardCardSeries

* Return models used in scaffolding in GET /app/:id
* Return models added to an app page
* Return models referenced by queries on app pages

When scaffolding, we should be automatically adding the
insert,update,delete implicit actions.

* Add order by on test

Co-authored-by: metamben <103100869+metamben@users.noreply.github.com>

* [Apps] Model Actions

Adding migration and endpoints for model_action.

* model action execution

Changing the execution endpoints again, still working on implicit action
execution tests. Need to also test the parameters fetch.

* Re-add execution route for dashcard action_id until Front End catches up

* Add name to model_action GET

* Hydrate model-action on dashcards

* Rename and combine hydration of model_action to action

* Implicit action execution support. Use slug columns as parameter ids

* Go through model-action when using GET /action

* FE integration fixes

* Move action execution to use new parameter shape

Now, parameters should be a map of parameter-id to value
Parameter-id should be mapped on the FE. So the action parameter-id not the unmapped
dashboard parameter-id.

* Bring ModelAction into copy code

* Generate target of implicit actions to match http actions

* Update action test to place action on model

* Add unique constraint to model_action for (card_id, slug)

* Add missing require

* model_action.slug needs to be varchar for mysql constraint

* Fix test from unique constraint

* Consistent ordering for tests

* Addressing Code Reviews and Scaffold changes

* Add type: implicit to implicit model actions

* Update src/metabase/api/model_action.clj

Co-authored-by: metamben <103100869+metamben@users.noreply.github.com>

* Allow changing card_id with PUT to dashcard for model-actions

* Update tests after merge

* Fix implicit delete and add better test coverage

* Addressing review changes

Add PUT test
Add schema for dashboard-id and dashcard-id params
Fix select ordering in test

* Model Actions: Frontend (#25646)

* save actions in models

* list model actions on detail page (#25648)

* Link Dashcards to model-based actions (#25770)

* Link dashcards to model-based actions

* address review comments

* Data apps model actions implicit creator (#25807)

* Link dashcards to model-based actions

* address review comments

* Link dashcards to model-based actions

* Allow implicit actions to be used like explicit ones

* address review comments

Co-authored-by: Anton Kulyk <kuliks.anton@gmail.com>

Co-authored-by: metamben <103100869+metamben@users.noreply.github.com>
Co-authored-by: Ryan Laurie <30528226+iethree@users.noreply.github.com>
Co-authored-by: Anton Kulyk <kuliks.anton@gmail.com>
Co-authored-by: Kyle Doherty <5248953+kdoh@users.noreply.github.com>

* draft fix

* test

* revert change to email testing helper to fix tests

* Fix static viz not rendering Lato font

* Move function re font registering to metabase.pulse.render.style namespace

* Add UUID param to version info checks

* add test and minor tweaks

* Update test/metabase/task/upgrade_checks_test.clj

Co-authored-by: Cal Herries <39073188+calherries@users.noreply.github.com>

* add is-prod check

Co-authored-by: Cal Herries <39073188+calherries@users.noreply.github.com>

[Fixes #25704]

* untangled date time ranges (and filters)

* more tests for relative dates

* Update test/metabase/driver/common/parameters/dates_test.clj

Co-authored-by: Cam Saul <1455846+camsaul@users.noreply.github.com>

* respond to review + remove an extra tap>

* remove java time

* with-clock is not allowed inside parallel tests.

Co-authored-by: Cam Saul <1455846+camsaul@users.noreply.github.com>

* Make Database details NOT NULL; add defaults for Database created_at and updated_at

* Add default value for `:details` to SerDes load-xform

* Fix MySQL UTF-8 test

* Reset changes and go thru everything again to make sure it makes sense

* Test fixes 

* indentation only

* Fix creation check for mysql

* Test that all persisting dbs can persist

* Erran tap> (whoops)

* `POST /api.ee/serialization/serialize/data-model` endpoint

* Sort namespaces

* Don't use `tru` for endpoint params validation

* Implement advanced date/time/zone manipulation, part 1

Incorporate new functions into MBQL and add tests:
 - get-year
 - get-quarter
 - get-month
 - get-day
 - get-day-of-week
 - get-hour
 - get-minute
 - get-second

* Fix BigQuery implementations to call extract

Mark as not supported in legacy driver

* Add date extraction fns for Postgres

* Disable in MongoDB (for now at least)

Disable in BigQuery (legacy driver)

Add implementations for presto-jdbc

* Misc cleanup from Jeff's PR

* Update Jeff's implementation of bigquery-cloud-sqk

* Reorganized tests

* Mongo

* Oracle

* Sqlserver

* Sqlite

* Add casting supports for presto

* Remove Jeff's implementation of presto-jdbc because  its parent is
sql-jdbc

* Update presto-jdbc tests to use the same catalog for all datasets

* Add date extraction functions to the expression editor (#25382)

* make sure the semantic type of aggregated columns are integer

* no recursive call in annotate for date-extract func

* get-unit -> temporal-extract(column, unit)

* desguar nested datetime extraction too

* Refactor static waterfall chart to use a single entrypoint

* Consolidate static watetrfall chart

* Update waterfall chart text size to match combo-chart

* Move `<Values />` from XYChart/ to be under components/

Since it'll be used by waterfall chart

* Fix `<Values />` key error (forgot to add key attribute)

* Show data point values on waterfall chart

* Fix failed BE tests

* Fix test -> wrong `waterfall-type` used. And test both types

The waterfall component was changed to take a type argument, instead of having 2 functions for nearly identical
output. The failing test just passed the wrong string to the key argument.

* Change waterfall keys from strings to Clojure keys

* Remove redundant test. Categorical also handled in `waterfall-test`

* Fix clj-kondo lint error

* Cleanup conditional statements

* Miscellaneous code cleanup

* Update waterfall test to render data point values

* Fix waterfall chart data point values not center in the bar

* Make some API for static waterfall chart with total saner

Co-authored-by: Adam James <adam.vermeer2@gmail.com>

* Switch to humane-are

* Fix some busted `are`s

* Remove unused namespaces

* Implement DB specific execution permissions

Separate execution permissions from data permissions as a new dimension
like application features. Create a new endpoint for getting and setting
these permissions.

* Substitute persisted queries in parameter card references

```sql
select o.total, o.quantity, model_p.category, model_p.title
from orders o
left join {{#14-pg-products}} model_p -- reference to products model
on o.product_id = model_p.id
limit 4
```

Testing:

I'm unhappy with how verbose the test is. At some point we're going to
need a better, standardized way to test persistence. But the test makes
a persisted table of the test-data.categories
table (test/metabase/test/data/dataset_definitions/test-data.edn). It
runs some sql to update the values in the table to turn `"Winery"` ->
`"Winery from cached table"`, joins the two tables together to have both
at once.

There's a lot of moving pieces and the setup is a bit verbose. There's
also no obvious place where these types of tests should go. We'll need
to consolidate them in the future (and extend them).

* persisted macro

* docstring for `persisted-info-native-query`

* alignments and small nits

* Reuse persistence macro

* clj-kondo cleanups

* remove clj-kondo change before rebase

* Introduce /execution/ permission

* the fix

* adding some tests

* alternative solution in which we don't merge metadata for aggregation fields at all

* wording for the docstring

* use partial= and remove smart quotes. Thanks Cam

* Handle bigint and boolean casts properly

- Add JSON -> SerializedJSON to `database-type->base-type`

* adds tests json bigint and boolean types for mysql

* Fix most of the tests

- still need to figure out :type/BigInteger vs :type/Integer

* fiddling with tests

* handle json booleans via default mechanism

* fix tests

* remove not(not(...)) from test

* bonk the test

- also fixes ddl.execute-with-timeout-test thrown? regex

* remove inline def

* try to get logs off ci

* don't test json on mariadb -- that never worked

* cleanup tests

* revert change to driver/database-supports? :mysql :nested-field-columns

* respond to code review comments

* 347 errors, 34 warnings

* 89 errors, 66 warnings

* 63 errors, 39 warnings

* 52 errors, 33 warnings

* 23 errors, 22 warnings

* 13 errors, 4 warnings

* Fix the last few errors.

* Sort Kondo config

* allow disabling custom geojson via env var

* add env var to docs and tweak setting description

* wip changes

* new endpoint

* fix test

* more test fixes

* add test for EE settings permissions

* try to fix another BE test

* address ngoc's comment

* fix cypress test

* fix another cypress test to hit new settings endpoint

* allow turning off mysql json unwrapping

- somehow this value was set to always be true, but it should be toggle-able.

- also for ignoring INIT=... on the conn string

* remove toc generation from api docs

* update api docs

* update test

* Don't use persisted model tables for segmented users

This actually isn't a bug, but due to very subtle and arbitrary reasons.

For background about why we need to ensure this never happens, we cannot
use persisted models when sandboxing is at play. A simple example is as
follows: make a model on a products table that does not select the
category. Have a sandbox on category such that someone can only see
products of category "Gizmo". the model lacks the category column but we
insert a where clause that still works. When the model is persisted,
there is no category column in the underlying table so sandboxing cannot
possibly work: the data necessary to filter is no longer associated with
the rest of the data in the model.

The fix for this is quite simple: in
`metabase.query-processor.middleware.fetch-source-query` we only splice
in the persisted query if the user is not a segmented user (product name
for sandboxing).

```clojure
(and persisted-info/*allow-persisted-substitution*
     (not (segmented-user?))  ;; <----- new check
     (:active card)
     (:definition card)
     (:query_hash card)
     (= (:query_hash card) (persisted-info/query-hash (:dataset_query card)))
     (= (:definition card) (persisted-info/metadata->definition (:result_metadata card)
                                                                (:table_name card)))
     (= (:state card) "persisted"))
```

Technical details about why this bug did not manifest

When swapping out a card__<id> to a source query, if its a model we will
see if it is persisted, and if so, we will use the native sql to select
from the persisted table. It does this by adding the native sql at a key
called `:persisted-info/native` and a middleware
`#'qp.persistence/substitute-persisted-query` walks the query replacing
the query with the native:

```clojure
;; metabase.query-processor.middleware.persistence

    (mbql.u/replace query
      (x :guard (every-pred map? :persisted-info/native))
      {:native (:persisted-info/native x)})
```

There is also a middleware that walks through the query looking for
tables with gtaps on them and replacing them. By change, the sandboxing
middleware runs immediately before the substitute-persisted middleware!

```clojure
   ;; literally the previous middleware
   (resolve 'ee.sandbox.rows/apply-sandboxing)
   #'qp.persistence/substitute-persisted-query
```

If you swap the order of these two sandboxing is broken. As is, it
"works" but not by design, just by happenstance. The sandboxing
middleware just did not know that the `:persisted-info/native` key meant
that a native query was to be substituted. In the reverse order, the
native query is already substituted and there is no change for the
sandboxing to occur.

The obvious fix is to ensure that we never even attempt to use the
persisted tables and that is what this PR does.

* Eastwood doesn't like shadowing like this

* Rearrange check order for tests

`segmented-user?` throws if there is no bound user. A test in
`fetch-source-query-test` was failing because there was no user bound,
but it wasn't attempting to swap out a persisted table, it just didn't
expect to need a user.

Moving it lower lets it short circuit on other bits that are bound to
fail (definition, query_hash, etc) requiring persistence before we check
for a bound user

* first pass at LDAP setup improvements

* fix lint errors

* fix LDAP api tests

* WIP test for new setting setter

* fix setting test

* set ldap-enabled to true in ldap server macro

* try to fix java11 tests

* Update src/metabase/integrations/ldap.clj

Co-authored-by: metamben <103100869+metamben@users.noreply.github.com>

* add transaction

* remove ldap-ever-enabled? setting and revert some of the logic that is no longer necessary

* set ldap-enabled via the ldap api and add tests

* fix tests and lint

* fix error on settings save

* fix cypress test

* actually fix cypress

Co-authored-by: metamben <103100869+metamben@users.noreply.github.com>

* initial prometheus sketch

need env variable to start: MB_PROMETHEUS_SERVER_PORT=9191

sample of output:

```
'# HELP jvm_threads_current Current thread count of a JVM
'# TYPE jvm_threads_current gauge
jvm_threads_current 81.0
'# HELP jvm_threads_daemon Daemon thread count of a JVM
'# TYPE jvm_threads_daemon gauge
jvm_threads_daemon 36.0
'# HELP jvm_threads_peak Peak thread count of a JVM
'# TYPE jvm_threads_peak gauge
jvm_threads_peak 81.0
'# HELP jvm_threads_started_total Started thread count of a JVM
'# TYPE jvm_threads_started_total counter
jvm_threads_started_total 104.0
'# HELP jvm_threads_deadlocked Cycles of JVM-threads that are in deadlock waiting to acquire object monitors or ownable synchronizers
'# TYPE jvm_threads_deadlocked gauge
jvm_threads_deadlocked 0.0
```

request:

```
❯ http localhost:9191/metrics
HTTP/1.1 200 OK
Content-Length: 7329
Content-Type: text/plain; version=0.0.4; charset=utf-8
Date: Wed, 31 Aug 2022 16:13:38 GMT
Server: Jetty(9.4.48.v20220622)

'# HELP jvm_gc_collection_seconds Time spent in a given JVM garbage collector in seconds.
'# TYPE jvm_gc_collection_seconds summary
jvm_gc_collection_seconds_count{gc="G1 Young Generation",} 41.0
jvm_gc_collection_seconds_sum{gc="G1 Young Generation",} 0.586
jvm_gc_collection_seconds_count{gc="G1 Old Generation",} 0.0
jvm_gc_collection_seconds_sum{gc="G1 Old Generation",} 0.0
'# HELP jvm_threads_current Current thread count of a JVM
'# TYPE jvm_threads_current gauge
```

* Log on unparseable prometheus port

* Clean up prometheus and save test

* typehint

* Jetty collector

* Reset system to nil when shutting down

* c3p0 stats

* Clean up, document, and add tests

* Error message for failure to bind to port

Starting up with prometheus port set to the main webserver port to get
the error:

```
MB_JETTY_PORT=3006 MB_DB_CONNECTION_URI="postgres://..." \
MB_PROMETHEUS_SERVER_PORT=3006 java -jar locally-built.jar
```
yields the following error:

```shell
2022-09-09 10:08:52,000 INFO metabase.core :: Setting up prometheus metrics
2022-09-09 10:08:52,002 INFO metabase.prometheus :: Starting prometheus metrics collector
2022-09-09 10:08:52,016 INFO metabase.prometheus :: Starting prometheus metrics web-server on port 3,006
2022-09-09 10:08:52,036 ERROR metabase.core :: Metabase Initialization FAILED
clojure.lang.ExceptionInfo: Failed to initialized Prometheus on port 3,006 {:port 3006}
[stacktrace ...]
Caused by: java.io.IOException: Failed to bind to 0.0.0.0/0.0.0.0:3006
[stacktrace ...]
Caused by: java.net.BindException: Address already in use
```

* Test for error message

* Move prometheus to analytics folder

* Str port so log does not add commas to number

eg:
> Starting prometheus metrics web-server on port 9,191

* make some test functions private

* docstring on defsetting

* align lets

* ns docstring changes

include the env variable for ease of understanding

* sort ns

* remove some reflection warnings

* reorder FE lines due to some new linter

* Cleanup: i18n descriptions, typo, List/of

* bit more concise c3p0 collection

* no longer need the helper `->array`

* clean up doseq for c3p0 measurements