* Remove the title from all panes and put it in the sidebar header component, next to the back button

* Return cards with db metadata

* Load all questions into DatabaseTablesPane

* Only load questions from the selected database

* Create ModelPane

* remove mapDispatchToProps

* Style ModelPane and reuse FieldPane for model fields

* Use FieldList for TablePane and ModelPane

* Load connected tables

* Create ConnectedTablesList

* Restyle DatabaseTablesPane

* Adjust styles

* smaller ModelPaneIcon

* ModelPaneText font weights

* description spacing

* icon colors and spacing

* appease the linter for real

* Only load models for a given database

* Add model ID next to each model

* Remove TODO

* Undo changes to database api

* Refactor files

* Add _target="blank" for model "See it" link

* Pluralize copy

* Fix i18n

* Change back TablePane from tsx -> jsx

* Tidy

* Don't show models if there aren't any

* Tidy (consistency)

* Add BE tests

* Do a read-check for the database

* Revert "Add BE tests"

This reverts commit 8d18f4e12636e32fda2b5b670124d69237fce5bd.

* Revert "Only load models for a given database"

This reverts commit 028ba43f20d55fef0aaf484d3557e75ae5501758.

* Change endpoint to database/:id/models

* Tidy

* Use name in SQL, not display name

* Add e2e test for models pane

* Tidy

* Fix e2e test - no more "back" text after redesign

* Convert SchemaPane to tsx

* Revert "Do a read-check for the database"

This reverts commit 550fc0eb31268d204b6dc7c853b9d2fc7ad50ecc.

* Fix endpoint for nil edit info

* Fix listQuestions path

* Rename test

* Remove prn

* Batch of suggested changes

* Convert ModelPane to typescript

* Convert to tsx

* Replace any

* Tidy

* Add permissions tests for GET api/database/:id/models

* Remove redundant code

* Fix typecheck error for getQuestionFromCard

* Rename models -> sortedModels

* Remove comment

* Use useMemo

* Remove /api/database/:id/models endpoint, use api/search instead

* Correct docstring

* Remove separate listModelsForDatabase endpoint

Co-authored-by: Maz Ameli <maz@metabase.com>

* untangled date time ranges (and filters)

* more tests for relative dates

* Update test/metabase/driver/common/parameters/dates_test.clj

Co-authored-by: Cam Saul <1455846+camsaul@users.noreply.github.com>

* respond to review + remove an extra tap>

* remove java time

* with-clock is not allowed inside parallel tests.

Co-authored-by: Cam Saul <1455846+camsaul@users.noreply.github.com>

* Make Database details NOT NULL; add defaults for Database created_at and updated_at

* Add default value for `:details` to SerDes load-xform

* Fix MySQL UTF-8 test

* Reset changes and go thru everything again to make sure it makes sense

* Test fixes 

Destroy open connections when resetting app DB; add `ReentrantReadWriteLock` around app DB `getConnection` (#21664)

* Destroy open connections when resetting app DB

* Use no-op dummy app DB

* Ok, don't shut down the app DB because it breaks in-memory app DBs

* Stop/start task scheduler

* Test fix ?

* Another test fix 

* Simplified impl

* Revert task changes

* Fix connection-pool destroying code

* Use ReentrantReadWriteLock for simpler implementation

* More doc

* indentation only

* Fix creation check for mysql

* Test that all persisting dbs can persist

* Erran tap> (whoops)

* `POST /api.ee/serialization/serialize/data-model` endpoint

* Sort namespaces

* Don't use `tru` for endpoint params validation

* Implement advanced date/time/zone manipulation, part 1

Incorporate new functions into MBQL and add tests:
 - get-year
 - get-quarter
 - get-month
 - get-day
 - get-day-of-week
 - get-hour
 - get-minute
 - get-second

* Fix BigQuery implementations to call extract

Mark as not supported in legacy driver

* Add date extraction fns for Postgres

* Disable in MongoDB (for now at least)

Disable in BigQuery (legacy driver)

Add implementations for presto-jdbc

* Misc cleanup from Jeff's PR

* Update Jeff's implementation of bigquery-cloud-sqk

* Reorganized tests

* Mongo

* Oracle

* Sqlserver

* Sqlite

* Add casting supports for presto

* Remove Jeff's implementation of presto-jdbc because  its parent is
sql-jdbc

* Update presto-jdbc tests to use the same catalog for all datasets

* Add date extraction functions to the expression editor (#25382)

* make sure the semantic type of aggregated columns are integer

* no recursive call in annotate for date-extract func

* get-unit -> temporal-extract(column, unit)

* desguar nested datetime extraction too

* Refactor static waterfall chart to use a single entrypoint

* Consolidate static watetrfall chart

* Update waterfall chart text size to match combo-chart

* Move `<Values />` from XYChart/ to be under components/

Since it'll be used by waterfall chart

* Fix `<Values />` key error (forgot to add key attribute)

* Show data point values on waterfall chart

* Fix failed BE tests

* Fix test -> wrong `waterfall-type` used. And test both types

The waterfall component was changed to take a type argument, instead of having 2 functions for nearly identical
output. The failing test just passed the wrong string to the key argument.

* Change waterfall keys from strings to Clojure keys

* Remove redundant test. Categorical also handled in `waterfall-test`

* Fix clj-kondo lint error

* Cleanup conditional statements

* Miscellaneous code cleanup

* Update waterfall test to render data point values

* Fix waterfall chart data point values not center in the bar

* Make some API for static waterfall chart with total saner

Co-authored-by: Adam James <adam.vermeer2@gmail.com>

* Implement DB specific execution permissions

Separate execution permissions from data permissions as a new dimension
like application features. Create a new endpoint for getting and setting
these permissions.

* Substitute persisted queries in parameter card references

```sql
select o.total, o.quantity, model_p.category, model_p.title
from orders o
left join {{#14-pg-products}} model_p -- reference to products model
on o.product_id = model_p.id
limit 4
```

Testing:

I'm unhappy with how verbose the test is. At some point we're going to
need a better, standardized way to test persistence. But the test makes
a persisted table of the test-data.categories
table (test/metabase/test/data/dataset_definitions/test-data.edn). It
runs some sql to update the values in the table to turn `"Winery"` ->
`"Winery from cached table"`, joins the two tables together to have both
at once.

There's a lot of moving pieces and the setup is a bit verbose. There's
also no obvious place where these types of tests should go. We'll need
to consolidate them in the future (and extend them).

* persisted macro

* docstring for `persisted-info-native-query`

* alignments and small nits

* Reuse persistence macro

* clj-kondo cleanups

* remove clj-kondo change before rebase

* Introduce /execution/ permission

* the fix

* adding some tests

* alternative solution in which we don't merge metadata for aggregation fields at all

* wording for the docstring

* use partial= and remove smart quotes. Thanks Cam

* Handle bigint and boolean casts properly

- Add JSON -> SerializedJSON to `database-type->base-type`

* adds tests json bigint and boolean types for mysql

* Fix most of the tests

- still need to figure out :type/BigInteger vs :type/Integer

* fiddling with tests

* handle json booleans via default mechanism

* fix tests

* remove not(not(...)) from test

* bonk the test

- also fixes ddl.execute-with-timeout-test thrown? regex

* remove inline def

* try to get logs off ci

* don't test json on mariadb -- that never worked

* cleanup tests

* revert change to driver/database-supports? :mysql :nested-field-columns

* respond to code review comments

* 347 errors, 34 warnings

* 89 errors, 66 warnings

* 63 errors, 39 warnings

* 52 errors, 33 warnings

* 23 errors, 22 warnings

* 13 errors, 4 warnings

* Fix the last few errors.

* Sort Kondo config

* allow disabling custom geojson via env var

* add env var to docs and tweak setting description

* wip changes

* new endpoint

* fix test

* more test fixes

* add test for EE settings permissions

* try to fix another BE test

* address ngoc's comment

* fix cypress test

* fix another cypress test to hit new settings endpoint

* allow turning off mysql json unwrapping

- somehow this value was set to always be true, but it should be toggle-able.

`display: action-button` => `display: action`

* remove toc generation from api docs

* update api docs

* update test

* Don't use persisted model tables for segmented users

This actually isn't a bug, but due to very subtle and arbitrary reasons.

For background about why we need to ensure this never happens, we cannot
use persisted models when sandboxing is at play. A simple example is as
follows: make a model on a products table that does not select the
category. Have a sandbox on category such that someone can only see
products of category "Gizmo". the model lacks the category column but we
insert a where clause that still works. When the model is persisted,
there is no category column in the underlying table so sandboxing cannot
possibly work: the data necessary to filter is no longer associated with
the rest of the data in the model.

The fix for this is quite simple: in
`metabase.query-processor.middleware.fetch-source-query` we only splice
in the persisted query if the user is not a segmented user (product name
for sandboxing).

```clojure
(and persisted-info/*allow-persisted-substitution*
     (not (segmented-user?))  ;; <----- new check
     (:active card)
     (:definition card)
     (:query_hash card)
     (= (:query_hash card) (persisted-info/query-hash (:dataset_query card)))
     (= (:definition card) (persisted-info/metadata->definition (:result_metadata card)
                                                                (:table_name card)))
     (= (:state card) "persisted"))
```

Technical details about why this bug did not manifest

When swapping out a card__<id> to a source query, if its a model we will
see if it is persisted, and if so, we will use the native sql to select
from the persisted table. It does this by adding the native sql at a key
called `:persisted-info/native` and a middleware
`#'qp.persistence/substitute-persisted-query` walks the query replacing
the query with the native:

```clojure
;; metabase.query-processor.middleware.persistence

    (mbql.u/replace query
      (x :guard (every-pred map? :persisted-info/native))
      {:native (:persisted-info/native x)})
```

There is also a middleware that walks through the query looking for
tables with gtaps on them and replacing them. By change, the sandboxing
middleware runs immediately before the substitute-persisted middleware!

```clojure
   ;; literally the previous middleware
   (resolve 'ee.sandbox.rows/apply-sandboxing)
   #'qp.persistence/substitute-persisted-query
```

If you swap the order of these two sandboxing is broken. As is, it
"works" but not by design, just by happenstance. The sandboxing
middleware just did not know that the `:persisted-info/native` key meant
that a native query was to be substituted. In the reverse order, the
native query is already substituted and there is no change for the
sandboxing to occur.

The obvious fix is to ensure that we never even attempt to use the
persisted tables and that is what this PR does.

* Eastwood doesn't like shadowing like this

* Rearrange check order for tests

`segmented-user?` throws if there is no bound user. A test in
`fetch-source-query-test` was failing because there was no user bound,
but it wasn't attempting to swap out a persisted table, it just didn't
expect to need a user.

Moving it lower lets it short circuit on other bits that are bound to
fail (definition, query_hash, etc) requiring persistence before we check
for a bound user

* first pass at LDAP setup improvements

* fix lint errors

* fix LDAP api tests

* WIP test for new setting setter

* fix setting test

* set ldap-enabled to true in ldap server macro

* try to fix java11 tests

* Update src/metabase/integrations/ldap.clj

Co-authored-by: metamben <103100869+metamben@users.noreply.github.com>

* add transaction

* remove ldap-ever-enabled? setting and revert some of the logic that is no longer necessary

* set ldap-enabled via the ldap api and add tests

* fix tests and lint

* fix error on settings save

* fix cypress test

* actually fix cypress

Co-authored-by: metamben <103100869+metamben@users.noreply.github.com>

* initial prometheus sketch

need env variable to start: MB_PROMETHEUS_SERVER_PORT=9191

sample of output:

```
'# HELP jvm_threads_current Current thread count of a JVM
'# TYPE jvm_threads_current gauge
jvm_threads_current 81.0
'# HELP jvm_threads_daemon Daemon thread count of a JVM
'# TYPE jvm_threads_daemon gauge
jvm_threads_daemon 36.0
'# HELP jvm_threads_peak Peak thread count of a JVM
'# TYPE jvm_threads_peak gauge
jvm_threads_peak 81.0
'# HELP jvm_threads_started_total Started thread count of a JVM
'# TYPE jvm_threads_started_total counter
jvm_threads_started_total 104.0
'# HELP jvm_threads_deadlocked Cycles of JVM-threads that are in deadlock waiting to acquire object monitors or ownable synchronizers
'# TYPE jvm_threads_deadlocked gauge
jvm_threads_deadlocked 0.0
```

request:

```
❯ http localhost:9191/metrics
HTTP/1.1 200 OK
Content-Length: 7329
Content-Type: text/plain; version=0.0.4; charset=utf-8
Date: Wed, 31 Aug 2022 16:13:38 GMT
Server: Jetty(9.4.48.v20220622)

'# HELP jvm_gc_collection_seconds Time spent in a given JVM garbage collector in seconds.
'# TYPE jvm_gc_collection_seconds summary
jvm_gc_collection_seconds_count{gc="G1 Young Generation",} 41.0
jvm_gc_collection_seconds_sum{gc="G1 Young Generation",} 0.586
jvm_gc_collection_seconds_count{gc="G1 Old Generation",} 0.0
jvm_gc_collection_seconds_sum{gc="G1 Old Generation",} 0.0
'# HELP jvm_threads_current Current thread count of a JVM
'# TYPE jvm_threads_current gauge
```

* Log on unparseable prometheus port

* Clean up prometheus and save test

* typehint

* Jetty collector

* Reset system to nil when shutting down

* c3p0 stats

* Clean up, document, and add tests

* Error message for failure to bind to port

Starting up with prometheus port set to the main webserver port to get
the error:

```
MB_JETTY_PORT=3006 MB_DB_CONNECTION_URI="postgres://..." \
MB_PROMETHEUS_SERVER_PORT=3006 java -jar locally-built.jar
```
yields the following error:

```shell
2022-09-09 10:08:52,000 INFO metabase.core :: Setting up prometheus metrics
2022-09-09 10:08:52,002 INFO metabase.prometheus :: Starting prometheus metrics collector
2022-09-09 10:08:52,016 INFO metabase.prometheus :: Starting prometheus metrics web-server on port 3,006
2022-09-09 10:08:52,036 ERROR metabase.core :: Metabase Initialization FAILED
clojure.lang.ExceptionInfo: Failed to initialized Prometheus on port 3,006 {:port 3006}
[stacktrace ...]
Caused by: java.io.IOException: Failed to bind to 0.0.0.0/0.0.0.0:3006
[stacktrace ...]
Caused by: java.net.BindException: Address already in use
```

* Test for error message

* Move prometheus to analytics folder

* Str port so log does not add commas to number

eg:
> Starting prometheus metrics web-server on port 9,191

* make some test functions private

* docstring on defsetting

* align lets

* ns docstring changes

include the env variable for ease of understanding

* sort ns

* remove some reflection warnings

* reorder FE lines due to some new linter

* Cleanup: i18n descriptions, typo, List/of

* bit more concise c3p0 collection

* no longer need the helper `->array`

* clean up doseq for c3p0 measurements

* Make table column width reasonable

Truncate column name that is longer than 16 characters

* Add a test to check header truncation of tables

Also adjusted the shape of the header fn's return data a bit -> the `[:th ... ]` vectors aren't wrapped in a seq anymore.

* Fix wrong max column character length 



* Update static viz table style

* Address PR comment, simplifying the code

Co-authored-by: Adam James <adam.vermeer2@gmail.com>

* Allow slugs in question template tags, like `{{#123-a-question}}`

* Update CardTagEditor to add slug names to the query text, instead of IDs

* Fix dispatchApiErrorEvent prop for entity object loader

* Ignore errors loading names of referenced questions

* Fix e2e tests

* Implement autocomplete for model slugs

* Prioritize models above questions

* Make the ace_autocomplete popup slightly wider to account for longer question names

* Reorder requires

* Remove unnecessary ? from endpoint string

* Add e2e test for card tag autocomplete

* Fix NativeQueryEditor not swapping back to standard completers

* Refactor replaceCardSlug

* Add e2e test for updating tags when card name is changed

* Refactor updateReferencedQuestionNames

* Add slug method to Question class

* Use map for autocomplete API results

* Rename endpoint path

* Rename for consistency

* Add docstrings

* Update slug() to guard against undefined names

* Fix read permissions for the endpoint

* Add backend test

* Fix unit tests

* Fix permissions error

* Make regex simpler

* Undo change of value on QuestionPicker

* Fix tests

* Fix models e2e test

* Simplify updateReferencedQuestionNames

* Fix bugs in updateReferencedQuestionNames

* Rename variable

* Move NativeQuery methods to new native-query utils module

* Fix typescript errors

* Fix e2e tests

* Fix e2e test

* Fix BE tests

* Actually fix BE test

* Refactor updateQuestionTagNames

* Align comments

* Reorder autocomplete results

* Refactor: move all query text tag utils in one place

* Add tests for regex

* Add unit tests for cardIdFromTagName

* Add schema to autocomplete_suggestions endpoints

* Add perms checks to autocomplete tests

* Add unit test for updateCardTagNames

* Rename updateQuestionTagNames -> updateCardTagNames

* Tidy card-autocomplete-suggestions-test

* Rename questionReference-> cardTag for consistency

* Rename card-reference -> card-tag for consistency

* Use ternary operator for comprehension

* Fix broken swapCompleters

* Use prefix match on name for 123-foo case

* thinking

* killing it

* lab image bundle for lines proper

* linter shutting up

* get rid of viz-settings binding at all

* Remove accidentally re-added categorical-area

* Merged master, and temporarily comment out some tests

* Simplify conditional for detecting chart type

* Remove a few old fns/symbols

* Clean up the detect-chart a bit

* Remove sparkline. It's an old concept that's not used

We can acheive similar results with :line directly. If we care about a trend, we can use a :smartscalar.

* Fix up tests that now also use combo charts

* Missed a key change for area test

* Get rid of deprecated sparkline stuff

* Remove commented out sparkline tests

* Delete this function. I was using it in the REPL, it's not needed

Co-authored-by: Adam James <adam.vermeer2@gmail.com>

Cards can depend on other Cards as their `:source-table`, but the code
to extract `serdes-dependencies` from the MBQL query did not
capture that case.

Many entities have `creator_id` and similar fields. `User`s are not
serialized. Foreign keys to users are serialized as email addresses.

During deserialization in a different instance (eg. a local dev instance
importing a dump from a prod instance) many such users may not exist.
This change creates new `User` entities on the fly with empty names,
generated passwords, and the email set.

- Limit the scanning of directories and files to those named after
  models; don't try to ingest `.git`, `README.md`, etc.
- `table_id` and `collection_id` are optional on Cards
- Deserialization was not resolving some deeply nested `:field`s inside MBQL
  queries.

* [Apps] Add scaffolded pages to existing apps

Reuse app scaffolding in order to add to existing apps. This will not
check that the tables you choose have not been previously added to the app.

Also update scaffolding:
- Add back to list button
- Update list question to order by newest row on top.
- Use linkType: "page" instead of linkType: "dashboard"

* Fix H2 test, no need to add app-id to name

* Remove unused require

* Addressing review comments - suggested test uncovered an ordering bug

* Put priority back to ascending

* Implement navigation between data app pages (#25366)

* Add `DataAppPagePicker`

* Rename `QuestionDashboardPicker`

* Add page to `EntityCustomDestinationClickBehavior`

* Fix comment

* Use ellipsis character

* Fix custom destination type

* Add "page" custom destination to pickers

* Pass router props to dashboard click actions

* Remove not used function arguments

* Implement "page" custom destination click behavior

* Fix "page" click behavior isn't treated as valid

* Reorganize `LinkedEntityPicker`

* Fix drills from the homepage

Co-authored-by: Anton Kulyk <kuliks.anton@gmail.com>