* Resolve nested questions in native queries

Fixes #25988.

* Decouple checking ddl from classifying h2 stmts

- should enable followup for easily blocking more kinds of queries
- check all statements to make sure they aren't "ddl".

* fix classify-query

* linter fixes + get-field refactor

* return the CommandInterface values as ints

Reach into the CommandList when needed

* docstring wording

* catch invalid queries -- they can't be classified

* respond to review comments

- fix see: comment
- remove CommandList typehint
- rename command-list -> command
- skip checking disallowed ddl commands when query is `nil`

then also:
- fix a linter error
- add a test for `nil` query ddl command checking

* rename command-list-type -> first-command-type

* fix schema + handle h2-parser build failure

* Fix action loading and extraction of action type

* Remove comment

* Fix handling required parameters for public basic actions (#28415)

* Fix presenting required parameters

* Add tests

* Add explanation text to the action editor (#28406)

* Add explanation text to the action editor

* Replace link

* Add missing `nock.cleanAll` calls (#28441)

* [CI] Improve `codenotify` workflow (#28432)

* Add path filtering to codenotify workflow

* Upgrade action to its latest version (v0.6.4)

* [E2E] Prevent app permission flakes (#28445)

* Ensure dashcards are deleted after actions are deleted (#28446)

(c.f. discussion at https://metaboat.slack.com/archives/C04FG88HL95/p1676649795131089

)

* Show not found view when opening archived model (#28443)

* Add `isArchived` method to `Question` class

* Show nice not found view when opening archived model

* [CI] Unify E2E tests workflows (#28431)

* [Ci] Fix wildcard glob pattern in path filters (#28447)

[ci skip]

* Remove dupe tests

* Add database_id to implicit actions

* Tidy

* Update select-actions docstring

* Add test for database_id in action dashcards

* Remove database_id before insert and update

* whitespace

* Refactor select-actions

* Add database_id to action-create-test

* Add test for create+update+select for implicit actions

* Fix test

* Remove conditional

* style

---------

Co-authored-by: Anton Kulyk <kuliks.anton@gmail.com>
Co-authored-by: Nemanja Glumac <31325167+nemanjaglumac@users.noreply.github.com>
Co-authored-by: Tim Macdonald <tim@metabase.com>

* If a model is converted to a card, archive the associated actions
* If an action is unarchived and its model _is_ a card, throw an error

* MISC improvements

* Update test/metabase/models/secret_test.clj

Co-authored-by: metamben <103100869+metamben@users.noreply.github.com>

* Update .clj-kondo/hooks/clojure/test.clj

Co-authored-by: metamben <103100869+metamben@users.noreply.github.com>

* Update .clj-kondo/hooks/clojure/test.clj

Co-authored-by: metamben <103100869+metamben@users.noreply.github.com>

---------

Co-authored-by: metamben <103100869+metamben@users.noreply.github.com>

* Improves consistency of substring in MBQL

Standard SQL SUBSTRING() uses a 1-based start index, with various drivers
handling a 0 start index in different ways. To make the behavior of substring
consistent across drivers, we will always convert a 0 start index in MBQL
to a 1 in SQL. The ->honeysql implementation of substring has been generalized
and drivers can now override just the name of the substring function, or the
entire expression if necessary. SQL Server now calls the parent implementation.

Resolves #12445

* Fixes substring from 0 in Mongo

* Adds arg to substring parent call

* Fix docs and helper text for substring()

* Adds schema to require positive integer index for substring

* WIP, adds frontend validation for substring index arg

* Adds frontend test for substring arg validation

* Updates substring test to expect exception on index 0

* Update src/metabase/driver/sql/query_processor.clj

Co-authored-by: metamben <103100869+metamben@users.noreply.github.com>

* Use s/if instead of s/conditional

* Fix substring call in mysql substring spec

* Reverts driver and sql.qp changes for substring index

* Reverts changes to dispatch-by-clause-name-or-class

* Fixes typeinferencer test substring index

* Adds test for canonicalizing substring

* Fixes substring canonicalization with extra args

* Updates string extracts test to account for substring canonicalization

* Rewrites resolver validation application

* Adds test case for frontend expression validation errors

* Fixes frontend compiler spec for new substring validation

---------

Co-authored-by: metamben <103100869+metamben@users.noreply.github.com>

* Misc QP test improvements

* Add test

This'll be revisited as part of v47

[Fixes #28425]

* Fix linter

* Make sure we're inlining num in substitution and sum aggregations

* Set hsql version

* Fixing tests

* Fixing tests after the h2 conversion

* Use h2x/extract in h2, fix test differences

* Fixing tests for honeysql2

* Only numbers get inlined

* More test fixes

* Fixing tests

* Undo bad commit

* Inline Ratio must produce doubles or h2 will do integer division where 1/2 = 0

* Fix bad case statement

* We can't run native queries with AS for oracle

---------

Co-authored-by: Cam Saul <github@camsaul.com>
Co-authored-by: Cam Saul <1455846+camsaul@users.noreply.github.com>

* Make joins work

* Run fuzzy-field-info-test only against H2

* Set driver/*driver* when performing actions

* Restrict expressions+joins-test to DBs supporting date arithmetics

* Fix ordering of (top level) fields in mongo tables

* Explicitly exclude mongo from tests requiring foreign key support

* Optimize away $sort stage for implicit order-by.

* add dashcard testing mocks

* improve virtual card overlays

* update types

* add search entity types

* add metabase entity support to link cards

* simplify link displays

* update types and support collections and tables

* use edit button to replace entity

* test entity links

* fix rebase

* update types

* address review comments

* handle restricted link entities

* Hydrate link cards info (#28297)

* hydrate link cards info

* we need to hydrate collections too

* Address Cal's comments:
- model use keywrod -> string
- use model hydration
- comments here and there

* Link cards serdes (#28321)

---------

Co-authored-by: Ngoc Khuat <qn.khuat@gmail.com>

(c.f. discussion at https://metaboat.slack.com/archives/C04FG88HL95/p1676649795131089)

* Removes :presto driver and tests
* Merges :presto-common into :presto-jdbc
* Adds migration to update presto databases to presto-jdbc

---------

Co-authored-by: Cam Saul <github@camsaul.com>

* Updates param parser to ignore SQL comments
* Adds code to disable SQL comment handling in Mongo native query param parsing

* Fix null parameters for implicit actions

* Fix test for dashboard and add action_id GET /api/dashboard/:id/cards endpoint schema

* Fix endpoint

* Update src/metabase/models/action.clj

Co-authored-by: Ngoc Khuat <qn.khuat@gmail.com>

---------

Co-authored-by: Ngoc Khuat <qn.khuat@gmail.com>

* Oracle Honey SQL 2

* Remove debugging function

* Fix concat()

* Oracle should use numbers for booleans when loading test data

The `personal_owner_id` field is now added to the Collection when computing collection ancestors.

* Static Viz Line/Area/Bar Should now respect set Y-axis min/max

The values for y-axis min and max are available to the static-viz js, it is just a matter of pulling them into the y
domain calculations.

This PR also adds a rendering test to the backend unit tests to confirm that the setting is respected through the
render pipeline.

* Make tests better by not re-rendering for every node lookup.

Just render each hiccup tree once, and search that tree multiple times instead. Just straight up a better way to do this test!

* Make variable names clearer so they don't get mistaken for fns

Added logic to get column ids from model metadata if possible.

* Fix `escape-join-aliases` middleware not properly deduplicating all aliases

* Fix join resolution order

* Appease Eastwood

* Test fixes 

* Ambiguous join aliases inside join conditions should prefer the joins from source queries

* One more test

* adding v2 data and query paths into the parser

* add example test

* adjust graph test output

- seems kinda wrong, could use some guidance

* add some more tests

* temporarily add rcf tests

* balance parens in deps.edn

* dirty commit + making progress

- need to make sure the spec is correct on this

* tests for permission graph v2 api

* Fix :data and :query keys in permissions graph v2

- adds tests for v2 graph read endpoint
- rename some malli schemas
- remove a few simple defendpoint-schema

* transfer rcf/tests into deftests

- fix a bug with native data permission path parsing
- remove inline defs
- add more tests for v2 style permission path parsing

* take out c.t.l/log

* formatting

* move more tests around

* lint fix

* update docstrings

- add / remove newlines

* Add public_uuid and made_public_by_id to actions table & endpoints to enable/disable the public sharing on individual actions (#27721)

* Add migration for public_uuid (indexed) and made_public_by_id

* Add UUIDString schema

* Add endpoints for enabling/disabling sharing of actions

* Test that the new fields are returned with `GET /api/action/:id`

* Remove validCheckSum on migration

* Remove trailing whitespace

* Fix DELETE test

* Fix tests

* Please migration linter

* Update the default public_uuid every test run

* Please migration linter

* Add FK onDelete cascade

* Replace ü

* Add GET endpoint and post-select for action

* Revert "Add GET endpoint and post-select for action"

This reverts commit 8cc8b57d6034146dd726b54bc4199830ec1fda21.

* Fix merge

* Reorder migrations

* Update test for GET `api/action?model-id=<id>` endpoint to include public sharing keys (#27802)

* Add GET: /api/public/action/:uuid endpoint (#27781)

* Add test

* Remove non-public columns and check for 404

* Fix docstring

* Rename for clarity

* Fix missing ns

* Remove unneccessary keys from action

* Update test

* Remove unused refer

* Reorder migrations

* Use mt

* Add require and refactor

* Use mt

* Add endpoint for executing a public action (#27793)

* Add endpoint and test

* Add more tests and TODOs

* Use mt

* Reorder migrations

* Remove unused require

* Rate limit actions at 1 per second

* Fix the tests for the throttle

* Refactor tests

* Fix test

* Fix docstring

* Add test for failed execution if actions are disabled

* Use crowberto in tests

* Fix using crowberto in tests

* Fix cyclic load dependency

* Add GET api/action/public endpoint

* Use defendpoint

* Add action-api-test

* Add model_id to GET action/public response

* Update GET api/action/public test for model_id

* Add actions to public sharing admin settings (#27675)

* Add actions to public sharing admin settings

* Add public action in public sharing admin settings tests

---------

Co-authored-by: Mahatthana (Kelvin) Nomsawadi <me@bboykelvin.dev>

[Fixes #28298]

* Add action.archived and search

Constituent commits:

Basic action archiving and search; no perms

Only show actions in search with appropriate permissions

archive test

Make search tests more tolerant of local DB state

Show model that actions come from in search results

Test action search properly

Per Conor/Bruno, actions have higher search precedence than pulses

Don't return archived actions from the main GET endpoint

Clean things up based on PR review

Don't show archived actions in most endpoints

Remove actions from search so this can be merged sooner

Delete associated DashboardCard when Action is archived

Add a few more tests

* Check at GET /action/:uuid

* Optimize queries

* Fix and add tests

* Add tests for public_links

* Remove tap>

* Use t2

* More of ngoc's suggestions

* use nil for default connection

* Use post-select

* Use select-one instead

Include database settings in GET "/api/database" and GET "/api/database/:id" response and respect `:visibility` (#28150)

* Apply visibility to database settings

* Refactor: Apply same pattern to writable visibilities

* Remove extra whitespace

* Update user-readable-values-map docstring

* Fix syntax error

* Update JSON encoding to filter settings even if the database is writeable

* Update test

* Add database endpoint tests

* Update to-json docstring

* Update comment

* Fix kondo errors

* Update geojson env var test

* Fix tests

* Change comment

* Rename to writable-settings and other suggestions

* Un-private test settings

* Typo

* Change temp-defaults to have setting be {} not nil

* Update settings only when settings is non-nil

* Fix

* Fix tests

* Fix test

* Fix test

* Support Honey SQL 2 for SQL QP

* Sort namespace

* Make Postgres/Redshift be Honey SQL 2-only

* Test fix 

* Fix busted

* Redshift test fixes 

* Sort namespaces

* Fix Postgres convert_timezone

* Fix linter config.

* Switch MySQL driver to Honey SQL 2

* Fix one last test failure

* Redshift test fix

* Test fixes and some extra validation etc.

* Update common.clj

* Update common.clj

* Update common.clj

* Add missing reflection warning

* Add missing reflection warning

* Use new namespace names

* Fix a few things.

* We should error on double-compilation of Honey SQL forms.

* Fix a bunch of nonsense

* Move Token features check fixes to a different branch

* Test fixes

* Fix double-compilation of raw forms

* Test fix 

* Test fix 

* Eliminate unneeded casts

* Fix syncing JSON table in MySQL 5.7

* Don't error on sync for tests that are supposed to test sync error handling

* update mu/defn kondo config 2 support lsp renaming

* update mu/defn kondo config 2 support lsp renaming

* add docstrings

* fix linting error

* fix "unused binding" error in test fxn

* fix some calls to mu/defn

* Address feedback on #28017

* Fix evil code order

* avoid n+1 in search api

* remove the mapping from model to the old toucan model

* use model vars instead of keyword

* Mind = blown 

* Remove printls

* Simplify things

* Better exampls

* Update dev.clj

* Support Honey SQL 2 for SQL QP

* Sort namespace

* Make Postgres/Redshift be Honey SQL 2-only

* Test fix 

* Fix busted

* Redshift test fixes 

* Sort namespaces

* Fix Postgres convert_timezone

* Fix linter config.

* Fix one last test failure

* Redshift test fix

* Test fixes and some extra validation etc.

* Update common.clj

* Update common.clj

* Update common.clj

* update description

* update api docs with script

* Better API error message for malli schemas

* add tests

* fix failed test

* fix the backward tests

* add api localized error message test

* add require and fix test