Fixes #30262.

* Use type/JSON instead of type/SerializedJSON

* Tidy migration

* Update migration

* Fix rollback migration for h2

* whitespace

* Add test for migration

* Fix test

* Add rollback

* whitespace

* Test JSONB type as well as JSON

* Don't fingerprint JSON columns

* Remove comment, that might be wrong in the future

* Update test

* Add tests for fingerprinting

* Use base-type JSON for fingerprinting base query

* Add test for visibility-type=details-only

* undo .

* Change migration id

* Fix migration test

* Merge master

* Exclude mariadb from tests

* Make is-mariadb? public

* Migration for adding nfc_enabled

* Add nfc_enabled to field settings

* Update describe-nested-field-columns to only unfold fields that have not been disabled

* Remove spy

* Tidy

* Fix

* Clear nested fields immediately if folding is disabled

* Clear nested fields on nfc_enabled change

* Trim trailing whitespace

* Tidy

* Fix

* Add enable-json-unfolding-test

* Move to field api test

* nfc_enabled -> json_unfolding

* Tidy test

* Make json-unfolding in database details just the default for new settings

* Restore original visibility-type logic

* Fix

* Sync field json_unfolding according to db json_folding

* Rename to json-unfolding-default

* Add test for the case when the json-unfolding is false for the database

* Implement default json unfolding for first sync

* Update comment

* Update comment

* Restore PUT field

* Migration for populating json_unfolding for mysql and postgres

* Remove migration and use default on the frontend instead

* Update json-unfolding database setting copy

* Move Unfold JSON setting under semantic type setting

* Separate sentences with spaces

* Capitalize

* Restore handleChangeSemanticType

* Use base_type TYPE.JSON instead

* Change order of migrations

* Add h2 migration

* Add test for migration

* Update test description

* Remove validCheckSum

Co-authored-by: Ngoc Khuat <qn.khuat@gmail.com>

* Use ?? instead of ||

* Fix tests

* Remove outdated serdes stuff

* Remove unnecessary and

* Fix mysql migration

* Remove unused require

* Fix test

* Add false default value for json_unfolding

* Fix json-unfolding nil case

* Other suggestions

* whitespace

* Tidy describe-nested-field-columns

* Update comment

* Remove unused clear-nested-fields!

* Remove unused return value

* Fix H2 migration to use base_type not database_type

* Always set json-unfolding during sync

* Fix test

* Fix test

* Fix test

* Add comment explaining nested-field-column support for MySQL

* Fix tests

* Fix tests

* Fix test

* Fix test

* Default json_unfolding to false for new fields

* Fix merge

* Add json_unfolding to mock tables

* Don't capitalize prepositions

* Update setting description

* whitespace

* whitespace

* Fix fetch_metadata

* Fix fetch_metadata

* Fix tests

* Fix test

* Fix clj-kondo

* Remove postgres database-supports test

* Fix postgres test

* Fix postgres test

* Fix mysql migration

* Fix clj-kondo

* Don't test mariadb

* Fix mysql test

* Fix mysql json-unfolding nil case

* Add comments to test

* Add upterm step to mariadb test

* Move upterm step before tests

* Fix postgresql migration

* move upterm step to test-driver action

* Remove upterm step from drivers.yml

* Comment out everything else in test-driver

* Fix mariadb migration

* Remove upterm action

* Whitespace

* Only handle JSONObjects with JSON_VALUE, not JSONArray

* Fix fields.sync_metadata/update-field-metadata-if-needed!

* Add comment explaining `json-unfolding-default`

* Add comment for JSON_VALUE test

* Update json-unfolding-default-true-test to use fresh db

* Fix test descriptions

* Don't capitalize setting display name

* Fix update json_unfolding

* Fix unfolding json for only one JSON column, not all

* Add e2e test for JSON unfolding setting

* Fix test

* whitespace

* Fix test

* Fix json-unfolding for nested field columns

* Fix mysql and postgres tests for nested field columns

* Coerce nil json-unfolding to false for sync_metadata

* Add test for nil json-unfolding for sync_metadata

* Fix test

* Don't update json_unfolding from intial sync

* Add json_unfolding to update-field-test

* Fix tests

* Use cypress-testing-library commands

* Fix e2e test

* Fix e2e test

* Add json-simple to deps

* Update uberjar.yml

* import org.json.simple

* Replace SQL migrations and H2 UDF with a clojure migration

* Remove json-simple from deps

* Restore uberjar.yml

* Rename to set-json-unfolding!

* Make the DB JSON unfolding setting override the field setting, instead of being a default.

* Remove unused var

* Fix error

* Remove custom clojure migration

* Fix migration

* Fix migration again

* Fix migration and kondo

---------

Co-authored-by: Ngoc Khuat <qn.khuat@gmail.com>

[ci skip]

Renamed Label to meet best practices.

[ci skip]

* add metabot tracking

* remove redundant metabot_page_view event

* mock metabot feedback endpoint

* feedback, add one prop

* Hide column types and expose a function (isa) to work with them

Part of #30054.

Effective and semantic type information are removed from display-info.
This forces the user of the API to use explicit type checks using the
new function isa.

Co-authored-by: Jeff Bruemmer <jeff.bruemmer@gmail.com>

* First pass at recent_views stored in user local setting

This modifies the view log event handler to write a user's views to their local settings map (stored as json in the
appdb).

some notes on what causes a view

:card-query
 - occurs when collection with pinned cards/models is opened. maybe shouldn't count as a view

:card-read
 - happens when you view a card, like, load it in the UI and are looking at it

:card-read AND :card-query
 - happens when you view a model, like, load it in the UI and are looking at it

:dashboard-read
 - happens when you view a dashboard

:card-create
 - when you save a card
 - this will probably cause double counts because it will save and then trigger a :card-read right away too

:table-read
 - when viewing a table.
 - when selecting a db/table in dropdowns in the Question editor

I think we need the view log topics to be much smaller: #{:card-read :dashboard-read :table-read}

* WIP test for the recent_views user local setting.

This isn't totally correct yet. I think it doesn't clean up after itself correctly. That's pending.

And, I have to investigate because :table-read isn't working either. Still worth pushing this progress!

* Test that works! All potential view events need :actor_id

* Working test for /api/recent_views

* Order of views in user-local recent-views works now

If a user views something they've recently viewed, the homepage order doesn't change. Now it will, as in, the most
recently viewed item will always be the first item in the vector.

* One time recent_views expensive query done in the user-recent-views getter

The logic of filling the recent-views setting was first written in the activity api namespace. Now it's handled where
the setting is defined, which lets anyone else consuming the setting not worry about filling it if its empty.

The setting can be reset with `nil` which will cause the query to run again. Otherwise, if the list of recent items is
an empty vector, that is considered valid and will not re-run the query.

Updating the setting per view is handled still in the view log handler, implemented in the private function `update-users-recent-views!`.

* Add `most-recently-viewed-dashboard` setting

The most recently viewed dashboard for that user, in the past 24 hours.

This is not meant to be a permanent solution to providing this bit of info for the user, rather, it's a 'stop gap'
while the audit tables are reworked.

* Add test for most recent dashboard setting

* Pass :context to the view log fns

This lets us filter out views triggered by pinned cards.

* Fix tests failing after adding :context to view metadata

* See if card-query can help us filter out pinned item views

* Tests should send :card-query events

* Since we use :card-query, we have to filter out dashboard context too

This should prevent views being recorded if someone just opens a dashboard.

* ViewLog Cleanup to see if it'll let other tests pass

* drop non-viz percy tests, add a workflow that assigns visual label which triggers percy tests runs

* remove push trigger in require-percy-test workflow

* fix typo

* driveby type improvement

* Fix not showing single or multiple values options

When a parameter is connected to a question with template tags

* Review: unit tests instead

* Fixes inlining of numeric params in native queries
* Updates namespaces to use sql.qp/with-driver-honey-sql-version

Inferencing for models was switched from an LLM to an embedding selector. Unfortunately, this did not do a very good job as too much context was not retained. This change goes back to using the LLM for model inferencing.

* Backend for allow disabling auto-apply filters in slow dashboards (#30069)

* Dumb BE change before proper BE code is implemented

* Add some backend tests for auto_apply_filters

- included in revisions
- included in serdes

fixup some tests that didn't expect it

* Fix BE for public dashboards

---------

Co-authored-by: dan sutton <dan@dpsutton.com>

* Prevent auto apply filters on slow dashboards (#29569)

* Able to toggle auto-apply dashboard filters

Make FormField be able to show title with new style

* Only apply filters when clicking apply button

* Add E2E test

* Fix dashboard double fetch cards onload with parameters query

* Remove puzzle driven development comments

* Move logic from HoC to redux 1/x

* Make apply filter button works on embeddings

* Fix x-ray dashboard parameters not working

* Fix unit tests

* Add E2E tests to public dashboard sharing

* Add signed embedding E2E tests

* Fix the apply button transition not working when clicked

Since there's another CSS applied when the button is hovered.

* Reviews: Fix x-ray dashboard actions

Remove the need of 3-states boolean in `setParameterValue` action for
x-ray dashboards. This fixes so that BE sends x-ray dashboards with
`auto_apply_filters: true`.

* Review: remove duplicate `aria-label`

* Review: standadize `draftParameterValues` default values

* Review: Fix `aria-label` to not change when filter values change

* Review: avoid passing `ReactNode` as a prop

Also fix transition not working and apply button shift when adding a
filter.

* Move selectors and actions to their own file

* Review: Don't animate (un)mount of FilterApplyButton

* Address: Clean up eslint ignore on tests

* Review: Only test 1 scenario in E2E

Since testing other scenarios are unnecessary since they're repeat the
same thing

* Review: Use E2E helper to clean up test

* Review: Reuse selector

* Review: Use e2e helpers to clean up the test

* Fix typo

* Ensure Apply button won't appear unnecessarily (#30255)

---------

Co-authored-by: dan sutton <dan@dpsutton.com>

* Add GitHub issues config

This removes the need for a separate "Questions and help" issue template!

* Convert "Bug report" into a form

One of the challenges of metabot is creating input suitable for prompts of different sizes. You can, with very large models and dbs, create prompts that are not runnable. This PR contains a function that reduces the size of table prompts (this should actually happen very rarely, but it is ugly when it does) until it finally gets to a reasonably sized prompt or just silently returns nothing (in this case you won't be able to prompt engineer the particular table -- use a model instead). It also moves memoization to the right place for tables to reduce workload and capture the right segments of data needed for prompt engineering.

* Add `OrderByClauseDisplayInfo` type

* Add display info types

* Fix column group functions

* Update `displayInfo` function typing

* Make order-by functions aware of the stage index

* Remove duplicated `OrderByDirection` type

* Add missing newline

* Make order-by functions aware of sort direction

* Add wrapper for `change_direction` function

* Use MLv2 for `StructuredQuery's` `hasSorts` method

* Add `QueryColumnPicker` component

* Use MLv2 in `SortStep` component

* Disable `OrderBy` class validation method

* Patch E2E tests

* Prefer "order by" over "sort"

* Remove not longer relevant tests

* Fix pivot table breakouts check

* Adjust for `fk_reference_name`

* Revert E2E test changes

* Fix column group functions usage

* Revert redundant changes

* Remove alias `sort` method

* Remove `canAddSort` method

* Remove `sortOptions` method

* Stop using `clearSort` in the notebook editor

* Remove `cleanSorts` method

* Return plain MBQL from query `sorts` method

* Remove `OrderBy` class

* Mark remaining order-bys methods as deprecated

* Remove `addSort` usage (one more left)

* Remove `SortWidget` component

* Rewrite comment

* Fix import

* move pre-update-check-sandbox-constraints to defenterprise

* fix failing test so that it runs with :sandboxes feture enabled

* Don't load metabase.query-processor-test.test-mlv2 until we load QP

* Add schema and tests for `:value`

* Don't skip tests for #29895

* Conversion logic for `:value`

* pMBQL normalization should handle `:effective-type`

* Options should validate `:base-type`, `:effective-type`, etc. if present

* Use `:effective-type` for type checking if present

The current state is a collection of hacks to preserve some legacy
behavior (eg. normalization sets a blank `widget-type` to `"category"`)
and it doesn't work properly right now.

An example of a currently-broken case:
- `SELECT * FROM Orders WHERE {{tag}}`
- Set it to a Field Filter and select People > LONGITUDE
  - Or some other field for which type we don't have a widget.
- It shows "None" as the widget type, and the widget isn't shown at the
  top.
- Save the question and reopen it.
- Now the widget is visible (and a bit broken) as a `"category"` type.

This is because on reading from the appdb it gets normalized.

This PR uses `"none"` properly for cases where there are no widget
options for the field type (eg. longitude), and updates all the usage
sites to properly handle `"none"` rather than missing/undefined.

Existing saved questions with a blank `widget-type` will continue to
"work" as they currently do.