* drop unused computation_job table

* drop unused computation_job_result table

This reverts commit 9da0bd76.

* initial prototype w/out statement count parameter

* new approach

* default-database-role driver method

* migration for connection_impersonations table

* conn impersonation model, API and tests

* impersonation fetch and deletion endpoints

* switch test to t2 with-temp

* read conn impersonation settings from DB

* fix merge issue

* add impersonated key to data perms graph and treat it the same as full self-service access

* include impersonated key in returned permissions graph

* make sure impersonated graph passes StrictDataPerms validation

* fix boolean logic

* make sure impersonated keyword doesnt cause error when deleting gtaps

* clear impersonations as necessary when perms graphs changes

* add impersonation support for postgres

* fix typo

* make sure impersonation updates are a non-lazy seq

* add impersonated-user? fn

* fix impersonation api tests

* fix snippet tests

* fix build & exclude connection impersonations from serialization

* switch a test to use t2.with-temp

* add with-impersonations helper and util tests

* move macro and add a connection impersonation driver-level test for postgres

* fix rebase issue

* more tests and code reorganization

* add snowflake test

* clarify comment

* fix lint errors

* fix final kondo error

* reorganization

* fix one test

* fix lint errors

* revert change to sql_jdbc.execute from bad merge

* make sure perms for all users gets reset after conn impersonation tests

* ignore exceptions when restoring perms

* fix postgres test

* refactor to address bryan's comment

* add note about new methods to database changelog

* driver method refactor

* update audit log view migration

* separate mysql and fix h2

* typo

* fix views

---------

Co-authored-by: Luiz Arakaki <luiz.arakaki@metabase.com>

* add the migrations

* edit comment

* add 'if exists' to drop view statements

* update yaml: move sql to the right spot

* fix migrations and add new test

* fix migration IDs in test

* dont fail migration if FK already exists

* rewrite without table aliases to try to fix error on mysql/mariadb

* try to rewrite again

* revert to prior version of query

* another attempt at a rewrite

* fix typo

* fix typo

* bump ci

* replace LIKE with =

* forward migration for size_x and col

* done?

* test fixes

* adds some document

* migration should not contains comment

* mysql and postgres use different syntax in casting

* makes it work on mariadb

* update grid columns

* update specs

* update specs

* use float for postgres

* postgresql not postgres

* add migration for revision with model=Dashboard

* row is reserved in h2 too

* handle edge case with rollback migration where we have a 1x1 card that located at the shrinkage line

* kondo

* Cal's feedback

* add the code we used to generate random array

* clean up

* use the algorithm to dynamicall generate test data

* no more height migrations

* grid ratio

* unmigrate height in cypress specs

* update specs

---------

Co-authored-by: Aleksandr Lesnenko <alxnddr@gmail.com>

* Adds audit v2 view migrations to liquibase

* bit of cleanup

* Add reverse migration for AddJoinAliasToVisualizationSettingsFieldRefs

* Limit selections to query_type="query"

* Make like strings more specific

* Fix where clause

* Further optimize where clause

* Rename and refactor

* Update test name

* Fix indentation and naming consistency

* add migration for typed collections

* add hydration of collections to card+dashboard api

- adds it for:
  - GET /api/question/:id
  - GET /api/dashboard/:id

* fix test that checks for collection shape

* get tests working with hydrated collection

* typo fixes

* fix more tests

- PUT requests don't hydrate :collection

* update tests to fix param_values {} vs nil inconsistencies

* Update underlying records drill - bring to top, label changes (#29984)

* Zoom in Drill (#30208)

* Add updated zoom drill title

* Fix e2e tests

* Add title unit tests

* Fix for Records drill UI when it is the only one displayed

* Review fixes

* Add pivot drill common popover (#30275)

* Add grouping for Pivot drill options

* Reuse filterField type

* Fix date formatting after rebase

* Move Automatic insight action to a common popover (#30370)

* Refactor ChartClickActionsView, reuse it component in drills (#30706)

* 26836 separate views for quick cell click filters (#30532)

* Added separate views for QuickFilterDrill

* Add tests, fix styles issues

* 26836 contains/does not contain quick filter for tables (#30778)

* Add contains/does not contain quick filter for tables

* Self review fixes

* Fix for column header filter search options cropped

* Fix for icon margin difference for contains / does not contain filters

* Fix contains filter actions icon styles

* Add quick filters support for aggregated queries

* Not show contains filter for joined fields

* Fix typecheck

* Adjust string field quick filter section title to have column name (#30888)

* 26836 Add View details action for quick filters (#30967)

* Add View details action to quick filters

* Self review fixes

* Fix e2e tests

* Fix review issues

* indexed entities schema

* schema

* endpoint with create and delete

* hooked up tasks

* first tests and hooking it all together

* some e2e tests

* search

* search

had to rename value -> name. The search stuff works well for stuff that
looks the same. And previously was renaming `:value` to name in the
search result, but it uses the searchable-columns-for-model for which
values to score on and wasn't finding :value. So easier to just let name
flow through.

* include model_pk in results

* send pk back as id, include pk_ref

* rename fk constraint (had copied it)

* populate indexed values with name not value

* GET http://localhost:3000/api/model_index?model_id=135

* populate indexed values on post

done here for ease of demo. followup work will schedule the task
immediately, but for now, do it on thread and return info

* test search results

* fix t2 delete syntax on h2

* fix h2

* tests work on h2 and postgres app dbs

* ...

Fix column settings being duplicated when fields have the same `id-or-name`, but different `join-alias` (#27487)

* Migrate legacy field refs in result_metadata

* Add expression and field_ref

* Update testing description

* Fix clj-kondo

* Exclude empty result_metadata from the query

Removed colloquial use of "your" in x-rays where appropriate.

* add column for marking the audit database

* create audit db on app startup

On app startup, in enterprise mode, always check that there is an
up-to-date audit database with is_audit set to true. up-to-date means
that it matches the app-db. If the audit db doesnt exist, or doesn't match up with
the app-db, it will be created. otherwise no-op

- refactor GET /api/database/ into defendpoint

* add a few tests

* add more tests + return kw for no-op

* Merge branch 'master' into bcm-add-app-db-as-audit-db

* fix migrations file

* lint fix + remove def

* keep the old audit db around

- nil -> ::no-op

* small macro fix

* Merge branch 'master' into bcm-add-app-db-as-audit-db

* Merge branch 'master' into bcm-add-app-db-as-audit-db

* move ensure-db-exists! into its own namespace

- avoids circlar dependency on app startup

* fix tests

* lint fix

* inspect test values

* use mdb.env/env to initialize audit db

* try returning the app-db connection pool

- instead of making a doppelganger in metabase database

* get the is_audit column when looking up dbs in qp

- this key is needed to return the appdb for the "Audit DB"'s connection

* clean up audit-db install/update/noop logic

* remove audit db from GET api/database/

* Merge branch 'master' of github.com:metabase/metabase into bcm-add-app-db-as-audit-db

* fix linter + inline extra function

* fix a test

* Merge branch 'master' into bcm-add-app-db-as-audit-db

* don't test actions on audit db 

* Revert "don't test actions on audit db "

This reverts commit 581ed1ab9495de7b46bccb8b1ab19f66a8fd106d.

* for tests to work low IDs are basically reserved

- use a higher id when installing audit db

* use defenterprise for ensure-audit-db-installed!

* Merge branch 'master' into bcm-add-app-db-as-audit-db

* Dashboard Tab entity (#29802)

* add dashboard tab entity

* Dashboard tabs CRUD (#29861)

* Dashboard tabs frontend (#30189)

* rename `DashboardHeader.tsx` to `DashboardHeaderView` to avoid collision with `DashboardHeader.jsx`

commit-id:7cdfd86a

* add tabs to dash

commit-id:d3c2fa74

* improve code structure and other fixes

* rename `dashboardtab_id` to `dashboard_tab_id`

* remove explicit `position` logic

* improve code quality

* add renaming functionality

* add two new tabs when creating first tab

* hide tabs if there is only one

* add unit test for `DashboardTabs` component

* refactor

* fix type errors

* add `tabId` to text cards

* add `tabId` to link cards

* add `tabId` to action button

* use disabled state when only one tab remains

* refetch cards with filter value when changing tabs

* add e2e test

* fix broken e2e tests

* add horizontal scrolling

* fix unit test

* fix create tab button being on right

* fix performance issue when switching tabs

* add `ttag` to `tabs.ts`

* use `aria-label` in e2e test helper

* fix type error after rebasing

* making ordered_tabs optional for tests purposes

* fix failing unit tests

* fix failing e2e tests

* fix flaky revision history e2e test

* fix type error after merging feature branch with master and rebasing

* fix revisions unit test

* fix `ActionParmatersInputsForm.tsx` to stop `actions-on-dashboards.cy.spec.js` from flaking

---------

Co-authored-by: Ngoc Khuat <qn.khuat@gmail.com>

---------

Co-authored-by: Emmad Usmani <emmadusmani@berkeley.edu>

* Create a migration for updating column_settings

* Make fn private

* Update test description

* Fix clj-kondo

* Migrate the entire vizualization_settings

* Add test for normalization and migration

* Update migration class name

* Use generate-string to improve key readability

* Add testing strings

* Fix migration

* transduce

* Add another test checking that `expected` is actually the up-to-date version

* Rework migration to only update field refs

* Use run!

* allow formData uploads

* basic CSV uploads from collections

* add dropzone support for collections

* Add CSV uploads feature and have postgres and H2 support it (#29636)

* Add csv uploads feature and have postgres support it

* Add h2 support

* CSV: schema detection (#29650)

* CSV / Admin Upload Settings (#29709)

* allow formData uploads

* basic CSV uploads from collections

* add dropzone support for collections

* allow formData uploads

* basic CSV uploads from collections

* add dropzone support for collections

* Add CSV uploads feature and have postgres and H2 support it (#29636)

* Add csv uploads feature and have postgres support it

* Add h2 support

* Add defsettings

* Add table prefix setting

* add data upload settings

* Add newline

* update settings mock

* Whether or not uploads are enabled

---------

Co-authored-by: Cal Herries <39073188+calherries@users.noreply.github.com>
Co-authored-by: Callum Herries <hi@callumherries.com>

* Create table and insert rows from CSV uploads for postgres (#29677)

* WIP create table and insert rows

* Add newline

* Use multimethods

* WIP

* Fix postgres

* Fix postgres impl

* Use detect-schema and add test

* Fix load-from-csv arglists

* Undo whitespace change

* Use empty db instead

* Add table and field tests

* Remove prn

* Uniquify names

* newline

* Add test for float

* Add a test for the table contents

* Tidy load-from-csv

* Drop the table if loading the csv fails

* Drop the table only if loading the csv fails

* whitespace

* Address SQL injection vulnerability

* Use the slugified file-name as the table-name

* Fix kondo

* Using absolute path instead of relative

* Fix kondo

* Detect types for empty CSV files appropriately (#29754)

* Create models from CSV uploads (#29966)

* Wire up CSV uploads, sync the new table

Still need to create a model

* Check database support for CSV uploads better

* Another test

* Create model after sync

* Fix request with formData

* Fix tests

* Fix tests

* Revert "Fix request with formData"

This reverts commit 66f146a41cf42feb34dbb03ed8566fa4220d0642.

* Add whitespace

---------

Co-authored-by: Tim Macdonald <tim@tsmacdonald.com>

* Fix postgres support for uploads (#29922)

* Check write perms for collection (#29976)

* Rename metabase.csv -> metabase.upload (#29975)

* Rename metabase.csv -> metabase.upload

* Add docstring

* Rename alias for release.common.upload

---------

Co-authored-by: Callum Herries <hi@callumherries.com>

* WIP - 29598 csv status toast (#29757)

* temp

* generalize StatusListing types

* Putting a pin in this for now

* poorly done typing

* generalizing upload function names

* point to real API

* Fixing Storybook

* Permissions for Status Listing

* use formdata and fix types

---------

Co-authored-by: Ryan Laurie <iethree@gmail.com>

* Don't support redshift (#29981)

* Handle uploads to the root collection (#29985)

* Allow the uploads schema to be nil (for MySQL, etc.) (#29987)

* CSV upload: fix stack overflow in detect-schema (#29983)

* Fix stack overflow error on detect schema

* map + vec -> mapv

---------

Co-authored-by: Tim Macdonald <tim@metabase.com>

* reload collection data after upload (#30009)

* MySQL support for uploads (#29829)

* Add MySQL support for uploads

* Rename arg to upload-type

* Use new ns name

* Allow no table-name prefix to be specified and fix tests

---------

Co-authored-by: Tim Macdonald <tim@tsmacdonald.com>

* Fix setting descriptions (#30023)

* Batch CSV insertion for better performance (#30028)

* CSV upload e2e tests (#30005)

* add CSV upload e2e tests

* update tests

* H2 support for uploads (#29828)

* Add uploads support for H2

* Rename arg

* whitespace

* Use new metabase.upload ns name

* Fix tests for H2

* Make table and field matching case insensitive

* Fix kondo error

---------

Co-authored-by: Tim Macdonald <tim@tsmacdonald.com>

* Clear CSV upload state after a while (#30014)

* clear upload status after 5s

* make async behavior clearer

* CSV Upload Dates and Datetimes Types (#30076)

* Simple dates

* Simple datetimes

* Fix coalesce for date and datetime

* Add mysql and h2 types

* Fix test

* Misc cleanup

* Fix test

---------

Co-authored-by: Tim Macdonald <tim@tsmacdonald.com>

* Fix Kondo (rebase gone wrong)

* MySQL test fix

* Fix order of Metabot setting

* adding unit tests to upload actions and reducers (#30161)

* adding unit tests to upload actions and reducers

* use fetch-mock

* Adding component unit test

* Check CSV Upload permissions on the frontend (#30094)

* CSV / Admin Upload Settings (#29709)

* allow formData uploads

* basic CSV uploads from collections

* add dropzone support for collections

* allow formData uploads

* basic CSV uploads from collections

* add dropzone support for collections

* Add CSV uploads feature and have postgres and H2 support it (#29636)

* Add csv uploads feature and have postgres support it

* Add h2 support

* Add defsettings

* Add table prefix setting

* add data upload settings

* Add newline

* update settings mock

* Whether or not uploads are enabled

---------

Co-authored-by: Cal Herries <39073188+calherries@users.noreply.github.com>
Co-authored-by: Callum Herries <hi@callumherries.com>

* properly check if user can upload to a collection

* do some db checking

* don't check admin status for upload notification

* misc cleanup

* fix rebase

* add loggedin check to statusListing

also combines container + view component into a single file

* update unit tests

* update tests

---------

Co-authored-by: Cal Herries <39073188+calherries@users.noreply.github.com>
Co-authored-by: Callum Herries <hi@callumherries.com>

* Csv/misc bugfixes (#30219)

* Support empty header names

* Handle long CSV names more appropriately

Handle duplicate column names in CSVs

* Test fix

* Partition CSV inserts a little more intelligently

* Test fix

* Remove 'duplicate' from duplicated column names

* More intelligent sampling

* Add comment re. Postgres table-name-length-limit

* Quote column names, clean up tests

When unquoted, we couldn't use reserved words (true, false, group, etc.)
as column names

* Fix test

Now that we quote things, H2 is case-sensitive

* Use dialect-appropriate quoting

Also clean up some card-filtering tests so they work with a dirty DB

* Fix deduplication

* Remove use of setting/get

* Remove unneeded migration

---------

Co-authored-by: Callum Herries <hi@callumherries.com>

* Fix test (#30383)

---------

Co-authored-by: Ryan Laurie <iethree@gmail.com>
Co-authored-by: Cal Herries <39073188+calherries@users.noreply.github.com>
Co-authored-by: Ryan Laurie <30528226+iethree@users.noreply.github.com>
Co-authored-by: Callum Herries <hi@callumherries.com>
Co-authored-by: Nick Fitzpatrick <nick@metabase.com>

* initial changes

* fix test

* add ldap

* fix tests

* update migrations

* swap

* swap ldap

* fix test

* update test

* fix test

* fix test

* address comments

* update

* Use type/JSON instead of type/SerializedJSON

* Tidy migration

* Update migration

* Fix rollback migration for h2

* whitespace

* Add test for migration

* Fix test

* Add rollback

* whitespace

* Test JSONB type as well as JSON

* Don't fingerprint JSON columns

* Remove comment, that might be wrong in the future

* Update test

* Add tests for fingerprinting

* Use base-type JSON for fingerprinting base query

* Add test for visibility-type=details-only

* undo .

* Change migration id

* Fix migration test

* Merge master

* Exclude mariadb from tests

* Make is-mariadb? public

* Migration for adding nfc_enabled

* Add nfc_enabled to field settings

* Update describe-nested-field-columns to only unfold fields that have not been disabled

* Remove spy

* Tidy

* Fix

* Clear nested fields immediately if folding is disabled

* Clear nested fields on nfc_enabled change

* Trim trailing whitespace

* Tidy

* Fix

* Add enable-json-unfolding-test

* Move to field api test

* nfc_enabled -> json_unfolding

* Tidy test

* Make json-unfolding in database details just the default for new settings

* Restore original visibility-type logic

* Fix

* Sync field json_unfolding according to db json_folding

* Rename to json-unfolding-default

* Add test for the case when the json-unfolding is false for the database

* Implement default json unfolding for first sync

* Update comment

* Update comment

* Restore PUT field

* Migration for populating json_unfolding for mysql and postgres

* Remove migration and use default on the frontend instead

* Update json-unfolding database setting copy

* Move Unfold JSON setting under semantic type setting

* Separate sentences with spaces

* Capitalize

* Restore handleChangeSemanticType

* Use base_type TYPE.JSON instead

* Change order of migrations

* Add h2 migration

* Add test for migration

* Update test description

* Remove validCheckSum

Co-authored-by: Ngoc Khuat <qn.khuat@gmail.com>

* Use ?? instead of ||

* Fix tests

* Remove outdated serdes stuff

* Remove unnecessary and

* Fix mysql migration

* Remove unused require

* Fix test

* Add false default value for json_unfolding

* Fix json-unfolding nil case

* Other suggestions

* whitespace

* Tidy describe-nested-field-columns

* Update comment

* Remove unused clear-nested-fields!

* Remove unused return value

* Fix H2 migration to use base_type not database_type

* Always set json-unfolding during sync

* Fix test

* Fix test

* Fix test

* Add comment explaining nested-field-column support for MySQL

* Fix tests

* Fix tests

* Fix test

* Fix test

* Default json_unfolding to false for new fields

* Fix merge

* Add json_unfolding to mock tables

* Don't capitalize prepositions

* Update setting description

* whitespace

* whitespace

* Fix fetch_metadata

* Fix fetch_metadata

* Fix tests

* Fix test

* Fix clj-kondo

* Remove postgres database-supports test

* Fix postgres test

* Fix postgres test

* Fix mysql migration

* Fix clj-kondo

* Don't test mariadb

* Fix mysql test

* Fix mysql json-unfolding nil case

* Add comments to test

* Add upterm step to mariadb test

* Move upterm step before tests

* Fix postgresql migration

* move upterm step to test-driver action

* Remove upterm step from drivers.yml

* Comment out everything else in test-driver

* Fix mariadb migration

* Remove upterm action

* Whitespace

* Only handle JSONObjects with JSON_VALUE, not JSONArray

* Fix fields.sync_metadata/update-field-metadata-if-needed!

* Add comment explaining `json-unfolding-default`

* Add comment for JSON_VALUE test

* Update json-unfolding-default-true-test to use fresh db

* Fix test descriptions

* Don't capitalize setting display name

* Fix update json_unfolding

* Fix unfolding json for only one JSON column, not all

* Add e2e test for JSON unfolding setting

* Fix test

* whitespace

* Fix test

* Fix json-unfolding for nested field columns

* Fix mysql and postgres tests for nested field columns

* Coerce nil json-unfolding to false for sync_metadata

* Add test for nil json-unfolding for sync_metadata

* Fix test

* Don't update json_unfolding from intial sync

* Add json_unfolding to update-field-test

* Fix tests

* Use cypress-testing-library commands

* Fix e2e test

* Fix e2e test

* Add json-simple to deps

* Update uberjar.yml

* import org.json.simple

* Replace SQL migrations and H2 UDF with a clojure migration

* Remove json-simple from deps

* Restore uberjar.yml

* Rename to set-json-unfolding!

* Make the DB JSON unfolding setting override the field setting, instead of being a default.

* Remove unused var

* Fix error

* Remove custom clojure migration

* Fix migration

* Fix migration again

* Fix migration and kondo

---------

Co-authored-by: Ngoc Khuat <qn.khuat@gmail.com>

* Backend for allow disabling auto-apply filters in slow dashboards (#30069)

* Dumb BE change before proper BE code is implemented

* Add some backend tests for auto_apply_filters

- included in revisions
- included in serdes

fixup some tests that didn't expect it

* Fix BE for public dashboards

---------

Co-authored-by: dan sutton <dan@dpsutton.com>

* Prevent auto apply filters on slow dashboards (#29569)

* Able to toggle auto-apply dashboard filters

Make FormField be able to show title with new style

* Only apply filters when clicking apply button

* Add E2E test

* Fix dashboard double fetch cards onload with parameters query

* Remove puzzle driven development comments

* Move logic from HoC to redux 1/x

* Make apply filter button works on embeddings

* Fix x-ray dashboard parameters not working

* Fix unit tests

* Add E2E tests to public dashboard sharing

* Add signed embedding E2E tests

* Fix the apply button transition not working when clicked

Since there's another CSS applied when the button is hovered.

* Reviews: Fix x-ray dashboard actions

Remove the need of 3-states boolean in `setParameterValue` action for
x-ray dashboards. This fixes so that BE sends x-ray dashboards with
`auto_apply_filters: true`.

* Review: remove duplicate `aria-label`

* Review: standadize `draftParameterValues` default values

* Review: Fix `aria-label` to not change when filter values change

* Review: avoid passing `ReactNode` as a prop

Also fix transition not working and apply button shift when adding a
filter.

* Move selectors and actions to their own file

* Review: Don't animate (un)mount of FilterApplyButton

* Address: Clean up eslint ignore on tests

* Review: Only test 1 scenario in E2E

Since testing other scenarios are unnecessary since they're repeat the
same thing

* Review: Use E2E helper to clean up test

* Review: Reuse selector

* Review: Use e2e helpers to clean up the test

* Fix typo

* Ensure Apply button won't appear unnecessarily (#30255)

---------

Co-authored-by: dan sutton <dan@dpsutton.com>

Better metabot logging, especially around generated token counts. This will be particularly useful as we determine what size of prompts we can generate.

* Add safe DDL commands to the whitelist for actions

* Run 'GRANT ALTER ANY SCHEMA TO GUEST;' on sample database

* Grant permissions for test databases and add test for sample database

* Fix test

* Fix test

* Add test for adding/removing columns

Added both FE and BE tooling for integration with LLMs, called "metabot". FE adds prompts for querying models and databases. BE takes these prompts, calls a LLM, parses the response, and returns the result.