There were two problems here:

- wrong args to `mdb/migrate!` (passing the db-type as well as the data
source), and

- in MySQL, the changelog table name is not `:databasechangelog`. Use
`metabase.db.liquibase/changelog-table-name` to get the actual table name.

* Handles card events for CRU (not D) => recent view

* add tests

* avoid some errors in dev

We expect recent-views/post-process to return nil occasionally (for
permissions reasons, or more), so let's filter those out instead of
printing a noisy unactionable warning

* uncomment and fix test

* fix some cypress tests + docs improvements

* update more cypress tests

* more cypress test updates

* adding a card puts it into your recents now

* fix notebook-data-source

* fix notebook-data-source

* fixing more e2e tests

* fix command palette e2e tests

* run prettier

* fix style issue

* update notebook-data-source tests

* update homepage e2e tests

* Next question is in recents, now

* update command palette: skip the first pagedown, since it is a noop

* check the buttons for their respective names

* ensure that the model is there, and no questions are

* Remove unused `metabase.api.transform`

* Copy the deps graph improvements from my other PR

* New combined X-Rays API namespace

* Fix kondo error

* Squashed commit

* Update config

* Backport the kondo ignore stuff from other PR.

* adds watch alias, which reloads BE on file changes

* watch dev, src, test + every path in `:extra-paths`

Co-authored-by: Emmad Usmani <emmadusmani@berkeley.edu>
Co-authored-by: Adam James <adam.vermeer2@gmail.com>
Co-authored-by: Mark Bastian <markbastian@gmail.com>
Co-authored-by: Jesse Devaney <22608765+JesseSDevaney@users.noreply.github.com>
Co-authored-by: Anton Kulyk <kuliks.anton@gmail.com>

* Wow

* Test fix 

* Fixes

* Actions should use strings for column names (fix :update-row and :create-row normalization)

* MLv2 schema should check against keys for the other query type

* Ok, have I fixed things?

* More fixes 

* Fix indentation

* Another round of test fixes. 

* Hopefully the last few test fixes 

* We need to test normalization for queries that have keyword keys as well.

* Fix Cljs i18n namespaces

* Sort namespaces

* Only test against H2

* Rename `metabase.mbql` to `metabase.legacy-mbql`

* Fix Kondo warnings

* Test fixes 

* Register MBQL clause schemas and test fixes 

* Test fixes and PR feedback

* Test fix

* Remove the normalization tests

* Test fixes 

* Fix kondo

* Fix import

* Another fix 

* Merge

* FIXES

* Add another missing REQUIRE

* first move to a db api

* mdb.spec into mdb

* mdb.u/isa -> mdb.query/isa

* mdb.u/qualify -> mdb.query/qualify

* last of mbql.u -> mbql.query and mbql.u

* remove last of the mdb.connection/db-type and use from mdb/db-type

* remove the last non-cmd usages of mdb.connection

had some circularity in

models.interface requires mdb/db-type for sql flavored "now"
mdb requires db/setup
which requires db/custom_migrations to ensure they are on cp
custom migration requires json functions in models.interface

so had to do a dynamic require for the db-type

* get db-type from mdb not mdb.env

* invoke `(mdb/app-db)` and not pass a var. lots of things fail

* mdb.setup/migrate! into mdb with potemkin

* remove get-connection. call .getConnection on the app-db

perhaps invites clumsy usage outside of `with-open` if it's a top level
function

* namespace docstrings

* last few and grab db.env/db-file into api ns

* lint cleanup

* addresses PR comments

* Drop report_card.dataset [WIP] [ci skip]

* Drop report_card.dataset

* Some test fixes

* More test fixes

* All OSS tests should be fixed now?

* Remove check for :dataset key from snake-hating-map

* Remove NOCOMMIT stuff

* Remove unused namespaces

* PR feedback

* Migrate dataset attribute in autocomplete suggestions (#39054)

* FE - Migrate `dataset: true` to enum value in Bookmarks (#39056)

* Fix Card["type"] - Bookmark["card_type"] mapping and add an extra assertion

* Fix card-type check out of raw query

* Fix the test

* Fix tests after merge

* Log body when unexpected response code

* Get logging in CI

* Relax is_upload schema for latest mariadb

---------

Co-authored-by: Kamil Mielnik <kamil@kamilmielnik.com>
Co-authored-by: Case Nelson <case@metabase.com>
Co-authored-by: Alexander Polyankin <alexander.polyankin@metabase.com>

* QP enterprise middleware should use `defenterprise`

* Test fix? 

* Remove `:rff` from the Query Processor context map

* Sort namespaces

* Simplify QP entrypoints part 1

* Test fixes 

* Test fixes 

* Make sure defenterprise dispatch is done for every QP run

* B I G  QP context overhaul

* WIP

* Remove qp.context.default

* Test fix 

* PR feedback

* QP context overhaul [WIP] [ci skip]

* QP context overhaul [WIP] [ci skip]

* Fix duplicate preprocesses

* Some test fixes.

* More test fixes [ci skip]

* Source card resolution middleware should be preprocessing middleware, not around [ci skip]

* Test fixes [WIP] [ci skip]

* Remove `test-qp-middleware`

* Remove `qp/compile`

* Fix Kondo warnings/cleanup

* Remove `mt/compile`

* Fix most tests

* Mostly everything working?

* 4 failures, 1 error

* Eliminate QP context `reducedf`

* 3 failures, 0 errors <3

* All tests are passing <3

* Remove dead debug-qp code

* Fix test failure

* Remove async context

* Remove raisef

* Forget it, just remove context entirely [ci skip]

* Fix Kondo errors

* Fix Kondo warnings

* Fix Kondo warnings

* Fix Kondo warnings

* Remove empty namespace

* Fix some reflection warnings

* Update dox [ci skip]

* Test fix 

* Fix pivot QP

* Misc improvements

* Test fix 

* Improvements and test fixes

* More test fixes 

* Test fixes 

* Address PR feedback

* Test fixes 

* More test fixes and docstring improvements

* Appease Eastwood

* Test fixes 

* More test fixes

* Test fixes 



* Revert test parallelization in metabase.models.params.chain-filter-test

* Update src/metabase/async/util.clj

Co-authored-by: metamben <103100869+metamben@users.noreply.github.com>

---------

Co-authored-by: metamben <103100869+metamben@users.noreply.github.com>

* load analytics only when changed using a checksum

- over audit deserialization files

* update docstrings

* fix testing wording

* fix nil analytics-resource-dir

* in tests, when you delete the audit db, set the checksum back to 0

* add a comment

* WIP hashing ia content from inside a jar

* Do the unzip sooner, still hashes the file system

Co-authored-by: John Swanson <john.swanson@metabase.com>

* cleanup java-time -> java-time.api

---------

Co-authored-by: John Swanson <john.swanson@metabase.com>

Sometimes a test passes when run by itself, but fails due to ordering
issues. This is typically caused by another test failing to properly
clean up after itself, leaving the database in a dirty state. For
example, we might change permissions and forget to change them back, so
that another test hits an unexpected permissions failure.

Finding the cause of these has been a source of pain for me, so I wrote
a small snippet to do it for me.

If you have a test that passes when run by itself, but fails when run
along with all the other tests, you can run
`(dev/find-root-test-failure! #'my-ns/my-intermittent-failure-test')`.

It will run *all* tests. After each one, it'll run the test you passed
in. Once that test starts failing, it'll alert you about which test
caused it to fail.

This still throws an exception on trying to directly remove the final
JOIN condition. But if the condition is being removed because a field it
depends on is being removed (eg. a custom column, a breakout, a
previous stage aggregation) then the entire JOIN is also removed, in the
usual cascading fashion.

Fixes #36690.

* Add dev fn to render dashboards with static-viz renderers

Run `(dev.render-png/render-dashboard-to-html 1)` to render the dashboard with id 1 to a handy html file.

This file will render each dashcard in the dashboard 3 ways:
 - as a png, like you would see in Slack or in an email body (if it's a chart)
 - as html/svg, like you'd see in the email body (for tables).. might also be helpful to inspect the svg output from
 the graaljs interpreter
 - 10 row table representing what the .csv attachment for the dashcard would look like

Note: to get this branch working properly, there are a couple lines commented out to pass the linter... it's a work in
progress and will be cleaned up for better use.

Todo:
 - [ ] avoid with-redefs to pass the linter
 - [ ] add a preview endpoint that can be used so that you don't need to run a Clojure repl to get the same result.

* Eliminate with-redefs to allow csv 'render'

* Make a 'preview' ns to power `api/pulse/preview_dashboard/:id`

* Make dev.render-png dashboard preview fns the same as the preview ns

* Dashboard Subscription preview ns now also uses csv attachment code

This change now uses the csv attachment code to include an html table representing what we expect to see from csv exports.

* Add dev fn to render dashboards with static-viz renderers

Run `(dev.render-png/render-dashboard-to-html 1)` to render the dashboard with id 1 to a handy html file.

This file will render each dashcard in the dashboard 3 ways:
 - as a png, like you would see in Slack or in an email body (if it's a chart)
 - as html/svg, like you'd see in the email body (for tables).. might also be helpful to inspect the svg output from
 the graaljs interpreter
 - 10 row table representing what the .csv attachment for the dashcard would look like

Note: to get this branch working properly, there are a couple lines commented out to pass the linter... it's a work in
progress and will be cleaned up for better use.

Todo:
 - [ ] avoid with-redefs to pass the linter
 - [ ] add a preview endpoint that can be used so that you don't need to run a Clojure repl to get the same result.

* Eliminate with-redefs to allow csv 'render'

* Make a 'preview' ns to power `api/pulse/preview_dashboard/:id`

* Make dev.render-png dashboard preview fns the same as the preview ns

* Dashboard Subscription preview ns now also uses csv attachment code

This change now uses the csv attachment code to include an html table representing what we expect to see from csv exports.

* Collect all element styles into a single style tag to use with the server's nonce

This allows the dashboard_preview endpoint to render properly by gathering all of the element styles into a single
style tag, which can then be given the server's nonce value so that CSP doesn't strip the style out.

This is useful for the preview endpoint so that we get an accurate picture of what our tables will look like in
emails, plus it makes the presentation look just a bit more readable.

* dev render_png fns match output used on the endpoint now too.

* Hickory dependency out of dev into regular deps

* Move sample-dataset.edn into test-data.edn

Replace (mt/dataset sample-dataset with (mt/dataset test-data

Replace more references to sample-dataset with test-data

Fix the format of the combined data

Might revert: remove test that doesn't apply, now that there's 1 DB

Make api.database-test pass, though expected could be wrong

* Update unit tests for combined dataset

* Bump Bigquery version from v3_ to v4_

* Bump v3_test-data to v4_test-data

* Remove wrapping with mt/dataset

Alphabetize fk-mappings

* Remove needless whitespace edits

* Fix failed e2e_test by removing reference to sample-dataset

* Fix tests for Mongo, Oracle, Snowflake, and Presto

* Fix unit tests again for Mongo, Presto, and Snowflake

* Will revert: run driver tests on my fork

* Will revert: run driver tests on my draft fork

* Revert "Will revert: run driver tests on my draft fork"

This reverts commit 078c8af1.

* Revert "Will revert: run driver tests on my fork"

This reverts commit e3e2922f.

* Make the postgres driver test pass again by reverting a change

* Apply Case Nelson's patch to fix Athena test

https://github.com/metabase/metabase/pull/36064#issuecomment-1824837705
Props @snoe

* Remove (mt/dataset sample-dataset) from new tests

* Maybe fix Athena driver tests

Revert needless deletions of (mt/dataset test-data
Maybe those deletions caused the
failing Athena driver tests.
But those deletions are out of scope
for this PR either way.

* Remove sample-dataset reference from a test in the master merge

* In card_test.clj, replace sample-dataset with test-data

* Merge in master, resolve conflict in pivot-from-model-test

* Merge in master, resolve conflict in pivot-from-model-test

* Must revert: allow databse creation

To see if it will fix:
https://github.com/metabase/metabase/actions/runs/7120808552/job/19388845720?pr=36064#step:3:441


Athena database creation is disabled: not creating database v2_test_data. Tests will likely fail.

* Revert "Must revert: allow databse creation"

This reverts commit d4c8f129.

* Add back in (mt/dataset where I deleted it

But replace sample-dataset with test-data.
This might be wrong, or maybe it's not needed in
so many places.
But it could show if this is the problem.

* Fix the failed sync test

---------

Co-authored-by: Case Nelson <case@metabase.com>

* Updating honeysql alias in dev ns

This just fixes a broken require to align with the work in #36670.

Fixes #36757

* a more refined first crack at adding perm-graph access for db or group id

* new routes: filter data-perm-graph on db or group

add tests

* remove inline defs

* code review responses

* fix typo

* fix the other tests

* update the tests to use the right malli schema

* reuse private vars in tests

* pull graph checker into test util ns and apply it

* Adding results_metadata to qp middleware

This adds two middlewares to `metabase.query-processor.middleware.results-metadata`:
- `inject-result-metadata`: Adds `result_metadata` from the context, if present, into the query map during preprocessing.
- `merge-existing-metadata`: A post-processing middleware that merges `results_metadata` (if present) from the query data into the metadata.

This has the effect of preserving existing metadata, which is particularly important for user-curated metadata, such as semantic type overrides.

This may also facilitate addressing the following TODO in the same ns:

```
;; 1. Is there some way we could avoid doing this every single time a Card is ran? Perhaps by passing the current Card
;;    metadata as part of the query context so we can compare for changes
```

* Preserving result_metadata in query processor

This PR uses `qp.util/combine-metadata` to simplify the combination of provided `:result_metadata` with computed metadata. It also updates pulse rendering code to use this new code path and adds and updates tests to vet this logic.

* fmt

* Modifying PR to document and use existing fns

The current qp pipeline will correctly propagate custom metadata if a question is derived from a model or if the model is processed like so:

```clojure
(qp/process-query
  (assoc-in dataset_query [:info :metadata/dataset-metadata] result_metadata))
```

This PR fixes rendering unit tests to clarify this and provides a much better working example in the `dev.render-png` ns.

* Updating test

* Adding Pulse w/Metadata Tests

This pr adds a unit test that constructs a dashboard and simulates a pulse email, then checks the resulting HTML for correctness.

A dependency on the [hickory](https://github.com/clj-commons/hickory) library was added to dev. This allows us to easily parse HTML into data and navigate the parse tree.

Note that it does NOT check the attachments for formatting correctness.

Spelling and import consistency changes were also made.

* Adding Percent Metadata Type

This adds the :type/Percentage metadata type to Metabase. It does
not currently do any fingerprinting. It just enables setting of the
type for the FE.

* Making :type/Percentage a semantic type as well as a decimal.

* support Percentage setting in the semantic type editor, infer percentage column formatting default

* Support for static-viz rendering when user metadata is `:type/Percentage`

This PR adds in user-defined metadata in `prep-for-html-rendering` so that `number-formatter` can properly render the column as a percentage. This may actually fix a family of issues as user-defined metadata doesn't appear to be used anywhere in this ns. We still need to consider cases where the formatting is specified by the viz settings, but this one step in the right direction.

* Percent semantic types render as percents in tables

This PR adds logic to properly capture a percent semantic type and renders it as such in a table static viz. Note that the `number-formatter` function in `metabase.pulse.render.common` has logic that renders the percent as a truncated int. IDK if this the desired long term behavior. I'll have to ask product that, but if we just make the change it could have potential impacts on places that expect this behavior. It seems like it would make more sense to be like Excel and allow each column to set its significant digits.

* Revert "support Percentage setting in the semantic type editor, infer percentage column formatting default"

This reverts commit e001e6312e6f0bd5700c1d9eb28c88829d87a70a.

---------

Co-authored-by: Aleksandr Lesnenko <alxnddr@gmail.com>

Use `(dev.fe-helpers/current-query)` from a CLJS REPL to get the query currently open in the UI.

Thanks to @dpsutton for the help here! I investigated an alternative approach of reusing the machinery that we use in
tests to determine whether the database already exists, but that turned out to rely on a private atom that stored a
list of databases created by the tests. I didn't want to muck around with those internals, and it turned out to be
simpler than expected to make `can-connect?` work (although I did need to catch the exception thrown when we couldn't
connect, which seems a little odd - I would have expected `can-connect?` to catch that and return `false`).

Fixes https://github.com/metabase/metabase/issues/9962