I didn't consider that we might not have permissions necessary to
disable triggers.

* new version of data_access migration + run ANALYZE beforehand

* relax constraints on migration timestamps

* more migration rewrites

* fix bug in migration

* fix another bug and remove extraneous logging

* reorder migrations to fix ID validation

* start adapting the changes to mysql as well

* wip

* add more ANALYZE calls

* revert logging changes

* finish duplicating work for mysql + a bit of clean up

* create H2 versions of migrations without the ANALYZE calls

* remove ANALYZE call from view_data.sql (probably not necessary)

* fix mysql syntax

* fix another mysql bug

* another mysql fix

* Don't exclude inactive tables for data_access

* Don't write rows for Administrators group

* Don't write rows for Admins (H2/MySQL)

* disable triggers before perms migration

---------

Co-authored-by: John Swanson <john.swanson@metabase.com>

* Add column_settings migration

* Remove old getColumnKey

* Remove old getColumnKey

* Fix export

* Change comment

* Change comment

* Change comment

* Change comment

* Add RemoveFieldRefsFromDashboardCardColumnSettings

* fix

* chore

* chore

* chore

* chore

* chore

* chore

* chore

* chore

* chore

* chore

* chore

* Fix

* Fix

* Fix tests

* Fix tests

* Fix tests

* Fix tests

* Fix tests

* Fix tests

* Update loki

* Update loki

* Fix the homepage dashboard being loaded everywhere

* Fix the homepage dashboard being loaded everywhere

* Fix tests

* Fix tests

* Fix tests

* Fixes

* Fix tests

* Fix tests

* Fix tests

* Fix

* Fix

* Fixes

* Fixes

* Fixes

* Fixes

* Fixes

* Fixes

* Fixes

* Fixes

* Fixes

* Fixes

* Fixes

* Fixes

* Fixes

* Fixes

* Fixes

* Fixes

* Fixes

* Fixes

* Fixes

* Fixes

* Fixes

* Fixes

* Fixes

* Fixes

* Fixes

* Fixes

* Fixes

* Fixes

* Fixes

* Fixes

* Fix typo

* Fix types

* Improve migration performance

* Fixes

* Revert changes

* Revert changes

* Revert changes

* Revert changes

* Remove migration

### Description

This change should fix some explosions in stats due to phantom fields, while also making things a bit more rigorous.

- Record the table schema in query_field - especially useful for unknown tables
- Allow null table names in query_field - they might be phantoms, or they might not
  - We still have the option to filter them out in tools we build on top
- Tighten up matching logic to consider schema

* Update popover dark theme on part of ParameterFieldWidget (#45813)

* Add filter visual tests

* Simplify embed CSS colors

* [WIP] theme filter popover

* Move theme to global node, so it affects portal too

* Fix embed colors

* Theme part of ParameterFieldWidget

* Add story for parameter with dropdown

* Update loki snapshots

* Update more popover style

* Update loki snapshots

* Remove outdated snapshots

* Update loki snapshots with production build

* Fix loki not working with `userEvent`

* Theme mantine popover

* Improve dark theme text-brand visibility

* Update loki snapshots from master change

* Fix new colors not working in SDK

* Fix flaky Loki tests

* Filter popover dark theme part 2 (#46284)

* Theme Date all options

* wip add month and year filter

* Fix wrong CSS custom property declaration

* Test hover state

* Update Loki snapshots

* Theme date month and year

* Add quarter and year tests

* Theme date single and date range

* Style relative date

* Update frontend/src/metabase/admin/datamodel/components/filters/pickers/DatePicker/DatePickerHeader.styled.tsx

Co-authored-by: Denis Berezin <denis.berezin@metabase.com>

---------

Co-authored-by: Denis Berezin <denis.berezin@metabase.com>

* Complete theming filter popover night mode (#46365)

* Style unit of time and number widget

* Attempt to fix flaky Loki snapshots

* Address white background checkbox on dark theme

* Style search filter single value

* Fix mis color

* Update snapshots

* Remove duplication on play functions since they're long

* Milestone 2 polishing (#46382)

* Matches the footer padding with header padding

* Update snapshots

* Add E2E to prevent further problem

* Try to reduce flaky Loki tests

* Remove unused color shades (#46421)

* Update no results illustration to work better on dark theme

* Update loki snapshots

---------

Co-authored-by: Denis Berezin <denis.berezin@metabase.com>

* Squash

* Clean up query modifier for validating card references (#46283)

* Fixup bad git merge on test expectation

* Tweak docstrings and de-densify logic

* Adds a middleware that sets 2 dynamic vars

- These vars will be used to insert context for embedding analytics

* migration adding embedding_client and embedding_version to qe and vl tables

* clarify function api

* headers come in -> dynamic vars -> written to db

When the vars are set we insert their values into query execution and
query_log under embedding_client and embedding_version.

* update update migrations

* add predconditions to migrations

* fix tablenames + dataset_test

* Code Review responses

- remove commented out requires
- clarify test for merge-count-maps

* Add `report_dashboard.last_viewed_at`

- defaults to `now()` (it's intended to be used as a timestamp after which
we're *sure* the item hasn't been used, not a definitive timestamp we're
sure the dashboard was actually used at)

- updated whenever the QP processes a query for a dashcard

---------

Co-authored-by: Ngoc Khuat <qn.khuat@gmail.com>

* refactor: Simplify code (#44275)

* Remove dead code

* Remove values same as default options

* Simplify hook API: Removing unused argument

* review: Remove redundant code

* Update look and feel copy (#44277)

* Update static embed tab name

* Update copies + rearrange settings

* Use Mantine Select

* Fix E2E tests by applying new copies

* Fix unit tests

* Fix more E2E tests

* review: Add a comment on Mantine `value` quirk

* Adjust spacing and relevant tests

* Move `transparent` theme to its own toggle option (#44362)

* Reorder the function to match the visual order

* Make function name consistent

* Introduce the new background option (do nothing yet)

* Remove redundant exports

* Tighten up values to reduce confusion

* Make the parameters order consistent

* Only show background option on dashboads (spec)

* Allow dashboard embed background to be transparent

* Make filter background and header/footer transparent

* Fix type error

* Fix unit tests

* Fix E2E tests

* Fix checkerboard pattern not showing when no background

* Add unit tests

* Add missing copy update

* Remove remnants from outdated solutions within the PR

* Improve code comment references

* Fix E2E tests since the last copy change

* Fix type errors from merging master

* Update light theme style (#44488)

* Add background to dashboard filter on light theme

* Make position sticky works with transparent background

* Fix sticky filters not working on theme=transparent

* Fix unit tests

* Simplify IntersectionObserver components + fix E2E tests

* Add storybook for embedded dashboard

* Clarify the use of ExplicitSize context

* Remove unused code

* Formatting

* Review: Better mock `isWithinIframe`

* Update dark theme to match the new design (#45134)

* Update light theme checkerboard colors

* Update checkerboard colors

* Consolidate dashboard light/dark border style

* Use `--mb-color-text-primary` for light theme dashboard

* Update embed dashboard dark theme primary color

* Migrate dashboard text-secondary color

* Migrate text-tertiary color

* Migrate gold, axis, and split line to CSS custom properties

* Fix failed e2e tests

* Fix static viz not working

* Update loki snapshots

* Remove unnecessary CSS custom property

* Make diff more consistent

* Address review: reusing type

* Support backward compatibility when passing `theme=transparent` (#45214)

* Match preview query with the actual code query

* Handle backward compatibility when `theme=transparent` on dashboard

* Add unit tests

* Address review: Make tests less repetitive

* Update Metabase banner upsell UI (#45424)

* Prepare folder for nested components

* Update upsell UI

- add `fullWidth` variant
- align title to the left

* Fix Upsell banner having horizontal scroll on full width

* Add new Metabase banner upsell component

* Fix unit tests

* Address review: Fix typo

Co-authored-by: Ryan Laurie <30528226+iethree@users.noreply.github.com>

* Address review: Move upsel UI to `admin/upsells`

Co-authored-by: Ryan Laurie <30528226+iethree@users.noreply.github.com>

* Remove unnecessary ESLint disable comment

We can remove it since the file is in admin folder, and is already excluded from the rule.

* Address review: remove default value property

Co-authored-by: Denis Berezin <denis.berezin@metabase.com>

* Address review: Extract magic number

---------

Co-authored-by: Ryan Laurie <30528226+iethree@users.noreply.github.com>
Co-authored-by: Denis Berezin <denis.berezin@metabase.com>

* More UI updates (#45450)

* Update dark theme text colors to match the new design

* Update powered by metabase logo

* Update loki snapshots

* Fix unit tests

* Fix formatting

* Fix wrong dark theme metabase logo badge color

* Address review: Compress SVG with svgo

Co-authored-by: Denis Berezin <denis.berezin@metabase.com>

* Fix E2E tests

* Update loki snapshots

* Clarify Metabase base colors with comments

---------

Co-authored-by: Denis Berezin <denis.berezin@metabase.com>

* Add more tests (#45628)

* Add static embed question stories

* Add loki snapshots

* Add transparent theme for static dashboard tests

* Add transparent theme tests for static questions

* Assert `background` option in analytics tests

* Remove tests for logic that has been moved

* Update loki snapshots

* Fix failed E2E tests from merging master

---------

Co-authored-by: Ryan Laurie <30528226+iethree@users.noreply.github.com>
Co-authored-by: Denis Berezin <denis.berezin@metabase.com>

* Alert on asset download error

* moving function to metabase global

* removing alert, adjusting e2e test

* update CSP

* Nicer error messages for linting migrations

In general, just throw exceptions instead of using clojure spec here.

You'll only get one error per run, but that seems fine.

I did keep some specs around, but run them slightly differently. Rather
than validating the whole collection of changeSets at once with `s/+`,
just `doseq` through the changeSets and validate each one separately.

That way, the value that's presented as erroneous is much smaller (a
single changeSet) and it's easier to see what went wrong.

* Update bin/lint-migrations-file/test/lint_migrations_file_test.clj

Co-authored-by: Noah Moss <32746338+noahmoss@users.noreply.github.com>

* Update bin/lint-migrations-file/src/lint_migrations_file.clj

Co-authored-by: Noah Moss <32746338+noahmoss@users.noreply.github.com>

* Require preConditions for certain liquibase change types to encourage idempotence (#44578)

Co-authored-by: Ngoc Khuat <qn.khuat@gmail.com>

* remove `!`s from non-side-effecty fns

* add preConditions to a few v51 migrations

---------

Co-authored-by: Noah Moss <32746338+noahmoss@users.noreply.github.com>
Co-authored-by: Ngoc Khuat <qn.khuat@gmail.com>

Per discussion in Slack, I made a mistake with the migration IDs as I wanted to backport to 50 but used 51 IDs.
This PR uses 50 IDs and places the migrations in the correct place.

Not sure if it's mergable in this state because Stats has run with the PR containing the incorrect migration IDs.

* User Parameter Values Are Unique Per Dashboard

Fixes: #43001

Might be related: #44858

User Parameter Values previously stored just the `parameter-id` and `user-id` as data to look up the last used
parameter value. This isn't sufficient as the parameter id is not guaranteed to be unique, in particular this is true
when a dashboard is duplicated: the parameters are copied to the new dashboard without changing the parameter id at
all.

This means that we need to also store the dashboard-id in the user_parameter_value table and use that to update/get
the last used value.

The migration removes existing entries to the user_parameter_value table because I want a non-nullable constraint on
the dashboard_id column, and existing entries will not have a value. The only way to backfill those values would be to
look through every dashboard containing parameters, and for every parameter check for a matching ID. Even if you can
do this, there's no way to disambiguate if the same paramter exists on 2 dashboards anyway, so one of them will be
wrong. I think it's not worth the trouble considering that removing the entries in this table doesn't break anything;
it just becomes a mild inconvenience that a user has to select a filter value again (since the dashboard will use the
default value).

* alter test to check for uniqueness across dashboard

This test makes 2 dashboards with parameters of the same ID and asserts that changing the value on dashboard-a does
not change the value on dashboard-b.

* adjust migration to pass linter rules

* remove the extra rollback on migration

* Update src/metabase/models/user_parameter_value.clj

Co-authored-by: bryan <bryan.maass@gmail.com>

* Update src/metabase/models/user_parameter_value.clj

Co-authored-by: bryan <bryan.maass@gmail.com>

* adjust parameters in test so we don't get logged warnings

* put update!/insert! in a transaction to avoid any race conditions

---------

Co-authored-by: bryan <bryan.maass@gmail.com>

* Wrap non-latin characters in a span specifying working font

Fixes: #38753

CSSBox seems to have a bug where font fallback doesn't work properly. This is noticeable when a font does not contain
glyphs that are present in the string being rendered. For example, Lato does not have many international characters,
so the rendered version of tables (that show up in Slack messages) will not render properly, making the card
unreadable.

Since this appears to be a downstream bug, I've opted to work around this limitation by wrapping any non-latin
characters in a <span> that specifies the font family to be sans-serif, which should contain the glyphs to properly
render.

This leaves Lato in place for other characters.

For now, I figured it's worth trying this solution over using Noto for 2 reasons:
- we can keep Lato, which has been the decided font style for the app for some time (this keeps things consistent
where possible)
- the Noto font containing all glyphs is quite a large font (>20mb, I think) and it would be nice to avoid bundling
that if we can.

* stub installed fonts test

* typo

* Do wrapping, but now per-string, and in Clojure data not html string

I've decided that a reasonable solution is to still wrap strings containing non-lato characters. But it's not done
with str/replace to the html string but rather in a postwalk over the Hiccup data prior to rendering.

Seems like a decent compromise of issues without patching CSSBox or fixing upstream (may be good to do, but will take
longer to get a fix in).

* add test checking that glyphs render correctly

* Add a test that directly checks the wrapping fn

* Change the string to keep the linter quiet

* Change how we check if string can be rendered to faster method, new Lato Font

With Sanya's help, the way I check if a given string is renderable with Lato is now faster.

Also use the full Lato font, not just the 'latin' lato so we can cover more chars

* change lato so that it loads the fn even in a fresh test run

* Writing migration notes

* adds the endpoint and a test

* add recents endpoint, todo: tests

* add post recents endpoint

* return recent views for both lists, but make the endpoint work

* Make recent-views context aware

- pruning is context aware, only checks for the recently-inserted
  context
- Adds endpoints:
    - POST to create selection recents,
    - GET activity/recents
      - requres context query param to be one of:
      - all, views, selections

- Adds context arg to update-users-recent-views!
- Cleans up arg schema for update-users-recent-views

* impl GET api/activity/recents

- return recent-selections and recent-views from
- send context == collection from pinned card reads

* update callsites of recent-views/update-users-recent-views!

- to use :view param where necessary

* fixes models/recent-view tests

* adds more activity/recent-view tests

* wip

- fix whitespace linter

* Fix command palette e2e test

- reuse util snake-keys

* updates fe to use new recents endpoints

* fixes fe type issue

* snake-keys -> deep-snake-keys

- I've been betrayed by lsp rename

* snake-keys -> deep-snake-keys

- I've been betrayed by lsp rename

* log selection events for created collections

* mysql doesn't allow default values for TEXT types

* log a recent view on data-picker selection

* decouple view-log context from card-event view context

* fix a doc typo

* stop double logging recent-views on POST

* maybe fixes some tests

* some e2e fixes

* fix mysterious divide by zero during score search

* fix divide by zero possibilities everywhere in score-items

metabase.api.activity/score-items used to throw when there weren't any
items being scored (even though there's a `(when (seq items) ...)` check)

* more test fixes

* fix more e2e tests, + rename endpoint in tests

* fix oopsie

* fixes a few more tests

* address review comments/questions

* allow for a comma delimited list in qps like ?context=views,selections

returns all recent view items with those contexts, most recent first
under the `:recents` key.

* refactors FE around new endpoint

* fixes for unit tests

* use ms/QueryVectorOf for context

* fix models/recent_views tests

* use multiple query params instead of comma delimited value on FE

* ignore timestamp when deduping recents

* review comments + test fixing

* update docstring

* fix api/activity_test s

* actually dedupe

* add test for deduping by context

* e2e fix: shows up-to-date list of recently viewed items after another page is visited

* e2e fix: should undo the question replace action

* e2e fix: should replace a dashboard card question (metabase#36984)

* e2e fix: should preselect the most recently visited dashboard

* fix 6 more e2e tests

* fixes fe type check and unit failure

* renames unit test mocking function

* fix a flaky e2e test

* widen Item to accept str or kw where sensible

- allow strings or keywords for moderated_status, and authority_level

* simplify impl + add test

* add view-log to events schema

* add collection context to view log

* fix the final 2 failing e2e tests

* click dashboard tab when the user has can-read? on recent entities

---------

Co-authored-by: Sloan Sparger <sloansparger@gmail.com>

* Improve the Trash

Ok, so I had a realization at the PERFECT time, immediately after the RC
cutoff. Great job, brain!

Here's the realization. For the Trash, we need to keep track of two
things:

- where the item actually is located in the hierarchy, and

- what collection we should look at to see what permissions apply to the
item.

For example, a Card might be in the Trash, but we need to look at
Collection 1234 to see that a user has permission to Write that card.

My implementation of this was to add a column,
`trashed_from_collection_id`, so that we could move a Card or a
Dashboard to a new `collection_id`, but keep track of the permissions we
actually needed to check.

So:

- `collection_id` was where the item was located in the collection
hierarchy, and

- `trashed_from_collection_id` was where we needed to look to check
permissions.

Today I had the realization that it's much, much more important to get
PERMISSIONS right than to get collection hierarchy right. Like if we
mess up and show something as in the Trash when it's not in the Trash,
or show something in the wrong Collection - that's not great, sure. But
if we mess up and show a Card when we shouldn't, or show a Dashboard
when we shouldn't, that's Super Duper Bad.

So the problem with my initial implementation was that we needed to
change everywhere that checked permissions, to make sure they checked
BOTH `trashed_from_collection_id` and `collection_id` as appropriate.

So... there's a much better solution. Instead of adding a column to
represent the *permissions* that we should apply to the dashboard or
card, add a column to represent the *location in the hierarchy* that
should apply to the dashboard or the card.

We can simplify further: the *only time* we want to display something in
a different place in the hierarchy than usual is when it was put
directly into the trash. If you trash a dashboard as a part of a
collection, then we should display it in that collection just like
normal.

So, we can do the following:

- add a `trashed_directly` column to Cards and Dashboards, representing
whether they should be displayed in the Trash instead of their actual
parent collection

- use the `collection_id` column of Cards and Dashboards without
modification to represent permissions.

There's one main downside of this approach. If you trash a dashboard,
and then delete the collection that the dashboard was originally in,
what do we do with `dashboard.collection_id`?

- we have to change it, because it's a foreign key

- we can't set it to null, because that represents the root collection

In this initial implementation, I've just cascaded the delete: if you
delete a dashboard and then delete a collection, the dashboard will be
deleted. This is not ideal. I'm not totally sure what we should do in
this situation.

* Rip out all the `trashed_from_collection_id`

* Migration to delete trashed_from_collection_id

* fixes

* don't move collections

And don't allow deleting collections

* only show cards/dashboards with write perms

* Show the correct archived/unarchived branch

* some cleanup

* add a todo for tomorrow

* Fix for yesterday's TODO

* more wip

* refactor

* memoize collection info

* move around memoization a bit

* fix schema migration test

* oops, delete server.middleware.memo

* Use a migration script (postgres only for now)

* Fix some tests

* remove n+1 queries in `collection-can-restore`

* fix test

* fix more tests, and x-db migration script

* fix h2 rollback

* fix mysql/mariadb migration

* lint

* fix some mariadb/mysql tests

* fix h2 rollback

* Fix mysql rollback

* Fix Postgres migration

* "Real" `trash_operation_id` UUIDs from migration

* fix mariadb migration

* Separate MySQL/MariaDB migrations

* trashed directly bit->boolean

* Remove `trashed_from_*` from migrations

* Rename `api/updates-with-trashed-directly`

Previously named `move-on-archive-or-unarchive`, which was no longer
accurate since we're not moving anything!

* Add `can_delete`

* Delete test of deleted code

* Can't move anything to the Trash

The Trash exists as a real collection so that we can list items in and
such without needing special cases. But we don't want anything to
_actually_ get moved to it.

* integrates can_delete flag on FE

* Update src/metabase/models/collection.clj

Co-authored-by: Noah Moss <32746338+noahmoss@users.noreply.github.com>

* Update src/metabase/search/impl.clj

Co-authored-by: Noah Moss <32746338+noahmoss@users.noreply.github.com>

* Better name for `fix-collection-id`

Also consolidated it into a single function. It takes the
trash-collection-id as an argument to prevent a circular dependency.

* s/trashed/archived/ on the backend

In the product, we want to move to "trashed" as the descriptor for
things that are currently-known-as-archived.

It'd be nice to switch to that on the backend as well, but that'll
require quite a lot of churn on the frontend as we'd need to change the
API (or have an awkward translation layer, where we call it `trashed` on
the backend, turn it into `archived` when sending data out via the API,
and then the FE turns around and called it "trashed" again).

For now, let's just be consistent on the backend and call it `archived`
everywhere. So:

- `archived_directly` instead of `trashed_directly`, and
- `archive_operation_id` instead of `trash_operation_id`

* Fix up a couple docstrings

* Rename `visible-collection-ids` args

for `collection/permissions-set->visible-collection-ids`:

- `:include-archived` => `:include-archived-items`

- `:include-trash?` => `:include-trash-collection`

* select affected-collection-ids in the tx

* Stop dealing with `:all` visible collection ids

This used to be a possible return value for
`permissions-set->visible-collection-ids` indicating that the user could
view all collections. Now that function always returns a set of
collection IDs (possibly including "root" for the root collection) so we
don't need to deal with that anymore.

* Don't use separate hydration keys for collections

Dispatch on the model of each item (but still keep them in batches for
efficiency.)

* vars for collectable/archived-directly models

* Round up loose `trashed_directly`s

* Fix a test

* Use `clojure.core.memoize` instead of handrolled

It's slightly different (TTL instead of exactly per-request) but that
should be fine.

* FE lint fixes

* 

 e2e test fix

---------

Co-authored-by: Sloan Sparger <sloansparger@gmail.com>
Co-authored-by: Noah Moss <32746338+noahmoss@users.noreply.github.com>