This was throwing errors trying to `nippy/freeze!` an Atom.

That was because in certain circumstances a pMBQL query can end up in
`(:json_query metadata)` for a query, and those can contain an atom
due to `metabase.lib.cache` and both `CachedMetadataProvider` and
`InvocationTracker`.

I added `lib.query/serializable` as a single location to strip out
anything that isn't serializable, such as atoms. (And the entire
`:lib/metadata`, which is a wasted effort to serialize.)

Remove localization for some additional metrics that were missed in 2882692b.

* Fix reference in deprecation notice

* Add lock_filled icon

* Add view-only tag to the question header for questions with hidden source tables

* Simplify the useHasHiddenSourceTables helper

* Add typings to useHasHiddenSourceTables

* Show 'view-only' badge when the question is not editable

* Set up card endpoints

* Add test for hidden tables

* Add e2e test for View-only tag

* Add visibilityType to TableDisplayInfo

* Use unfiltered table display info to determine which tables are hidden

* Rename util to hooks

* Use rem for border-radius

* Sign in in beforeEach

* Add tests for more visibility types

* Fix unit tests

* Remove empty file

* Add test for unavailable cards

* Remove unnecessary card endpoint

* Add unit test for inaccessible card

* Move ViewOnlyTag to it's own file

* Add more unit tests for ViewOnly tag

* Use MBQL to create queryies where possible

* Fix a Couple bugs related to pivot exports

 - pivot-grouping value, when not an int (in the case of Oracle dbs), would cause exports to contain zero rows. This
 fixes that
 - json formatted/unformatted for the dataset API endpoints (unsaved questions) was not being used, so all exports
 were formatted

* json and csv use (int group) to check row inclusion

* grouping change fix tests

* json formatted/unformatted test

* xlsx formatters are correct when pivot-grouping col is removed

* cljfmt

* fix up failing tests

* fix test

* Update src/metabase/api/dataset.clj

* Modularize metabase.pulse; add metabase.pulse.core API namespace

* Rename metabase.pulse.send-test

* Split email result attachment stuff into its own namespace

* Update dashboard-subscription-test

* Allow metabase.email.result-attachment externally... for now

* More code cleanup

* Decouple models.card and email.messages

* Decouple models.collection and metabase.email.messages

* Hopefully the last fix 

* Fix #48647 

* Fix event handler

* :render/text, not :pulse/text

* Test fixes 

* Test fix 

* RESPECT the modules

* Remove unused namespace

* Adds query_executions_by_source_24h and entity_id_translations_last_24h

- add docstring

* clear eid translation count in stats-post-cleanup

* remove ->> with after? it's confusing

* dissoc the right path

== Behaviour without this fix ==

See #48881

== Behaviour with this fix ==

1. Metabase starts with an empty application database, and a config file
   containing configuration for a connected database
3. By the time it reaches "INFO metabase.task :: Task scheduler started",
   `qrtz_triggers` contains `metabase.task.sync-and-analyze.trigger.1`
   (sample DB) and `metabase.task.sync-and-analyze.trigger.2` (DB from
   config file)

Closes: https://github.com/metabase/metabase/issues/48881


Co-authored-by: Noah Moss <noah@metabase.com>

* Use type date for case expression when there are Date and DateTime args

* Use logic for getting case expr type from annotate middleware

* Add e2e test

* Add bigquery test

* Update test

* cljfmt and comment

* Adjust e2e test

* Fix expression stage

* Add test

* Parallel test

* Adjust test

* Adjust test

* Use metric's aggregation :name

* Add test

* fix: Always cast json number types as decimal

Fixes #48507

* Fix tests

* Fix tests

* Fix test

Fixes #48639

* Let non-admins see tables in recents

```shell
❯ http get "localhost:3000/api/activity/recents?context=views" Cookie:"metabase.SESSION=ba..3d" -pb
{
    "recents": [
        {
            "can_write": false,
            "database": {
                "id": 6,
                "initial_sync_status": "complete",
                "name": "pg restaurants"
            },
            "description": null,
            "display_name": "Restaurants",
            "id": 112,
            "model": "table",
            "name": "restaurants",
            "table_schema": "public",
            "timestamp": "2024-10-15T22:11:39.412100Z"
        },
        {
            "can_write": false,
            "database": {
                "id": 6,
                "initial_sync_status": "complete",
                "name": "pg restaurants"
            },
            "description": null,
            "display_name": "Reviews",
            "id": 110,
            "model": "table",
            "name": "reviews",
            "table_schema": "public",
            "timestamp": "2024-10-15T19:53:19.999445Z"
        }
    ]
}
```

We were getting permissions of a "fake" table. Instead, let the db
select it and get the proper stuff. Worked for admins because
mi/can-read? is presumably always true for tables, and worked in tests
because we mocked mi/can-read?

* bump ci

* Add new promethues counter for :metabase-metrics/adjust-errors

To record errors in metabase.query-processor.middleware.metrics/adjust

* Add prometheus metrics test to metrics middleware tests

* Also inc :metabase-metrics/adjust-errors when throwing an exception

* Make splice-compatible-metrics private

It doesn't seem to be used outside of metabase.query-processor.middleware.metrics.

* Add test for :metabase-metrics/adjust-errors when exception is thrown

* Move inc-and-throw! helper into metabase.analytics.prometheus

* Add inc-and-throw! test to prometheus tests

* Rename inc-test to inc!-test and use approx= and metric-value helpers

* Add try/catch to metrics adjust and record metric errors in a single place

Record promethues metrics in a single place in metabase.query-processor.middleware.metrics and ensure we record
failures for any exception throw.

* Improve testing description string

* Rename :metabase-metrics/adjust-errors to :metabase-query-processor/metrics-errors

* Move metrics adjustment try/catch into a new helper

This ensures we count all exceptions generated by adjust-metric-stages.

* Add new prometheus metric to count number of MB metrics seen by the QP

This new metric is intended to count the number of metrics that the query processor metrics middleware sees and
attempts to adjust.

* Add more metrics metrics tests

- Ensure that queries with no metrics do not incremement either counter
- Ensure that successful calls to adjust do not increment the error counter
- Ensure that exceptions throw from other libs also increment the error counter

* Rename test to adjust-prometheus-metrics-test and add helper to reduce code dup

* If no metrics are found immediately return query in metrics middleware adjust

Don't attempt to adjust query if we don't find any :metric clauses.

* Appease cljfmt

* Improve metric description

Co-authored-by: metamben <103100869+metamben@users.noreply.github.com>

* Improve epsilon test in approx=

Co-authored-by: metamben <103100869+metamben@users.noreply.github.com>

* Remove adjust-metric-stages-counting-errors

This function was an attempt to avoid double-counting errors, but probably not worth the effort.

See

https://github.com/metabase/metabase/pull/48598#discussion_r1800197964
https://github.com/metabase/metabase/pull/48598#discussion_r1800200421

* Count metrics directly when mocking prometheus/inc!

Based on PR feedback here:

https://github.com/metabase/metabase/pull/48598#discussion_r1800207674

* Remove prometheus/inc-and-throw!

Value prop was always questionable, but now that it's only called from a single place, just remove it and inline the
call to prometheus/inc!. Easy to add back later if it proves generally useful.

PR discussion thread

https://github.com/metabase/metabase/pull/48598#discussion_r1799728596

* Rename match-one-metric to find-first-metric

https://github.com/metabase/metabase/pull/48598#discussion_r1803321025

* Rename :metabase-query-processor/metrics-errors to metric-errors

https://github.com/metabase/metabase/pull/48598#discussion_r1803313818

* Throw an exception rather than logging a warning if we fail to adjust a metric

https://github.com/metabase/metabase/pull/48598#discussion_r1803328482



---------

Co-authored-by: metamben <103100869+metamben@users.noreply.github.com>

* add prometheus tracking for sdk-embedding

* indentation

* improve track-sdk-response api

* register :metabase-sdk/response-{ok,error} + test

* avoid cyclic dependency whe nusing prometheus

fixes:
[ /metabase/api/common ]
<- /metabase/models/setting
<- /metabase/analytics/settings
<- /metabase/analytics/prometheus
<- /metabase/analytics/sdk
<- /metabase/models/view_log
<- /metabase/events/schema
<- /metabase/events
<- [ /metabase/api/common ]
<- /metabase/public_settings/premium_features
<- /metabase/auth_provider
<- /metabase/driver
<- /dev/debug_qp
<- /dev

* nix dependency: events.schema --> models.view-log

- make them both read context from view-log-impl instead

* cljfmt

* adds data + schema for metrics stats ping

* remove comment

* annotate todos

* fill in the rest of the metrics values

- and add defaults

* fix some definitions + use a single timestamp

* shuffle stuff around to appease the namespace linter

- we aren't reaching into the api API, so the linter is more of a
  formality here.

* update docstring

* fix jsonschema, maybe

* review responses

- revert changes to 1-0-0
- add metrics section to new file: 1-0-1
- bump ::instance_stats to "1-0-1"
- add tags into 1-0-1
- make the code return tags (empty for now until we know what to tag things.)

- also fix test that broke from shuffling settings around

* remove a footgun

* add tags to metrics,

add grouped_metrics to jsonschema 1-0-1

format 1-0-1

* indent

* cljfmt

* version should match filename

* update instance stats 1-0-1 schema

* require `tags` in grouped_metric + snowcat comment

* fix formatting noise

* update schema to make it valid

* remove grouped_metrics from instance_stats schema

* cljfmt appeasement

* unbin cache_num_queries_cached value

- alphabetize metrics generation

* we can now guarantee metric values will be ints

* jsonschema for instance_uuid, settings, and grouped_metrics

* add analytics_uuid and make it required

* lint + alphabetize instance stats json schema

* update setting key type

- add maxLength to some strings

* lint jsonschema

- Add description to setting.items.tags
- add maxLength, and {min,max}imum to setting.items.value

* Bump instance stats to 2-0-0

- remove analytics_uuid string length == 36 check
- adds assertion to ensure required fields are set (and it passes)
- adds info for embedding settings

* adds a grouped-metric to stats ping

- adds length info to the schema to pass jsonschema linting

* cljfmt

* Incremental Pivot Processing for Exports

WIP

Fixes pivot exports for CSV and xlsx.

The CSV export should use less memory by incrementally building up the data structure and aggregating necessary row
data right away, so the memory overhead becomes only as large as the total pivot result.

In cases where the pivot rows/cols do combine into many many columns and rows, this can still be a large set of data,
but it should behave much better now in most cases.

The Excel export is a little more straightforward: create the export rows in the same fashion, streaming one row at a
time, and just post-process the sheet to add the pivot table in one shot at the end.

* WIP adding row totals.

* aggregate totals as rows are added

Row, column, section, and grand totals are all aggregated as each row is added.
This means the final step of building pivot output becomes just an exercise of lookups/arrangement, no further
aggregation is needed.

* CSV pivot works per-row, export respects formatting

This is a big step forward; we don't need to hold the entire dataset in memory, we instead aggregate a row's data into
the pivot datastructure, which only holds onto:

- unique values for each pivot-row in a sorted set
- unique values for each pivot-col in a sorted set
- grand total for each measure N values, where N is number of measures, ususally 1 or 2
- row totals for each combination of each pivot-row * N measures
- col totals for each combination of each pivot-col * N measures
- totals for each 'section', determined by unique values of first pivot-row * N measures
- values for each measure in every 'cell'; Row Combos * Col Combos * N Measures

So, there can still be a decent amount of data to store; but it will never hold onto all of the 'raw rows' from the
dataset.

We can never completely guarantee that Row Combos * Col Combos * N Measures remains small, but two things let us move
forward anyway:

- there's now visible feedback in the app that the download is running (or if it's failed)
- Pivot table utility diminishes rapidly with huge output anyway; users still need to curate/set up their data
- effectively to improve the table's utility, so we can assume that a slow-to-download pivot table is also slow to
- use/less effective, and will likely be something the user doesn't want (as often).

* some test fixes

* now, if we export 'raw pivot rows', they don't show pivot-grouping

and they also don't include the 'extra' rows for totals/subtotals/grand totals (any row with pivot-grouping > 0).

This means that now the non-pivot version of a pivot table export will match what a user sees if they change the viz
to a regular table.

* remove old test

* re-incorporate some changes from master

* fix csv for non-pivots due to oversight in my changes

This is just a temporary change, I think I should clean up this bit of the code a little, I can probably make it a
little more readable and use some cleaner logic regarding if the rows are 'raw pivot rows' or not.

* start moving format_rows to POST bod, add pivot_results too

There's still wiring work to do, but this starts to add format_rows and pivot_results to POST body for the various API
endpoints. Also modify tests to improve coverage/consistency across downloads and alerts/subscriptions.

The tests will not pass on this commit, but fixes will be incoming

* native pivot tables in xlsx

* add precondition to pass migration linter

* try to get migrations fixed

* pasing pivot-results through api and attachments

* fix tests for format_rows in BODY vs query param

* tests!

* might have the tests all fixed now

* the pivoted export now respects col/row totals settings

* add test coverage for public questions and dashboards

* col and row totals work as expected

* build-pivot refactor for clarity

* docstring change + tiny refactor in helper fn

* see if dashcard download works with format_rows

* csv pivot handles nil values

* pass format_rows and pivot_results in :params not :body

* fix some other tests

* pivot-grouping col filtered out of xlsx

* pivot-grouping-col removed for all rows

* configurable pivot exports and attachments (#47880)

* exports fe

* specs

* ui

* specs

* format/unformatted now works for xlsx

* format test changes for xlsx formatting

* embedding endpoints accept pivot_results

* cljfmt and eslint fix

* empty

* embedding test should have formatting defaulted to true

* embed test fixes

* Use `Chip` for export settings widget

* downloads e2e test fix

* fix public download limit test

* public card download defaults

* fix public download defaults in some tests

* Fix visual test

---------

Co-authored-by: Aleksandr Lesnenko <alxnddr@users.noreply.github.com>
Co-authored-by: Noah Moss <32746338+noahmoss@users.noreply.github.com>
Co-authored-by: Anton Kulyk <kuliks.anton@gmail.com>

* fix iframe dashcards crash subscriptions

* add a test to ensure iframes are filtered out of subscriptions

---------

Co-authored-by: Adam James <adam.vermeer2@gmail.com>
Co-authored-by: adam-james <21064735+adam-james-v@users.noreply.github.com>

* [Notification] Migrate user invited email (#48215)

* [Notification] Migrate alert create email (#48292)

* [Notification] Migrate slack token error email (#48333)

* Implement inactive field removal

* Use information_schema for describe-database

* Unrelated fix: set dynamic-dataset-loading defaults to true

* cljfmt

* Fix test

* Collapse `metabase.shared.*` namespaces

* Fix Kondo warnings

* Does updating the stories-data keys fix the failing tests?

* Appease msgcat

* Appease msgcat

* Fix typo

* Make the build happy

* Appease fslint

* Remove `MB_API_KEY` env var

A bit awkwardly, we never set `:deprecated` on the setting before. We
can retroactively deprecate this as of v50.

I'm keeping the setting purely to emit the warning message on startup.

* version and channel query params for version info

https://github.com/metabase/metabase/issues/48615

* omit blanks from query params for version info