* Keep Filter Values for enabled Parameters in Embedding

Fixes: #37914

Previously, we removed locked parameters entirely. This is to prevent leaking potentially sensitive values.

However, for the situation where the same field backs 1 locked or disabled paramter and 1 enabled parameter, we do
still want to send the paramater values, because the enabled parameter implies that the values are permissible to see
in the embed.

So, this change will still remove parameters based on their 'disabled/locked/enabled' status, but will NOT remove the
linked field ids if they're also being used by 'enabled' parameters.

This results in the backend correctly sending necessary parameter values to the embed, where the frontend can then
render the appropriate UI instead of falling back to just text filters.

* test the case where a locked and enabled param share same field

* Address feedback. Added comment to explain classify fn

* Exclude implicitly joinable columns from visible-columns in fk-columns-to

* Add test

* Update description

[P1 BUG FIX] Cumulative aggregations with multiple breakouts should reset counts when values of breakouts other than change (#41340)

* Fix add-alias-info matching for nominal field literal refs

* Minor cleanup

* Minimize diff

* A bit more cleanup

* Cumulative aggregations w/ multiple breakouts should reset counts (#2862)

* Update dox

* Do the nesting FIRST before adding alias info.

* Remove unused

* Fix add-alias-info matching for nominal field literal refs

* Minor cleanup

* Minimize diff

* A bit more cleanup

* Convert `auto-bucket-datetimes` middleware to MLv2

* New Kondo rule time.

* Test & lint fixes 



* Fix typos

* Apply suggestions from code review

Co-authored-by: metamben <103100869+metamben@users.noreply.github.com>

---------

Co-authored-by: metamben <103100869+metamben@users.noreply.github.com>

* collection and db breadcrumbs

* compact breadcrumbs

* adding navigation keybindings

* recent_views and search changes to always include collection or db name

* removing breadcrumbs, display parent name

* removing unneeded components

* e2e and FE unit tests

* waitFor

* e2e tests for keybindings

* help

* be tests passing

* PR feedback and type adjustments

* unit test adjustment

* Test created_at and schema fields
* Coerce nil schema to empty string

* Fix embed download endpoint

We have added format_rows as a query parameter. In most endpoints this just works fine.

Here, however, the problem is that we use query parameters to pass parameter values. We run some validation over these
provided parameter values (eg. don't allow a user to over ride a 'locked' parameter). Since we have these 2 different
use cases for query parameters, the validation was failing on :format_rows, which isn't expected to exist at all.

Here, I dissoc the :format_rows key prior to this validation step, so normal validation can continue.

* Add test that uses this card embedding endpoint

* Nested queries have limits which can defeat their purpose

Consider a query

```sql
select count(*) from {{#199}}
```

This query should return the number of distinct rows in the query
defined by 199. But it's actually limited by the excel limit of
1048575. And that's because when the value of `{{#199}}` is expanded it
has that limit applied as normal.

```clojure
qp=> (let [card-id 199] ;; use a valid card id for you
       (-> {:database 1,
            :type :native,
            :native {:query "select count(*) from {{ref}}"
                     :template-tags {:ref {:card-id card-id
                                           :type :card
                                           :name "ref"
                                           :display-name "ref"}}}
            :middleware {:disable-max-results? true}}
           qp.compile/compile
           :query
           (metabase.db.query/format-sql )
           println))
select
  count(*)
from
  (
    SELECT
      "PUBLIC"."ORDERS"."ID" AS "ID",
      "PUBLIC"."ORDERS"."TOTAL" AS "TOTAL"
    FROM
      "PUBLIC"."ORDERS"
    LIMIT
      1048575
  )
```

But we can suppress this limit when substituting a query inside yielding

```sql
select
  count(*)
from
  (
    SELECT
      "PUBLIC"."ORDERS"."ID" AS "ID",
      "PUBLIC"."ORDERS"."TOTAL" AS "TOTAL"
    FROM
      "PUBLIC"."ORDERS"
  )
```

And this is proper because we want to limit the _outer_ query, not
internal queries.

* Remove limit from test expectation

* stupid trailing space

* another subquery test

* Use helper function to disable limit middleware

* Fix #39138 again

Fixes #40934

Got unfixed somehow in #40578. seems like some file renames caught it off guard

But the long and short is that its possible for a persisted model to end
up in the state "refreshing" (possible an instance restart during
refreshing). The refreshing job doesn't look for these so they become
effectively invisible.

Since the job to refresh them will only run one at a time cluster wide,
any jobs that are in the "refreshing" state when the refresher begins to
refresh are necessarilly stuck (no one else could currently be
refreshing them). So we can just add them to the queue of models to
refresh.

```clojure
(jobs/defjob ^{org.quartz.DisallowConcurrentExecution true ;; <----
               :doc "Refresh persisted tables job"}
  PersistenceRefresh [job-context]
  (refresh-job-fn! job-context))
```

* Ensure ee/oss pathways are taken in tests

Annoying little footgun here. CI does not run with an ee token, so in
order to ensure that test pathway goes through ee version of a
defenterprise we _must_ use the `mt/with-premium-features
{:cache-granular-controls}`,  but we also want to ensure that it goes
through the oss version.

So two options, a `doseq` on both features (empty set and the feature
that triggers this). But want a test in the enterprise folder as well to
ensure.

The real trickiness comes from running tests at a repl and CLI. My REPL
always has an ee token in it. My command line always lacks that as
well. So want to be explicit about the token features in effect at test
time.

That's why I'm essentially duplicating tests (ee in ee folder, oss in
regular pathway)

* Support parsing {{variable}}s and such in queries

[Fixes #39954]

[Fixes #39953]

* H2 compilation improvements: avoid unneeded casting

* Legacy MBQL schema improvements

* Add additional test for wrap-value-literals

* Simpler version of the fixes from #36068

* Test and lint fixes 

* Pivot QP use MLv2

* Wow

* Test fix 

* Fixes

* Actions should use strings for column names (fix :update-row and :create-row normalization)

* MLv2 schema should check against keys for the other query type

* Ok, have I fixed things?

* More fixes 

* Fix indentation

* Another round of test fixes. 

* Hopefully the last few test fixes 

* We need to test normalization for queries that have keyword keys as well.

* Fix Cljs i18n namespaces

* Sort namespaces

* Only test against H2

* Test fixes 

* Register MBQL clause schemas and test fixes 

* Test fixes and PR feedback

* Test fix

* Remove the normalization tests

* Test fixes 

* Fix kondo

* PR feedback

* Resurrect remove-all-breakouts

* Test fixes 

These type checks are useful when constructing clauses with full MLv2
metadata available, but when converting from legacy the `:field` refs
have only `:base-type`.

If a column has its metadata overridden to eg. coerce a string or
integer into a date, its `:base-type` will remain `:type/Integer` or
`:type/Text` and the `:effective-type` will be `:type/Date` etc.

This change skips the type check while converting from legacy, and
while checking `can-run`. Eventually this should be replaced with
tracking `:effective-type` on refs or using real metdata, instead of
relying on the `:base-type` only.

Fixes #41122.

We need to serialize it for any card, since when a normal question depends on a native question - their result_metadata won't be updated when opening a dashboard, which leads to front-end not understanding how to connect the parameters from a dashboard to a card, and errors during queries.

* Group axes with the correct map shape.

Prior to this, any time axes were all explicitly set :left or :right by the user, the `group-axes-at-once` function
would incorrectly return the map in the following form:

`{:left ["a" "b" "c"]}` which is incorrect. It must instead look like this:

`{"a" :left "b" :left "c" :left}`

This fixes the map shape and permits the user to explicitly set all axes to the left or to the right.

* Add a test to confirm the axis settings are respected and do render

* Wow

* Test fix 

* Fixes

* Actions should use strings for column names (fix :update-row and :create-row normalization)

* MLv2 schema should check against keys for the other query type

* Ok, have I fixed things?

* More fixes 

* Fix indentation

* Another round of test fixes. 

* Hopefully the last few test fixes 

* We need to test normalization for queries that have keyword keys as well.

* Fix Cljs i18n namespaces

* Sort namespaces

* Only test against H2

* Rename `metabase.mbql` to `metabase.legacy-mbql`

* Fix Kondo warnings

* Test fixes 

* Register MBQL clause schemas and test fixes 

* Test fixes and PR feedback

* Test fix

* Remove the normalization tests

* Test fixes 

* Fix kondo

* Fix import

* Another fix 

* Merge

* FIXES

* Add another missing REQUIRE

* More Metric => LegacyMetric renaming

* metabase.models.metric-important-field => metabase.models.legacy-metric-important-field

* Even more renaming

* More renaming

* Even more fixes.

* More renaming

* Wow, even more renaming!!

* Use window functions for SQL drivers for cumulative count and sum

* H2/Snowflake/Redshift implementation

* Test fixes 

* Use SELECT expression positions as the default window function implementation

* Some test fixes.

* Fix bugs in add-alias-info

* MongoDB fixes

* Another pass

* BigQuery should use positions?

* BigQuery almost works 100% now.

* MLv2 fix conversion behavior for expressions with type info

* Add transformations for nesting cumulative aggregations in expressions

* Minor tweaks

* MLv2 should escape aliases using driver/escape-alias (dynamically bind this)

* Kondo fix

* Lint fixes 

* More test fixes and revert test changes

* Update some tests to use correct case for nominal Field literal refs

* Wow! BigQuery is fixed! Remove upgrade-field-literals middleware

* Disable tests for the failing drivers, we can fix in follow-on PRs.

* Enable BigQuery :window-functions

* Extra info

* Add GH issue numbers

* Update more GH issue numbers

* Fix SQL Server!!!! wooo

* I THOUGHT WE WERE DISABLING SPARK SQL!!!!!!!!!!1

* Cleanup to get things ready for review.

* Snowflake + Vertica shouldn't use output column numbers for ORDER BY in OVER

* Code cleanup

* Snowflake + Vertica shouldn't use output column numbers for ORDER BY in OVER

* Code cleanup

* Test fixes 

* Much simpler BigQuery implementation

* Add GH issue context

* Remove commented-out code

* Remove unused add-stage-at-index

* Documentation

* Test fix? 

* Test fix 

* Legacy MBQL documentation

* Do not commit dox

* Subject the legacy MBQL stuff to the same stricter linting rules as MLv2

* PR feedback

This optional fourth argument will disregard all joins at `position` and
later when suggesting conditions. This is useful when editing a join's
RHS, which wipes out the condition. We don't want to generate a
suggested condition that's based on keys from the RHS of the join we
just edited, or from later joins.

Fixes #40916. Part of #40890.

Fixes #41023.

New endpoint accepts entity and entity ids in form of `database=1&dashboard=2&question=3`. If you don't supply
`&include=overrides`, then it tries to find configs directly referencing supplied entities and updates their
`invalidated_at`. If you supply `&include=overrides`, all the referenced cards are
updated (`report_card.cache_invalidated_at`).

Cache strategies then select the maximum `invalidated_at` in their logic. EE-only.

resolves #40548

[Fixes #13922]

* MLv2 normalization: add :lib/uuid automatically to options maps if it's missing

* Remove unneeded normalization code in QP

* Fix 

* Ignore invalid metadata providers when passed in from the REST API

* Use window functions for SQL drivers for cumulative count and sum

* H2/Snowflake/Redshift implementation

* Test fixes 

* Use SELECT expression positions as the default window function implementation

* Some test fixes.

* Fix bugs in add-alias-info

* MongoDB fixes

* Another pass

* BigQuery should use positions?

* Disable tests for the failing drivers, we can fix in follow-on PRs.

* Extra info

* Add GH issue numbers

* Update more GH issue numbers

* Fix SQL Server!!!! wooo

* I THOUGHT WE WERE DISABLING SPARK SQL!!!!!!!!!!1

* Cleanup to get things ready for review.

* Snowflake + Vertica shouldn't use output column numbers for ORDER BY in OVER

* Code cleanup

* Test fix?