* Add stage-number parameter to NativeParameterDimensionTarget

* Update unit test

* Recognize native query dimension parameters with stage-number

* Update test setup

---------

Co-authored-by: Tamás Benkő <tamas@metabase.com>

* Disable `?locale` on static embeds/public links

* Fix E2E tests

* Add a new line for consistency

* Don't dissoc `:id` in `before-update`

There seems to be a nasty footgun in toucan2 here.

Say you're executing an `update!` command on a set of IDs, e.g.:

```
(t2/update! :model/Card :id [:in 1 2] {:view_count 1})
```

This works if:

- both cards 1 and 2 have `view_count=1` already

- both cards 1 and 2 have `view_count!=1`

However.

If one of the two cards has `view_count=1` and another has a different
view_count, then (if the `before-update` method doesn't have the primary
key attached) Toucan emits a call to update *every card in the
database*, without a where clause at all.

* FE - Allow to use in dashboard filters both pre- and post- last aggregation columns (#46670)

* Extract getFilterStageIndexes

* Update comment

* Include 2 last stages in filterable columns in getParameterColumns

* Move getJoinQueryHelpers to test-helpers

* Add basic query creation

* Add breakout columns

* Make tableName optional

* Implement 1-stage test, refactor assertions

* Rename

* Move filter utils up

* Use getGroupItems and appendStageIfAggregated in getFilterableColums

* Introduce Lib.filterGroups

* Revert "Introduce Lib.filterGroups"

This reverts commit 5f8b2d09.

* Don't reuse getGroupItems or appendStageIfAggregated

* Move getFilterStageIndexes out of Lib

* Format code

* Revert types move

* Revert functions move

* Improve diff

* Group dashcard mapping options

* Adjust existing getParameterColumns usages (except unit tests)

* Remove redundant call

* Fix filtering

* Update unit tests

* Format code, remove TODO

* Add stage index assertions

* Add model test for 1-stage query

* Add partial boilerplate for 2-stage query

* Add a 2-stage case

* Add a 2-stage case for model

* Update unit test

* Fix click behavior mapping

* Remove duplicated functions

* Remove complex return type

* Move appendStageIfAggregated and getFilterStageIndexes to metabase-lib/filter.ts

* Re-group columns based on stage index instead of using ML-returned references for comparison

* Add new options to dimension types

* Use the new stage-number option

* Rename appendStageIfSummarized to ensureFilterStage and add explanatory comment

* Remove duplicated ensureFilterStage after merge

* FE - e2e - Cover dashboard filters with tests (#46958)

* Extract getFilterStageIndexes

* Update comment

* Include 2 last stages in filterable columns in getParameterColumns

* Move getJoinQueryHelpers to test-helpers

* Add basic query creation

* Add breakout columns

* Make tableName optional

* Implement 1-stage test, refactor assertions

* Rename

* Move filter utils up

* Use getGroupItems and appendStageIfAggregated in getFilterableColums

* Introduce Lib.filterGroups

* Revert "Introduce Lib.filterGroups"

This reverts commit 5f8b2d09.

* Don't reuse getGroupItems or appendStageIfAggregated

* Move getFilterStageIndexes out of Lib

* Format code

* Revert types move

* Revert functions move

* Improve diff

* Group dashcard mapping options

* Adjust existing getParameterColumns usages (except unit tests)

* Remove redundant call

* Fix filtering

* Update unit tests

* Format code, remove TODO

* Add stage index assertions

* Add model test for 1-stage query

* Add partial boilerplate for 2-stage query

* Add a 2-stage case

* Add a 2-stage case for model

* Update unit test

* Fix click behavior mapping

* Remove duplicated functions

* Remove complex return type

* Remove unused @ts-expect-error directive
- use createQuestionAndDashboard helper instead of command
- sort imports

* Add boilerplate for query stages test suite

* Add createDashboard

* Try out the new function

* Organize code

* Add descriptions

* Move visitDashboard call out of createDashboard

* Move appendStageIfAggregated and getFilterStageIndexes to metabase-lib/filter.ts

* Re-group columns based on stage index instead of using ML-returned references for comparison

* Add boilerplate for base questions

* Add assertions for mapping options

* Simplify verifyDashcardMappingOptions interface

* Add assertions for all types of parameters

* Update test name

* Format code

* Add new options to dimension types

* Use the new stage-number option

* Add Q2 creation

* Introduce helper constants

* Rename appendStageIfSummarized to ensureFilterStage and add explanatory comment

* Add cases for models

* Extract subroutines

* Add Q3

* Extract createQ2Query and createQ3Query

* Extract createQ1uery, remove createM1

* Optimize verifyPopoverMappingOptions

* Add more assertions

* Add model assertions

* Add tooltip assertion

* Break large functions down

* Restructure tests to prevent memory-related crashes

* Refactor

* Update comment

* Update comment

* Remove createQ1

* Remove createQ2, createQ3

* Refactor

* Refactor

* Comment out failing assertions

* Add Q3 assertions

* Add Q3 assertions

* Add comment

* Introduce createAndVisitDashboardWithQueryMatrix

* Add Q4

* Fix assertions

* Add repro tags

* Add comment

* De-hardcode indexes

* Add Q5-Q8

* Rename

* Add Q5

* Add Q6

* Add Q7 & Q8

* Add Q9

* Fix structure

* Reuse Lib.ensureFilterStage

* Fix unit of time parameters

* Support explicit and implicit stage numbers for temporal-unit params

* Extend test with temporal unit parameter with stage number

* Update unit tests

* Let stage-path handle nil (default) stage-number

---------

Co-authored-by: Tamás Benkő <tamas@metabase.com>

* Allow filtering on columns from all stages in dashboards (#48254)

* Allow filtering on columns from all stages in dashboards

* Update tests

* Clean up

* Update unit tests

* Distinguish multiple "Summaries" column groups in dashboard filter mapping (#48261)

* Allow filtering on columns from all stages in dashboards

* Update tests

* Clean up

* Update unit tests

* Distinguish multiple "Summaries" column groups

* Fix imports

* Remove redundant attribute

* Update import

* Allow filter params targeting native stages (#48379)

* Move filters on nested native stages to parent stage in expand-mbql-params

Similar to the existing handling in move-join-condition-to-source-query.

This fixes filter params that directly target a native stage with a non-negative :stage-number, e.g. when applying
dashboard filters to a card where the card query is native.

Fixes #48258

* Add test for filter params explicitly targeting nested native queries

* FE - e2e - Cover dashboard drills with tests (#48289)

* Allow filtering on columns from all stages in dashboards

* Update tests

* Clean up

* Update unit tests

* Distinguish multiple "Summaries" column groups

* Fix imports

* Remove redundant attribute

* Fix type

* Add a test

* Add a test

* Reduce flakiness

* Fix race conditions with last_used_param_values
- see https://metaboat.slack.com/archives/C010L1Z4F9S/p1727947712609579

* Add a temporary workaround

* Fix name

* Fix applying parameters in Question

* Revert "Fix type"

This reverts commit 67f6778d.

* Rename and reorder tests

* Add more tests

* Update tests to avoid hitting https://github.com/metabase/metabase/issues/46774

* Add Q8 tests and fix tests for Q9

* Update assertions

* Update assertions because of https://github.com/metabase/metabase/issues/48339

* Improve test names

* Add test

* Skip failing test

* Add more tests

* Add test

* Remove redundant code

* Remove invalid cases

* Add test

* Add tests

* Add more assertions

* Add more assertions

* Add assertions

* Add assertions

* Use helper functions

* Use helper functions

* Nest helpers consistently throughout the file

* Decrease flakiness

* Revert "Add a temporary workaround"

This reverts commit 48ed8770.

* Add a temporary workaround

* Add tests

* Add assertions

* Add tests

* Update unit tests

* Make temporary workaround safer

* Add TODOs to github links

* Extract a function to improve readability

* Simplify getParametersMappedToDashcard

* Fix failing tests by calling Lib.ensureFilterStage in the correct place and properly identifying the originalQuestion so that Question.prototype.isDirtyComparedTo returns expected result

* Remove redundant code

* Quick dirty fix to run in CI

* Format code

* Format code

* Update unit tests

* Fix test setup

* Revert "Remove redundant code"

This reverts commit 35b47a10.

* Use Lib.areLegacyQueriesEqual

* Create a new e2e test group: dashboard-filters-2 (#48610)

* Allow filtering on columns from all stages in dashboards

* Update tests

* Clean up

* Update unit tests

* Distinguish multiple "Summaries" column groups

* Fix imports

* Remove redundant attribute

* Fix type

* Add a test

* Add a test

* Reduce flakiness

* Fix race conditions with last_used_param_values
- see https://metaboat.slack.com/archives/C010L1Z4F9S/p1727947712609579

* Add a temporary workaround

* Fix name

* Fix applying parameters in Question

* Revert "Fix type"

This reverts commit 67f6778d.

* Rename and reorder tests

* Add more tests

* Update tests to avoid hitting https://github.com/metabase/metabase/issues/46774

* Add Q8 tests and fix tests for Q9

* Update assertions

* Update assertions because of https://github.com/metabase/metabase/issues/48339



* Improve test names

* Add test

* Skip failing test

* Add more tests

* Add test

* Remove redundant code

* Remove invalid cases

* Add test

* Add tests

* Add more assertions

* Add more assertions

* Add assertions

* Add assertions

* Use helper functions

* Use helper functions

* Nest helpers consistently throughout the file

* Decrease flakiness

* Revert "Add a temporary workaround"

This reverts commit 48ed8770.

* Add a temporary workaround

* Add tests

* Add assertions

* Add tests

* Update unit tests

* Make temporary workaround safer

* Add TODOs to github links

* Extract a function to improve readability

* Simplify getParametersMappedToDashcard

* Fix failing tests by calling Lib.ensureFilterStage in the correct place and properly identifying the originalQuestion so that Question.prototype.isDirtyComparedTo returns expected result

* Remove redundant code

* Quick dirty fix to run in CI

* Format code

* Format code

* Update unit tests

* Fix test setup

* Create a new e2e test group because the old one was timing out in CI

* Add the group to GH action

* Revert "Remove redundant code"

This reverts commit 35b47a10.

* Use Lib.areLegacyQueriesEqual

* FE - e2e - Cover combination of unit of time parameter + non-last-stage filters with tests (#48726)

* Add JSDoc for entityPickerModalItem

* Add addToDashboard parameter to saveQuestion

* Add JSDoc for saveQuestion

* Fix import

* Add a test reproducing the bug

* Try to fix unit of time parameters

* Fix wrong stage-index for temporal unit parameters on composed ad hoc questions

* Fix ad-hoc question naming when there is an empty extra filter stage

* Fix unit tests

* FE - Do not add extra filtering stage for pivot tables (#48975)

* Avoid using Lib.ensureFilterStage for pivoted questions

* Fix unit test setup

* Add tests for pivot tables

* Optimize tests

* Test filter modal

* Revert redundant changes

* [WIP] Dashboard filter columns be fixes (#48906)

* Revert "Add a temporary workaround"

This reverts commit 4ea37487.

* Fix some of the issues and unskip some of the E2E tests

* Acknowledge that metrics are transparent

* Fix breakouts by the same column with different temporal-units

* Use non-negative stageIndex for temporal unit parameters

* Add stage if there is an explicit reference to stage after the last

Instead of duplicating the ensure-filter-stage logic, just check if there is a parameter referencing the filter
stage (which is always one after the last in the query). If there is such a reference, add that stage.

---------

Co-authored-by: Kamil Mielnik <kamil@kamilmielnik.com>

* Require only breakouts when adding new stage in ensure-filter-stage (#49067)

* Only require breakouts when adding new stage in ensure-filter-stage

Previously, this function required both breakouts and aggregations.

Closes #48339

* Update e2e tests after changes to Lib.ensureFilterStage

This resolves all-but-one of the TODOs related to #49339

Related to #49022

* Implementing parameter_mappings migration (#49256)

* FE - e2e - Update tests after `Lib.ensureFilterStage` changes (#49222)

* Unskip #19744 repro

* Remove sanity checks

* Update comment

* Fix typo

* Remove invalid test

* Ignore pivoted "table" viz and and inline Question.prototype.isPivoted

* Remove test for pivoted table

* FE - e2e - Click behavior (#49239)

* Unskip #19744 repro

* Remove sanity checks

* Update comment

* Fix typo

* Remove invalid test

* Ignore pivoted "table" viz and and inline Question.prototype.isPivoted

* Remove test for pivoted table

* Add a test

* Disable columns from non-last stage in click behavior

* Finish the test

* Refactor

* FE - e2e - Cover public & embedded dashboards with tests  (#49280)

* Add test for public dashboard

* Add test for embedded dashboard

* Add comment

* Add test for 2nd stage column

* Add test for 2nd stage aggregation column

* Add test for 2nd stage breakout column

* Add assertions for models

* Handle no auto-pivoting of ad-hoc questions from drill thrus

---------

Co-authored-by: Alexander Polyankin <alexander.polyankin@metabase.com>

* Add missing test setup

* Address some review comments (#49429)

* FE - Show columns from all stages in click behavior (target question)  (#48828)

* Add JSDoc for entityPickerModalItem

* Add addToDashboard parameter to saveQuestion

* Add JSDoc for saveQuestion

* Fix import

* Add a test reproducing the bug

* Try to fix unit of time parameters

* Fix wrong stage-index for temporal unit parameters on composed ad hoc questions

* Fix ad-hoc question naming when there is an empty extra filter stage

* Fix unit tests

* Fix non-unique keys

* Add stage-number to click behavior dimension target

* Call Lib.ensureFilteringStage in click behavior

* Use describeEE because click behavior tests use setTokenFeatures("all")

* Stop testing implementation details

* Stop testing implementation details

* Stop testing implementation details

* Modernize test and make it less flaky

* Address flake source

* Fix SAMPLE_DB_ID

* Unskip repro for #19744

* Add a test

* Update test name

* Introduce verifyNotebookQuery

* Use verifyNotebookQuery

* Use verifyNotebookQuery

* Move verifyNotebookQuery to helpers

* Assert items count in verifyNotebookQuery

* Extract helper functions

* Rename helpers

* Refactor

* Consistent comments

* Extract verifyVizTypeIsLine

* Add scrollIntoView calls for extra safety

* Add TODO

* Use verifyNotebookQuery

* Make stages optional

* Add createMultiStageQuery

* Remove redundant scrollIntoView calls
- We don't need them because the assertions are not using "be.visible"

* Improve test setup

* Add more assertions

* Add more assertions

* Add basic join support to verifyNotebookJoins

* Unnest code

* Implement join assertions

* Implement expressions and sort assertions

* Simplify code

* Add commented-out assertions

* Yes, it is correct
- it's stageIndex which sourceColumn comes from in sourceQuery

* Conditionally apply filter stage

* Allow mapping to all stages in click behavior

* Remove redundant argument

* Simplify diff

* Simplify diff

* Update test setup

* Remove TODO related to metabase#49282
- Closes #49282

* Support fetching param values on filter stage (#50120)

* Support fetching param values on filter stage

Fixes #48613

* Fix temporal-unit-parameters

* Enrich dimension targets in click_behavior with stage-numbers (#49911)

* Enrich dimension targets in click_behavior with stage-numbers

Fixes #49110

* Add missing stage-number

* Remove changeSynchronousBatchUpdateSetting references

* Update assertions to include time bucket

* Use mbql.u/legacy-last-stage-number in metabase.query-processor.card

---------

Co-authored-by: Tamás Benkő <tamas@metabase.com>
Co-authored-by: appleby <86076+appleby@users.noreply.github.com>
Co-authored-by: metamben <103100869+metamben@users.noreply.github.com>
Co-authored-by: Alexander Polyankin <alexander.polyankin@metabase.com>

* prevent the UI from breaking if there is an invalid enum value set by MB_LOADING_MESSAGE

* add test case

* add enum validation on backend

* oops - fix set-value-of-type! call

---------

Co-authored-by: Noah Moss <noahbmoss@gmail.com>

* Normalize :temporal_units in action, card, and dashboard entries

Fixes #50371

* sankey prototype

* formatting, tooltips

* layout, formatting

* handle invalid settings

* use metabase color palette

* add missing data for click behavior, hovers, add clicks support

* smart sankey default columns selection

* check more columns

* fix tooltip shows incorrect values for nodes

* polish, specs

* extend the viz settings type with sankey settings

* add isSensible to SankeyChart

* specs on sankey default columns computation

* import correct column predicates

* revert outdated changes in graph.js

* sankey colors

* format values in sankey tooltip

* change default sankey edge color to source

* increase the default size of sankey dashboard cards

* improve error messages

* use echarts option to hide overlapping labels

* increase padding

* click behavior, tooltip values

* fixes

* fix rebase: getFriendlyName got removed

* layout, labels

* polish

* remove non-existing viz type

* update spec

* polish

* add capability for sending bug reports

* fix typing, linting and testing errors

* fix failures in unit and e2e tests

* fix ts error

* add bug report RTK

* move MB_BUG_REPORT_ENABLED handling to backend

* move bug reporter setting to global settings

* fix type errors

* fix bug-report-enabled setting visibility

* add options map to slack postMessage funcion

* fix failing tests

* fix post-chat-message handling

* fix type error

* rename bug reporting setting

Fixes #48721

* hide pivoted exports under env var

* only export pivoted when the env var is true

Also added tests to confirm.

* must specify :export for public settings

* spec

* change setting to :internal since it's only via env var

* use correct visibility add setting to settings.yaml for serialization

---------

Co-authored-by: Adam James <adam.vermeer2@gmail.com>

* Fix serdes of missing cards in click behavior

When exporting a dashboard, we export its dashcards, including cards
referenced inside its viz_settings.

The problem is that viz_settings might have references to
now-nonexistent cards.

This is an initial stab at cleaning up the problem, specifically for
click behavior. If Dashboard D has Dashcard DC, which has a
click-behavior that links to Card C, then when we collect the
descendants of D, we'll include a reference to C - even if C has been
deleted.

So this makes two changes: first, when calculating the descendants of a
dashcard, don't include links to cards that don't exist. And, when
exporting the click behavior, remove the whole thing entirely if the
card it points to doesn't exist.

Make model ranking dynamic, so that:

1. We can experiment with it on-the-fly in Stats.
2. We can make it context dependent in future.

Co-authored-by: Nick Fitzpatrick <nickfitz.582@gmail.com>

* Rank based on when the current user last viewed the item

* Fix tests

* Don't try to fetch dbs for zero cards in api.table/batch-fetch-card-query-metadatas

Closes #48461

Ensure we don't throw an exception by attempting to fetch the Databases associated with potentially-deleted card ids in
api.table/batch-fetch-card-query-metadatas.

In particular, this ensures we don't generate an exception if the user does the following

1. Create a card "Source Card"
2. Create another card "Derived Card" with a :source-table of "Source Card"
3. Add Derived Card to a dashboard
4. Permanently delete (not just archive) the Source Card

* Add test for virtual-table query-metadata with deleted source card

* Add test for dataset query_metadata with deleted source card

* Add test for dashboard query-metadata with deleted source card

* Add test for card query-metadata with deleted source card

* Add metabase.api.test-util and use it in query-metadata-with-deleted-card tests

* Replace copy/pasted debug helpers with select-query-metadata-keys-for-debugging

* [xlsx-export] Cache the custom datetime formatter

* [xlsx-export] Optimize functions in m.q_p.pivot.postprocess

* cljfmt

---------

Co-authored-by: Adam James <adam.vermeer2@gmail.com>

* Add temporal unit to display name

* Remove temporal unit adjustments from FE

* formatColumn to displayNameForColumn

* Factor out getFriendlyName

* Adjust BE tests

* Add require

* Adjust FE unit tests

* Adjust temporal-unit-parameters.cy.spec.js

* Adjust dashboard-reproductions.cy.spec

* Convert formatColumn occurrence

* Adjust legacy display name logic generation

After `getFriendlyName` was removed from FE, capitalization
of display name should happen here. Logic is similar, should
be adjusted prior merging the PR.

* Adjust embedding-linked-filters.cy.spec.js

* Adjust joins-reproductions.cy.spec.js

* Adjust metrics-editing.cy.spec.js

* Adjust timelines-question.cy.spec.js

* Adjust offset.cy.spec.ts

* TMP Avoid duplication of temporal unit in display name in MLv2 code

* Probably FE fix: add default temporal bucket only if not present

* WIP Add ends-with-temporal-unit? function

Current strategy is to add temporal units to display names if same is not already present

* Adjust display-name-method :metadata/column

Append TU to display name only if different to what's already present

* Adjust FE: show temporal unit that's present on field instead of default on hover on QueryColumnPicker element

* Return also temporal-unit from visible-columns

* Enable tempora-unit to be returned from previous-stage-or-source-visible-columns

* WIP Adjust multi breakout multi stage e2e multiple-column-breakouts.cy.spec

* Adjust multiple-column-breakouts.cy.spec.ts

* It was not a bug.

* Revert changes in _BaseBucketPickerPopover

* Restore temporal-unit for next stages removal in stage.cljc

* Add inherent-temporal-unit

* Adjust display name temporal bucket logic

* Rename to inherited-temporal-unit

* Shutdown too broad display name capitalization

* Eastwood

* Adjust pivot-rows-order-test

* Adjust pivot-non-numeric-values-in-aggregations

* Adjust inherent to inherited

* Rename to inherited

* Add inherent-temporal-unit only to returned-columns

* Update e2e/test/scenarios/dashboard-filters/temporal-unit-parameters.cy.spec.js

* Update e2e/test/scenarios/question/multiple-column-breakouts.cy.spec.ts

* Update consistent-date-formatting-test

* Update dashboard/x-rays.cy.spec.js

* Update waterfall.cy.spec.js

* Return month default for default inherited-temporal-unit

* Update reproductions-3.cy.spec.js

* Update downloads/downloads.cy.spec.js

* Update legend.cy.spec.js

* Update pie_chart.cy.spec.js

* Update click-behavior.cy.spec.js

* Update embedding-linked-filters.cy.spec.js

* Return QueryColumnPicker's condition to original state

* Remove reundant export

* Update e2e/test/scenarios/question/multiple-column-breakouts.cy.spec.ts

* Comment

* Update nested.cy.spec.js

* Update e2e/test/scenarios/sharing/sharing-reproductions.cy.spec.js

* Adjust to TS types

* Adjust for cljfmt

* Capitalize

* Add propagation tests

* Add inherited-temporal-unit propagation for expressions

* Add default inherited-temporal-unit test cases

* Add qp temporal-bucket-test ns

* Simplify default unit logic

* Comment

* Use NULL_DISPLAY_VALUE

* Comment

* Comment

* Comment

* Comment

* Comment

* Comment

* Remove redundant code (for now)

* Update comments

* Comment

* Restore way too daring simplification

* cljfmt

* Adjust capitalize

* Update comment

* Update test

* Address remarks in tests

* Update test

* Address remarks

* typo