* Simplify epic template to require easier editing

* Remove superflouous 'if any'

* Rename engineering doc to technical design doc

* Add link for testing plan

* Remove feature branch sentence

* feat(sdk): migrate existing sdk tests to cypress component testing

* Self review

* Trigger CI tests

* Add dummy job generator

* [DO NOT MERGE] Remove driver test config from the list of backend changes

* Add missing runs-on

* Avoid too much nesting

* Revert changes to file paths and README

* FE - Allow to use in dashboard filters both pre- and post- last aggregation columns (#46670)

* Extract getFilterStageIndexes

* Update comment

* Include 2 last stages in filterable columns in getParameterColumns

* Move getJoinQueryHelpers to test-helpers

* Add basic query creation

* Add breakout columns

* Make tableName optional

* Implement 1-stage test, refactor assertions

* Rename

* Move filter utils up

* Use getGroupItems and appendStageIfAggregated in getFilterableColums

* Introduce Lib.filterGroups

* Revert "Introduce Lib.filterGroups"

This reverts commit 5f8b2d09.

* Don't reuse getGroupItems or appendStageIfAggregated

* Move getFilterStageIndexes out of Lib

* Format code

* Revert types move

* Revert functions move

* Improve diff

* Group dashcard mapping options

* Adjust existing getParameterColumns usages (except unit tests)

* Remove redundant call

* Fix filtering

* Update unit tests

* Format code, remove TODO

* Add stage index assertions

* Add model test for 1-stage query

* Add partial boilerplate for 2-stage query

* Add a 2-stage case

* Add a 2-stage case for model

* Update unit test

* Fix click behavior mapping

* Remove duplicated functions

* Remove complex return type

* Move appendStageIfAggregated and getFilterStageIndexes to metabase-lib/filter.ts

* Re-group columns based on stage index instead of using ML-returned references for comparison

* Add new options to dimension types

* Use the new stage-number option

* Rename appendStageIfSummarized to ensureFilterStage and add explanatory comment

* Remove duplicated ensureFilterStage after merge

* FE - e2e - Cover dashboard filters with tests (#46958)

* Extract getFilterStageIndexes

* Update comment

* Include 2 last stages in filterable columns in getParameterColumns

* Move getJoinQueryHelpers to test-helpers

* Add basic query creation

* Add breakout columns

* Make tableName optional

* Implement 1-stage test, refactor assertions

* Rename

* Move filter utils up

* Use getGroupItems and appendStageIfAggregated in getFilterableColums

* Introduce Lib.filterGroups

* Revert "Introduce Lib.filterGroups"

This reverts commit 5f8b2d09.

* Don't reuse getGroupItems or appendStageIfAggregated

* Move getFilterStageIndexes out of Lib

* Format code

* Revert types move

* Revert functions move

* Improve diff

* Group dashcard mapping options

* Adjust existing getParameterColumns usages (except unit tests)

* Remove redundant call

* Fix filtering

* Update unit tests

* Format code, remove TODO

* Add stage index assertions

* Add model test for 1-stage query

* Add partial boilerplate for 2-stage query

* Add a 2-stage case

* Add a 2-stage case for model

* Update unit test

* Fix click behavior mapping

* Remove duplicated functions

* Remove complex return type

* Remove unused @ts-expect-error directive
- use createQuestionAndDashboard helper instead of command
- sort imports

* Add boilerplate for query stages test suite

* Add createDashboard

* Try out the new function

* Organize code

* Add descriptions

* Move visitDashboard call out of createDashboard

* Move appendStageIfAggregated and getFilterStageIndexes to metabase-lib/filter.ts

* Re-group columns based on stage index instead of using ML-returned references for comparison

* Add boilerplate for base questions

* Add assertions for mapping options

* Simplify verifyDashcardMappingOptions interface

* Add assertions for all types of parameters

* Update test name

* Format code

* Add new options to dimension types

* Use the new stage-number option

* Add Q2 creation

* Introduce helper constants

* Rename appendStageIfSummarized to ensureFilterStage and add explanatory comment

* Add cases for models

* Extract subroutines

* Add Q3

* Extract createQ2Query and createQ3Query

* Extract createQ1uery, remove createM1

* Optimize verifyPopoverMappingOptions

* Add more assertions

* Add model assertions

* Add tooltip assertion

* Break large functions down

* Restructure tests to prevent memory-related crashes

* Refactor

* Update comment

* Update comment

* Remove createQ1

* Remove createQ2, createQ3

* Refactor

* Refactor

* Comment out failing assertions

* Add Q3 assertions

* Add Q3 assertions

* Add comment

* Introduce createAndVisitDashboardWithQueryMatrix

* Add Q4

* Fix assertions

* Add repro tags

* Add comment

* De-hardcode indexes

* Add Q5-Q8

* Rename

* Add Q5

* Add Q6

* Add Q7 & Q8

* Add Q9

* Fix structure

* Reuse Lib.ensureFilterStage

* Fix unit of time parameters

* Support explicit and implicit stage numbers for temporal-unit params

* Extend test with temporal unit parameter with stage number

* Update unit tests

* Let stage-path handle nil (default) stage-number

---------

Co-authored-by: Tamás Benkő <tamas@metabase.com>

* Allow filtering on columns from all stages in dashboards (#48254)

* Allow filtering on columns from all stages in dashboards

* Update tests

* Clean up

* Update unit tests

* Distinguish multiple "Summaries" column groups in dashboard filter mapping (#48261)

* Allow filtering on columns from all stages in dashboards

* Update tests

* Clean up

* Update unit tests

* Distinguish multiple "Summaries" column groups

* Fix imports

* Remove redundant attribute

* Update import

* Allow filter params targeting native stages (#48379)

* Move filters on nested native stages to parent stage in expand-mbql-params

Similar to the existing handling in move-join-condition-to-source-query.

This fixes filter params that directly target a native stage with a non-negative :stage-number, e.g. when applying
dashboard filters to a card where the card query is native.

Fixes #48258

* Add test for filter params explicitly targeting nested native queries

* FE - e2e - Cover dashboard drills with tests (#48289)

* Allow filtering on columns from all stages in dashboards

* Update tests

* Clean up

* Update unit tests

* Distinguish multiple "Summaries" column groups

* Fix imports

* Remove redundant attribute

* Fix type

* Add a test

* Add a test

* Reduce flakiness

* Fix race conditions with last_used_param_values
- see https://metaboat.slack.com/archives/C010L1Z4F9S/p1727947712609579

* Add a temporary workaround

* Fix name

* Fix applying parameters in Question

* Revert "Fix type"

This reverts commit 67f6778d.

* Rename and reorder tests

* Add more tests

* Update tests to avoid hitting https://github.com/metabase/metabase/issues/46774

* Add Q8 tests and fix tests for Q9

* Update assertions

* Update assertions because of https://github.com/metabase/metabase/issues/48339

* Improve test names

* Add test

* Skip failing test

* Add more tests

* Add test

* Remove redundant code

* Remove invalid cases

* Add test

* Add tests

* Add more assertions

* Add more assertions

* Add assertions

* Add assertions

* Use helper functions

* Use helper functions

* Nest helpers consistently throughout the file

* Decrease flakiness

* Revert "Add a temporary workaround"

This reverts commit 48ed8770.

* Add a temporary workaround

* Add tests

* Add assertions

* Add tests

* Update unit tests

* Make temporary workaround safer

* Add TODOs to github links

* Extract a function to improve readability

* Simplify getParametersMappedToDashcard

* Fix failing tests by calling Lib.ensureFilterStage in the correct place and properly identifying the originalQuestion so that Question.prototype.isDirtyComparedTo returns expected result

* Remove redundant code

* Quick dirty fix to run in CI

* Format code

* Format code

* Update unit tests

* Fix test setup

* Revert "Remove redundant code"

This reverts commit 35b47a10.

* Use Lib.areLegacyQueriesEqual

* Create a new e2e test group: dashboard-filters-2 (#48610)

* Allow filtering on columns from all stages in dashboards

* Update tests

* Clean up

* Update unit tests

* Distinguish multiple "Summaries" column groups

* Fix imports

* Remove redundant attribute

* Fix type

* Add a test

* Add a test

* Reduce flakiness

* Fix race conditions with last_used_param_values
- see https://metaboat.slack.com/archives/C010L1Z4F9S/p1727947712609579

* Add a temporary workaround

* Fix name

* Fix applying parameters in Question

* Revert "Fix type"

This reverts commit 67f6778d.

* Rename and reorder tests

* Add more tests

* Update tests to avoid hitting https://github.com/metabase/metabase/issues/46774

* Add Q8 tests and fix tests for Q9

* Update assertions

* Update assertions because of https://github.com/metabase/metabase/issues/48339



* Improve test names

* Add test

* Skip failing test

* Add more tests

* Add test

* Remove redundant code

* Remove invalid cases

* Add test

* Add tests

* Add more assertions

* Add more assertions

* Add assertions

* Add assertions

* Use helper functions

* Use helper functions

* Nest helpers consistently throughout the file

* Decrease flakiness

* Revert "Add a temporary workaround"

This reverts commit 48ed8770.

* Add a temporary workaround

* Add tests

* Add assertions

* Add tests

* Update unit tests

* Make temporary workaround safer

* Add TODOs to github links

* Extract a function to improve readability

* Simplify getParametersMappedToDashcard

* Fix failing tests by calling Lib.ensureFilterStage in the correct place and properly identifying the originalQuestion so that Question.prototype.isDirtyComparedTo returns expected result

* Remove redundant code

* Quick dirty fix to run in CI

* Format code

* Format code

* Update unit tests

* Fix test setup

* Create a new e2e test group because the old one was timing out in CI

* Add the group to GH action

* Revert "Remove redundant code"

This reverts commit 35b47a10.

* Use Lib.areLegacyQueriesEqual

* FE - e2e - Cover combination of unit of time parameter + non-last-stage filters with tests (#48726)

* Add JSDoc for entityPickerModalItem

* Add addToDashboard parameter to saveQuestion

* Add JSDoc for saveQuestion

* Fix import

* Add a test reproducing the bug

* Try to fix unit of time parameters

* Fix wrong stage-index for temporal unit parameters on composed ad hoc questions

* Fix ad-hoc question naming when there is an empty extra filter stage

* Fix unit tests

* FE - Do not add extra filtering stage for pivot tables (#48975)

* Avoid using Lib.ensureFilterStage for pivoted questions

* Fix unit test setup

* Add tests for pivot tables

* Optimize tests

* Test filter modal

* Revert redundant changes

* [WIP] Dashboard filter columns be fixes (#48906)

* Revert "Add a temporary workaround"

This reverts commit 4ea37487.

* Fix some of the issues and unskip some of the E2E tests

* Acknowledge that metrics are transparent

* Fix breakouts by the same column with different temporal-units

* Use non-negative stageIndex for temporal unit parameters

* Add stage if there is an explicit reference to stage after the last

Instead of duplicating the ensure-filter-stage logic, just check if there is a parameter referencing the filter
stage (which is always one after the last in the query). If there is such a reference, add that stage.

---------

Co-authored-by: Kamil Mielnik <kamil@kamilmielnik.com>

* Require only breakouts when adding new stage in ensure-filter-stage (#49067)

* Only require breakouts when adding new stage in ensure-filter-stage

Previously, this function required both breakouts and aggregations.

Closes #48339

* Update e2e tests after changes to Lib.ensureFilterStage

This resolves all-but-one of the TODOs related to #49339

Related to #49022

* Implementing parameter_mappings migration (#49256)

* FE - e2e - Update tests after `Lib.ensureFilterStage` changes (#49222)

* Unskip #19744 repro

* Remove sanity checks

* Update comment

* Fix typo

* Remove invalid test

* Ignore pivoted "table" viz and and inline Question.prototype.isPivoted

* Remove test for pivoted table

* FE - e2e - Click behavior (#49239)

* Unskip #19744 repro

* Remove sanity checks

* Update comment

* Fix typo

* Remove invalid test

* Ignore pivoted "table" viz and and inline Question.prototype.isPivoted

* Remove test for pivoted table

* Add a test

* Disable columns from non-last stage in click behavior

* Finish the test

* Refactor

* FE - e2e - Cover public & embedded dashboards with tests  (#49280)

* Add test for public dashboard

* Add test for embedded dashboard

* Add comment

* Add test for 2nd stage column

* Add test for 2nd stage aggregation column

* Add test for 2nd stage breakout column

* Add assertions for models

* Handle no auto-pivoting of ad-hoc questions from drill thrus

---------

Co-authored-by: Alexander Polyankin <alexander.polyankin@metabase.com>

* Add missing test setup

* Address some review comments (#49429)

* FE - Show columns from all stages in click behavior (target question)  (#48828)

* Add JSDoc for entityPickerModalItem

* Add addToDashboard parameter to saveQuestion

* Add JSDoc for saveQuestion

* Fix import

* Add a test reproducing the bug

* Try to fix unit of time parameters

* Fix wrong stage-index for temporal unit parameters on composed ad hoc questions

* Fix ad-hoc question naming when there is an empty extra filter stage

* Fix unit tests

* Fix non-unique keys

* Add stage-number to click behavior dimension target

* Call Lib.ensureFilteringStage in click behavior

* Use describeEE because click behavior tests use setTokenFeatures("all")

* Stop testing implementation details

* Stop testing implementation details

* Stop testing implementation details

* Modernize test and make it less flaky

* Address flake source

* Fix SAMPLE_DB_ID

* Unskip repro for #19744

* Add a test

* Update test name

* Introduce verifyNotebookQuery

* Use verifyNotebookQuery

* Use verifyNotebookQuery

* Move verifyNotebookQuery to helpers

* Assert items count in verifyNotebookQuery

* Extract helper functions

* Rename helpers

* Refactor

* Consistent comments

* Extract verifyVizTypeIsLine

* Add scrollIntoView calls for extra safety

* Add TODO

* Use verifyNotebookQuery

* Make stages optional

* Add createMultiStageQuery

* Remove redundant scrollIntoView calls
- We don't need them because the assertions are not using "be.visible"

* Improve test setup

* Add more assertions

* Add more assertions

* Add basic join support to verifyNotebookJoins

* Unnest code

* Implement join assertions

* Implement expressions and sort assertions

* Simplify code

* Add commented-out assertions

* Yes, it is correct
- it's stageIndex which sourceColumn comes from in sourceQuery

* Conditionally apply filter stage

* Allow mapping to all stages in click behavior

* Remove redundant argument

* Simplify diff

* Simplify diff

* Update test setup

* Remove TODO related to metabase#49282
- Closes #49282

* Support fetching param values on filter stage (#50120)

* Support fetching param values on filter stage

Fixes #48613

* Fix temporal-unit-parameters

* Enrich dimension targets in click_behavior with stage-numbers (#49911)

* Enrich dimension targets in click_behavior with stage-numbers

Fixes #49110

* Add missing stage-number

* Remove changeSynchronousBatchUpdateSetting references

* Update assertions to include time bucket

* Use mbql.u/legacy-last-stage-number in metabase.query-processor.card

---------

Co-authored-by: Tamás Benkő <tamas@metabase.com>
Co-authored-by: appleby <86076+appleby@users.noreply.github.com>
Co-authored-by: metamben <103100869+metamben@users.noreply.github.com>
Co-authored-by: Alexander Polyankin <alexander.polyankin@metabase.com>

* prevent the UI from breaking if there is an invalid enum value set by MB_LOADING_MESSAGE

* add test case

* add enum validation on backend

* oops - fix set-value-of-type! call

---------

Co-authored-by: Noah Moss <noahbmoss@gmail.com>

* make the collection clean up alert show if content 3 months stale, rather than 6

* make test more robust

* Normalize :temporal_units in action, card, and dashboard entries

Fixes #50371

* sankey prototype

* formatting, tooltips

* layout, formatting

* handle invalid settings

* use metabase color palette

* add missing data for click behavior, hovers, add clicks support

* smart sankey default columns selection

* check more columns

* fix tooltip shows incorrect values for nodes

* polish, specs

* extend the viz settings type with sankey settings

* add isSensible to SankeyChart

* specs on sankey default columns computation

* import correct column predicates

* revert outdated changes in graph.js

* sankey colors

* format values in sankey tooltip

* change default sankey edge color to source

* increase the default size of sankey dashboard cards

* improve error messages

* use echarts option to hide overlapping labels

* increase padding

* click behavior, tooltip values

* fixes

* fix rebase: getFriendlyName got removed

* layout, labels

* polish

* remove non-existing viz type

* update spec

* polish

* Document `null` vs `undefined` setting type

* Improve the comment

* use -1 for stage index for breakout and aggregation list

* add e2e tests

* remove sum of total in e2e

* use test id matcher

* add issue to e2e name

Co-authored-by: Kamil Mielnik <kamil@kamilmielnik.com>

* move stageIndex to a separate variable

---------

Co-authored-by: Kamil Mielnik <kamil@kamilmielnik.com>

* Partition Postgres jobs (alternate approach)

* Make sure we're still testing SSL

* Improved Postgres job naming

* Unify banners

* Fix and update unit tests

* add capability for sending bug reports

* fix typing, linting and testing errors

* fix failures in unit and e2e tests

* fix ts error

* add bug report RTK

* move MB_BUG_REPORT_ENABLED handling to backend

* move bug reporter setting to global settings

* fix type errors

* fix bug-report-enabled setting visibility

* add options map to slack postMessage funcion

* fix failing tests

* fix post-chat-message handling

* fix type error

* rename bug reporting setting

* updating @reduxjs/toolkit

* kick ci

---------

Co-authored-by: Ryan Laurie <iethree@gmail.com>

Fixes #48721

* track items being moved to trash

* fix spelling

* fix unit tests

* hide pivoted exports under env var

* only export pivoted when the env var is true

Also added tests to confirm.

* must specify :export for public settings

* spec

* change setting to :internal since it's only via env var

* use correct visibility add setting to settings.yaml for serialization

---------

Co-authored-by: Adam James <adam.vermeer2@gmail.com>

* note about metrics permissions

* clarify viewing and editing

* clollection

On MySQL 8.0.17, the following is prohibited:

```
WITH RECURSIVE MyRecursiveCTE (...) AS
(
  SELECT ... FROM bar UNION ALL
  SELECT ... FROM bar JOIN MyRecursiveCTE ON ...
)
```

Instead, you need to do:

```
WITH RECURSIVE MyRecursiveCTE (...) AS
(
  SELECT ... FROM bar UNION ALL
  SELECT ... FROM MyRecursiveCTE JOIN bar ON ...
)
```

Basically, just putting the CTE on the left vs. the right side of the
join (which is exactly equivalent under the hood) fixes the issue.

I can't actually find any documentation of this change, but it seems
that sometime between MySQL 8.0.17 and MySQL 8.0.33, MySQL started
allowing this, perhaps just an optimization where it now notices the
equivalence and does the right thing.

I originally tested this on what I believed was the oldest version of
MySQL we supported, but didn't read carefully enough. We recommend
8.0.33 or higher here:

https://www.metabase.com/docs/latest/databases/connections/mysql

But that's not for the AppDB - for that, we recommend 8.0.17 or higher.

* add alert to prompt admins to clean up a collection

* clean up

* pr feedback

* fix i18n issue

* copy changes