The original attempt was #33777.

As revealed in #33799, the issue hasn't been fully fixed.
Take a look at this run: https://github.com/metabase/metabase/actions/runs/6110515399

The skipped test names are now in this format: e2e-tests-collectionsslow-ee
Let's see how do we create a name:

name: e2e-tests-${{ matrix.folder }}${{ matrix.context }}-${{ matrix.edition }}
So it's obvious that both folder and context are included, when in reality we need only one of the two.

The flaw in the logic was trying to remove the runner from the matrix. At least ONE matrix parameter needs to be different in the slow context. We achieve this by setting the runner to ubuntu-latest instead of ubuntu-22.04. This should now finally fix the names.

[ci skip]

* Update additional context for E2E stubbed/skipped tests

* Include two new E2E groups to stubbed/skipped E2E tests

* Run slow E2E tests using faster runner

* Tag slow tests

* Add additional x-ray tests to the slow batch

This log shows the output of the files changed:
https://github.com/metabase/metabase/actions/runs/6087912385/job/16537680230#step:3:602

```
Changes output set to ["shared_sources","frontend_sources","frontend_specs","frontend_all","backend_sources","backend_specs","backend_all","sources","e2e_specs","e2e_all","codeql","i18n","visualizations"]
```

Yet, the `download-uberjar` job ran, and the `build` never ran.
This resulted in E2E test failures due to missing FE changes.

The fault in the logic introduced in #33190 was thinking that
`needs.files-changed.outputs.e2e_specs == 'true'` means the only output,
when in reality it means one of many outputs.

This PR should fix this by making sure that `needs.files-changed.outputs.e2e_all != 'true'`
is also respected. This translates to - if there were no source code changes.

[ci skip]

* PoC: Download previously saved uberjar for E2E workflow

* Separate `e2e_specs` and `e2e_all` file path triggers

* Fix file path result oputput

* Refine E2E tests conditional trigger

* Further refine E2E tests conditional trigger

* Download the uberjar directly where it's needed

* Work around inability to dynamically define job ouputs

* First obtain the artifact, then run anything else

* Map job outputs

https://docs.github.com/en/actions/using-jobs/defining-outputs-for-jobs

* Apply changes to visual regression tests

* Final logic cleanup

* Work around the API rate limit

* Split `admin` E2E group and merge it with `auditing`

* Run new `admin-2` group in CI

* Move dashboard bookmarks spec to `organization`

* Move dashboard filter related specs to appropriate group

* Create new `dashboard-cards` E2E group

based on https://github.com/orgs/community/discussions/26303 and my experience, workflows with `always()` will not cancel even if a new run of the same workflow appears with concurrency and cancel-in-progress:true. Official documentation suggests using `!cancelled()` instead

* Install Chrome immediately after checkout

* Switch to Chromium 111.0.5563.146

As of two days ago we started seeing Chrome crashing in E2E tests.
An example of the failed run: https://github.com/metabase/metabase/actions/runs/5665970958/job/15354475344

```
We detected that the Chromium Renderer process just crashed.

This is the equivalent to seeing the 'sad face' when Chrome dies.

This can happen for a number of different reasons:

- You wrote an endless loop and you must fix your own code
- You are running Docker (there is an easy fix for this: see link below)
- You are running lots of tests on a memory intense application.
    - Try enabling experimentalMemoryManagement in your config file.
    - Try lowering numTestsKeptInMemory in your config file.
- You are running in a memory starved VM environment.
    - Try enabling experimentalMemoryManagement in your config file.
    - Try lowering numTestsKeptInMemory in your config file.
- There are problems with your GPU / GPU drivers
- There are browser bugs in Chromium
```

I'm not quite sure whether this is a bug in Chrome v115 or some interplay between Cypress and this particular Chrome version, but running all tests on older Chrome version v114 seems to be working just fine.
https://github.com/metabase/metabase/actions/runs/5677579526?pr=32688

Proposal
Let's merge this ASAP as a hotfix to unblock everyone, and then we'll follow-up when there's a clear resolution of this problem (most likely upstream).

* untangle ldap circular dependency

* only try to require ee code when ee is available

* LDAP login e2e tests

* fix eslint errors

* Update e2e/test/scenarios/admin/settings/sso/ldap.cy.spec.js

Co-authored-by: Nemanja Glumac <31325167+nemanjaglumac@users.noreply.github.com>

* update command in e2e-ldap-helpers.js

---------

Co-authored-by: Nemanja Glumac <31325167+nemanjaglumac@users.noreply.github.com>

* Start all tests without a token

* Update tests

* Fix tests in `admin` group

* Fix tests in `dashboard-filters` group

* Fix tests `visualizations` group

* Batch fixes

* Fix database banner snowplow test

* Fix full-app mobile view banner test

* Simplify and remove `isPremiumActive` check

* Fix the helper

* Fix helper to fail fast

* Fix tests

* Ensure token scope is set

* Ensure CYPRESS_ env tokens are present

* Set token only if running Metabase ee

* Improve JSDoc

* Slightly tweak log message

* Update E2E dev guide

* Update cross-version tests

* Remove premium token from Cypress backend setup

* Improve a comment

* Fix test

* Update embedding copy

---------

Co-authored-by: Alexander Polyankin <alexander.polyankin@metabase.com>

The previous setting accidentally executed a scheduled job **every minute**
on Sunday, when we actually wanted it to run only once on that day.

This PR fixes it and it will now run at 10PM every Sunday.
https://crontab.guru/#0_22_*_*_0

[ci skip]

Replay is adding a significant overhead and is making every single run
fail. This is messing with our statistics and analytics.

While the Replay.io team is working on some optimizations that would
remove the overhead, we can significantly dial down the frequency of
these runs. We'll switch to running them only once per week, on Sunday.

For now...

I forgot to do this in #31432.
As the result, some PRs have been stuck in CI due to the required check.

This PR will run the fallback job for `actions` E2E group when changed
files don't require E2E tests to run.

[ci skip]

* Extract `questionDetails`

* Move `public-sharing` to the `sharing` folder

* Remove redundant test

This part was covered in `public-sharing.cy.spec.js`.

* Create a separate `public-dashboard` E2E spec

- Extract relevant part from `public.cy.spec.js`
- Make all tests in `public-dashboard.cy.spec.js` run in isolation

* Make tests in `public.cy.spec.js` run in isolation

* Remove redundant wait

* Limit the query results to speed test up

* Merge public questions E2E tests

* Merge `downloads` with the `sharing` E2E group

* Remove `downloads` from the folder list

* Refactor `public-dashboard.cy.spec.js`

- Clean up linter warnings by using semantic selectors
- Speed test up by using API

* Move core actions E2E tests to a separate folder

* Run new `actions` E2E group in CI

Metabase actions deserve their own E2E group, as the test suite around
them is expected to grow. This commit introduces `actions` E2E group to
the CI.

We're seeing increasing number of timeouts in CI without any significant
changes in the test suite. This could be due to GitHub's runners performance.

[ci skip]

[ci skip]

@jaril informed me that Replay.io is missing metadata
from Metabase E2E tests. Apparently trying to access
system environment variable from Cypress support file
didn't work.

The solution for this is to prefix the environment variable
with `CYPRESS_` and then to read it using `Cypress.env()`

This is what Cypress documentation suggests:
https://docs.cypress.io/guides/guides/environment-variables#Option-3-CYPRESS_

[ci skip]

* Install `replay.io` library

* Register `replay.io` in the Cypress config

* Run E2E tests using Replay chromium browser but only in CI

* Upload Replay.io recordings to the dashboard

* Manually install Replay.io browser

* Always upload recordings

* Pass in a custom test run id

* Disable asserts and send replays to a separate team

* Upload OSS recordings as well

* Use specific Ubuntu version

* Record and run Replay.io on `master` only

* Do not toggle CI browsers in the config

* Test run: pass `replay-chromium` browser as CLI flag in a run

* Fix multi-line command

* Use replay plugin conditionally

* Set the flag correctly

* Require node process

* Remove sourcemap vars

* Record using replay.io on schedule

* Explicitly name replay runs

---------

Co-authored-by: Jaril <jarilvalenciano@gmail.com>

* WIP action data type tests

* DRY up the actions tests

* lots of type tests

* make the tests work

* reorganize tests

* WIP action data type tests

* DRY up the actions tests

* lots of type tests

* make the tests work

* reorganize tests

* reduce some flakes

* better time formatting in action forms

* sort out all the timezones 

* update after rebase

* rebase disaster

* Build Script overhaul

* Remove stray deps.edn

* Include :ci profile for build and release tests

* Fix release script

* Update markdown dox

* Update another dox

* Tweakz

* Don't pin new version of data.xml to core project, just for build scripts

* Ignore unrecognized options

* Fix i18n/enumerate

* Register new E2E path alias

* Move E2E support files

* Move E2E runner files

* Move E2E snapshot related files

* Move E2E visual regression tests

* Move E2E `downloads` scenarios

* Move E2E `admin` scenarios

* Move E2E `auditing` scenarios

* Move E2E `bining` scenarios

* Move E2E `ccollections` scenarios

* Move E2E `cross-version` scenarios

* Move E2E `custom-column` scenarios

* Move E2E `dashboard` scenarios

* Move E2E `dashboard-filters` scenarios

* Move E2E `dashboard` reproductions

* Move E2E `embedding` scenarios

* Move E2E `filters` scenarios

* Move E2E `joins` scenarios

* Move E2E `models` scenarios

* Move E2E `native-filters` scenarios

* Move E2E `native` scenarios

* Move E2E `onboarding` scenarios

* Move E2E `organization` scenarios

* Move E2E `permissions` scenarios

* Move E2E `question` scenarios

* Move E2E `visualizations` scenarios

* Move E2E `sharing` scenarios

* Move E2E `docker-compose`

* Fix commands imports

* Fix helpers imports

* Fix downloads reproductions imports

* Fix integration imports

* Move visual regression tests

* Various fixes

* Move `validate-e2e-files` script

* Add eslint rules for E2E folder

* Improve file paths

* Move `cross-version` tests into scenarios

* Path fixes

* Register E2E alias in tsconfig

* Fix failing tests after rebase

[ci skip]

Resolves #27885

Resolves #18301