Improve developers guide related to E2E tests (#37132)

* Add section about `Lodash`

* Fix typo

* Expand the standard development flow section

* Expand the running options section

* Expand running on Docker section

* Add section for tests that require SMTP server

* Clarify the viewport resizing options

* Javascript -> JavaScript

* Delete redundant README section

* Explain `contains` vs `find` vs `get` better

docs/developers-guide/e2e-tests.md

@@ -10,31 +10,40 @@ Metabase uses Cypress for “end-to-end testing”, that is, tests that are exec
 
 Metabase’s Cypress tests are located in the `e2e/test/scenarios` source tree, in a structure that roughly mirrors Metabase’s URL structure. For example, tests for the admin “datamodel” pages are located in `e2e/test/scenarios/admin/datamodel`.
 
+Our custom Cypress runner builds its own backend and creates a temporary H2 app db. Both are destroyed when this process is killed. The reserved default port is `4000` on the local host. There is nothing stopping you from running your local Metabase instance on `localhost:3000` at the same time. This might even be helpful for debugging purposes.
+
 ### Standard Development Flow
-1. Run `yarn build-hot` to continuously build the frontend
+1. Continuously build the frontend
+
+    a. If you need only the frontend, run `yarn build-hot`
 
-2. then `yarn test-cypress-open --browser=electron` to open the Cypress application where you can execute tests you are working on
+    b. If you want to run a local Metabase instance alongside Cypress, the easiest way to achieve this is by using `yarn dev` or `yarn dev-ee` (both rely on frontend hot reloading under the hood)
 
+2. In a separate terminal session (without killing the previous one) run `yarn test-cypress-open`. This will open a Cypress GUI that will let you choose which tests to run. Alterantively, take a look at more running options below.
 
 ### Running Options
 
 To run all Cypress tests programmatically in the terminal:
-```
+```sh
 yarn run test-cypress-run
 ```
 
-You can run a specific set of scenarios by using the `--folder` flag, which will pick up the chosen scenarios under `e2e/test/scenarios/`.
+You can run a specific set of scenarios by using a custom `--folder` flag, which will pick up the chosen scenarios under `e2e/test/scenarios/`.
 
-```
+```sh
 yarn run test-cypress-run --folder sharing
 ```
 
-You can quickly test a single file only by using the `--spec` flag.
+You can quickly test a single file only by using the official `--spec` flag.
 
-```
+```sh
 yarn test-cypress-run --spec e2e/test/scenarios/question/new.cy.spec.js
 ```
 
+You can specify a browser to execute Cypress tests in using the `--browser` flag. For more details, please consult [the official documentation](https://docs.cypress.io/guides/guides/launching-browsers).
+
+Specifying a browser makes most sense when running Cypress in a _run_ mode. On the other hand, Cypress _open_ mode (GUI) allows one to easily switch between all available browsers on the system. However, some people prefer to specify a browser even in this scenario. If you do this, keep in mind that you are merely preselecting an initial browser for Cypress but you still have the option to choose a different one.
+
 ## Anatomy of the Test
 
 Cypress test files are structured like Mocha tests, where `describe` blocks are used to group related tests, and `it` blocks are the tests themselves.
@@ -63,12 +72,19 @@ Try to avoid repeatedly testing pieces of the application incidentally. For exam
 ### `contains` vs `find` vs `get`
 
 Cypress has a set of similar commands for selecting elements. Here are some tips for using them:
-* `contains` is case-sensitive to the text *in the DOM*. If it’s not matching text you’d expect, check that CSS hasn’t updated the case.
-* `contains` matches substrings, so if you see “filter by” and “Add a filter”, `contains(“filter”)` will match both. To avoid these issues, you can either pass a regexp that pins the start/end of the string or pass a selector in addition to the string: `.contains(selector, content)`.
-* `find` will let you search within your previous selection. `get` will search the entire page even if chained.
+- [`contains`](https://docs.cypress.io/api/commands/contains) is (by default) case-sensitive to the text *in the DOM*. If it’s not matching text you’d expect, check that CSS hasn’t updated the case. You can explicitly tell it to ignore the case with the following option `{ matchCase: false }`.
+    - `contains` matches substrings. Given two strings “filter by” and “Add a filter”, `cy.contains(“filter”);` will match both. To avoid these issues, you can either pass a regexp that pins the start/end of the string or scope a string to a specific selector: `cy.contains(selector, content);`.
+- [`find`](https://docs.cypress.io/api/commands/find) will let you search within your previous selection.
+- [`get`](https://docs.cypress.io/api/commands/get) will search the entire page even if chained, unless you explicitly tweak the `withinSubject` option.
 
 ### Increase viewport size to avoid scrolling
-Sometimes Metabase views are a bit large for Cypress’s default 1000x660 viewport. This can require you to scroll for tests to work. To avoid that, you can increase the viewport size for a specific test by calling `cy.viewport(width, height)`.
+Sometimes Metabase views are a bit large for Cypress’ default 1280x800 viewport. This can require you to scroll for tests to work. For example, virtualized tables will not even render the contents outside of the viewport. To avoid these problems, increase the viewport size for a specific test. Unless you're specifically testing how the application behaves on a window resize, please avoid using the `cy.viewport(width, height);` in the middle of the test. Set the viewport width/height using the optional Cypress test config instead. This config works with both `describe` and `it` blocks.
+
+```js
+describe("foo", { viewportWidth: 1400 }, () => {});
+
+it("bar", { viewportWidth: 1600, viewportHeight: 1200 }, () => {})
+```
 
 ### Code reloading vs test reloading
 When you edit a Cypress test file, the tests will refresh and run again. However, when you edit a code file, Cypress won’t detect that change. If you’re running `yarn build-hot`, the code will rebuild and update within Cypress. You’ll have to manually click rerun after the new code has loaded.
@@ -77,7 +93,7 @@ When you edit a Cypress test file, the tests will refresh and run again. However
 One great feature of Cypress is that you can use the Chrome inspector after each step of a test. They also helpfully provide a helper that can test out `contains` and `get` calls. This helper creates new UI that prevents inspecting from targeting the correct elements. If you want to inspect the DOM in Chrome, you should close this helper.
 
 ### Putting the wrong HTML template in the Uberjar
-`yarn build` and `yarn build-hot` each overwrite an HTML template to reference the correct Javascript files. If you run `yarn build` before building an Uberjar for Cypress tests, you won’t see changes to your Javascript reflected even if you then start `yarn build-hot`.
+`yarn build` and `yarn build-hot` each overwrite an HTML template to reference the correct JavaScript files. If you run `yarn build` before building an Uberjar for Cypress tests, you won’t see changes to your JavaScript reflected even if you then start `yarn build-hot`.
 
 ### Running Cypress on M1 machines
 
@@ -86,12 +102,20 @@ This is caused by the `@bahmutov/cypress-esbuild-preprocessor` that is using `es
 
 ### Running tests that depend on Docker images
 
-A subset of our tests depend on the external services that are available through the Docker images. At the time of this writing, those are three supported external QA databases, Webmail and LDAP server. It's tedious to have five Docker containers running locally. An escape hatch is provided for people that do not care about these tests, but still need to run specs containing them locally. Run this command:
+A subset of our tests depend on the external services that are available through the Docker images. At the time of this writing, those are the three supported external QA databases, Webmail, Snowplow and LDAP servers. It's tedious to have all these Docker containers running locally. An escape hatch is provided for people who do not care about these tests, but still need to run specs containing them locally. Run this command:
 
-- `yarn test-cypress-run --env grepTags="-@external" --spec path/to/spec/foo.cy.spec.js`
+```sh
+yarn test-cypress-run --env grepTags="-@external" --spec path/to/spec/foo.cy.spec.js
+```
 
 Please note the minus sign before the `@external` tag. For more details, consult [the official documentation](https://github.com/cypress-io/cypress-grep#filter-with-tags).
 
+If you want to or need to run these tests, there is a handy option that does the heavy lifting for you:
+
+```sh
+yarn test-cypress-open-qa
+```
+
 ### Running tests with Snowplow involved
 
 Tests that depend on Snowplow expect a running server. To run them, you need to:
@@ -99,6 +123,27 @@ Tests that depend on Snowplow expect a running server. To run them, you need to:
 - run Snowplow locally: `docker-compose -f ./snowplow/docker-compose.yml up -d`
 - pass env variables to the test run: `MB_SNOWPLOW_AVAILABLE=true MB_SNOWPLOW_URL=http://localhost:9090 yarn test-cypress-open`
 
+### Running tests that require SMTP server
+
+Some of our tests, that depend on the email being set up, require a local SMTP server. We use `maildev` Docker image for that purpose. At the time of this writing the image we use is `maildev/maildev:2.1.0`. It should be safe to always use the `:latest` image in your local development. Run this command:
+
+```sh
+docker run -d -p 1080:1080 -p 1025:1025 maildev/maildev:latest
+```
+
+### Cypress comes with `Lodash` for free
+
+We don't need to have [Lodash](https://lodash.com/) in our direct dependencies to be able to [use it with Cypress](https://docs.cypress.io/api/utilities/_). It is aliased with an underscore and its methods can be accessed with `Cypress._.method()`. We can use `_.times` method to stress-test a certain test (or a set of tests) locally.
+
+```js
+// Run the test N times
+Cypress._.times(N, ()=> {
+  it("should foo", ()=> {
+    // ...
+  });
+});
+```
+
 ## DB Snapshots
 
 At the beginning of each test suite we wipe the backend's db and settings cache. This ensures that the test suite starts in a predictable state.

e2e/test/scenarios/README.md

 # Metabase Scenarios
 
-## Running
-
-- If you are running tests that include `alert > email_alert`, run `docker run -d -p 1080:1080 -p 1025:1025 maildev/maildev:2.0.5` in terminal first for setting up email through your localhost
-
 ## Metabase QA DB Tests
 
 If you're running Cypress locally and you need to write or run tests for one of the [supported databases](https://github.com/metabase/metabase-qa), make sure you have docker running and start cypress using `yarn-test-cypress-open-qa`.