* tmp: patched monger for 4.11.1 mongo java driver

* tmp: Update monger utils

Aggregation probably wont work now, but we are not using those from monger anyway. With this change in place I'm able to load needed namespaces and create test-data dataset successfully.

Commit contains lot of condo errors that should be resolved while porting.

* WIP: Monger removed in favor of java driver wrapper

* Update java driver wrapper

* Update srv test

* Update comments

* Use non keywordized run-command for `dbms-version`

* Fix according to e2e tests

* Cleanup

* Separate `java-driver-wrapper` into multiple namespaces

* Fix semantic type inference for serialized json

* Fix options for run-command

* Cleanup json namespace

* Cleanup conversion namespace

* Cleanup operators

* Update kondo in operators ns

* Cleanup connection namespace

* Cleanup mongo namespace

* Cleanup util namespace

* Add todo

* Move session related code to util

* Cleanup database namespace

* Update docstring for conn string generation

* Update docstrings

* Update tests

* Update linter for with macros

* Update modules/drivers/mongo/src/metabase/driver/mongo/connection.clj

Co-authored-by: metamben <103100869+metamben@users.noreply.github.com>

* Update modules/drivers/mongo/src/metabase/driver/mongo/connection.clj

Co-authored-by: metamben <103100869+metamben@users.noreply.github.com>

* Use transient in from-document for building a map

Co-authored-by: metamben <103100869+metamben@users.noreply.github.com>

* Update can-connect to use let form

- Avoid nested `with-mongo-database` call.
- Avoid creating a set used for searching for database name.

* Remove redundant set use from describe-database

* Change from-document keywordize default to false

* Remove log message translation

* Update maybe-add-ssl-context-to-builder! to always return builder

* Indent from-document

* Remove redundant ConvertToDocument extensions

* Use oredered-map in do-find's sort

* Use ex-info in details-normalized

* Add imports and update comment in execute ns

* Update fixme comment

* Pass opts instead of a selection to from-document in do-find

* Avoid unnecessary double dot call

* Update connection test ns according to review remarks

* Make tests parallel

* Docstring update

* Update docstring

---------

Co-authored-by: metamben <103100869+metamben@users.noreply.github.com>

* Add new Question.prototype.setDefaultDisplay boilerplate
- rename old one to _setDefaultDisplay

* Handle native queries

* Handle table & scalar displays

* Add aggregationColumn wrapper

* Add breakoutColumn wrapper

* Add semanticType to ColumnDisplayInfo

* Add skeleton for the rest of setDefaultDisplay

* Add effectiveType to ColumnDisplayInfo

* Extract getDefaultDisplay out of Question

* Add test suite for getDefaultDisplay

* Add a test case for simple queries

* Add a test case for 1 aggregation and no breakouts

* Add a test case for 1 aggregation, 1 breakout and state column

* Format code

* Update test query

* Add a test case for 1 aggregation, 1 breakout and country column

* Update test names

* Simplify setup

* Add date-related tests

* Extract isState, isCountry, deal with breakout info vs column info

* Handle dates

* Move helpers down

* Add tests for binning

* Add explanatory variable

* Handle 1 aggregation and 2 breakouts, at least 1 of which is date

* Handle 1 aggregation and 2 breakouts by coordinates

* Add a test case for default return

* Use parameterless aggregation

* Handle categories

* Optimize: do not call Lib.displayInfo for all breakouts

* Optimize: extract getBreakoutInfos and call it only when necessary

* Do not group tests

* Remove redundant createQueryWithClauses calls

* Rename getBreakoutInfos to getBreakoutColumnInfos

* Fix Question.prototype.updateSettings typing

* Simplify test setup

* Simplify code

* Use accounts table

* Remove Question.prototype._setDefaultDisplay

* Rename getDefaultDisplay to defaultDisplay and expose it in metabase-lib/v2

* Update test names

* Consolidate conditions

* Fully migrate Category and Date checks

* Fix typo

* Split unrelated conditions to make it look like in the original function

* Reorder arguments to match the convention

* Rename getBreakoutColumnInfos to getBreakoutsWithColumnInfo

* Prevent categorizing extraction of/truncation to year as temporal-extraction

Fixes #37172.

* Move display.ts to viz/display.ts

* Use existing Lib.is* functions

* Make a change to trigger CI

* Revert "Make a change to trigger CI"

This reverts commit 328becd8e659073aa2f0daffd6011fa785bfa99e.

* Do the same for any display-info calculation

---------

Co-authored-by: Tamás Benkő <tamas@metabase.com>

* Adding geographic coordinate formatting to exports

Prior to this PR, exports as csv, excel, and json files as well as html in emails all treated longitude and latitude values as plain floats while the UI rendered the coordinates as degrees NSEW. This PR adds a new function, `format-geographic-coordinates`, in `metabase.formatter` and applies it to these tabular exports.

The function is applied as follows:

- For everything but excel exports, a new case  in `metabase.formatter/create-formatter` was added that renders all `:type/Coordinate`s
- For excel exports, `metabase.query-processor.streaming.xlsx/maybe-parse-coordinate-value` was added in the `add-row!` function and follows the same pattern as `maybe-parse-temporal-value`

Unit tests were added to cover all three export formats as well as html rendering in pulses (emails).

* Fixing `metabase.pulse.render.body-test` for lon/lat formatting.

Commit 8201e545 introduced failing tests (e.g. `clj -X:dev:ee:ee-dev:drivers:test :only metabase.api.card-test/update-card-with-type-and-dataset-test`).

They all seem to originate from NPEs in `model-hydrate-based-on-upload` on the first condition in the `and` clause in the `based-on-upload` anonymous function. This PR just makes that first condition null-safe with a `some->` threading macro.

That is, **if** there is a table with multiple PKs, and another table with
multiple FKs for those PKs, **then** any `:=` filters for other PKs on
that table should be preserved.

This allows using 1 or more quick-filter drills to add `FOO_ID = 7` filters
for some of the foreign table's PKs, followed by a fk-details drill that
jumps to the filtered view of that table.

Progress towards #36253.

* Hide unmapped filters for public/embedded dashboards

* Add E2E tests

* Add missing null check

* Rename filters in E2E tests

* Make `hiddenParameterSlugs` consistent

* Use `selectedTabId` from state

* Remove newline

* Fix E2E tests

* Fix test setup

* Fix name shortcuts

* add Hebrew support

* Prevent categorizing extraction of/truncation to year as temporal-extraction

Fixes #37172.

* Move all files related to the `ViewHeader` component to a separate folder

* Rename utils and convert to TS

* Move `View.jsx` back to the common top level folder

* Add `index.ts` to the `ViewHeader` structure

* Move `ExploreResultsLink` component

* Move `FilterHeaderButton` component

* Move `HeaderBreadcrumbs` component

* Move `QuestionNotebookButton` component

* Move `QuesitonDataSource` component

* Move `ConvertQueryButton` component

* Move `QuestionDescription` component

* Move `QuestionActions` component

* Move `QuestionSummarizeWidget` component

* Remove unused function

* Move `QuestionFilters` component

* Remove unused eslint directive

* Fix accidental empty export

* Export all components from a single `index.ts`

* Add required label to dashboard parameter settings pane
* Enable required behavior for parameter widgets on the dashboard
* Support reverting back to default value for all widget types
* Add and update e2e tests

* Move `canExploreResults` out of the Question prototype

* Move `canExplore` to `ViewHeaderUtils` file

* Add back unit tests for `canExplore`

* Fixing static-viz exports of saved x-ray dashboards

When dashboards were generated with x-rays, the `:visualization_settings` were not properly filled in. In particular, the `:graph.dimensions` and `:graph.metrics` were being populated from either the raw template values or not at all. The following namespace demonstrates this:

```clojure
(ns tickets.38350-broken-xray-export
  (:require
    [dev.render-png :as render-png]
    [metabase.models :refer [Card]]
    [metabase.test :as mt]))

(mt/dataset test-data
  (mt/with-temp [Card {bad-card-id :id}
                 {:display :row
                  :dataset_query {:database (mt/id)
                                  :type     :query
                                  :query    {:source-table (mt/id :orders)
                                             :aggregation  [[:count]]
                                             :breakout     [[:field
                                                             (mt/id :products :category)
                                                             {:source-field (mt/id :orders :product_id)}]]}}
                  :visualization_settings {:graph.series_labels [nil]
                                           :graph.dimensions [{:ProductCategoryMedium {}}]
                                           :graph.colors ["#EF8C8C"]
                                           :graph.metrics [nil]}}
                 Card {good-card-id :id}
                 {:display :row
                  :dataset_query {:database (mt/id)
                                  :type     :query
                                  :query    {:source-table (mt/id :orders)
                                             :aggregation  [[:count]]
                                             :breakout     [[:field
                                                             (mt/id :products :category)
                                                             {:base-type :type/Text
                                                              :source-field (mt/id :orders :product_id)}]]}}
                  :visualization_settings {:graph.dimensions ["CATEGORY"]
                                           :graph.metrics ["count"]}}]
    (render-png/render-card-to-png bad-card-id)
    (render-png/render-card-to-png good-card-id)))
```

The FE seemed to have some level of error checking and was robust enough to recover from these bad `:visualization_settings`.

The solution is to correctly populate these values. This required updating the `:dimension-name->field` value in each card to include mappings derived from both metrics and dimensions and then using the right logic in the `visualization-settings` function.

* Removed unneeded `->legacy-MBQL` on values in `:mbql/query` dispatch

Adding `r.metadata.source.branch === "master" &&` to `randomlyUploadAll` for master-only uploads.

Also adding more info to log message.

Part of https://github.com/metabase/metabase/issues/36497

### Description

In https://github.com/metabase/metabase/pull/36744 we tracked dashcard replacement with a new event name, but did not add this event name to the dashboard snowplow schema.

In this PR we add the name to the schema, and update an e2e test to ensure the event is being tracked.

### How to verify

Follow this [guide](https://www.notion.so/metabase/Snowplow-integration-5da1f874beda4153b4fccfa6c1e77caa) to run snowplow, then try replacing a dashcard locally, you should see the log in the console.

### Demo

https://github.com/metabase/metabase/assets/37751258/e8566f09-b9b8-47de-b215-fe0e25c7d920

### Checklist

- [x] Tests have been added/updated to cover changes in this PR

Part of https://github.com/metabase/metabase/issues/38208

### Description

Adds snowplow event tracking for dashboard card and tab duplication. No snowplow schema changes were made, we just added a couple of new event names.

### How to verify

Follow this [guide](https://www.notion.so/metabase/Snowplow-integration-5da1f874beda4153b4fccfa6c1e77caa) on running snowplow locally, then go to a dashboard and duplicate some cards and tabs, confirm the logs appear.

### Demo

https://github.com/metabase/metabase/assets/37751258/db59337d-a3aa-4d5d-afd0-fdd9998b13ed


### Checklist

- [x] Tests have been added/updated to cover changes in this PR

The options make caching
`metabase.lib.metadata.calculation/visible-columns` difficult, but the
common case is to call it without options on things like query stages.

That case is expensive, common, and reusable, so this is a great spot to
cache it.

Note that `metabase.lib.js/visible-columns` is also cached in the same
way, but this does not really overlap. The JS wrapper actually populates
the `selected: true` flags as well, and returns a JS array. This
slightly modified value is what gets cached there, and it's enough extra
work (calling `returned-columns` and `mark-selected`) that it's worth
having both. Structural sharing of the untouched or lightly-edited column
maps means the memory impact is modest.

Fixes #38312.

Adds additional logic per [this Slack thread](https://metaboat.slack.com/archives/C052LBPDAF3/p1706613621488479) to upload flake replays.

* Make the MLv2 display-name include the path of nested fields
* Hide binning and bucketing info when displaying breakoutables

Fixes #35938.

Most drills don't appear for structured (eg. JSON, XML) columns.

This adds a new canned query and canned clicks for such a column
(by lying in the metadata), then fixes a few test expectations
plus one broken drill.

Progress towards #36253.

More progress towards #36253.

* Add e2e test for metabase#38354

* Rename file to match issue id

* Tag the test as external since it uses QA DBs

* Fix ORDERS_ID

Resolves #38301

* Migrate syncStructuredQueryColumnsAndSettings to MLv2

* Migrate syncStructuredQueryColumnsAndSettings to MLv2

* Fix types

* Support converting aggregation references to legacy references

Fixes #37698.

* Update TS wrapper

* Pass query and stageIndex to legacyQuery

* Pass query and stageIndex to legacyQuery

* Remove no longer used columnDimensionWithName

* Update assertion in failing test

* Replace non-existing Question.prototype.isNative & Question.prototype.isStructured calls

---------

Co-authored-by: Tamás Benkő <tamas@metabase.com>
Co-authored-by: Kamil Mielnik <kamil@kamilmielnik.com>

* auto-adjust native query dates to be at the start of the date range

* re-work trend charts compute function to handle updated native query date formatting

* add test to validate the native query date formatting

* fix unit tests

* reorganize test

This also adds a couple of hand-written test cases in addition to
testing the canned set of queries against most of the drills.

Progress towards #36253.

This previous got rebuilt from scratch over and over, since different
queries being handed the same JS `Metadata` instance were returning
completely separate `MetadataProvider`s.

The FE replaces the `Metadata` instance as new metadata arrives from the
BE, so the `metabase.lib.cache/side-channel-cache` which is attached to
the input object is implicitly invalidated when the old `Metadata` is
thrown away.