* Add metadata reload on question to model conversion

* Add e2e test

when running an x-ray on tables, we were sending back the following
parameter mappings:

BEFORE with bug:
```clojure
core=> (let [table (t2/select-one 'Table :id 1)]
         ;; products table
         (-> (automagic-analysis table {})
             :ordered_cards first
             (select-keys [:parameter_mappings :card])
             (update-in [:card] #(select-keys % [:dataset_query]))))
{:parameter_mappings ({:parameter_id "-1185138340",
                       :target [:dimension
                                [:field
                                 "CREATED_AT"  ;; <----- field reference by name
                                 {:base-type :type/DateTime}]],
                       :card_id G__246876}
                      {:parameter_id "241800831",
                       :target [:dimension
                                [:field
                                 "CATEGORY"    ;; <----- field reference by name
                                 {:base-type :type/Text}]],
                       :card_id G__246876}),
 :card {:dataset_query {:type :query,          ;; <----- mbql query
                        :database 1,
                        :query {:source-table 1,
                                :breakout ([:field
                                            64
                                            {:temporal-unit :quarter-of-year}]),
                                :aggregation (["count"])}}}}
```

After this change we have the following parameter mappings which use ids
as field references:

```clojure
core=> (let [table (t2/select-one 'Table :id 1)]
              ;; products table
              (-> (automagic-analysis table {})
                  :ordered_cards first
                  (select-keys [:parameter_mappings :card])
                  (update-in [:card] #(select-keys % [:dataset_query]))))
{:parameter_mappings ({:parameter_id "-1185138340",
                       :target [:dimension [:field 64 nil]], ;; <-- by id
                       :card_id G__186662}
                      {:parameter_id "241800831",
                       :target [:dimension [:field 58 nil]], ;; <-- by id
                       :card_id G__186662}),
 :card {:dataset_query {:type :query,
                        :database 1,
                        :query {:source-table 1,
                                :breakout ([:field
                                            64
                                            {:temporal-unit :quarter-of-year}]),
                                :aggregation (["count"])}}}}
```

This change was introduced in our refactoring to support x-rays on
models with commit message

* metabase.automagic-dashboards.filters - Fix to build-fk-map so that
parameters show up on model x-rays

I'm not sure what the reasoning was but we can revisit it when we want
to do some filters on models

* metabase.automagic-dashboards.core - Docs, cleanup, and destructuring for clarity
metabase.api.search - Adding fix to realize search results
metabase.automagic-dashboards.filters - Fix to build-fk-map so that parameters show up on model x-rays
metabase.automagic-dashboards.populate - Fix typo
metabase.api.search-test - Unit tests for search fix

* Brought over tab saving of transient x-ray dashboard code to save-transient-dashboard!

Added initial cut of tabbed x-ray generation from mapping over linked entities to the automagic-dashboards api

* update signature

we'll need some filtering criteria for the tabs. we don't just want an
x-ray of orders, we want an x-ray of _Customer's orders_

* Added query-filter to /model_index/:model-index-id/primary_key/:pk-id API

* Fixed id propagation

* clean up the logic a little bit

* Intermediate step to trap bad generated queries

* First stab at queries with joins for x-rays

take a model, find joinable tables and create joins back to the model's
underlying table so we can filter where pk=selected-id.

```clojure
{:database_id 1,
 :database-id 1,
 :table_id 2,
 :dataset_query {:database 1,
                 :type :query,
                 :query {:source-table 2,
                         :where [:=
                                 [:field
                                  62
                                  {:join-alias "ORDERS__via__PRODUCT_ID"}]
                                 1],
                         :joins [{:alias "ORDERS__via__PRODUCT_ID",
                                  :fields :none,
                                  :strategy :left-join,
                                  :condition [:=
                                              [:field
                                               62
                                               {:join-alias "ORDERS__via__PRODUCT_ID"}]
                                              [:field 40 nil]],
                                  :source-table 1}]}}}
```

Note the where clause and the join, but selecting `:fields :none` from
the join. so it's

```sql
select j.*
from joinable j
left join model m
where m.id = <user-selection>
```

But x-rays of these queries strip out our joins :(

Here's what comes back in the automagic dashboard

```clojure
{:type :query,
 :database 1,
 :query {:source-table 2,
         :filter [:and
                  [:=
                   [:field
                    62
                    nil]
                   1]],
         :breakout ([:field
                     41
                     {:temporal-unit :month}]),
         :aggregation (["avg"
                        [:field
                         39
                         nil]])}}
```

It's a breakout by month of orders.created_at (field 41) aggregated by
orders.quantity (field 39) where products.id (field 62 of table) =
1. But there's no join to products any longer :(

And there's this nice little comment

```clojure
(defn- maybe-enrich-joins "Hack to shove back in joins when they get
automagically stripped out by the question decomposition into metrics"
```

So that's no great. More investigating to do

* Added in `:source-field` for our x-ray tabs so that filters would succeed.

* Merged fixes to filters.

* Adding in entity x-ray dashboard title

* Fixed entity x-ray description.

* Added model-link card to each tab in the dynamically generated entity x-rays.

* Adding min-width calculation for source model link card.

* Adding tests for min-width calculation for source model link card.

* Fixing import indentation

* tests for x-rays on indexed models

* Adding tests to metabase.api.automagic-dashboards-test to assert the existence of model link cards and meaningful dashboard names and descriptions.

* include model_index_id in search results for indexed entity

had the rest of the changes, just not the id of the model_index

* use negative integers, not gensyms for tab-ids

FE wants negative integers for ids that do not exist in the db yet. some
of the cards and such still use gensym and that seems to be ok for now.

* fix tabs on x-ray dashboard in FE

* add x-ray button to `SearchResult` component

* clean up styles for tabs in `AutomaticDashboardAppInner`

* Added tests to understand some of the logic in `field-candidates.` The selection of a candidate is variable depending on the specification of the dimension, the field type, the table type, how the context is populated, etc. These tests attempt to illustrate some of the logic.

* Adding tests in which multiple fields match to multiple dimensions.

* Added tests for bind-dimensions and much of the logic it calls.

* Removing `:transient_name` and `:transient_filters` from generated entity x-ray to ensure the right information is displayed on the generated dashboard.

* Formatting

* Formatting

* Formatting

* Formatting

* Fixing quoted instance of Card

* wslint

* Hook for `metabase.api.automagic-dashboards-test/with-indexed-model`

* wip

* wslint

* lint warnings on ns requires

unused ns qp
using a deprecated namespace so give an override

* better error message for no linked entities

they used to be in the let-404 so you would get a quite confusing error
message. Now it's a 400 and will show "No linked entities" if you expand
the error message. We still need to make this better. But this is an
obvious improvement on a 404.

* Adding default entity dashboard for when no linked entities are found.

* fix search tests

* call count is 16 now

* Fixing unit test

* Fixed the query filter in `metabase.api.automagic-dashboards` so that `create-linked-dashboard` doe not produce bad queries.

* remove model index destructure

* Assert that all query dashcards have the correct filter.

We're no longer making joins back to the model's underlying
table. Recognizing that we were joining on linked table's fk to the
model's underlying table's pk and then searching where pk = <pk-value>,
we can just filter where fk = <pk-value>, omit the join.

So these tests just grab all of the linked tables and assert that one of
those filters is found.

eg, suppose a model based on products. the linked tables are orders and
reviews. And rather than the queries:

```sql
select o.*
from orders o
left join products p
on p.id = o.product_id
where p.id = <pk-value>
```

we can just do

```sql
select *
from orders
where product_id = <pk-value>
```

And similar for reviews. And for each query in the dashboard we're
looking for one of the following:

```clojure
, #{[:= $orders.product_id 1] [:= $reviews.product_id 1]}
```

* Intermediate stage of fixing our tests

* Added missing `is` clauses to a few tests. This surfaced some minor formatting changes that would have broken the tests had they been working. Updated the tests to correspond to the changes (just has to do with pluralization).

* Fixed untyped event and added mock search data.

* Fixing entity x-rays for a single linked entity

* Adding logic to provide only a single linked entity view for entity x-rays if only one is generated.

* lint fix

* add tests for 1 linked-table = 0 tabs case

* Updated `create-linked-dashboard` with a threading macro for clarity and to be more clojuric.

* split single test into three

cam made a new linter to look for test forms over a certain
size. There's no great way to run only a single testing form so this
helps make things more atomic

* More tests to understand bind-dimensions and the functions it depends on.

* Added test for three column query linking all columns to the same underlying table.

* Handle expression refs in indexed-models

Custom columns look like the following:

```clojure
{:expressions {"full-name" [:concat <first-name> <last-name>]}
```

To index values, we need a sequence of primary key and the associated
text value. So we select them from a nested query on the model. But the
model's way to refer to an expression and queries _on_ the model are
different. To the model, it's an expression. To queries based on the
model, it's just another field.

And have field refs in the result_metadata `[:expression
"full-name"]`. But when selecting from a nested query, they are just
string based fields: `[:field "full-name" {:base-type :type/Text}]`.

old style query we issued when fetching values:

```clojure
{:database 122
 :type :query
 :query {:source-table "card__1715"
         :breakout [[:field 488 {:base-type :type/Integer}]
                    [:expression "full name"]] ;; <-- expression
         :limit 5001}}
```

new style after this change:

```clojure
{:database 122
 :type :query
 :query {:source-table "card__1715"
         :breakout [[:field 488 {:base-type :type/Integer}]
                    [:field "full name" {:base-type :type/Text}]]
         :limit 5001}}
```

* Normalize field references

The schema was expecting valid mbql field refs (aka vectors and
keywords) but was getting a list and string (`[:field 23 nil]`
vs ("field" 23 nil)`). So normalize the field ref so we can handle the
stuff over the wire

this nice little bit of normalization lived in models.interface and
comped two functions in mbql.normalize. A few places reused it so moved
it to the correct spot.

* Better error message in `fix-expression-refs`

handles [:field ...] and [:expression ...] clauses. Seems unlikely that
aggregations will flow through here as that's not a great way to label a
pk. But i'll add support for that in a follow up

* Move `normalize-field-ref` below definition of `normalize-tokens`

`normalize-tokens` is `declare`d earlier, but we aren't using the var as
a var, but in a def we derefence it. But that's nil as it hadn't been
defined yet. Caused lots of failures downstream, including in cljs land

```clojure
user=> (declare x)
user=> (def y x) ;; <- the use in a def gets its current value
user=> (def x 3) ;; <- and it's not reactive and backfilling
user=> y
```

* Added tests for `has-matches?` and `resolve-overloading`

* Adding filter examples into the `resolve-overloading` and `has-matches"` tests.

Filter and metric matching logic is identical, but these examples are added for completeness and illustration.

* tests WIP trying to fix merge changes

* Removing tap

* Removing duplicate def

* Added tests for card-candidates

---------

Co-authored-by: dan sutton <dan@dpsutton.com>
Co-authored-by: Emmad Usmani <emmadusmani@berkeley.edu>

* refactor use-list-select hook

- toggleAll(items) felt unintuitive as some items could be selected while simultaneously unselecting others
  - switched to selectOnlyTheseItems(items) to be more explicit on what the function is doing
- updated consumers of useListSelect to use the new API

* convert to functional component

* convert HoC ListSelect to hook useListSelect

* convert redux connects to useSelectors

* refactor ArchiveApp and improve types

* remove unused eslint disable line

* update useListSelect because of changes

* convert ArchivedItem to TypeScript

* refactor Search.loadList to useSearchListQuery

* add E2E tests for archive page

* refactor

- remove dead code
- refactor duplicated code

* fix type error

* fix variable usage

* fix broken scroll update

* Add the official build for release workflow

[ci skip]

* Store full-length SHA-1 commit hash in an artifact

* Trigger pre-release when the official build workflow finishes

* Download the official uberjar instead of building it again

* Adapt `check-commit` job to use the release artifact

* Adapt `check-uberjar-health` job to use the release artifact

* Adapt `run-sanity-check` job to use the release artifact

* Adapt `containerize` job to use the release artifact

* Adapt `verify-docker-pull` job to use the release artifact

[ci skip]

* Checkout the repo

We need the folder structure to satisfy the JAR path.
[ci skip]

* Make `check-uberjar-healt` dependend on the commit check job

* Fix commit-id file path

[ci skip]

* Fix `containerize` reference to the release artifact commit output

[ci skip]

* Fix `release-artifact` outputs

[ci skip]

* Move useLeaveConfirmation hook to global hooks directory

* Use useBeforeUnload in useLeaveConfirmation

* Remove redundant custom onConfirm prop

* Convert useLeaveConfirmation to TS

* Fix router & route props in PermissionsPageLayout

* Refactor useLeaveConfirmation into LeaveConfirmationModal

* Move withRouter from PermissionsPageLayout to LeaveConfirmationModal

* Remove redundant _.compose call

* Sort props

* Fix build
- ./bin/i18n/update-translation-template did not recognize the syntax

* Sort props

* Stop making my eyes bleed!

* Fix Kondo warnings

* Fix broken drill thrus

* Fix some inputs

* WHY ARE WE CONSTRUCTING ALL THESE CLAUSES BY HAND =(

* [MLv2] Column metadata to legacy-field-ref

Fixes #33622

Pulled from https://github.com/metabase/metabase/pull/32698



* [experimental] Allow selecting MLv2 metadata directly from app DB using Toucan 2 shenanigans

* Fix file that changed in merge

---------

Co-authored-by: Cam Saul <github@camsaul.com>

disable "download results" and "edit question" from xray dashcards that have not been saved (#33638)

* attempt at detecting if we're in xray

* use dashboard.id to detect if inside xrays

* e2e test for xray 33637

* isXray passed down as prop from AutomaticDashboard

* isXray moved to shouldRender

* test: add unit test for #33637

* refactor: put all conditional rendering logic inside DashCardMenu.shouldRender

* remove e2e test as we prefer unit

* refactor: rename shouldShowDashCardMenu -> shouldShowDashCardMenu as per pr feedback

The original attempt was #33777.

As revealed in #33799, the issue hasn't been fully fixed.
Take a look at this run: https://github.com/metabase/metabase/actions/runs/6110515399

The skipped test names are now in this format: e2e-tests-collectionsslow-ee
Let's see how do we create a name:

name: e2e-tests-${{ matrix.folder }}${{ matrix.context }}-${{ matrix.edition }}
So it's obvious that both folder and context are included, when in reality we need only one of the two.

The flaw in the logic was trying to remove the runner from the matrix. At least ONE matrix parameter needs to be different in the slow context. We achieve this by setting the runner to ubuntu-latest instead of ubuntu-22.04. This should now finally fix the names.

[ci skip]

* Fix object detail crashing when a table from foreign key is hidden

* Add basic Object detail unit test

* Add the rest of the test

* Make fetch mock idiomatic

* Address feedback: Improve test name clarity

Co-authored-by: Kamil Mielnik <kamil@kamilmielnik.com>

* Address feedback: Improve test readability and correctness

* Fix type error

---------

Co-authored-by: Kamil Mielnik <kamil@kamilmielnik.com>

Co-authored-by: Mahatthana (Kelvin) Nomsawadi <me@bboykelvin.dev>
Co-authored-by: Mahatthana Nomsawadi <mahatthana.n@gmail.com>

We had to adjust E2E tests when we removed H2 support.
That was done in #33212.

But this broke the pre-release workflow.
That workflow doesn't have necessary docker services needed for this test
(postgres and mysql).

On top of that, adding both databases in this test is kind of redundant,
and definitely an overkill. We just want to make sure we can add a database.
So, why not use SQLite that doesn't require external services?

This PR accomplishes exactly that.
It simplifies and speeds up this particular test by using SQLite database.

* Remove unused "No changes to permissions will be made." confirmation in PermissionsEditBar
- Remove shouldConfirmCancel prop

* Sort props

* Update additional context for E2E stubbed/skipped tests

* Include two new E2E groups to stubbed/skipped E2E tests

* Don't delete a connection impersonation policy when non-data perms are modified

* refactor

* apply similar fix to gtap code

* set feature flag in test

# Primary objective: From an indexed entity, located the items referencing that entity and make a dashboard for that entity. This will make several dashboards, one for each entity, so the strategy is to combine these all into a single dashboard with tabs. If there are no linked entities as simple dashboard indicating that nothing interesting is linked. If there is a single linked entity we create a single dashboard (no tab) but with the same linked entity title.

A few of the main nses that were affected and other bullet point changes:

* metabase.api.automagic-dashboards - The vast majority of the work to create the unified dashboard is found here.
* metabase.automagic-dashboards.core - Docs, cleanup, and destructuring for clarity
* metabase.api.search - Adding fix to realize search results
* metabase.automagic-dashboards.filters - Fix to build-fk-map so that parameters show up on model x-rays
* metabase.automagic-dashboards.populate - Fix typo
* metabase.api.search-test - Unit tests for search fix
* Brought over tab saving of transient x-ray dashboard code to save-transient-dashboard!

---------

## Overall summary

The primary entry point to these changes can be found in `metabase.api.automagic-dashboards` at:

```clojure
(api/defendpoint GET "/model_index/:model-index-id/primary_key/:pk-id" ...
```

A simple reproduction of dashboard creation from indexed entities is shown here:

```clojure
(let [model-index-id 54 ;; This and pk-id will be specific to some indexed entity in your system
      pk-id          1]
  (binding [api/*current-user-permissions-set* (delay #{"/"})
            api/*current-user-id*              1]
    (let [model-index       (t2/select-one ModelIndex :id model-index-id)
          model             (t2/select-one Card (:model_id model-index))
          model-index-value (t2/select-one ModelIndexValue
                                           :model_index_id model-index-id
                                           :model_pk pk-id)
          linked            (#'api.magic/linked-entities
                              {:model             model
                               :model-index       model-index
                               :model-index-value model-index-value})
          dashboard         (#'api.magic/create-linked-dashboard
                              {:model             model
                               :linked-tables     linked
                               :model-index       model-index
                               :model-index-value model-index-value})]
      dashboard)))
```

---------

## Fixed the query filter in `metabase.api.automagic-dashboards` so that `create-linked-dashboard` doe not produce bad queries.

We're no longer making joins back to the model's underlying
table. Recognizing that we were joining on linked table's fk to the
model's underlying table's pk and then searching where pk = <pk-value>,
we can just filter where fk = <pk-value>, omit the join.

So these tests just grab all of the linked tables and assert that one of
those filters is found.

eg, suppose a model based on products. the linked tables are orders and
reviews. And rather than the queries:

```sql
select o.*
from orders o
left join products p
on p.id = o.product_id
where p.id = <pk-value>
```

we can just do

```sql
select *
from orders
where product_id = <pk-value>
```

And similar for reviews. And for each query in the dashboard we're
looking for one of the following:

```clojure
, #{[:= $orders.product_id 1] [:= $reviews.product_id 1]}
```

---------

## Handle expression refs in indexed-models

Custom columns look like the following:

```clojure
{:expressions {"full-name" [:concat <first-name> <last-name>]}
```

To index values, we need a sequence of primary key and the associated
text value. So we select them from a nested query on the model. But the
model's way to refer to an expression and queries _on_ the model are
different. To the model, it's an expression. To queries based on the
model, it's just another field.

And have field refs in the result_metadata `[:expression
"full-name"]`. But when selecting from a nested query, they are just
string based fields: `[:field "full-name" {:base-type :type/Text}]`.

old style query we issued when fetching values:

```clojure
{:database 122
 :type :query
 :query {:source-table "card__1715"
         :breakout [[:field 488 {:base-type :type/Integer}]
                    [:expression "full name"]] ;; <-- expression
         :limit 5001}}
```

new style after this change:

```clojure
{:database 122
 :type :query
 :query {:source-table "card__1715"
         :breakout [[:field 488 {:base-type :type/Integer}]
                    [:field "full name" {:base-type :type/Text}]]
         :limit 5001}}
```

---------

## Normalize field references

The schema was expecting valid mbql field refs (aka vectors and
keywords) but was getting a list and string (`[:field 23 nil]`
vs ("field" 23 nil)`). So normalize the field ref so we can handle the
stuff over the wire

this nice little bit of normalization lived in models.interface and
comped two functions in mbql.normalize. A few places reused it so moved
it to the correct spot.

* Better error message in `fix-expression-refs`

handles [:field ...] and [:expression ...] clauses. Seems unlikely that
aggregations will flow through here as that's not a great way to label a
pk. But i'll add support for that in a follow up

* Move `normalize-field-ref` below definition of `normalize-tokens`

`normalize-tokens` is `declare`d earlier, but we aren't using the var as
a var, but in a def we derefence it. But that's nil as it hadn't been
defined yet. Caused lots of failures downstream, including in cljs land

```clojure
user=> (declare x)
user=> (def y x) ;; <- the use in a def gets its current value
user=> (def x 3) ;; <- and it's not reactive and backfilling
user=> y
```

* Don't capture `declare`d vars in a def

need late binding. when the function is called, those vars will have a
binding. But if we use a `def`, it grabs their current value which is an
unbound var.

---------

Co-authored-by: dan sutton <dan@dpsutton.com>
Co-authored-by: Emmad Usmani <emmadusmani@berkeley.edu>

* Add caching
* Hand roll simplify-compound-filter

* Run slow E2E tests using faster runner

* Tag slow tests

* Add additional x-ray tests to the slow batch

* inital commit

* fix tests

* update test

* fix tests

* fix tests

* fix tests

* fix tests

* update test

* update tests

* update tests

* address changes

* fix tests

* fix test

* fix tess

---------

Co-authored-by: adam-james <21064735+adam-james-v@users.noreply.github.com>

The build is failing on master due to :cause "No such var: lib.join/current-join-alias"
This breaking change was introduced in #33574.

Searching for current-join-alias shows that all other references use a different namespace. Namely, lib.join.util.
This PR uses that namespace instead.

The function was moved in #32698. Both PRs have been opened, and have had CI green before the merge, but got out of sync. A merge queue would've prevented this.

This log shows the output of the files changed:
https://github.com/metabase/metabase/actions/runs/6087912385/job/16537680230#step:3:602

```
Changes output set to ["shared_sources","frontend_sources","frontend_specs","frontend_all","backend_sources","backend_specs","backend_all","sources","e2e_specs","e2e_all","codeql","i18n","visualizations"]
```

Yet, the `download-uberjar` job ran, and the `build` never ran.
This resulted in E2E test failures due to missing FE changes.

The fault in the logic introduced in #33190 was thinking that
`needs.files-changed.outputs.e2e_specs == 'true'` means the only output,
when in reality it means one of many outputs.

This PR should fix this by making sure that `needs.files-changed.outputs.e2e_all != 'true'`
is also respected. This translates to - if there were no source code changes.

[ci skip]

* Extract ObjectDetailBody to a separate file

* Extract ObjectDetailHeader to a separate file

* Extract ObjectDetailView to a separate file
- extract ObjectDetailWrapper.styled.tsx

* Extract ObjectDetailHeader unit tests

* Extract ObjectDetailBody unit tests

* Rename ObjectDetail.unit.spec.tsx to ObjectDetailView.unit.spec.tsx

* Remove the non-null assertion operator

* Use relative import

* Introduce hasFks

* Do not loadFKReferences() when there was no previous zoomed row
- this function will be called in useMount hook's callback

* Use project-wide != null pattern

* Remove loadFKReferences() on mount because it is duplicated later by useEffect

* Extract ObjectRelationships.styled

* Extract ObjectDetailTable.styled

* Fix imports

* Add repro test for #32720