* Fix unable to disable scan field values (#41170)

* Migration to remove scan field values triggers for DBs that turn it off (#41348)

---------

Co-authored-by: Ngoc Khuat <qn.khuat@gmail.com>

Co-authored-by: Nicolò Pretto <info@npretto.com>

* fix progress bar in query builder

* add spec to verify regression fix

Co-authored-by: Jesse Devaney <22608765+JesseSDevaney@users.noreply.github.com>

* note on model caching

* update link

* blockquote

Co-authored-by: Jeff Bruemmer <jeff.bruemmer@gmail.com>

Co-authored-by: Jeff Bruemmer <jeff.bruemmer@gmail.com>

* static embedding

* Update docs/embedding/start.md



---------

Co-authored-by: Alex Yarosh <alexandra@metabase.com>
Co-authored-by: Jeff Bruemmer <jeff.bruemmer@gmail.com>

Co-authored-by: Nick Fitzpatrick <nickfitz.582@gmail.com>

See: https://security.snyk.io/vuln/SNYK-JAVA-ORGBOUNCYCASTLE-6612984



Co-authored-by: Chris Truter <crisptrutski@users.noreply.github.com>

fix header being rendered (with its border) on dashboard embedded with titled=false and only one tab (#41399) (#41407)

* fix header being rendered (with its border) on dashboard embedded with titled=false and only one tab

* embedframe-header -> embed-frame-header

Co-authored-by: Nicolò Pretto <info@npretto.com>

* Keep Filter Values for enabled Parameters in Embedding

Fixes: #37914

Previously, we removed locked parameters entirely. This is to prevent leaking potentially sensitive values.

However, for the situation where the same field backs 1 locked or disabled paramter and 1 enabled parameter, we do
still want to send the paramater values, because the enabled parameter implies that the values are permissible to see
in the embed.

So, this change will still remove parameters based on their 'disabled/locked/enabled' status, but will NOT remove the
linked field ids if they're also being used by 'enabled' parameters.

This results in the backend correctly sending necessary parameter values to the embed, where the frontend can then
render the appropriate UI instead of falling back to just text filters.

* test the case where a locked and enabled param share same field

* Address feedback. Added comment to explain classify fn

Co-authored-by: adam-james <21064735+adam-james-v@users.noreply.github.com>

* Exclude implicitly joinable columns from visible-columns in fk-columns-to

* Add test

* Update description

Co-authored-by: lbrdnk <lbrdnk@users.noreply.github.com>

Co-authored-by: Cal Herries <39073188+calherries@users.noreply.github.com>

* increase activation distance for dashboard parameter dragging (#41123)

* increase activation distance for dashboard parameter dragging

* updating drag sensitivity for other places

* adjusting e2e tests

* backporting testid for grabber

---------

Co-authored-by: Nick Fitzpatrick <nick@metabase.com>
Co-authored-by: Nick Fitzpatrick <nickfitz.582@gmail.com>

Co-authored-by: Chris Truter <crisptrutski@users.noreply.github.com>

* drop note on sandboxes

* model caching

* reinstate model caching sandbox warning

Co-authored-by: Jeff Bruemmer <jeff.bruemmer@gmail.com>

Co-authored-by: Raphael Krut-Landau <raphael.kl@gmail.com>

This middleware shouldn't throw exceptions when `limit` or `offset`
query parametesr aren't integers.

Co-authored-by: John Swanson <john.swanson@metabase.com>

* Use MLv2 to determine whether a query can be previewed

* Add E2E repro for #40608

* Expand E2E repro

* Fix test

* Make sure the step is active and visible before offering to preview its query

* Expand E2E test

* Address review comment - use `getNotebookStep` helper

Co-authored-by: Nemanja Glumac <31325167+nemanjaglumac@users.noreply.github.com>

Co-authored-by: Nemanja Glumac <31325167+nemanjaglumac@users.noreply.github.com>

Co-authored-by: Nick Fitzpatrick <nick@metabase.com>

* Fix embed download endpoint

We have added format_rows as a query parameter. In most endpoints this just works fine.

Here, however, the problem is that we use query parameters to pass parameter values. We run some validation over these
provided parameter values (eg. don't allow a user to over ride a 'locked' parameter). Since we have these 2 different
use cases for query parameters, the validation was failing on :format_rows, which isn't expected to exist at all.

Here, I dissoc the :format_rows key prior to this validation step, so normal validation can continue.

* Add test that uses this card embedding endpoint

Co-authored-by: adam-james <21064735+adam-james-v@users.noreply.github.com>

* Nested queries have limits which can defeat their purpose

Consider a query

```sql
select count(*) from {{#199}}
```

This query should return the number of distinct rows in the query
defined by 199. But it's actually limited by the excel limit of
1048575. And that's because when the value of `{{#199}}` is expanded it
has that limit applied as normal.

```clojure
qp=> (let [card-id 199] ;; use a valid card id for you
       (-> {:database 1,
            :type :native,
            :native {:query "select count(*) from {{ref}}"
                     :template-tags {:ref {:card-id card-id
                                           :type :card
                                           :name "ref"
                                           :display-name "ref"}}}
            :middleware {:disable-max-results? true}}
           qp.compile/compile
           :query
           (metabase.db.query/format-sql )
           println))
select
  count(*)
from
  (
    SELECT
      "PUBLIC"."ORDERS"."ID" AS "ID",
      "PUBLIC"."ORDERS"."TOTAL" AS "TOTAL"
    FROM
      "PUBLIC"."ORDERS"
    LIMIT
      1048575
  )
```

But we can suppress this limit when substituting a query inside yielding

```sql
select
  count(*)
from
  (
    SELECT
      "PUBLIC"."ORDERS"."ID" AS "ID",
      "PUBLIC"."ORDERS"."TOTAL" AS "TOTAL"
    FROM
      "PUBLIC"."ORDERS"
  )
```

And this is proper because we want to limit the _outer_ query, not
internal queries.

* Remove limit from test expectation

* stupid trailing space

* another subquery test

* Use helper function to disable limit middleware

Co-authored-by: dan sutton <dan@dpsutton.com>

Co-authored-by: Cal Herries <39073188+calherries@users.noreply.github.com>

Co-authored-by: Noah Moss <32746338+noahmoss@users.noreply.github.com>

Co-authored-by: Cal Herries <39073188+calherries@users.noreply.github.com>

* Restore tabs display for embedded dashboard without title

* Add unit test

* Remove empty file

---------

Co-authored-by: Denis Berezin <denis.berezin@metabase.com>
Co-authored-by: Mahatthana Nomsawadi <mahatthana.n@gmail.com>

Co-authored-by: Mahatthana (Kelvin) Nomsawadi <me@bboykelvin.dev>

These type checks are useful when constructing clauses with full MLv2
metadata available, but when converting from legacy the `:field` refs
have only `:base-type`.

If a column has its metadata overridden to eg. coerce a string or
integer into a date, its `:base-type` will remain `:type/Integer` or
`:type/Text` and the `:effective-type` will be `:type/Date` etc.

This change skips the type check while converting from legacy, and
while checking `can-run`. Eventually this should be replaced with
tracking `:effective-type` on refs or using real metdata, instead of
relying on the `:base-type` only.

Fixes #41122.

Co-authored-by: Braden Shepherdson <braden@metabase.com>

Co-authored-by: Cam Saul <1455846+camsaul@users.noreply.github.com>

Co-authored-by: Jeff Bruemmer <jeff.bruemmer@gmail.com>

* Update RTL to v12

* fixing unit tests

---------

Co-authored-by: Nick Fitzpatrick <nickfitz.582@gmail.com>

I have edited the list of "Download results."

Co-authored-by: Abed Habli <154609371+abdhabli@users.noreply.github.com>

Co-authored-by: Alexander Solovyov <alexander@solovyov.net>

* Group axes with the correct map shape.

Prior to this, any time axes were all explicitly set :left or :right by the user, the `group-axes-at-once` function
would incorrectly return the map in the following form:

`{:left ["a" "b" "c"]}` which is incorrect. It must instead look like this:

`{"a" :left "b" :left "c" :left}`

This fixes the map shape and permits the user to explicitly set all axes to the left or to the right.

* Add a test to confirm the axis settings are respected and do render

Co-authored-by: adam-james <21064735+adam-james-v@users.noreply.github.com>

Co-authored-by: Jeff Bruemmer <jeff.bruemmer@gmail.com>

This logic was trying to match `[name join-alias]` pairs previously, but
that won't work since the later stage field doesn't have a `:join-alias`
on it. This falls back to matching on only the ID or name, but only if
it's unambiguous.

See #40252.

Co-authored-by: Braden Shepherdson <braden@metabase.com>

Co-authored-by: Noah Moss <32746338+noahmoss@users.noreply.github.com>

Co-authored-by: Uladzimir Havenchyk <125459446+uladzimirdev@users.noreply.github.com>

* Enable not emptyable arguments for is-empty and not-empty clauses (#40957)

* Enable not emptyable arguments for is-empty and not-empty clauses

* Add review remarks

* Shorten dataset name so it is usable with some drivers

* Disable test for Athena

* Handle possibly missing base type

* Update comment

* Missing base type field is treated as not emptyable

* Update desugar middleware

Now middleware adds base type to fields with integer id that are missing that first. This ensures correct function of is-empty expansion.

* Update desugar middleware

Cleanup base types necessary for desugaring is-empty

* Update desugar middleware cleanup

* Update test

* Adjust tests

* Remove now redundant e2e test

* Update comment

* Backport new emptyable definition and derivates

---------

Co-authored-by: lbrdnk <lbrdnk@users.noreply.github.com>