return collections in same order from /api/collections/tree and /api/collections/:id/items (#40044) (#40263)

There were two problems here:

- one endpoint was sorting in the database, but not putting 'analytics'
last.

- one endpoint was sorting in the database, but then sorting again in
clojure, which had subtly different behavior for special characters

Fixes #39965

Co-authored-by: John Swanson <john.swanson@metabase.com>

Co-authored-by: Anton Kulyk <kuliks.anton@gmail.com>

Co-authored-by: Uladzimir Havenchyk <uladzimir.dev@gmail.com>

* Use expression fix on both pk-ref and value-ref

The gist: inside the query expressions are referred to as [:expression
"foo"]. But as a nested query, it's just another field, and the fact
that it was an expression is irrelevant and should not leak from that
stage. So "downstream" should just be [:field "foo" base-type].

Oddly, if you just fix one field or the other, the query would be
"valid", would log an error, but no valid values would be returned.

It seems like the invalid expression reference was just dropped so we
only selected a single column and those were removed (we run `(filter
valid-tuples?)` as we only want valid key/value pairs).

So let's add a `(> (count values) 0)` which was failing previously.

* Fix indexing error

When updating the model index, it diffs the existing values vs the new
values and deletes appropriate and adds the new values. But it was using
the wrong column when updating the model index values. It used `pk_ref`
but that table uses `model_pk`. This led to the following errors on
stats:

```sql
select pk_ref, value_ref, state, error from model_index where state = 'error'

pk_ref,value_ref,state,error
"[""field"",563028,null]","[""field"",563018,null]",error,"ERROR: column ""pk_ref"" does not exist Position: 68"
"[""field"",534030,null]","[""field"",534037,null]",error,"ERROR: column ""pk_ref"" does not exist Position: 68"
"[""field"",545945,null]","[""field"",545948,null]",error,"ERROR: column ""pk_ref"" does not exist Position: 68"
```

Now those errors will be fixed and the model indexes will correctly
track the values

* Bump indexing threshold to 25,000

Had been limited to 5,000 rows of model indexing. This was a made up
number as we went into this endeavor. But it's proving to be far too
small. Bumping to 25,000

* Parititon deletions and inserts

With a higher limit need to ensure that we don't send too large of a
query.

```clojure
model-index=> (t2/insert! ModelIndexValue
                          (map (fn [id]
                                 {:name           (str (gensym "value"))
                                  :model_pk       id
                                  :model_index_id 1})
                               (range 25000)))
Execution error (PSQLException) at org.postgresql.jdbc.PgPreparedStatement/<init> (PgPreparedStatement.java:105).
PreparedStatement can have at most 65,535 parameters. Please consider using arrays, or splitting the query in several ones, or using COPY. Given query has 75,000 parameters
```

* move comment and fix test failure explanation

comment should be next to the number? check. and also need to use
`mc/explain` which returns info rather than `mc/validate` which returns
a bool:

```clojure
;; bad
model-index-test=> (let [values [[1 "foo"] [2 "foo"] ["foo" "foo"]]]
                     (-> (mc/validate [:sequential [:tuple number? string?]] values)
                         (me/humanize)))
nil
;; good
model-index-test=> (let [values [[1 "foo"] [2 "foo"] ["foo" "foo"]]]
                     (-> (mc/explain [:sequential [:tuple number? string?]] values)
                         (me/humanize)))
[nil nil [["should be a number"]]]
```

Co-authored-by: dpsutton <dan@dpsutton.com>

* Fix case-sensitive option leaking to string/= (#40535)

Fixes #40383

If a native query parameter is set up with a string widget type that
supports a `case-sensitive` option, and that parameter is tied to a
dashboard filter that did not support that option, the option was still
leaking into the filter as an option where options were not expected.

For instance the native query parameter used `string/contains` and the
dashboard used `string/=`, the QP would try to build a filter like `[:=
field value {:case-sensitive false}]` which is an illegal filter.

* Fix moved fn

---------

Co-authored-by: Case Nelson <case@metabase.com>
Co-authored-by: Chris Truter <crisptrutski@users.noreply.github.com>

* 40232 sortable with popovers (#40378)

* custom sensor

* e2e test, moving sortable

* e2e test adjustment

* combining props

* backporting move DnDKitElement

* adjusting e2e test

---------

Co-authored-by: Nick Fitzpatrick <nick@metabase.com>
Co-authored-by: Nick Fitzpatrick <nickfitz.582@gmail.com>

Co-authored-by: Braden Shepherdson <braden@metabase.com>
Co-authored-by: Alexander Polyankin <alexander.polyankin@metabase.com>

Fixes #37517.

Co-authored-by: Braden Shepherdson <braden@metabase.com>

fix normalized chart crashes when trend lines enabled but there is no insights data (#40624) (#40627)

Co-authored-by: Aleksandr Lesnenko <alxnddr@users.noreply.github.com>

Co-authored-by: Luis Paolini <paoliniluis@gmail.com>

* Fix e2e utils

* Update e2e/support/commands/api/composite/createDashboardWithQuestions.js



* Remove unnecessary return



---------

Co-authored-by: Mahatthana (Kelvin) Nomsawadi <me@bboykelvin.dev>
Co-authored-by: Kamil Mielnik <kamil@kamilmielnik.com>

* Add reproduction for spaces breaking the search in filter pickers

* Search using display name too

* Search using display name in summarize picker

Co-authored-by: Romeo Van Snick <romeo@romeovansnick.be>

I started a much more complex fix with a job that tried to clean up
stale `refreshing` PersistedInfos... then talked to Dan, who pointed out
that because only one instance is running these jobs at any given
time, we can just add `refreshing` to the list of refreshable states.

Co-authored-by: Jeff Bruemmer <jeff.bruemmer@gmail.com>

Co-authored-by: Jeff Bruemmer <jeff.bruemmer@gmail.com>

Co-authored-by: Jeff Bruemmer <jeff.bruemmer@gmail.com>

Co-authored-by: Alex Yarosh <alexandra@metabase.com>

* In #37160 we added a migration to convert all datetime columns to
timestamp with time zone by using a fixed list of what to convert. but
it turns out some customer instances has a different list. So this PR
removed the fixed list and use a query to get a list of all datetime
columns.

Co-authored-by: Ngoc Khuat <qn.khuat@gmail.com>

Co-authored-by: Chris Truter <crisptrutski@users.noreply.github.com>

* Add tests for checking partial table privileges on PG (#40549)

* Include postgresql tables that are visible through column grants instead of full table grants for simple questions (#40034)

Co-authored-by: Ngoc Khuat <qn.khuat@gmail.com>

---------

Co-authored-by: Ngoc Khuat <qn.khuat@gmail.com>
Co-authored-by: Eric Jensen <ericcj@gmail.com>

Co-authored-by: Ngoc Khuat <qn.khuat@gmail.com>

* Redshfit: sync tables with partial select permission (#40421)

* uncomment since 40058 is backported

---------

Co-authored-by: Ngoc Khuat <qn.khuat@gmail.com>

* Funnel Chart Static-Viz should handle row data more robustly

The row data coming into the funnel chart render method can evidently have a few slightly different shapes, and may
need to be 'coordinated' with the :funnel.rows key from viz-settings (names and order are specified there).

This is a WIP to make the method more robust to different row data shapes. For example:

[[1 100] [2 200]] should work as well as [["A" 100] ["B" 200]] and correctly handle the viz-settings

WIP because I need to add a test or two, try to get a handle on the exact schema that is allowed for 'row shape' and
the funnel.row viz key, and see if there are failing cases that I haven't considered yet.

* Funnel rows should now work more effectively with possible raw-rows

* Add a test that checks the success of the render and correct order

* Update src/metabase/pulse/render/body.clj



* Update src/metabase/pulse/render/body.clj



* Addressing feedback from the review

 - renamed build-funnel-rows to funnel-rows
 - used 'funnel-viz' as the binding for the {:key "asdf" :name "asdf" :enabled true} maps from the viz settings
 - added docstring to the function to try clarify its 2 branches (keys vs indices on the rows)

---------

Co-authored-by: adam-james <21064735+adam-james-v@users.noreply.github.com>
Co-authored-by: Chris Truter <crisptrutski@users.noreply.github.com>

* Trend Chart Viz Should Use Proper Card execute in Rendering (#40464)

* Trend Chart Viz Should Use Proper Card execute in Rendering

WIP

Fixes: #39854

The trend chart follows a code path shared with multi-series static-viz. This means there was an implicit expectation
that a dashcard is associated with the card at all times. This isn't true for alerts, so the code is fixed to remove
the failure.

* Add a test

* add missed :breakout that actually made the test work

oops

* Adding comments to try clarify what's going on

* indentation

---------

Co-authored-by: Adam James <adam.vermeer2@gmail.com>
Co-authored-by: adam-james <21064735+adam-james-v@users.noreply.github.com>

* note about reorder

* Apply suggestions from code review



---------

Co-authored-by: Alex Yarosh <alexandra@metabase.com>
Co-authored-by: Jeff Bruemmer <jeff.bruemmer@gmail.com>

Co-authored-by: Cal Herries <39073188+calherries@users.noreply.github.com>

Co-authored-by: Cal Herries <39073188+calherries@users.noreply.github.com>

Co-authored-by: Callum Herries <calherries@gmail.com>

* Fix flaky whitelabel test (#40441)

* Fix flaky whitelabel test

* Update another test for `landing-page`

* Fix failed tests

This was because the new toast notification label isn't backported to 49
since it was part of the new feature

---------

Co-authored-by: Mahatthana (Kelvin) Nomsawadi <me@bboykelvin.dev>
Co-authored-by: Mahatthana Nomsawadi <mahatthana.n@gmail.com>

* sort expression functions by use case

* custom expression edits

* formatting cleanup

* fix link

* Apply suggestions from code review



* address comments

---------

Co-authored-by: Alex Yarosh <alexandra@metabase.com>
Co-authored-by: Jeff Bruemmer <jeff.bruemmer@gmail.com>

Co-authored-by: Nick Fitzpatrick <nick@metabase.com>

* Often slo won't be setup, but we should still delete the session

* test that we delete the session even when "/handle_slo" is not called

* cleanup test

* take the cookie from the session, not as a parameter

Co-authored-by: bryan <bryan.maass@gmail.com>

Fixes: #40306

Our datetime formatter relies on the `java-time.api`, for which there are many different, sometimes confusing, formatter patterns: https://docs.oracle.com/javase/8/docs/api/java/time/format/DateTimeFormatterBuilder.html#appendPattern-java.lang.String-

In this case, 'YYYY' is a week-of-year style year, which calculates which week a date falls into before returning the
year. Sometimes days near the start/end of a year will fall into a week in the wrong year.

For example, apparently 2023-12-31 falls into the 1st week of 2024, which probably not the year you'd expect to
see. What we probably do want is 'yyyy' which calculates what day of the year the date is and then returns the year
based off of that instead of the week number.

For an explanation, you can check out this SO answer: https://stackoverflow.com/a/46395342

 provides an explanation.

Co-authored-by: adam-james <21064735+adam-james-v@users.noreply.github.com>

Co-authored-by: Jeff Bruemmer <jeff.bruemmer@gmail.com>

Co-authored-by: Raphael Krut-Landau <raphael.kl@gmail.com>

Thanks, Jeff and Alex!

Co-authored-by: Abed Habli <154609371+abdhabli@users.noreply.github.com>

Co-authored-by: Alexander Polyankin <alexander.polyankin@metabase.com>

* Add alternative hostname support to snowflake

* move hostname support out of additional details

* Update modules/drivers/snowflake/src/metabase/driver/snowflake.clj



* Update modules/drivers/snowflake/src/metabase/driver/snowflake.clj



* Update modules/drivers/snowflake/test/metabase/driver/snowflake_test.clj



* opt-in

---------

* Clean up test

* Fix race condition

* Wait for GET dashboard + for sidebar to close

* Add wait

* Add a sad comment

Co-authored-by: Anton Kulyk <kuliks.anton@gmail.com>