* Add Timeseries insights even when Custom Columns exist

WIP

Fixes 46244

I don't know why it was (maybe still is) required that the :other columns be empty before providing insights.

* might be a solution to changing trendlines when data doesn't change

The insights code that powers the frontend trendline requires taking a sample of the dataset.

This sampling has some randomness associated by design, and should maintain its randomness. But it makes some sense to
use the exact same sample when the input has not changed at all, hence trying this memoize approach out.

Might not be the final solution, but it's a start

* add test to show that insights are computed

* use java.util.random with a seed to keep insights stable

* Add testcase from https://github.com/metabase/metabase/issues/43915



* Try a custom formatter if the default formatter for OffsetDateTime doesn't work

* Add testcase

* Make auxillary formatter private

* Catch a more specific exception

* Throw IllegalArgumentException on invalid dates

* Use strict resolver and fix test

* Empty commit to assign credit where it's due

---------

Co-authored-by: Chris Truter <chris@metabase.com>

* Dowload Row Limit Env Var

Adds `MB_DOWNLOAD_ROW_LIMIT` to enable changing the row limit on downloads and alert/subscription attachments (not the
rendered tables, but the .csv, etc.).

Based on: #44982 (Thanks, @r-kot)

Partially implements: #28144

The difference in this PR compared to #44982 is that the download limit applies to all downloads, the only exception
being when the limit is above `qp.i/absolute-max-results` (1048575, based on Excel's limitation); in such a case, the
user supplied limit is only used if the download is csv or json, and `qp.i/absolute-max-results` is used for xlsx.

This PR also fixes alert/subscription attachment limits; prior to this, they were set to the in-app limit of 2000
rows, but now they will follow the user supplied download-row-limit.

This PR also adds a test to the downloads-and-exports test namespace, confirming that they follow the supplied limit,
or the max limit if none is supplied.

* add test confirming the default limit works

* fix test to use the download-row-limit

* address review feedback

* Update src/metabase/public_settings.clj

Co-authored-by: Cal Herries <39073188+calherries@users.noreply.github.com>

* add test covering case where download-row-limit is unset

---------

Co-authored-by: Cal Herries <39073188+calherries@users.noreply.github.com>

* Export Formatting Correctness

WIP

Several bugs exist around the correctness of export formatting, this aims to fix them.

Fixes #27374

When a model has viz settings/metadata changed after save, the download won't incorporate said changes. At least it's
clear that scale doesn't work.

* number formatter merges settings from columns too

* column formatting on aggregated cols works in xlsx now too

* fixes 43040 by grabbing global-type-settings in csv formatter

* Last Used Param Value doesn't fill in with default value if cleared

Fixes #46368

The Last used parameter value is cleared when the frontend sends `nil` as the parameter's value. This works except in
cases where a user clears a parameter with a default value. In this case, we want to indicate that the parameter is
cleared, so we store `nil` in the appdb in such a case.

* Update src/metabase/query_processor/dashboard.clj

Co-authored-by: metamben <103100869+metamben@users.noreply.github.com>

* add assertion showing that 'nil' with default is still stored

---------

Co-authored-by: metamben <103100869+metamben@users.noreply.github.com>

Fixes: #46558

In #45575 we introduced casting uuid fields for drilldowns.

However we failed to include a check that the rhs was itself another
field of uuid type and so no casting should occur. This broke joins on
models when the database_type of the field may not be known.

* Try without broken refs

* Disable broken test

* Disable broken test

* fix: Handle tokens in strings when parsing params

Fixes #38051
Fixes #36262
Fixes #11755

* Fix handling snippets with single quotes in name

== Goal ==

Prevent admins of Metabase Cloud instances with Metabase Cloud Storage
from accidentally breaking their upload configuration, since in this
scenario that is automatically managed by Metabase Cloud to use the
attached data warehouse.

== How to test ==

On a subscription with Metabase Cloud Storage:
* The "Uploads" section in the left sidebar of `/admin/settings` does
  not show
* A "not found" page is shown at `/admin/settings/uploads` where one
  would otherwise expect the "Uploads Settings" page
* `PUT /api/setting` with an `uploads-settings` field in the body fails
  with `403 Forbidden` and the setting is not changed
* `PUT /api/setting/uploads-settings` fails with `403 Forbidden` and
  the setting is not changed
* `GET /api/setting` contains an element with `key: "uploads-settings"`
  and a value
* `GET /api/setting/uploads-settings` responds with the value
* `GET /api/session/properties` contains the `uploads-settings` field

Closes: https://github.com/metabase/harbormaster/issues/5121


Co-authored-by: Ryan Laurie <iethree@gmail.com>

* Add :relative-time-interval mbql function

* Add relativeDateFilterPartsRelativeTimeInterval

* Update display-name-method :relative-time-interval

* Update desugar-relative-time-interval

* Define relative-time-filter op

* Add or update tests

* Update tests

* Update display-name-method

* Update test

* Fix positive relative-time-interval shift

* Add checkins:1-per-day:60 dataset

* Update var name

* Bail out from optimization of relative-time-interval desugared filters

* Update dataset kondo hook

* Generate cache required for kondo hooks in kondo.sh

* Rename dataset

* perf: Optimize validation

* Bump Malli version

* perf: Optimize validation

### Description

This change should fix some explosions in stats due to phantom fields, while also making things a bit more rigorous.

- Record the table schema in query_field - especially useful for unknown tables
- Allow null table names in query_field - they might be phantoms, or they might not
  - We still have the option to filter them out in tools we build on top
- Tighten up matching logic to consider schema

* Fix coll permissions for audit collection

We've had a function in `models.collection` for a while that has taken a
permissions set and returned a set of collection IDs that the user has
permissions on. I refactored this recently, but didn't notice that it
was actually doing the permissions checks slightly incorrectly.
Specifically, because it only looks at the user's permissions and the
collection IDs and doesn't use `mi/can-read?` or `mi/can-write?`, it's
completely indifferent to whether a collection is an audit collection or
not.

This is arguably not a *permissions* issue: the two errors that could
come about here are:

- someone sees the audit collection even though the audit feature is
disabled, or

- someone is presented with the audit collection in a context where only
writable collections should be present - when they try to actually write
to it, it fails (since then we're doing the real permissions check).

The primary motivation for this fix was to prevent audit dashboards and
cards from appearing in the list of stale items.

* Fix hardcoded collection ID

We were creating a timeline in a fixed collection ID, that in tests
happened to be the ID of the Metabase Analytics collection.

* Update values-from-card-query

* Add test

* Update values-from-card-query

* Add missing issue no

* Avoid adding temporal-unit to lhs cols

* Use :default temporal-unit

* Simplify logic, update comments

* Update substitute-field-filter-test

* Update align-temporal-unit-with-param-type-test

* Update field-filter-date-test

* Use end-excludding gte lt filter for DateTime fields

* Update substitute-field-filter-test

* Update align-temporal-unit-with-param-type-test

* Update field-filter-date-test

* Update bigquery test + comment

* Update date-str->qp-aware-offset-dt

* Add guard for unexpected date string format

This is just for the completeness, I haven't encountered it.

* Address review remarks

* Update comment

* Squash

* Clean up query modifier for validating card references (#46283)

* Fixup bad git merge on test expectation

* Tweak docstrings and de-densify logic

This reverts commit 72a20181.

* Add :relative-time-interval mbql function

* Add relativeDateFilterPartsRelativeTimeInterval

* Update display-name-method :relative-time-interval

* Update desugar-relative-time-interval

* Define relative-time-filter op

* Add or update tests

* Update tests

* Update display-name-method

* Update test

* Fix positive relative-time-interval shift

* Allow filtering on collection

* Add `effective_ancestors` to collection data

Note: I removed the `qv=` comparator. I think it'.s better to be more explicit
about what exactly we're asserting. In particular, I hadn't updated the
`:count` when we were filtering on collection type, so the count was
still the total for *every* collection even though the results were being
restricted by the collection. However the `qv=-data` function was still
reporting that everything was OK, because it was accounting for other
data from the App DB by just asserting that the count we'd specified was
lower than the actual count. I'd argue it's better for tests to fail
locally because I have garbage in my app DB (which I can clean up) than
succeed when they really shouldn't.

* Consolidate effective ancestors

For the Search API, we're currently returning slightly different
ancestors:

- it never has the Root Collection, and

- the ancestors don't have a `personal_owner_id`

We could change the API (and maybe will soon - it seems pretty
unexpected to have `effective_ancestors` sometimes act one way and
sometimes act another) but for now we can just reshape things to look
the same as they did before, while using the same code under the hood.

Co-authored-by: Sloan Sparger <sloansparger@users.noreply.github.com>

* echarts pie

* setup rendering pipeline for static pie chart (#43547)

* setup rendering pipeline for static pie chart

* remove more donut code from BE

* compute viz settings for static pie chart (#43548)

* compute viz settings for static pie chart

* create chartModel for pie chart (#43549)

* create chartModel for pie chart

* render static pie chart (#43550)

* render static pie chart

* render static pie chart legend (#43551)

* disable animation

* alter static-viz pie chart be tests to account for echarts-impl

The old implementation had a different implementation for legends, separate from the 'categorical/donut' js
render. Echarts nicely combines the legend into the pie render, so we don't need to worry about that stuff anymore.

Instead, I've written a simple set of assertions on the echarts pie implementation now.

* Get rid of this test that was attempting to use old code path

* echarts pie

* One last spot where we don't need old pie stuff anymore.

* add loki specs

* save snapshots

* fix small slices disappearing

* update snapshots

* add repro story for exteremely small slice

* add snapshot

* fix rounded total

* fix percent formatting in legend

* fix date dimension formatting

* fix binned and relative date dimension labels on legend

* add specs and snapshots for null and unaggregated dimensions

* hide overlaping labels on chart

* fix null dimension color

* show error when metric column has a negative value

* move legend to botom

* make dot margin smaller

* add columns to legend

* sort legend vertically

* dont use grid if only one row for legend

* never hide legend in static viz

* fix empty column culling

* show single legend item

* add long dimension name story

* fix long dimension name in legend

* right justify percentages in legend

* remove negative error and re-add stories

* handle negative values

* create chart definition for dynamic pie chart

* render chart with responsiveness

* add comment for borderWidth calculation

* center legend

* fix legend centering

* add story for missing currency formatting

* fix missing currency formatting

* fix graalvm crashing

* fix column settings computation

* add another currency formatting repro

* fix wrong order of col settings computations

* convert colors to hex

* fix colors missing from viz settings

* temp todo comment

* fix colors in dynamic viz

* update snapshots

* fix crashing due to invalid dimension/metric setting

* fix chart crashing for boolean dimension with labels on chart

* add repro

* fix regression in label formatting

* fix labels on small slices

* update specs

* emphasis state for slice hover

* recreate old label hiding logic

* include height measurement in getIsLabelVisible

* move d3 slices to chart model

* fix labels on static viz

* wip basic hover state and tooltip

* fix emphasis style getting stuck by memoizing everything

* update total display value

* fix stuck emphasis by setting notMerge to false for pie only

* handle other slice in tooltip

* fix bugs in total graphic

* use OTHER_SLICE_KEY const in static viz legend

* implement legend and connect to hover state

* implement click actions

* fix slice not clickable

* handle all zero slices

* add repro

* update specs

* update pie_chart.cy.spec.js

* update chart_drill.cy.spec.js

* add repro for single column legend

* use minimum 2 legend columns

* fix legend text overlap

* use more margin for single row legend

* update specs

* add repro for 28568

* add repro for 38424

* fix percent label not showing on large slices

* hide total text when chart is small

* replace comment about animations

* show warning for mixed positive and negative values

* delete old chart files

* fix type errors

* update d3 import

* update specs

* remove branch from uberjar.yml

* dont put single slice into other

* fix lint errors and pie render test

* update more d3 imports

* remove unused settings

* revert preview-card-width change

* remove unused import

* remove majorWidth option from formatting

* use debounceLeading refresh mode in ChartWithLegend

* remove TOTAL_GRAPHIC_OPTION constant

* delete unit test for pie chart

* fix failing test by doing set comparison instead

* fix typo in comment

* remove SUNBURST_SERIES_OPTION constant

* fix lint by removing unused eslint-disable

* move default color computation to getValue

* fix static pie test to show that we will always render the legend

* update pie border color for night mode and embedding

* fix styles for embedding case and add stories

* skip embedding stories

---------

Co-authored-by: Adam James <adam.vermeer2@gmail.com>

* Make an internal API for retrieving stale content

This introduces the `metabase.stale` module, which has one public
function (so far): `metabase.stale/find-candidates`. This takes a set of
`collection-ids`, a cutoff date, and pagination/sort info.

Next, we introduce an HTTP API endpoint that will figure out the appropriate
set of `collection-ids` for a given user/request, call this function,
and return the results in the shape we want.

Also: for e2e tests, we need a way to mark a card or dashboard as stale. I
think the easiest way to do this is to just create a testing endpoint
that we can call to update the `last_used_at` or `last_viewed_at` of the
card/dashboard to set it to ~7 months ago.


---------

Co-authored-by: bryan <bryan.maass@gmail.com>