* Convert e2e-custom-column-helpers to TS

* Refactor createTimeline, createTimelineEvent & createTimelineWithEvents to TS

* Fix circular imports

Failing after changes in 13983247

Per slack conversation, ok to disable:

https://metaboat.slack.com/archives/C013N8XL286/p1724711461313089?thread_ts=1724414568.662469&cid=C013N8XL286

* Don't encrypt boolean settings (by default)

We have tooling to disable encryption on settings even when the
`MB_ENCRYPTION_SECRET_KEY` is set. Turn it on by default for all boolean
settings, which don't need to be encrypted.

I also optimized the code that decrypts settings on startup because I
didn't want to delay startup if someone had set a bunch of boolean
settings. With 20 set, the old version added about 200+ ms to startup,
about 10ms per boolean setting, whether or not it was encrypted or in
the DB already. The optimized version selects all the never-encrypt
values from the database at once (a bit silly, but we also just exclude
raw `true` and `false` values so we don't bother checking them) and
updates them if they're encrypted - this adds ~40ms to startup with 20
encrypted boolean settings (about 2ms per boolean setting) and ~5ms to
startup on subsequent runs, when no encrypted values are in the DB.

This testing is too strict to support some legitimate use cases that
have no workaround at present. We can bring back this type-checking
eventually, once it's possible to correctly express things like
automatic coercion of strings to numbers in `SUM` aggregations.

Fixes #44431.

* fix import order in the files with disabled rules

* remove hack

* minor

* serialization clarification

* Update docs/installation-and-operation/serialization.md

Co-authored-by: Jeff Bruemmer <jeff.bruemmer@gmail.com>

* Update docs/installation-and-operation/serialization.md

Co-authored-by: Jeff Bruemmer <jeff.bruemmer@gmail.com>

* Update docs/installation-and-operation/serialization.md

Co-authored-by: Jeff Bruemmer <jeff.bruemmer@gmail.com>

* Update docs/installation-and-operation/serialization.md

Co-authored-by: Jeff Bruemmer <jeff.bruemmer@gmail.com>

---------

Co-authored-by: Jeff Bruemmer <jeff.bruemmer@gmail.com>

Co-authored-by: Cam Saul <1455846+camsaul@users.noreply.github.com>

Seems that `glue:GetCatalogImportStatus` is also needed for the integration with Amazon Athena because there's multiple AccessDenied on cloudtrail if this permission is missing:
```
{
    "eventVersion": "1.09",
    "userIdentity": {
        "type": "IAMUser",
        "principalId": "***********************",
        "arn": "arn:aws:iam::*************:user/metabase",
        "accountId": "**********************",
        "accessKeyId": "*******************",
        "userName": "metabase"
    },
    "eventTime": "2024-04-11T08:29:00Z",
    "eventSource": "glue.amazonaws.com",
    "eventName": "GetCatalogImportStatus",
    "awsRegion": "eu-west-1",
    "sourceIPAddress": "***********",
    "userAgent": "DriverVersion/02.00.35.1001/JDBCVersion/4.2/PluginName/IAM, aws-sdk-java/1.12.339 Linux/5.10.213-201.855.amzn2.x86_64 OpenJDK_64-Bit_Server_VM/11.0.22+7 java/11.0.22 clojure/1.11.1 vendor/Eclipse_Adoptium cfg/retry-mode/legacy",
    "errorCode": "AccessDenied",
    "errorMessage": "An unknown error occurred",
    "requestParameters": null,
    "responseElements": null,
    "requestID": "*************************************",
    "eventID": "*************************************",
    "readOnly": true,
    "eventType": "AwsApiCall",
    "managementEvent": true,
    "recipientAccountId": "*************************************",
    "eventCategory": "Management",
    "tlsDetails": {
        "tlsVersion": "TLSv1.3",
        "cipherSuite": "TLS_AES_128_GCM_SHA256",
        "clientProvidedHostHeader": "glue.eu-west-1.amazonaws.com"
    }
}
```

* Update appearance.md

Adding information about how colors are selected by Metabase because there is the confusion that the order in which you select colors is the order you get on the chart

* Update appearance.md

Removed code reference and tried to simplify the context

* Update docs/configuring-metabase/appearance.md

Co-authored-by: Jeff Bruemmer <jeff.bruemmer@gmail.com>

---------

Co-authored-by: Jeff Bruemmer <jeff.bruemmer@gmail.com>

Co-authored-by: Cam Saul <1455846+camsaul@users.noreply.github.com>

fix(dashboard): In dashboard value source modal, after closing question picker modal, don't also close the first modal (#47242)

There are a few separate changes here:

- Migrations: add and populate indexed columns perm_value, perm_type, and collection_id to permissions

These fields allows us to efficiently run queries based on collection permissions in the DB without string manipulation. Keeping the table as permissions allows us to do this migration in-place. Note that in some cases collections may have been deleted from the database without deleting the associated permissions row (since there was no foreign key before). We need to be defensive here: if we have a permissions row without a corresponding collection, delete the row before running the rest of the migration.

- Write perm_value, perm_type, and collection_id for new collection permissions

A very simple before-insert method sets these fields before a collection permission is written to the DB.

- Replace collection/permissions-set->visible-collection-ids and collection/visible-collection-ids->honeysql-filter-clause with collection/honeysql-filter-clause

Previously, just about everywhere we used permissions-set->visible-collection-ids, what we were essentially doing was an in-app join: select all the collection IDs you have permission on, then convert it to a SQL clause like WHERE collection_id IN ( all of those collection IDs).

Replace both of these with honeysql-filter-clause, which uses the new fields we added to permissions above to construct a honeysql filter representing "all the collections I have permissions on", without needing to round-trip them to the application and back to the DB.

Of course, we can then write a function visible-collection-ids, which uses honeysql-filter-clause, for those cases where we do actually need the whole bunch in the application (we use this, for example, when constructing the effective-location for a collection).

I also added one more toggle to the VisibilityConfig that's passed into the honeysql-filter-clause (and used to be passed to permissions-set->visible-collection-ids), allowing you to select only the effective children of some collection.

* [MBQL lib] Map dashboard filters to last stage w/aggregations

Originally we used the last stage's columns for dashboard filters, but
then if you eg. use a custom expression to clean up the results of the
aggregation, then you can't filter at the most useful spot - before the
aggregations.

Fixes #19744. Related to #26244, #33958.

* Add stage-number option to the dimension not to the field

* Remove FE changes

* Support negative stage numbers

* Add tests for multi-stage parameter expansion

* Specify schema for dimension options

* Fix legacy MBQL schema

* Append stage when running cards with top level summary

* Fix schema

* Only ensure filter stage when the parameters say it's used

* Implement ensure-filter-stage

* Format clj

* Add ensure-filter-stage test

---------

Co-authored-by: Braden Shepherdson <braden@metabase.com>

* Fix DateTimeRange substitution for open ranges

* Add E2E repro for #47172

* cljfmt

* Add test for datetime column

* Update range boundary for open range

* Update test names

* Apply suggestions from code review

Co-authored-by: Kamil Mielnik <kamil@kamilmielnik.com>

* Fix e2e

---------

Co-authored-by: Nemanja <31325167+nemanjaglumac@users.noreply.github.com>
Co-authored-by: Kamil Mielnik <kamil@kamilmielnik.com>

* Add database type adjustments to server side generated temporal values

* Speculative base type change

* Infer timezone only from _actual_ user, not global user level

* Set timezone for actual user in test extensions

* Add test case

* Update type info generation

* Adjust server side relative datetime

* Update src/metabase/driver/sql/query_processor.clj

Co-authored-by: Braden Shepherdson <braden@metabase.com>

* Fix base type logic

* Fix typo

* Fix formatting

* Update user name handling

* cljfmt

* Address review comments

---------

Co-authored-by: Braden Shepherdson <braden@metabase.com>

Co-authored-by: Mahatthana (Kelvin) Nomsawadi <me@bboykelvin.dev>

* Fix #46846

* Don't use `-v` to check if script has arg since it doesn't work on all bash versions

* Fix typo

* feat: BigQuery nested fields

Updates dataset definitions to allow `nested-fields`

Adds the `::sql.qp/nfc-path` to include nfc-path in the field
identifier.

* Add bigquery to nested-fields feature. Rework geographical-tips to allow adding to bigquery

* Parse RECORD type into maps

* Fix tests

* Fix tests

* Fix formatting

* Handle arrays of records

* Fix formatting

* Conditionally add nested-fields

* Fix test

* Update docs/developers-guide/driver-changelog.md

Co-authored-by: metamben <103100869+metamben@users.noreply.github.com>

* Update docs/developers-guide/driver-changelog.md

Co-authored-by: metamben <103100869+metamben@users.noreply.github.com>

* Update modules/drivers/bigquery-cloud-sdk/src/metabase/driver/bigquery_cloud_sdk.clj

Co-authored-by: metamben <103100869+metamben@users.noreply.github.com>

---------

Co-authored-by: metamben <103100869+metamben@users.noreply.github.com>

* Strip expression metadata for field expression

Fixes #44940.

* reorganizes data-permissions.ts to be a folder split by a file per concern, moves some logic out into other utils files where it makes sense

* pr feedback and fix lints

feat(admin/performance): In Admin / Performance, add a tab where you can manage the caching policies of dashboards and questions (#42990)

Closes #42567

* Fix binning matching

* Add tests

* Add tests

* Add tests

* typo

* Update src/metabase/lib/equality.cljc

Co-authored-by: Braden Shepherdson <braden@metabase.com>

* Fix alignment

* Fix alignment

* Update src/metabase/lib/js.cljs

Co-authored-by: Braden Shepherdson <braden@metabase.com>

---------

Co-authored-by: Braden Shepherdson <braden@metabase.com>

* move default modal padding to general modal component

* small adjustments. Using base modal where apropriate

* shame