* [xlsx-export] Cache the custom datetime formatter

* [xlsx-export] Optimize functions in m.q_p.pivot.postprocess

* cljfmt

---------

Co-authored-by: Adam James <adam.vermeer2@gmail.com>

* Add temporal unit to display name

* Remove temporal unit adjustments from FE

* formatColumn to displayNameForColumn

* Factor out getFriendlyName

* Adjust BE tests

* Add require

* Adjust FE unit tests

* Adjust temporal-unit-parameters.cy.spec.js

* Adjust dashboard-reproductions.cy.spec

* Convert formatColumn occurrence

* Adjust legacy display name logic generation

After `getFriendlyName` was removed from FE, capitalization
of display name should happen here. Logic is similar, should
be adjusted prior merging the PR.

* Adjust embedding-linked-filters.cy.spec.js

* Adjust joins-reproductions.cy.spec.js

* Adjust metrics-editing.cy.spec.js

* Adjust timelines-question.cy.spec.js

* Adjust offset.cy.spec.ts

* TMP Avoid duplication of temporal unit in display name in MLv2 code

* Probably FE fix: add default temporal bucket only if not present

* WIP Add ends-with-temporal-unit? function

Current strategy is to add temporal units to display names if same is not already present

* Adjust display-name-method :metadata/column

Append TU to display name only if different to what's already present

* Adjust FE: show temporal unit that's present on field instead of default on hover on QueryColumnPicker element

* Return also temporal-unit from visible-columns

* Enable tempora-unit to be returned from previous-stage-or-source-visible-columns

* WIP Adjust multi breakout multi stage e2e multiple-column-breakouts.cy.spec

* Adjust multiple-column-breakouts.cy.spec.ts

* It was not a bug.

* Revert changes in _BaseBucketPickerPopover

* Restore temporal-unit for next stages removal in stage.cljc

* Add inherent-temporal-unit

* Adjust display name temporal bucket logic

* Rename to inherited-temporal-unit

* Shutdown too broad display name capitalization

* Eastwood

* Adjust pivot-rows-order-test

* Adjust pivot-non-numeric-values-in-aggregations

* Adjust inherent to inherited

* Rename to inherited

* Add inherent-temporal-unit only to returned-columns

* Update e2e/test/scenarios/dashboard-filters/temporal-unit-parameters.cy.spec.js

* Update e2e/test/scenarios/question/multiple-column-breakouts.cy.spec.ts

* Update consistent-date-formatting-test

* Update dashboard/x-rays.cy.spec.js

* Update waterfall.cy.spec.js

* Return month default for default inherited-temporal-unit

* Update reproductions-3.cy.spec.js

* Update downloads/downloads.cy.spec.js

* Update legend.cy.spec.js

* Update pie_chart.cy.spec.js

* Update click-behavior.cy.spec.js

* Update embedding-linked-filters.cy.spec.js

* Return QueryColumnPicker's condition to original state

* Remove reundant export

* Update e2e/test/scenarios/question/multiple-column-breakouts.cy.spec.ts

* Comment

* Update nested.cy.spec.js

* Update e2e/test/scenarios/sharing/sharing-reproductions.cy.spec.js

* Adjust to TS types

* Adjust for cljfmt

* Capitalize

* Add propagation tests

* Add inherited-temporal-unit propagation for expressions

* Add default inherited-temporal-unit test cases

* Add qp temporal-bucket-test ns

* Simplify default unit logic

* Comment

* Use NULL_DISPLAY_VALUE

* Comment

* Comment

* Comment

* Comment

* Comment

* Comment

* Remove redundant code (for now)

* Update comments

* Comment

* Restore way too daring simplification

* cljfmt

* Adjust capitalize

* Update comment

* Update test

* Address remarks in tests

* Update test

* Address remarks

* typo

* fix: Sync postgres materialized view fields

Fixes: #49912

* Disable test for redshift

* Fix formatting

* XLSX Pivot Export - Initialize Pivot Table with Small Area Ref

This PR changes the native pivot export so that we first initialize the pivot table with an area ref that's only the
first 2 rows. Prior, I used `AreaReference/GetWholecolumn` which had the side effect of using lots of memory.

I think this is related to how the Pivot Table's Cache is built up as you add rows/cols to the pivot definition. So,
if you keep the area ref only as wide as the number of columns in the raw data, and just 2 rows, then the cache can
stay small.

After adding row/column data to the pivot table, you can then modify the area ref to be all rows in the relevant
columns (`A:E` for example).

This keeps the pivot-table object much smaller in size.

* need the extra xml schemas to get this to work

I'd like to try find a way around including this extra dep, it's a 13mb jar (ish), and I think that's a little bigger
than I'd like

* oops, didnt mean to have this in

* Add test for pivot table initialization being fast and lean on memory

* remove time based assertion, add comments in impl

* cljfmt

* fix test

* Pesky little whitespace

---------

Co-authored-by: Oleksandr Yakushev <alex@bytopia.org>

Make sure the query to fetch result_metadata for native sandboxes doesn't use the streaming results handler (#50049)

* XLSX Pivot Exports Now Use The Sort Settings from Viz Settings

Prior to this PR, the xlsx pivot exports worked correctly except for the fact that no sorting (ascending or
descending) was applied to the pivot rows or columns. Now, those settings are used when they exist.

* only try to set sort when a correct setting exists

* add ooxml-full dep for correct classes

* Update src/metabase/query_processor/streaming/xlsx.clj

Co-authored-by: Noah Moss <32746338+noahmoss@users.noreply.github.com>

---------

Co-authored-by: Noah Moss <32746338+noahmoss@users.noreply.github.com>

Fixes: #50072

When we `describe-fields` we are passing db details, but sometimes we
have the database name in `dbname` rather than in `db` depending on the
connection specs and possibly environment. So we check for both now.

== Goal ==

Hide attached DWH database details from anyone incl. admins:
* Do not show them in the UI
* Do not permit to change them
* Do not serialize them
  - This is what we're adding for H2 snapshots (dumps) with this PR.

The aim is that customers cannot gain access to (parts of) credentials,
and they cannot break a feature they are paying for by changing
connection details.

== Implementation ==

In 592360c9 I wrongly understood that
database details would be omitted when dumping to H2, but this is only
true for dumping H2 databases.

Fix this by omitting database details also when dumping databases with
`is_attached_dwh` set.

== How to test ==

To prepare, download a H2 JAR matching the H2 version used by Metabase:
```
wget https://repo1.maven.org/maven2/com/h2database/h2/2.1.214/h2-2.1.214.jar
```

If the H2 version does not match, you will get an error when trying to
open the H2 shell:
```
Exception in thread "main" org.h2.jdbc.JdbcSQLNonTransientConnectionException: Unsupported database file version or invalid file header in file "[REDACTED]" [90048-232]
```

=== New behaviour ===

Setting the `is_attached_dwh` field hides the database details from H2 dumps:

1. Configure a database as described in https://www.metabase.com/docs/latest/configuring-metabase/config-file#databases.
   - In addition to the fields you would normally set, also set
     `is_attached_dwh: true`.
   - This also works when adding this flag to a database that previously
     did not have this flag set.
2. Start your Metabase instance.
3. Run `${metabase} dump-to-h2 ./dump-file-h2` to create the H2 snapshot file.
4. Run `java -cp h2-2.1.214.jar org.h2.tools.Shell -url "jdbc:h2:file:./dump-file-h2;ifexists=true"` to open a H2 shell.
5. Verify that `SELECT name,details FROM metabase_database;` shows `{}` for the database you added in step 1

=== Original behaviour ===

Behaviour without setting the `is_attached_dwh` field is unchanged:

1. Configure a database as described in https://www.metabase.com/docs/latest/configuring-metabase/config-file#databases.
   - Only set the fields you would normally set.  Do not set
     `is_attached_dwh` (or set it to `false`).
2. Start your Metabase instance.
3. Run `${metabase} dump-to-h2 ./dump-file-h2` to create the H2 snapshot file.
4. Run `java -cp h2-2.1.214.jar org.h2.tools.Shell -url "jdbc:h2:file:./dump-file-h2;ifexists=true"` to open a H2 shell.
5. Verify that `SELECT name,details FROM metabase_database;` shows a non-empty object (i.e. not `{}`) for the database you added in step 1

Fixes: 592360c9
Closes: https://github.com/metabase/harbormaster/issues/5526

* sync the attached datawarehouse

requires either "sync_db" true or a table name

it will error
```shell
❯ http post 'localhost:3000/api/notify/db/attached_datawarehouse' x-metabase-apikey:apikey -pb
Must provide `sync_db` or a `table_name` to sync
```

It will sync the db when sync_db is true [NOTE: this is outdated. now just omitting the table name will sync the db]
```shell
❯ echo '{"synchronous?": true, "sync_db": true}' | http post 'localhost:3000/api/notify/db/attached_datawarehouse' x-metabase-apikey:apikey -pb
{
    "success": true
}
```

It will sync existing tables
```shell
❯ http post 'localhost:3000/api/notify/db/attached_datawarehouse' table_name=existing schema_name=public x-metabase-apikey:apikey -pb
{
    "success": true
}
```

it will error if it cannot find a table
```shell
❯ http post 'localhost:3000/api/notify/db/attached_datawarehouse' table_name=new schema_name=public x-metabase-apikey:apikey -pb
{
    "cause": "Unable to identify table 'public.new'",
    "data": {
        "schema_name": "public",
        "status-code": 404,
        "table_name": "new"
    },
    "message": "Unable to identify table 'public.new'",
    "schema_name": "public",
    "table_name": "new",
    "trace": [],
    "via": [
        {
            "data": {
                "schema_name": "public",
                "status-code": 404,
                "table_name": "new"
            },
            "message": "Unable to identify table 'public.new'",
            "type": "clojure.lang.ExceptionInfo"
        }
    ]
}
```

if i create that table
```sql
attached=# create table new (id int);
CREATE TABLE
```

it will then find and sync it

```shell
❯ http post 'localhost:3000/api/notify/db/attached_datawarehouse' table_name=new schema_name=public x-metabase-apikey:apikey -pb
{
    "success": true
}
```

* formatting

* Tests for attached datawarehouse

* PR nits

typo in `thorw` -> `throw`
comment explaining why `find-and-sync-new-table` is always sync
gensym the db name in test
remove a binding for a fn `sync!` that's only called once
test `{:sync_db true}` syncs the whole db

* stupid formatting 

* Use absence of table_name as indicator to sync database

* Apply Column Sort To Pivot Sections

Fixes #49437

This PR fixes the dataset API so that column sorts (ascending/descending settings on pivot-rows and pivot-cols) are
properly parsed and passed through the QP.

After that, I've also modified the post processor to use the sort orders properly in pivoted exports.

* println debugging :)

* fixing a few tests

* fix a few more tests

* Try snowflake 3.19.0

* Deafault to java 21 for drivers

* Trigger CI

* Try running things with --add-opens to see if it solves our problems

* Fix `update-view-dashboard-timestamp-test`

* Try snowflake 3.19.0

* Deafault to java 21 for drivers

* Try running things with --add-opens to see if it solves our problems

* Fix `update-view-dashboard-timestamp-test`

* Update deps.edn

* Switch to Java 21

* Docker image needs to use --add-opens option

* Add note about change to Java 21 to driver changelog

---------

Co-authored-by: Nemanja <31325167+nemanjaglumac@users.noreply.github.com>
Co-authored-by: Luis Paolini <paoliniluis@gmail.com>

* Fixes https://github.com/metabase/metabase/issues/33037

* Align query and fix the linter

* Add blurb from C3P0 docs to comment for unreturnedConnectionTimeout

* Enable debugUnreturnedConnectionStackTraces in data-warehouse-connection-pool-properties

Closes #47981

* Add defsetting for jdbc-data-warehouse-debug-unreturned-connection-stack-traces

And only enable debugUnreturnedConnectionStackTraces in the C3P0 config if the setting is enabled. There is some
memory cost to enabling the setting, and since maxPoolSize is user-configurable, it seems safer to disable
debugUnreturnedConnectionStackTraces by default.

* Add metabase.logger/level-enabled?

* Log a warning if C3P0 logs are not configured correctly to log stack traces

Log a warning if the user enables the jdbc-data-warehouse-debug-unreturned-connection-stack-traces but does not raise
the log level for the com.mchange package to INFO or higher.

* Add docs for MB_JDBC_DATA_WAREHOUSE_DEBUG_UNRETURNED_CONNECTION_STACK_TRACES

* Improve markdown docs for MB_JDBC_DATA_WAREHOUSE_DEBUG_UNRETURNED_CONNECTION_STACK_TRACES

Co-authored-by: Jeff Bruemmer <jeff.bruemmer@gmail.com>

---------

Co-authored-by: Jeff Bruemmer <jeff.bruemmer@gmail.com>

* Keep Pivoted Option works in Subscriptions

Fixes: #49525

Even though the user could toggle 'Keep data pivoted` in the Sharing Sidebar UI, the setting was not being propagated
to the backend, so attachments would not be pivoted.

* add a cypress test

* add viz settings to get the pivot working

* fix for linter

* remove eslint ignore

---------

Co-authored-by: Aleksandr Lesnenko <alxnddr@gmail.com>

* Revert "[streaming-response] Use HttpEndpoint.isOpen() instead of reading a byte from TCP socket (#49460)"

This reverts commit 060baf0c.

* Disable connection reuse for requests that serve streaming responses