Skip to content
Snippets Groups Projects
Unverified Commit 998b28cd authored by adam-james's avatar adam-james Committed by GitHub
Browse files

Disable Pivot Export Post Processing Step (#44561)

This PR disables the 'make exports look pivoted' post processing step for csv and xlsx exports.

This is in response to a reported issue that Large Pivot Table downloads are failing entirely: #44556

A more hollistic fix and improvement to the post processing feature will be worked on, so the post processing code is
left untouched in this PR.

The dynamic vars in the csv and xlsx namespaces each default `false` and disable the export from running any post
processing. The var should remain `false` until we've got a plan for these types of exports, and there is no way for
users (either through ENV or UI) to change this, so the feature is effectively disabled.
parent 7a832787
No related branches found
No related tags found
No related merge requests found
......@@ -25,6 +25,16 @@
(u.date/format (t/zoned-date-time)))}
:write-keepalive-newlines? false}))
;; As a first step towards hollistically solving this issue: https://github.com/metabase/metabase/issues/44556
;; (which is basically that very large pivot tables can crash the export process),
;; The post processing is disabled completely.
;; This should remain `false` until it's fixed
;; TODO: rework this post-processing once there's a clear way in app to enable/disable it, or to select alternate download options
(def ^:dynamic *pivot-export-post-processing-enabled*
"Flag to enable/disable export post-processing of pivot tables.
Disabled by default and should remain disabled until Issue #44556 is resolved and a clear plan is made."
false)
(defmethod qp.si/streaming-results-writer :csv
[_ ^OutputStream os]
(let [writer (BufferedWriter. (OutputStreamWriter. os StandardCharsets/UTF_8))
......@@ -34,7 +44,7 @@
(reify qp.si/StreamingResultsWriter
(begin! [_ {{:keys [ordered-cols results_timezone format-rows? pivot-export-options]
:or {format-rows? true}} :data} viz-settings]
(let [opts (when pivot-export-options
(let [opts (when (and *pivot-export-post-processing-enabled* pivot-export-options)
(assoc pivot-export-options :column-titles (mapv :display_name ordered-cols)))
;; col-names are created later when exporting a pivot table, so only create them if there are no pivot options
col-names (when-not opts (common/column-titles ordered-cols (::mb.viz/column-settings viz-settings) format-rows?))]
......
......@@ -555,6 +555,16 @@
(.addColumnLabel pivot-table (get aggregation-functions idx DataConsolidateFunction/COUNT) idx))
wb))
;; As a first step towards hollistically solving this issue: https://github.com/metabase/metabase/issues/44556
;; (which is basically that very large pivot tables can crash the export process),
;; The post processing is disabled completely.
;; This should remain `false` until it's fixed
;; TODO: rework this post-processing once there's a clear way in app to enable/disable it, or to select alternate download options
(def ^:dynamic *pivot-export-post-processing-enabled*
"Flag to enable/disable export post-processing of pivot tables.
Disabled by default and should remain disabled until Issue #44556 is resolved and a clear plan is made."
false)
(defmethod qp.si/streaming-results-writer :xlsx
[_ ^OutputStream os]
(let [workbook (SXSSFWorkbook.)
......@@ -566,7 +576,7 @@
(reify qp.si/StreamingResultsWriter
(begin! [_ {{:keys [ordered-cols format-rows? pivot-export-options]} :data}
{col-settings ::mb.viz/column-settings :as viz-settings}]
(let [opts (when pivot-export-options
(let [opts (when (and *pivot-export-post-processing-enabled* pivot-export-options)
(pivot-opts->pivot-spec (merge {:pivot-cols []
:pivot-rows []}
pivot-export-options) ordered-cols))
......
This diff is collapsed.
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment