Skip to content
Snippets Groups Projects
This project is mirrored from https://github.com/metabase/metabase. Pull mirroring updated .
  1. Oct 19, 2021
  2. Oct 18, 2021
    • Pawit Pornkitprasan's avatar
      Fix X-Ray table field shown as "null" in the title (#18066) · 2c69dc73
      Pawit Pornkitprasan authored
      Field were not normalized before being processed
      resulting in the result being `null`
      
      Fixes #15737
      Unverified
      2c69dc73
    • Pawit Pornkitprasan's avatar
      Allow caching of fonts and images (#18239) · 3db89e2a
      Pawit Pornkitprasan authored
      Webpack generate multiple resources with the name of
      "/[md4-hash].ext". We should allow those to be
      cached.
      Unverified
      3db89e2a
    • Ariya Hidayat's avatar
      Revert "Circle CI: build a complete Uberjar on a release branch (#18370)" (#18544) · 7ea4b567
      Ariya Hidayat authored
      This reverts commit adb2f715 as it broke Uberjar builds on CircleCI.
      Unverified
      7ea4b567
    • Cam Saul's avatar
    • Ariya Hidayat's avatar
      Circle CI: build a complete Uberjar on a release branch (#18370) · adb2f715
      Ariya Hidayat authored
      Make sure that the built Uberjar contains translations etc.
      Unverified
      adb2f715
    • Howon Lee's avatar
      Tools for fixing errors problems with postgres semantics of limits (blank... · 9e388032
      Howon Lee authored
      Tools for fixing errors problems with postgres semantics of limits (blank display of error table) (#18432)
      
      Previously sometimes error table blanks out in postgres because of limit semantics of postgres. Get it to not do that by whacking the limit and doing the display of latest error another way.
      Unverified
      9e388032
    • Howon Lee's avatar
      Mongo custexp fixes: group by and filters (#18403) · 5c819b89
      Howon Lee authored
      Previously mongo custexps were just columns: you couldn't have group bys with them and you couldn't have filters with them. This allows those, because those weren't really tested before. Also enabled nemanja's tests for them.
      Unverified
      5c819b89
    • Noah Moss's avatar
    • Dalton's avatar
      Use Dimensions to access Fields in Parameters code (#18431) · a48733a0
      Dalton authored
      
      * fix virtual field access when connecting parameters to dimensions
      
      * fix tests
      
      * lint fix
      
      * maybe keep things a little more backwards compatible
      
      * fix cy test
      
      * stop messing with field array ids
      
      * look for virtual fields on nested questions + tests
      
      * fix dashboard mapping test
      
      * fix cy test
      
      * add parameterToMBQLFilter tests
      
      * remove direct FIELD_REF usage
      
      * revert a few changes
      
      * Update frontend/src/metabase-lib/lib/Dimension.js
      
      Co-authored-by: default avatarGustavo Saiani <gustavo@poe.ma>
      
      Co-authored-by: default avatarGustavo Saiani <gustavo@poe.ma>
      Unverified
      a48733a0
    • Ariya Hidayat's avatar
      Custom expression editor: use lexical-based highlighting (#18482) · 11be3959
      Ariya Hidayat authored
      This reduces the memory pressure since the spans (for the
      contentEditable) will be flat (before, they construct a tree).
      Unverified
      11be3959
    • dpsutton's avatar
      Ensure we are paginating resultsets (#18477) · a33fa568
      dpsutton authored
      * Ensure we are paginating resultsets
      
      Made big tables in both pg and mysql
      
      pg:
      ```sql
      create table large_table
      (
          id         serial primary key,
          large_text text
      );
      
      insert into large_table (large_text)
      select repeat('Z', 4000)
      from generate_series(1, 500000)
      ```
      
      In mysql use the repl:
      ```clojure
      
        (jdbc/execute! (sql-jdbc.conn/db->pooled-connection-spec 5)
                       ["CREATE TABLE large_table (id int NOT NULL PRIMARY KEY AUTO_INCREMENT, foo text);"])
      
        (do
          (jdbc/insert-multi! (sql-jdbc.conn/db->pooled-connection-spec 5)
                              :large_table
                              (repeat 50000 {:foo (apply str (repeat 5000 "Z"))}))
          :done)
      
        (jdbc/execute! (sql-jdbc.conn/db->pooled-connection-spec 5)
                       ["ALTER TABLE large_table add column properties json default null"])
      
        (jdbc/execute! (sql-jdbc.conn/db->pooled-connection-spec 5)
                       ["update large_table set properties = '{\"data\":{\"cols\":null,\"native_form\":{\"query\":\"SELECT
                       `large_table`.`id` AS `id`, `large_table`.`foo` AS `foo` FROM `large_table` LIMIT
                       1\",\"params\":null},\"results_timezone\":\"UTC\",\"results_metadata\":{\"checksum\":\"0MnSKb8145UERWn18F5Uiw==\",\"columns\":[{\"semantic_type\":\"type/PK\",\"coercion_strategy\":null,\"name\":\"id\",\"field_ref\":[\"field\",200,null],\"effective_type\":\"type/Integer\",\"id\":200,\"display_name\":\"ID\",\"fingerprint\":null,\"base_type\":\"type/Integer\"},{\"semantic_type\":null,\"coercion_strategy\":null,\"name\":\"foo\",\"field_ref\":[\"field\",201,null],\"effective_type\":\"type/Text\",\"id\":201,\"display_name\":\"Foo\",\"fingerprint\":{\"global\":{\"distinct-count\":1,\"nil%\":0.0},\"type\":{\"type/Text\":{\"percent-json\":0.0,\"percent-url\":0.0,\"percent-email\":0.0,\"percent-state\":0.0,\"average-length\":500.0}}},\"base_type\":\"type/Text\"}]},\"insights\":null,\"count\":1}}'"])
      
      ```
      
      and then from the terminal client repeat this until we have 800,000 rows:
      ```sql
      insert into large_table (foo, properties) select foo, properties from large_table;
      ```
      
      Then can exercise from code with the following:
      
      ```clojure
      (-> (qp/process-query {:database 5 ; use appropriate db and tables here
                              :query {:source-table 42
                                      ;; :limit 1000000
                                      },
                              :type :query}
                              ;; don't retain any rows, purely just counting
                              ;; so resultset is what retains too many rows
                             {:rff (fn [metadata]
                                     (let [c (volatile! 0)]
                                       (fn count-rff
                                         ([]
                                          {:data metadata})
                                         ([result]
                                          (assoc-in result [:data :count] @c))
                                         ([result _row]
                                          (vswap! c inc)
                                          result))))
                              })
           :data :count)
      ```
      
      PG was far easier to blow up. Mysql took quite a bit of data.
      
      Then we just set a fetch size on the result set so that we (hopefully)
      only have than many rows in memory in the resultset at once. The
      streaming will write to the download stream as it goes.
      
      PG has one other complication in that the fetch size can only be honored
      if autoCommit is false. The reasoning seems to be that each statement is
      in a transaction and commits and to commit it has to close resultsets
      and therefore it has to realize the entire resultset otherwise you would
      only get the initial page if any.
      
      * Set default fetch size to 500
      
      ;; Long queries on gcloud pg
      ;; limit 10,000
      ;; fetch size | t1   | t2   | t3
      ;; -------------------------------
      ;; 100        | 6030 | 8804 | 5986
      ;; 500        | 1537 | 1535 | 1494
      ;; 1000       | 1714 | 1802 | 1611
      ;; 3000       | 1644 | 1595 | 2044
      
      ;; limit 30,000
      ;; fetch size | t1    | t2    | t3
      ;; -------------------------------
      ;; 100        | 17341 | 15991 | 16061
      ;; 500        | 4112  | 4182  | 4851
      ;; 1000       | 5075  | 4546  | 4284
      ;; 3000       | 5405  | 5055  | 4745
      
      * Only set fetch size if not default (0)
      
      Details of `:additional-options "defaultRowFetchSize=3000"` can set a
      default fetch size and we can easily honor that. This allows overriding
      per db without much work on our part.
      
      * Remove redshift custom fetch size code
      
      This removes the automatic insertion of a defaultRowFetchSize=5000 on
      redshift dbs. Now we always set this to 500 in the sql-jdbc statement
      and prepared statement fields. And we also allow custom ones to persist
      over our default of 500.
      
      One additional benefit of removing this is that it always included the
      option even if a user added ?defaultRowFetchSize=300 themselves so this
      should actually give more control to our users.
      
      Profiling quickly on selecting 79,000 rows from redshift, there
      essentially no difference between a fetch size of 500 (the default) and
      5000 (the old redshift default); both were 12442 ms or so.
      
      * unused require of settings in redshift tests
      
      * Appease the linter
      
      * Unnecessary redshift connection details tests
      Unverified
      a33fa568
    • Ariya Hidayat's avatar
      Unskip the repro for 14776 (#18525) · 76a35846
      Ariya Hidayat authored
      The issue has been fixed in the past with PR 15839.
      Unverified
      76a35846
    • Alexander Lesnenko's avatar
      Fix missing card descriptions (#18501) · b1a74fff
      Alexander Lesnenko authored
      * fix missing description info icon on dashboard cards
      
      * update test
      Unverified
      b1a74fff
    • Alexander Polyankin's avatar
    • Anton Kulyk's avatar
      Friendly revision history messages (#17858) · 1018c5fc
      Anton Kulyk authored
      * Move revision helpers to own directory
      
      * Add simple utility to format revision messages
      
      * Add messages for basic dashboard cards changes
      
      * Handle null values for dashboard card actions
      
      * Add basic card series revision message support
      
      * Batch multiple changes in a single revision
      
      * Add `isValidRevision` helper
      
      * Return title and description instead a single string
      
      * Filter out unknown fields in revisions
      
      * Fix viz settings revision descriptions
      
      * Add helpers to revisions unit tests
      
      * Use new revision messages util
      
      * Filter out invalid or unknown revisions
      
      * Capitalize revision descriptions
      
      * Wrap new item name with double-quotes
      
      * Move revisions unit tests to source code directory
      
      * Add basic HistoryModal tests
      
      * Add getChangedFields helper
      
      * Revert getRevisionMessage return type back to str
      
      * Extend isValidRevision check
      
      * Fix getRevisionEventsForTimeline work with updated helper
      
      * Expsoe revision utils
      
      * Use new messages in HistoryModal
      
      * Remove getRevisionDescription function
      
      * Handle cases when revision's after / before state is null
      
      * Simplify getRevisionMessage
      
      * Use "description" instead of "message"
      
      * Fix dataset_query revision not parsed correctly
      
      * Filter out unknown field change types
      
      * Support collection_id change event
      
      * Return array of changes instead of batching in a single message
      
      * Return JSX from getRevisionEventsForTimeline
      
      * Fix UI
      
      * Remove console.log
      
      * Use "rearranged the cards" message
      
      * Fix e2e test using old revision messages
      
      * Prefer 'after' state to get changed fields
      
      * Fix timeline revision event
      
      * Fix translations
      
      * Add `key` prop to `jt`
      
      * Merge revision files
      
      * Add an option not to lowercase the capitalize str
      
      * Use updated capitalize function
      
      * Fix test string
      
      * Display question's "display" change messages
      
      * [ci nocache]
      
      * Fix tests
      
      * [ci nocache]
      Unverified
      1018c5fc
    • Anton Kulyk's avatar
      Add custom react-testing-library render function (#18353) · 01f442a8
      Anton Kulyk authored
      * Add custom @testing-library/react render wrapper
      
      * Migrate unit tests to custom render function
      
      * Rename helper to `renderWithProviders`
      
      * Remove irrelevant eslint rule disable
      Unverified
      01f442a8
  3. Oct 17, 2021
  4. Oct 16, 2021
  5. Oct 15, 2021
    • Jeff Evans's avatar
      Fix flaky `rotate-encryption-key!-test` (#18371) · 82b3147c
      Jeff Evans authored
      Fix flaky `rotate-encryption-key!-test`
      
      Bind a temporary site locale when running the test, and disable the settings cache
      
      Now, a full explanation of what was making the test flaky, for posterity:
      
      When this test runs, it first tries to create a blank app DB.  This is either postgres, h2, or mysql, depending on the driver under test, which is slightly confusing, but anyway...
      
      In order to populate all the app DB tables (since the "real" app DB has already been initialized at this point to even run the test), it first rebinds the appropriate dynamic vars for the db type, spec, connection, etc. and then loads a test fixture H2 file into the app DB.  This h2 test fixture file, evidently, contains all the current app DB tables, etc.
      
      When initializing the app DB from H2 (in `metabase.cmd.copy/copy!`), the code needs to internationalize certain loading message strings.  And in order to internationalize messages, it has to look up the current site locale, which is itself an application setting, which of course comes from the setting entity table.
      
      For the "regular" blank app DB scenario (i.e. when Metabase first starts with a blank app DB), there is some code in the `metabase.util.i18n.impl` namespace to handle this chicken and egg problem (basically, defaulting to "en" if it can't be loaded from the app DB because there is no app DB yet).  But after this tricky process finishes, that temporary, hacked lookup is replaced by the real one (since now, the app DB exists and the locale can just be loaded normally).
      
      For the purpose of our test, that is the problem.  That state (which swaps out the `site-locale-from-setting` fn) has already run (remember, we have already initialized the app DB to even run this test in the first place, and now we are swapping in a temporary one).  So the call to load the site locate from the settings table (which is needed to print the loading message to initialize this temp app DB) fails, and hence the test fails.
      
      Now, the reason this test was flaky, instead of just always failing, is because of the settings cache.  If the site locale had been loaded anywhere within some short time frame before this test needs to load it, to print init messages, then it succeeds!  But that doesn't always end up happening, of course, since it's effectively a race condition (setting cache expiration versus test execution timing).
      Unverified
      82b3147c
    • Howon Lee's avatar
      Make EE audit app frontend display error if queries fail · ff6a56bd
      Howon Lee authored
      Currently it only displays JS errors. Now, it still displays JS errors first but if there's a query error and no JS error it'll display the query error
      Unverified
      ff6a56bd
    • Alexander Lesnenko's avatar
      Revert removing ci commands (#18493) · 60bc250f
      Alexander Lesnenko authored
      Unverified
      60bc250f
    • Dalton's avatar
      refactor getMappingsByParameter function in metabase/meta/Dashboard.js (#18340) · 10de89ec
      Dalton authored
      * refactor getMappingsByParameter
      
      * handle dashcard.series and dashcard.parameter_mappings being undefined
      
      * pull out getMapping fn from getMappings
      
      * rmv hasDisjointValueSets code + ui warning
      Unverified
      10de89ec
    • Nemanja Glumac's avatar
      #18458 Repro: Caching does not respect user defined settings (#18479) · 75663e94
      Nemanja Glumac authored
      * Expand `openNativeEditor` helper for use with multiple databases
      Unverified
      75663e94
    • Alexander Lesnenko's avatar
    • Ariya Hidayat's avatar
    • Nemanja Glumac's avatar
    • Nemanja Glumac's avatar
  6. Oct 14, 2021
Loading