Skip to content
Snippets Groups Projects
This project is mirrored from https://github.com/metabase/metabase. Pull mirroring updated .
  1. May 17, 2022
    • Bryan Maass's avatar
      bumps outdated deps versions to be current + drop support for java 8 (#22567) · c1b73ed6
      Bryan Maass authored
      * bumps outdated deps versions to be current
      
      * un-upgrade h2 and jetty
      
      * un-upgrade joda-time and kixi/stats
      
      * drop Java 8 support in circle CI config
      
      - things that used to rely on be-tests-java-8-ee now rely on be-tests-java-11-ee
      
      * remove java 8 from github health check matrix
      
      * revert toucan to 1.17.0
      
      * revert mariadb java client to 2.7.5
      
      * Back to 18, and handle new behavior
      
      toucan used to just look in *.models.<model-name> for models and just
      give up apparently. I made a feature that toucan will look in a model
      registry to create models rather than using the convention
      https://github.com/metabase/toucan/commit/762ad69defc1477423fa9423e9320ed318f7cfe7
      
      
      but now we're getting errors in these tests about maps vs models.
      
      ```clojure
      revision_test.clj:154
      Check that revisions+details pulls in user info and adds description
      expected: [#metabase.models.revision.RevisionInstance{:is_reversion false,
                                                            :is_creation false,
                                                            :message nil,
                                                            :user
                                                            {:id 1,
                                                             :common_name "Rasta Toucan",
                                                             :first_name "Rasta",
                                                             :last_name "Toucan"},
                                                            :diff
                                                            {:o1 nil, :o2 {:name "Tips Created by Day", :serialized true}},
                                                            :description nil}]
        actual: (#metabase.models.revision.RevisionInstance{:description nil,
                                                            :is_creation false,
                                                            :is_reversion false,
                                                            :user
                                                            {:id 1,
                                                             :first_name "Rasta",
                                                             :last_name "Toucan",
                                                             :common_name "Rasta Toucan"},
                                                            :message nil,
                                                            :diff
                                                            {:o1 nil,
                                                             :o2
                                                             #metabase.models.revision_test.FakedCardInstance{:name
                                                                                                              "Tips Created by Day",
                                                                                                              :serialized
                                                                                                              true}}})
      ```
      
      The only difference here is `:o2` is a
      `metabase.models.revision_test.FakedCardInstance` but still has the same
      keys, `:name`, and `:serialized`. So all is well, we're just able to
      make the model.
      
      So a few different fixes. Some are use `partial=` which doesn't care
      about record/map distinction. Some are just make the model, and some are
      turning them into maps for revision strings (which more closely mimics
      what the real revision stuff does):
      
      ```clojure
      (defn default-diff-map
        "Default implementation of `diff-map` which simply uses clojures `data/diff` function and sets the keys `:before` and `:after`."
        [_ o1 o2]
        (when o1
          (let [[before after] (data/diff o1 o2)]
            {:before before
             :after  after})))
      
      (defn default-diff-str
        "Default implementation of `diff-str` which simply uses clojures `data/diff` function and passes that on to `diff-string`."
        [entity o1 o2]
        (when-let [[before after] (data/diff o1 o2)]
          (diff-string (:name entity) before after)))
      ```
      
      So all in all this change impacts nothing in the app itself, because
      those models follow convention and are correct in
      `metabase.models.<model-name>` and are thus "modelified":
      
      ```clojure
      revision-test=> (revision/revisions Card 1)
      [#metabase.models.revision.RevisionInstance{:is_creation true,
                                                  :model_id 1,
                                                  :id 1,
                                                  :is_reversion false,
                                                  :user_id 2,
                                                  :timestamp #object[java.time.OffsetDateTime
                                                                     "0x77e037f"
                                                                     "2021-10-28T15:10:19.828539Z"],
                                                  :object #metabase.models.card.CardInstance
                                                  {:description nil,
                                                   :archived false,
                                                   :collection_position nil,
                                                   :table_id 5,
                                                   :database_id 2,
                                                   :enable_embedding false,
                                                   :collection_id nil,
                                                   :query_type :query,
                                                   :name "ECVYUHSWQJYMSOCIFHQC",
                                                   :creator_id 2,
                                                   :made_public_by_id nil,
                                                   :embedding_params nil,
                                                   :cache_ttl 1234,
                                                   :dataset_query {:database 2,
                                                                   :type :query,
                                                                   :query {:source-table 5,
                                                                           :aggregation [[:count]]}},
                                                   :id 1,
                                                   :display :scalar,
                                                   :visualization_settings {:global {:title nil}},
                                                   :dataset false,
                                                   :public_uuid nil},
                                                  :message nil,
                                                  :model "Card"}]
      ```
      
      so the model/no-model is just arbitrary distinction in the test. All of
      them in the actual app are turned into models:
      
      ```clojure
      (defn- do-post-select-for-object
        "Call the appropriate `post-select` methods (including the type functions) on the `:object` this Revision recorded.
        This is important for things like Card revisions, where the `:dataset_query` property needs to be normalized when
        coming out of the DB."
        [{:keys [model], :as revision}]
        ;; in some cases (such as tests) we have 'fake' models that cannot be resolved normally; don't fail entirely in
        ;; those cases
        (let [model (u/ignore-exceptions (db/resolve-model (symbol model)))]
          (cond-> revision
          ;; this line would not find a model previously for FakedCard and
          ;; just return the map. But now the registry in toucan _finds_ the
          ;; model defintion and returns the model'd map
            model (update :object (partial models/do-post-select model)))))
      
      (u/strict-extend (class Revision)
        models/IModel
        (merge models/IModelDefaults
               {:types       (constantly {:object :json})
                :pre-insert  pre-insert
                :pre-update  (fn [& _] (throw (Exception. (tru "You cannot update a Revision!"))))
                :post-select do-post-select-for-object}))
      ```
      
      * try using mssql-jdbc 10.2.1.jre11
      
      - Important that we get off the jre8 version
      
      * various fixes that needn't be reverted
      
      * Revert "various fixes that needn't be reverted"
      
      This reverts commit 2a820db0743d0062eff63366ebe7bc78b852e81f.
      
      * go back to using circle ci's java 11 docker image
      
      * java-16 (?) -> java-17
      
      * Revert "go back to using circle ci's java 11 docker image"
      
      This reverts commit b9b14c535a689f701d7e2541081164288c988c4e.
      
      Co-authored-by: default avatardan sutton <dan@dpsutton.com>
      c1b73ed6
    • adam-james's avatar
      Whitelabelling users font family font setting (#22791) · 856c46e2
      adam-james authored
      
      * Add font utilities to derive list of available fonts from font dirs
      
      This PR adds two defsettings to facilitate Enterprise users' ability to set the font for their instance. The
      'available-fonts' can be grabbed by the frontend to populate the dropdown menu. The 'application-font' setting can be
      written by the frontend to save the chosen font.
      
      * Adding some fonts and font selector
      
      * Send  (("dirname", "font name"),) to frontend
      
      Instead of only sending a list of names, send a tuple of `[directory-name, Display Name]`
      
      * Adding half of the fonts
      
      * Font list derived from font directory/font file names
      
      This is an attempt to derive the 'Display Name' of a font by comparing the directory name to the file names. '_' is
      replaced with spaces, but '-' exists in the 'Lato' font directory name, and we may have naming confusion in the
      future. Basically, I want to be liberal with what font directory/file names we can receive.
      
      Not certain this is the right approach yet, but it will work and correctly provides names based on the fonts available.
      
      * Adding remaining fonts
      
      * Using Default Font family in visual components
      
      * Small change to fix some lint warnings
      
      * Removing Noto Sans JP
      
      * Move logging
      
      * Alter hardcoded font path
      
      * Added cypress test
      
      * Change Cypress Test to look for correct text
      
      * Added unit tests for fonts.clj, and simplified creating display name
      
      The display name approach is simple -> underscores become spaces, and names are split on dashes/first word is taken
      from the split.
      
      This might not be robust to future font installations, but will be a good v1
      
      * adding visual test
      
      * fixing cypress test
      
      * Actually add the unit tests
      
      * Fix lint problems
      
      * Add setter to application-font to prevent invalid font setting
      
      * Addressing PR review points
      
      - fixed 'normalize-font-dirname' description
      - 'contains-font-file?' returns true/false
      - 'available-fonts-test' now properly uses 'is'
      
      * Make font-path private
      
      * Lint error.
      
      * Modified fonts to use java.io/resource to hopefully work in a jar
      
      * Simplified Lato folder name
      
      * PR Cleanup
      
      * Address some backend feedback
      
      * Lint error from reflection with .toString on a path
      
      I removed the explicit .toString inside the lamdba functino and str/includes still works.
      
      * no longer need u.files in test ns
      
      Co-authored-by: default avatarNick Fitzpatrick <nick@metabase.com>
      856c46e2
    • Maz Ameli's avatar
      tag strings in time series chrome (#22797) · f66e0a8c
      Maz Ameli authored
      f66e0a8c
    • Case Nelson's avatar
      Add extra logging for persisted models (#22784) · bf5ca940
      Case Nelson authored
      Add inidividual logs for each model.
      Add final log that includes successes and errors.
      bf5ca940
    • metamben's avatar
      Prevent do-with-timeout from throwing Throwable return values (#22728) · 3521de88
      metamben authored
      Since the documentation of do-with-timeout doesn't mention that return
      values that are instances of Throwable will be thrown instead of being
      returned as other values, it's better not to do this.
      3521de88
    • Bryan Maass's avatar
      Revert backend api actions (#22755) · 3acbe649
      Bryan Maass authored
      * Revert "Revert new precondition"
      
      This reverts commit 1f97a4d6ccfdaa92ff82fd0256f08f3512c3c5e7.
      
      * Revert "plug in and alter tests to use with-actions-test-data"
      
      This reverts commit 2f33cb47b700b1da106adf54d41928c5956978d0.
      
      * Revert "Add Database-local Setting for enabling Actions for a specific Database"
      
      This reverts commit a52f8d0a478a31bb1f1b4a6693abf8f8e658d9cf.
      
      * Revert "Make sure api.actions is loaded"
      
      This reverts commit 7d7b913a7a0d64cbe2b6907f389d994a1130c0f5.
      
      * Revert "Add `experimental-enable-actions` feature flag (#22559)"
      
      This reverts commit 9b313752.
      
      * Revert "SQL JDBC Delete Row Action (#22608)"
      
      This reverts commit d6864f38.
      
      * Remove actions api docs
      
      * Revert "Destructible version of `test-data` dataset for testing writeback actions (#22692)"
      
      This reverts commit 15c57d9a.
      
      * keep around a few misc improvements
      
      * removes precondition
      
      - {:pre [(nil? (namespace symb))]}
      
      * various fixes that needn't be reverted
      3acbe649
    • Alexander Polyankin's avatar
    • Alexander Lesnenko's avatar
      094b29be
    • Howon Lee's avatar
  2. May 16, 2022
    • Anton Kulyk's avatar
      Revert writeback FE changes (#22759) · ca8456f7
      Anton Kulyk authored
      * Revert writeback changes
      
      * Almost missed that
      ca8456f7
    • Case Nelson's avatar
      Persist auto enable (#22756) · c5a31c48
      Case Nelson authored
      
      * Auto enable persistence for models
      
      When persistence is turned on for a db, we want to enable persistence
      caching for all models in the db.
      
      We do this by finding any models without a PersistentInfo at the top of
      the scheduled refresh task and creating one that will get picked up by
      the refresh.
      
      This necessitated introducing another "off" state on PersistedInfo that
      will get set from the front end, manually disabling persistence on a
      model.  This turns PersistedInfo into a marker so that when the refresh
      task runs again, these models will not be turned back on.
      
      The prune job will prune "off" or "deletable" PersistedInfo. Since we don't
      have a second "off-ing" state; the prune job will "drop if exists" the
      cache table each time. This may need to change.
      
      * Cherry-pick persist-refresh changes from persist-refresh-fail-email
      
      * Ready models when enabling persistence on db
      
      * Handle automatic model persistence in Tools table
      
      * Address review: insert-many instead of  doseq insert
      
      Co-authored-by: default avatarAnton Kulyk <kuliks.anton@gmail.com>
      c5a31c48
    • Case Nelson's avatar
      Persist model statement_timeout (#22690) · 9b949bc6
      Case Nelson authored
      * Persist model statement_timeout
      
      In order to safeguard against unknowingly or accidentally large models
      being persisted, we can leverage the postgres `statement_timeout`
      setting.
      
      This way, if a particular statement (likely `create table as`) takes too
      long to run, the database will kill it for us and reclaim resources.
      
      The timeout will be the lower of 10 minutes and an existing
      statement_timeout. The thinking here is that if an admin explicitly
      limitted metabase's statement runtime we shouldn't exceed it.
      
      * Rename query to avoid shadow problems
      9b949bc6
    • Case Nelson's avatar
      Add remarks to persistence ddl statements (#22757) · a662e90e
      Case Nelson authored
      Simply adds `-- Metabase\n` to the front of ddl queries. Eventually this
      should turn into a multimethod like `qp.util/query->remark` but until we
      write another ddl driver this will work.
      a662e90e
    • Gustavo Saiani's avatar
      d0d66c35
    • adam-james's avatar
      Add font utilities to derive list of available fonts from font dirs (#22612) · f106ebfd
      adam-james authored
      
      * Add font utilities to derive list of available fonts from font dirs
      
      This PR adds two defsettings to facilitate Enterprise users' ability to set the font for their instance. The
      'available-fonts' can be grabbed by the frontend to populate the dropdown menu. The 'application-font' setting can be
      written by the frontend to save the chosen font.
      
      * Adding some fonts and font selector
      
      * Send  (("dirname", "font name"),) to frontend
      
      Instead of only sending a list of names, send a tuple of `[directory-name, Display Name]`
      
      * Adding half of the fonts
      
      * Font list derived from font directory/font file names
      
      This is an attempt to derive the 'Display Name' of a font by comparing the directory name to the file names. '_' is
      replaced with spaces, but '-' exists in the 'Lato' font directory name, and we may have naming confusion in the
      future. Basically, I want to be liberal with what font directory/file names we can receive.
      
      Not certain this is the right approach yet, but it will work and correctly provides names based on the fonts available.
      
      * Adding remaining fonts
      
      * Using Default Font family in visual components
      
      * Removing Noto Sans JP
      
      * Small change to fix some lint warnings
      
      * Move logging
      
      * Alter hardcoded font path
      
      * Added cypress test
      
      * Change Cypress Test to look for correct text
      
      * adding visual test
      
      * fixing cypress test
      
      * Added unit tests for fonts.clj, and simplified creating display name
      
      The display name approach is simple -> underscores become spaces, and names are split on dashes/first word is taken
      from the split.
      
      This might not be robust to future font installations, but will be a good v1
      
      * Actually add the unit tests
      
      * Fix lint problems
      
      * Add setter to application-font to prevent invalid font setting
      
      * Addressing PR review points
      
      - fixed 'normalize-font-dirname' description
      - 'contains-font-file?' returns true/false
      - 'available-fonts-test' now properly uses 'is'
      
      * Make font-path private
      
      * Lint error.
      
      * PR Cleanup
      
      Co-authored-by: default avatarNick Fitzpatrick <nick@metabase.com>
      f106ebfd
    • Dalton's avatar
      Handle missing parameter `fields` array in ParameterValueWidget (#22656) · 7eca0f26
      Dalton authored
      * Move some util fns to utils file
      
      * add tests & remove redundant getFields fn
      7eca0f26
    • Gustavo Saiani's avatar
    • Alexander Polyankin's avatar
      8983b573
    • Ariya Hidayat's avatar
      Upgrade to PostCSS 8.4 (#22670) · cb2403de
      Ariya Hidayat authored
      cb2403de
    • Noah Moss's avatar
    • Luis Paolini's avatar
      Replace deprecated CircleCI image (#19423) · 7d2d862c
      Luis Paolini authored
      * New CI image
      * Upgrade to new Clojure CLI version + multi arch images + add git to Builder images [ci nocache]
      7d2d862c
    • Ariya Hidayat's avatar
      Upgrade Underscore.js to v1.13.3 (#22705) · 665800ec
      Ariya Hidayat authored
      665800ec
    • Alexander Lesnenko's avatar
      Fix inability to save to sub-collections when having no access to the parent (#22663) · 3d829101
      Alexander Lesnenko authored
      
      * Enable saving to subcollection when access to parent is revoked (#14052)
      
      * Formatting
      
      * fix adding questions to dashboards when parent is not shared
      
      * add lodash to dependencies
      
      Co-authored-by: default avatarChris Wu <chris@faros.ai>
      3d829101
  3. May 15, 2022
  4. May 14, 2022
  5. May 13, 2022
    • Nemanja Glumac's avatar
    • Case Nelson's avatar
      Update persist model api permissions for EE (#22601) · f302d582
      Case Nelson authored
      * Update persist model api permissions for EE
      
      On OSS, only Admins can enable model cache in `Settings/Cache`,
      enable database model cache in `Settings/Databases` and cache individual models.
      
      On EE/Pro, users with `Settings access` application permissions can enable model
      cache in `Settings/Cache` and users with `Manage database` can enable model
      cache for a database.
      
      * Add tests for application permissions
      f302d582
    • Case Nelson's avatar
      Upgrade build-drivers to Clojure 1.11.1 (#22709) · 5ae6f212
      Case Nelson authored
      Addressing
      Running clj -X:deps tree in bin/build-drivers the cli version is getting picked up
      as I switched clojure-cli versions between 1.11 and 1.10.
      
      I think this is because including metabase/metabase-core as a local/root no longer
      places the clojure dep within it at the top level.
      
      Now, the docs say
      > It is a top dep (top dep versions always win) or it is a new lib or a newer version of a known lib (Maven keeps only the first found version regardless of new/old)
      
      And so I would expect to see org.clojure/clojure in the deps tree under
      metabase/metabase-core but it's missing, which to me means clojure maybe treated
      specially if the root deps don't specify it.
      5ae6f212
    • metamben's avatar
      Replace "" with nil for Postgres UUIDs (#22667) · 2f208ff1
      metamben authored
      UUIDs are treated as strings and when checking is-empty/not-empty, we compare
      them against the empty string.
      But an empty string is not a valid UUID, so this comparison is useless for
      Postgres where a UUID field cannot have such a value. Assuming that a UUID
      field is only ever compared to the empty string in the context of
      is-empty/non-empty checks, replacing it with nil is OK.
      2f208ff1
    • dpsutton's avatar
      Record startup times (#22707) · ae032bb0
      dpsutton authored
      * Record startup times
      
      updated `dev/start!`. `mb.core/init!` itself sets up the db, loads
      plugins, and sets the init status so those were superfluous.
      
      * clean up dev namespace
      
      * testing doesn't call init so set a value
      ae032bb0
    • Cam Saul's avatar
      SQL JDBC Delete Row Action (#22608) · d6864f38
      Cam Saul authored
      
      * SQL JDBC Delete Action [ci skip]
      
      * Add note about parsing values [ci skip]
      
      * transition to using mbql for actions/row/:action
      
      - TODO: how to setup a qp store?
      - TODO: better way to keywordize filters?
      
      * some cleanup [ci skip]
      
      * better message on delet statement conflicts
      
      - and a test
      
      * mt/with-everything-store -> seed store manually [ci skip]
      
      * fix some test input
      
      * get more tests passing
      
      * add more validation tests
      
      * privatize catch-throw
      
      * plug in and alter tests to use with-actions-test-data
      
      Co-authored-by: default avatarBryan Maass <bryan.maass@gmail.com>
      d6864f38
    • dpsutton's avatar
      Don't reschedule syncs on startup (#22680) · a6c1b33c
      dpsutton authored
      * Don't reschedule syncs on startup
      
      Previously when we started up, for each database we compare its current
      schedule to its schedule in Quartz, and update the triggers if they were
      different. But we handle this change on db inserts and db updates so
      this is completely unnecessary. We did this back when quartz was in
      memory only and it remained when we persisted.
      
      We can remove this complexity. On an app-db with 400 postgres databases,
      this takes 46 seconds in the old world and appears as just another log
      statement now.
      
      Reports from the wild are even more distressing: the quartz tables got
      full of dead tuples causing queries to the triggers to take a long
      time. They also had 500+ databases. We query all triggers and linear
      search for each database and this process exploded up to as much as 8
      hours. Now we are not doing all of this extra work and startup should be
      quick again.
      
      * clj-kondo ignore unused private var
      
      * appease ns linter
      
      * remove clojure.core/binding and unused bindings
      
      * Rename `default-schedules` -> `default-randomized-schedule`
      
      makes it a bit more clear what's going on
      
      * Put randomizer in transducer over reducible query; tests
      
      basically a `run!` over a reducible query that also does a count for
      logging. Tests now check the three options:
      - default schedules, updated
      - non-default schedules, not updated timestamp verified that it is
      identical.
      - default schedules but user controls schedule: not updated, timestamp
      verified that it is identical.
      
      * Prevent randomized schedules from returning a default value
      
      it actually doesn't matter in the application but the tests are
      asserting that the newly randomized schedule is not one of the defaults
      it was changed from and that can fail sometimes. Doesn't matter
      logically but no biggie to simplify the tests.
      
      Otherwise they fail in (1/60)+(1/24) test runs which is actually
      decently frequently with 26 test suites per commit
      a6c1b33c
    • Jeff Bruemmer's avatar
      78981c9c
    • Diogo Mendes's avatar
    • Anton Kulyk's avatar
      Add basic insert/update row form UI (#22642) · db4ed5cf
      Anton Kulyk authored
      * Add stub `WritebackForm`
      
      * Connect QB to writeback flag state
      
      * Add `WritebackForm` to `QueryModals`
      
      * Add "New row" button
      
      * Add missing `Table` and `Field` type properties
      
      * Generate simple form out of field metadata
      
      * Fix form layout in a modal
      
      * Add field fingerprint types
      
      * Show "New row" button on raw tables too
      
      * Allow overwriting `FieldValuesWidget` renderers
      
      * FieldValuesWidget: prop to always use `TokenField`
      
      * Add class names to `TokenField`
      
      * Highlight input border on hover
      
      * Add category field picker for writeback form
      
      * Support initial values for `WritebackForm`
      
      * Add editing mode to `ObjectDetail`
      
      * Fix type errors
      
      * Fix `CategoryFieldInput` `onChange` return value
      
      * Ooops
      db4ed5cf
    • Alexander Polyankin's avatar
    • Howon Lee's avatar
      No open redirects for enterprise SSO's (#22622) · f3f9c627
      Howon Lee authored
      Open redirects means doing some sso with a built-in redirect, and redirecting into an unhappy place (aka, a non-MB place) afterwards so that someone gets phished or other bad things happen. This is already prevented for OSS sso's but not EE - prevents this for EE sso's by forcing redirects to be in MB `site-url` set domain.
      f3f9c627
Loading