Skip to content
Snippets Groups Projects
This project is mirrored from https://github.com/metabase/metabase. Pull mirroring updated .
  1. Sep 29, 2023
    • Nemanja Glumac's avatar
      Optimize changed files check for the backend.yml workflow (#34209) · c13328df
      Nemanja Glumac authored
      Before
      1. prepare-frontend
      2. prepare-backend
      3. build-static-viz
      4. files-changed-check
      5. static-viz-files-changed-check
      
      So, we're doing all these expensive operations before the step 4, even though they might not even be necessary.
      If files changed already outputs backend_all, this means we have to run these tests whether or not there were static-viz changes.
      
      After
      1. test which files changed
          - if backend_all output -> skip static-viz-bundle and start running tests straight away
          - else if there are no direct backend changes, have an additional check for static-viz
      2. static-viz-files-changed
          - if true -> run tests
          - else -> skip tests
      Unverified
      c13328df
    • Mark Bastian's avatar
      This is a small set of changes pulled from another PR whose contents (#34228) · 92faa8e2
      Mark Bastian authored
      probably won't be merged. The intent of these changes is to be
      explicit in what is passed into each of the card creation functions,
      ensure consistency of types, and eliminate redundant or non-required
      arguments.
      
      `build-dashcard` was the target of this simplification. The
      `available-values` argument is no longer necessary as that data is
      sourced from other arguments. the `satisfied-*` arguments are no
      longer seqs of satisfied filter, metric, and dimension definitions,
      but maps of names of those items to the definitions. This is only
      needed because satisfied metrics needs metrics names when adding
      metadata. However, all values were modified into a consistent shape.
      
      In other areas, if an aggregate object with several keys is passed
      to a function that uses only one element of the object, only the
      inner element is passed.
      
      In card-candidates, `:matches` is dissoced from `satisfied-dimensions`
      to remove the extra payload that isn't needed in that argument.
      This makes it clear that satisfied field values are passed into card
      creation functions via the `satisfied-bindings` and not cruft from
      earlier stage.
      
      As an aside, one potential fix that would substantially improve the
      readibility of this entire codebase would be to move the element
      names in the one key to value maps as is done for dimensions,
      metrics, filters, and cards into the elements as a named key.
      Carrying that shape forward from the yaml files requires a ton of
      first this or zipmap that to recover the names into whatever the
      current context is.
      Unverified
      92faa8e2
    • Denis Berezin's avatar
      Fix flaky ActionViz unit tests (#34250) · 6e5b6a7b
      Denis Berezin authored
      Unverified
      6e5b6a7b
    • Mahatthana (Kelvin) Nomsawadi's avatar
      Fix overlapping x-tick labels on static viz timeseries charts (#33707) · 70355597
      Mahatthana (Kelvin) Nomsawadi authored
      
      * Fix overlapping x-tick labels on static viz timeseries charts
      
      * Address review: make the condition easier to understand + fix failed cases
      
      * Update frontend/src/metabase/static-viz/components/XYChart/utils/ticks.ts
      
      Co-authored-by: default avatarJesse Devaney <22608765+JesseSDevaney@users.noreply.github.com>
      
      * Add a comment to explain why we have to fix timeseries ticks
      
      ---------
      
      Co-authored-by: default avatarJesse Devaney <22608765+JesseSDevaney@users.noreply.github.com>
      Unverified
      70355597
    • Roman Abdulmanov's avatar
      fixed & symbol escaping (#34243) · 4224bbea
      Roman Abdulmanov authored
      Unverified
      4224bbea
    • Kamil Mielnik's avatar
      Add custom warning message when leaving creating/editing a query action in... · 5c2279d4
      Kamil Mielnik authored
      Add custom warning message when leaving creating/editing a query action in model details page (#34192)
      
      * Replace useBeforeUnload with LeaveConfirmationModal in ActionCreator
      
      * Take action name and action settings changes into account when computing isDirty
      
      * Improve action name change detection
      
      * Rename handleFormSettingsChange to patchFormSettings
      
      * Rename handleActionChange to patchAction
      
      * Introduce a way to sync initial action state after creation/update
      
      * Sync initial action state after creation/update
      
      * Add a hack for re-rendering
      
      * Revert "fixing" isDirty flag
      
      * Replace delay hack with an effect
      
      * Improve ActionCreatorModal setup
      - use actual component, not the mock
      - move waiting for spinner outside of setup, as it is initialRoute-dependent
      - ensure history object is always present
      - add missing setupDatabaseEndpoints call
      
      * Add a test for showing custom unsaved changes warning modal
      
      * Update default mock query action's visualization_settings
      
      * Add a test for not showing custom unsaved changes warning modal
      
      * Fix isDirty flag evaluation
      
      * Group similar tests
      
      * Add tests for creating new action
      
      * Remove redundant spread
      
      * Use local action
      
      * Fix typo
      
      * Add a test for updating an action
      
      * Add a test for creating an action
      
      * Fix a flake
      
      * Add missing router in tests setup
      
      * Enable LeaveConfirmationModal only in ActionCreatorModal with a dedicated route
      - see action-related edge cases in https://github.com/metabase/metabase/issues/34114
      
      * Fix flaking test by waiting for modal to close
      
      * Make assertion consistent
      
      * Fix flaky unit tests
      
      * Refactor isDirty computation
      
      * Add a way to skip unsaved changes warning when submitting the modal
      
      * Remove setAction
      
      * Bring back old logic
      
      * Undo rename
      
      * Update comment
      
      * Update comment
      
      * Add areActionsEqual to deal with redundant "value" attribute of action parameter
      
      * Omit "hasVariableTemplateTagTarget", compute visualization settings diff based on form state
      
      * Revert unnecessary change
      
      * Revert unnecessary change
      
      * Rename getActionCrudParameters to getCleanParameters
      
      * Revert unnecessary change
      
      * Revert unnecessary change
      
      * Simplify isDirty computation with Question.prototype.isDirtyComparedTo
      
      * Revert leftover code
      
      * Remove duplicated assertion
      
      * Update test to detect correctness of isDirty flag when user reverts changes
      Unverified
      5c2279d4
    • Nemanja Glumac's avatar
      Add `:dataset` to the `POST /api/card` endpoint (#34218) · 85dbd9ac
      Nemanja Glumac authored
      Depending on this key's value, we might either create a model when saving a card, or not.
      
      It already exists in some tests and in the `create-card!` function but it was never explicitly defined in this endpoint.
      That is a shame because we generate API documentation based on these keys.
      Unverified
      85dbd9ac
    • Cam Saul's avatar
  2. Sep 28, 2023
  3. Sep 27, 2023
  4. Sep 26, 2023
    • adam-james's avatar
      Alert API sends all alert information to any user. Change this to onl… (#34067) · 0920254f
      adam-james authored
      * Alert API sends all alert information to any user. Change this to only show those pulses the user has created or is
      recipient of
      
      * Adjust tests to show that only admins see all alerts. Regular users only see their alerts.
      
      * Change docstring to describe behaviour of the endpoint.
      Unverified
      0920254f
    • Tim Macdonald's avatar
    • Cam Saul's avatar
      Remove deprecated `current-db-time` method (#32686) · 47a52b57
      Cam Saul authored
      
      * Remove deprecated current-db-time method
      
      * Address PR feedback
      
      * sync-timezone! should handle java.time.ZoneId and java.time.ZoneOffset
      
      * Allow time zone offsets as zone IDs
      
      * Fix tests and suppress clj-kondo on a few forms
      
      * Suppress warning on one more occurrence of db-default-timezone
      
      * Map offset Z to time zone UTC when syncing
      
      * Accept zone offsets, zone ids and strings for h2 and oracle
      
      * Mark snowflake as not supporting report timezone
      
      * Revert "Mark snowflake as not supporting report timezone"
      
      This reverts commit 70b15a60334fda707d695bc6c01621961f1c5aa8.
      
      * Fix report-timezone handling
      
      * Numeric timezone
      
      ---------
      
      Co-authored-by: default avatarmetamben <103100869+metamben@users.noreply.github.com>
      Co-authored-by: default avatarTamás Benkő <tamas@metabase.com>
      Unverified
      47a52b57
    • lbrdnk's avatar
    • Nick Fitzpatrick's avatar
      Only run BE tests on static viz FE changes (#33760) · 39acfd4a
      Nick Fitzpatrick authored
      * output static viz sources, update be workflow and file paths
      
      * build static viz before file check
      
      * extending file check timeout
      
      * fixing mistake
      
      * disable optimization when generating file paths
      
      * prefer offline
      
      * moving workflows to save 89505
      
      * Maybe caching the build?
      
      * removing minification
      
      * upload artifact instead of use cache
      
      * Add workflows back
      
      * reduce files changed timeout
      
      * removing unneeded yarn install
      Unverified
      39acfd4a
    • dpsutton's avatar
      X ray stage 1 (#34026) · 400850b9
      dpsutton authored
      * reintroduce the functions
      
      * using affinities
      
      had to update the affinities map to handle multiple definitions of the
      same affinity name.
      
      ```clojure
      core=> (let [affinities (-> ["table" "GenericTable"]
                                  dashboard-templates/get-dashboard-template
                                  dash-template->affinities-map)]
               (affinities "HourOfDayCreateDate"))
      [["HourOfDayCreateDate"
        {:dimensions ["CreateTimestamp"], :metrics ["Count"], :score 50}]
       ["HourOfDayCreateDate"
        {:dimensions ["CreateTime"], :metrics ["Count"], :score 50}]]
      ```
      
      Here there were two cards defined as "HourOfDateCreateDate". One looked
      for a createtimestamp, the other for a createtime. When treating the
      affinities as unique names, the one with createtime clobbered the
      createtimestamp and then it went unsatisfied.
      
      Now we keep both definitions around. The group by is stable so the
      timestamp comes first and in match that one will be matched.
      
      * i don't like reduced for some reason
      
      * fixup affinities
      
      1. Affinities were in wrong shape:
      
      before:
      
      ```clojure
      {"card-name" [["card-name" definition]
                    ["card-name" definition]]
       ,,,
      ```
      
      new shape:
      
      ```clojure
      {"card-name" [definition
                    definition]
       ,,,
      ```
      
      ex:
      
      ```clojure
      core=> (let [affinities (-> ["table" "GenericTable"]
                                  dashboard-templates/get-dashboard-template
                                  dash-template->affinities-map)]
               (affinities "HourOfDayCreateDate"))
      ({:dimensions ["CreateTimestamp"], :metrics ["Count"], :score 50}
       {:dimensions ["CreateTime"], :metrics ["Count"], :score 50})
      ```
      
      2. Erroring on unfindable filters/metrics
      using `{:filter [:dimension ::unsatisfiable]}` didn't work correctly
      because of this:
      
      ```clojure
      ;; it doesn't look for dimensions by keyword
      core=> (dashboard-templates/collect-dimensions [:dimension ::nope])
      ()
      core=> (dashboard-templates/collect-dimensions [:dimension (str ::nope)])
      (":metabase.automagic-dashboards.core/nope")
      ```
      
      * Updating dash-template->affinities-map to dash-template->affinities.
      
      Two key changes were made:
      - The card name is now a part of the 'affinity' object (under the `:affinity-name` key).
        This flattening should also make affinities easier to deal with and more flexible
        (e.g. arbitrary groupings).
      - All base dimensions are exploded out into these affinity objects under the `:base-dims` key.
        Note that this may result in growth of the number of affinities when named items are
        repeated with different definitions. For example, card names are not unique, resulting in
        N affinities per card. Also, metrics and filters need not be unique. GenericTable, for
        example, has 6 definitions of the Last30Days filter. This will result in 6X the number of
        affinities created for each card using that filter.
      
      This change encapsulates affinities, as before you'd need to know what dimensions underlied
      any given metric or filter. ATM, we do not package each filter or metric definition in each
      affinity object, but perhaps that would be worth doing in the future for complete encapsulation.
      
      This change also allows for very simple matching logic. To match card affinities, for example,
      you just filter all affinities for which the affinity dimensions are a subset of the provided
      dimensions.
      
      * Updated tests for new affinity code. Removed accidental cruft.
      
      * Change shape of affinities
      
      The "satisfied-affins" are of this shape:
      ```clojure
      {"AverageIncomeByMonth" #{"Income" "Timestamp"},
       "AverageDiscountByMonth" #{"Income" "Discount" "Timestamp"}}
      ```
      
      And they are ordered so it can drive the `make-cards` function in the
      future. The idea is for right now we'll look up the card based on
      affinity name, and when multiple cards found, the set of dimensions they
      depend on. And that will drive the layout.
      
      But in the future, just the affinity itself will drive how we make a
      card layout. This is the firs step towards that.
      
      * Satisfied-affinities shape is ordered map to vector of sets of dims
      
      ```clojure
      {"RowcountLast30Days" [#{"CreateTimestamp"}],
       "Rowcount" [#{}]}
      ```
      
      Since each could be met in a few definitions in multiple ways
      
      * Update comment
      
      ```clojure
      core=> (let [affinities (-> ["table" "GenericTable"]
                                  dashboard-templates/get-dashboard-template
                                  dash-template->affinities)]
               (match-affinities affinities {:available-dimensions {"JoinDate" :whatever}}))
      {"Rowcount" [#{}],
       "RowcountLast30Days" [#{"JoinDate"}],
       "CountByJoinDate" [#{"JoinDate"}],
       "DayOfWeekJoinDate" [#{"JoinDate"}],
       "DayOfMonthJoinDate" [#{"JoinDate"}],
       "MonthOfYearJoinDate" [#{"JoinDate"}],
       "QuerterOfYearJoinDate" [#{"JoinDate"}]}
      ```
      
      * comment block is helpful :octopus:
      
      
      
      * Drive `make-cards` from affinities
      
      old style was make-cards looped over all cards to see which ones were
      satisfied. Now we've taken a notion of "interestingness" which we call
      affinities: which dimensions are interesting in combination with each
      other. Right now these are derived from card combinations but that will
      change going forward.
      
      So now going into the make-cards loop are interesting combinations and
      we then grab a card-template from the combination. Again, it's a double
      lookup back to cards but this lets us break that cycle and come up with
      interesting card templates just based on the groupings themselves.
      
      in the future, we're going to want an affinity to produce multiple
      card-templates so this will become a mapcat of some sorts rather than a
      map.
      
      * Removing pre-check from card-candidates and corresponding unit test as this is not an invariant -- card-candidates should always be satisfied with our affinity mapping.
      
      * comment and docstring
      
      * Changed names for clarity (affinity -> affinity-sets) and modified
      the return value of match-affinities to have values of sets of sets
      rather than vectors of sets. This makes matching simpler and easier.
      
      Added a docstring to CardTemplateProvider and started adding some tests.
      
      * Revert "Changed names for clarity (affinity -> affinity-sets) and modified"
      
      This reverts commit dd2aef1fea8e6deb5f970e51f698e4b72fa97b32.
      
      * Something about either the cherry-pick or stale state made the previous
      change of affinities (as a vector of set) to affinity-sets (a set of sets)
      cause failures. It may just be that the implementation was broken and
      the tests passed due to a stale state ¯\_(ツ)_/¯.
      
      This picks out the clarity and doc changes and reverts the set of sets.
      
      * Adding schemas for the affinity functions in metabase.automagic-dashboards.core
      
      * Created the AffinitySetProvider protocol, which, given an item, will
      provide the set of affinities (a set of set of dimensions) required to
      bind to that item. The initial implementation reifies the protocol
      over a dashboard template and provides affinity sets for cards, but
      this protocol could be extended to provide affinities for whatever
      object we desire.
      
      The initial implementation looks like so:
      
      ```clojure
      (p/defprotocol+ AffinitySetProvider
        "For some item, determine the affinity sets of that item. This is a set of sets, each underlying set being a set of
        dimensions that, if satisfied, specify affinity to the item."
        (create-affinity-sets [this item]))
      
      (mu/defn base-dimension-provider :- [:fn #(satisfies? AffinitySetProvider %)]
        "Takes a dashboard template and produces a function that takes a dashcard template and returns a seq of potential
        dimension sets that can satisfy the card."
        [{card-metrics :metrics card-filters :filters} :- ads/dashboard-template]
        (let [dim-groups (fn [items]
                           (-> (->> items
                                    (map
                                      (fn [item]
                                        [(ffirst item)
                                         (set (dashboard-templates/collect-dimensions item))]))
                                    (group-by first))
                               (update-vals (fn [v] (mapv second v)))
                               (update "this" conj #{})))
              m->dims    (dim-groups card-metrics)
              f->dims    (dim-groups card-filters)]
          (reify AffinitySetProvider
            (create-affinity-sets [_ {:keys [dimensions metrics filters]}]
              (let [dimset                (set (map ffirst dimensions))
                    underlying-dim-groups (concat (map m->dims metrics) (map f->dims filters))]
                (set
                  (map
                    (fn [lower-dims] (reduce into dimset lower-dims))
                    (apply math.combo/cartesian-product underlying-dim-groups))))))))
      ```
      
      * Adding specs for dashcard, dashcards, context (minimal) and instrumenting
      make-cards. I'm wondering if we should move card-candidates into the
      layout-producer protocol since we're using known affinities to make a thing,
      not to make a baby step to make a thing. The one gotcha in this is that
      there's a positional index inserted in there which IDK how we use other
      than maybe layout ATM.
      
      * Added more schemas and externalized the matching of affinities to
      potential dimensions (or any map conforming to the a map of dimension
      names to matches of items). This generalizes our ability to match
      of affinity groups with "things" that we want to generate.
      
      * Renaming output of all-satisfied-bindings to satisfied-bindings to prevent variable shadowing error.
      
      ---------
      
      Co-authored-by: default avatarMark Bastian <markbastian@gmail.com>
      Unverified
      400850b9
    • Cal Herries's avatar
    • Tim Macdonald's avatar
      Use Bigints for CSV uploads (#34118) · ecf9d549
      Tim Macdonald authored
      [Fixes #34080]
      Unverified
      ecf9d549
    • Nicolò Pretto's avatar
      Revert "show columns added after the dashcard has been created in the viz... · 7e0eb68f
      Nicolò Pretto authored
      Revert "show columns added after the dashcard has been created in the viz settings (#33886)" (#34116)
      
      Unverified
      7e0eb68f
Loading