-
- Downloads
X ray stage 1 (#34026)
* reintroduce the functions * using affinities had to update the affinities map to handle multiple definitions of the same affinity name. ```clojure core=> (let [affinities (-> ["table" "GenericTable"] dashboard-templates/get-dashboard-template dash-template->affinities-map)] (affinities "HourOfDayCreateDate")) [["HourOfDayCreateDate" {:dimensions ["CreateTimestamp"], :metrics ["Count"], :score 50}] ["HourOfDayCreateDate" {:dimensions ["CreateTime"], :metrics ["Count"], :score 50}]] ``` Here there were two cards defined as "HourOfDateCreateDate". One looked for a createtimestamp, the other for a createtime. When treating the affinities as unique names, the one with createtime clobbered the createtimestamp and then it went unsatisfied. Now we keep both definitions around. The group by is stable so the timestamp comes first and in match that one will be matched. * i don't like reduced for some reason * fixup affinities 1. Affinities were in wrong shape: before: ```clojure {"card-name" [["card-name" definition] ["card-name" definition]] ,,, ``` new shape: ```clojure {"card-name" [definition definition] ,,, ``` ex: ```clojure core=> (let [affinities (-> ["table" "GenericTable"] dashboard-templates/get-dashboard-template dash-template->affinities-map)] (affinities "HourOfDayCreateDate")) ({:dimensions ["CreateTimestamp"], :metrics ["Count"], :score 50} {:dimensions ["CreateTime"], :metrics ["Count"], :score 50}) ``` 2. Erroring on unfindable filters/metrics using `{:filter [:dimension ::unsatisfiable]}` didn't work correctly because of this: ```clojure ;; it doesn't look for dimensions by keyword core=> (dashboard-templates/collect-dimensions [:dimension ::nope]) () core=> (dashboard-templates/collect-dimensions [:dimension (str ::nope)]) (":metabase.automagic-dashboards.core/nope") ``` * Updating dash-template->affinities-map to dash-template->affinities. Two key changes were made: - The card name is now a part of the 'affinity' object (under the `:affinity-name` key). This flattening should also make affinities easier to deal with and more flexible (e.g. arbitrary groupings). - All base dimensions are exploded out into these affinity objects under the `:base-dims` key. Note that this may result in growth of the number of affinities when named items are repeated with different definitions. For example, card names are not unique, resulting in N affinities per card. Also, metrics and filters need not be unique. GenericTable, for example, has 6 definitions of the Last30Days filter. This will result in 6X the number of affinities created for each card using that filter. This change encapsulates affinities, as before you'd need to know what dimensions underlied any given metric or filter. ATM, we do not package each filter or metric definition in each affinity object, but perhaps that would be worth doing in the future for complete encapsulation. This change also allows for very simple matching logic. To match card affinities, for example, you just filter all affinities for which the affinity dimensions are a subset of the provided dimensions. * Updated tests for new affinity code. Removed accidental cruft. * Change shape of affinities The "satisfied-affins" are of this shape: ```clojure {"AverageIncomeByMonth" #{"Income" "Timestamp"}, "AverageDiscountByMonth" #{"Income" "Discount" "Timestamp"}} ``` And they are ordered so it can drive the `make-cards` function in the future. The idea is for right now we'll look up the card based on affinity name, and when multiple cards found, the set of dimensions they depend on. And that will drive the layout. But in the future, just the affinity itself will drive how we make a card layout. This is the firs step towards that. * Satisfied-affinities shape is ordered map to vector of sets of dims ```clojure {"RowcountLast30Days" [#{"CreateTimestamp"}], "Rowcount" [#{}]} ``` Since each could be met in a few definitions in multiple ways * Update comment ```clojure core=> (let [affinities (-> ["table" "GenericTable"] dashboard-templates/get-dashboard-template dash-template->affinities)] (match-affinities affinities {:available-dimensions {"JoinDate" :whatever}})) {"Rowcount" [#{}], "RowcountLast30Days" [#{"JoinDate"}], "CountByJoinDate" [#{"JoinDate"}], "DayOfWeekJoinDate" [#{"JoinDate"}], "DayOfMonthJoinDate" [#{"JoinDate"}], "MonthOfYearJoinDate" [#{"JoinDate"}], "QuerterOfYearJoinDate" [#{"JoinDate"}]} ``` * comment block is helpful* Drive `make-cards` from affinities old style was make-cards looped over all cards to see which ones were satisfied. Now we've taken a notion of "interestingness" which we call affinities: which dimensions are interesting in combination with each other. Right now these are derived from card combinations but that will change going forward. So now going into the make-cards loop are interesting combinations and we then grab a card-template from the combination. Again, it's a double lookup back to cards but this lets us break that cycle and come up with interesting card templates just based on the groupings themselves. in the future, we're going to want an affinity to produce multiple card-templates so this will become a mapcat of some sorts rather than a map. * Removing pre-check from card-candidates and corresponding unit test as this is not an invariant -- card-candidates should always be satisfied with our affinity mapping. * comment and docstring * Changed names for clarity (affinity -> affinity-sets) and modified the return value of match-affinities to have values of sets of sets rather than vectors of sets. This makes matching simpler and easier. Added a docstring to CardTemplateProvider and started adding some tests. * Revert "Changed names for clarity (affinity -> affinity-sets) and modified" This reverts commit dd2aef1fea8e6deb5f970e51f698e4b72fa97b32. * Something about either the cherry-pick or stale state made the previous change of affinities (as a vector of set) to affinity-sets (a set of sets) cause failures. It may just be that the implementation was broken and the tests passed due to a stale state ¯\_(ツ)_/¯. This picks out the clarity and doc changes and reverts the set of sets. * Adding schemas for the affinity functions in metabase.automagic-dashboards.core * Created the AffinitySetProvider protocol, which, given an item, will provide the set of affinities (a set of set of dimensions) required to bind to that item. The initial implementation reifies the protocol over a dashboard template and provides affinity sets for cards, but this protocol could be extended to provide affinities for whatever object we desire. The initial implementation looks like so: ```clojure (p/defprotocol+ AffinitySetProvider "For some item, determine the affinity sets of that item. This is a set of sets, each underlying set being a set of dimensions that, if satisfied, specify affinity to the item." (create-affinity-sets [this item])) (mu/defn base-dimension-provider :- [:fn #(satisfies? AffinitySetProvider %)] "Takes a dashboard template and produces a function that takes a dashcard template and returns a seq of potential dimension sets that can satisfy the card." [{card-metrics :metrics card-filters :filters} :- ads/dashboard-template] (let [dim-groups (fn [items] (-> (->> items (map (fn [item] [(ffirst item) (set (dashboard-templates/collect-dimensions item))])) (group-by first)) (update-vals (fn [v] (mapv second v))) (update "this" conj #{}))) m->dims (dim-groups card-metrics) f->dims (dim-groups card-filters)] (reify AffinitySetProvider (create-affinity-sets [_ {:keys [dimensions metrics filters]}] (let [dimset (set (map ffirst dimensions)) underlying-dim-groups (concat (map m->dims metrics) (map f->dims filters))] (set (map (fn [lower-dims] (reduce into dimset lower-dims)) (apply math.combo/cartesian-product underlying-dim-groups)))))))) ``` * Adding specs for dashcard, dashcards, context (minimal) and instrumenting make-cards. I'm wondering if we should move card-candidates into the layout-producer protocol since we're using known affinities to make a thing, not to make a baby step to make a thing. The one gotcha in this is that there's a positional index inserted in there which IDK how we use other than maybe layout ATM. * Added more schemas and externalized the matching of affinities to potential dimensions (or any map conforming to the a map of dimension names to matches of items). This generalizes our ability to match of affinity groups with "things" that we want to generate. * Renaming output of all-satisfied-bindings to satisfied-bindings to prevent variable shadowing error. --------- Co-authored-by: Mark Bastian <markbastian@gmail.com>
Showing
- src/metabase/automagic_dashboards/core.clj 277 additions, 95 deletionssrc/metabase/automagic_dashboards/core.clj
- src/metabase/automagic_dashboards/schema.clj 189 additions, 0 deletionssrc/metabase/automagic_dashboards/schema.clj
- test/metabase/automagic_dashboards/core_test.clj 185 additions, 203 deletionstest/metabase/automagic_dashboards/core_test.clj
src/metabase/automagic_dashboards/schema.clj
0 → 100644
This diff is collapsed.
Please register or sign in to comment