Skip to content
Snippets Groups Projects
user avatar
Mark Bastian authored
* Fix clj-kondo

was getting an error:

```clojure
user=> (require ’[clj-kondo.hooks-api :as hooks])
Syntax error compiling at (clj_kondo/impl/analysis/java.clj:61:6).
Unable to find static field: ASM9 in interface org.objectweb.asm.Opcodes
```

checking where the bad version comes from:

```clojure
user=> (io/resource "org/objectweb/asm/Opcodes.class")
        "0x37e9d2e2"
        "jar:file:/Users/dan/.m2/repository/org/ow2/asm/asm-all/5.2/asm-all-5.2.jar!/org/objectweb/asm/Opcodes.class"]
```

and running

```
> clj -X:deps tree :aliases '[:dev :ee :ee-dev :drivers :drivers-dev :alpha :socket :morse :reveal]' > deps

❯ grep asm deps
  X org.ow2.asm/asm 9.4 :superseded
  . org.ow2.asm/asm-all 5.2
      . org.ow2.asm/asm 9.5 :newer-version
      . org.ow2.asm/asm-commons 9.5
        . org.ow2.asm/asm 9.5
        . org.ow2.asm/asm-tree 9.5
          . org.ow2.asm/asm 9.5
    X org.ow2.asm/asm 9.2 :older-version
```

The 9.4 and 9.5s are all close to each other. But the asm-all 5.2 is far
too old. That's brought in by eastwood. So i'll exclude it and see if it
works

* Can't throw error in api endpoint for linked entities x-ray

`create-linked-dashboard` creates a dashboard if there are no linked
entities, so this is unreachable code

* unify card and query code pathways.

now it's quite clear how card and query diverge:

```clojure
(defmethod automagic-analysis Card
  [card {:keys [cell-query] :as opts}]
  (let [root     (->root card)
        cell-url (format "%squestion/%s/cell/%s" public-endpoint
                         (u/the-id card)
                         (encode-base64-json cell-query))]
    (query-based-analysis root opts
                          (when cell-query
                            {:cell-query cell-query
                             :cell-url   cell-url}))))

(defmethod automagic-analysis Query
  [query {:keys [cell-query] :as opts}]
  (let [root       (->root query)
        cell-query (when cell-query (mbql.normalize/normalize-fragment [:query :filter] cell-query))
        opts       (cond-> opts
                     cell-query (assoc :cell-query cell-query))
        cell-url   (format "%sadhoc/%s/cell/%s" public-endpoint
                           (encode-base64-json (:dataset_query query))
                           (encode-base64-json cell-query))]
    (query-based-analysis root opts
                          (when cell-query
                            {:cell-query cell-query
                             :cell-url   cell-url}))))
```

* Adding frequencies to a test to prevent non-deterministic behavior

* Fixing linter check and ordering issue in test.

* Adding TODO future task to replace /rule/ with /dashboard-template/ in paths.

* Added large ns block to metabase.automagic-dashboards.core

* Fixing spelling mistakes

* Some docstrings and renaming for clarity.

* Added ->field and ->root (covering source) tests

* TODO - Fix minumum spelling error on standalone PR.

* Breaking up tests to understand what is on with bulk failures

* Broke make-cards into several logical steps so that the code is more readable. Added tests for each stage.

* Updating references to rules.clj for i18n generation.

* Inlined make-context into apply-dashboard-template. This makes the steps of creating dimensions, metrics, and filters more explicit. ATM we still need to resolve the logic around inject-root as this has dependence on the context and not the individual dimensions.

* put build stuff on classpath for lsp

* calculate base-context once before template loop

* Removed the inject-root multimethod in favor of simple functions. Introducing the idea of removing metric and filter candidates that claim dimensions that don't exist.

* Adding logic to pre-emptively remove unsatisfied metrics, filters, and dimensions.

In `card-candidates` the check:

```
(and
  (every? context-dimensions (map ffirst card-dimensions))
  (every? context-metrics card-metrics)
  (every? context-filters card-filters))
```

is now performed to ensure that the card is satisfied before moving forward.

* Renamed common-dimensions, common-metrics, and common-filters to satisfied-* for clarity. In card-candidates also changed context-* and card-* bindings to available-* and required-* for clarity.

* Teasing out the generated dimensions, metrics, and filters from the big-ball-of-mud context. `make-cards` now takes the base context as well as the available computed dimensions, metrics, and filters as a separate argument. Additionally, tests were added for the `(comp #(filter-tables % tables) dashboard-templates/->entity)` branch of `some-fn` in `potential-card-dimension-bindings`. However, I am not confident that this is ever called outside of the tests. In order to be called, the entity type for the table has to be a named dimension in a card, which I can't find any examples of in the templates.

* Pushing mega-context down to return value of apply-dashboard-template. Just need to take it apart where it is used downstream.

* apply-dashboard-template now just returns generated values and does not build up the base context.

* Adding tests for cases in which dimensions are defined in a native query on the dashcard template.

The only case where this happens is in the resources/automagic_dashboards/table/example.yaml and,
prior to this PR, had no test coverage.

It does add some weird complexities to the system (including bringing in dimensions from a different
approach vector) so perhaps we want to re-evaluate if we want this as a feature or not.

---------

Co-authored-by: default avatardan sutton <dan@dpsutton.com>
764fcd06
History
Code owners
Assign users and groups as approvers for specific file changes. Learn more.

i18n info

Building the backend pot file

Building the backend pot file can be done from the command line (from the project root directory):

❯ clojure -X:build:build/i18n
Created pot file at  cli.pot
Found 1393 forms for translations
Grouped into 1313 distinct pot entries

This is called from update-translation-template which builds frontend, backend, and automagic dashboard pot files and then merges them into one artifact with msgcat.

Developer information

We use a custom script to build up our backend pot (po template) file. Start a repl from this folder and the work is done in the src/i18n/enumerate.clj file.

It uses the very helpful library Grasp to use a spec and a list of sources to find all uses of trs, tru, etc. It returns the forms found from the source, along with metadata that includes the filenames and line numbers.

enumerate=> (def single-file (str u/project-root-directory "/src/metabase/util.clj"))
#'i18n.enumerate/single-file
enumerate=> (map (juxt meta identity)
                 (g/grasp single-file ::translate))
([{:line 81,
   :column 13,
   :uri "file:/Users/dan/projects/work/metabase/src/metabase/util.clj"}
  (trs
   "Maximum memory available to JVM: {0}"
   (format-bytes (.maxMemory (Runtime/getRuntime))))]
 [{:line 313,
   :column 33,
   :uri "file:/Users/dan/projects/work/metabase/src/metabase/util.clj"}
  (tru "Timed out after {0}" (format-milliseconds timeout-ms))]
 [{:line 504,
   :column 26,
   :uri "file:/Users/dan/projects/work/metabase/src/metabase/util.clj"}
  (tru "Not something with an ID: {0}" (pr-str object-or-id))])

Pot files can only have a single entry for each string, so these are grouped by the message and then we use the org.fedorahosted.tennera.jgettext library to emit the pot file.

You can build quick pot files with the helpful function create-pot-file!

enumerate=> (create-pot-file! single-file "pot.pot")
Created pot file at  pot.pot
{:valid-usages 3, :entry-count 4, :bad-forms ()}

which will output

# Copyright (C) 2022 Metabase <docs@metabase.com>
# This file is distributed under the same license as the Metabase package
#, fuzzy
msgid ""
msgstr ""
"Project-Id-Version: 1.0\n"
"Report-Msgid-Bugs-To: docs@metabase.com\n"
"POT-Creation-Date: 2022-07-22 14:03-0500\n"
"MIME-Version: 1.0\n"
"Content-Type: text/plain; charset=UTF-8\n"
"Content-Transfer-Encoding: 8bit\n"

#: /Users/dan/projects/work/metabase/src/metabase/sync/analyze/fingerprint/fingerprinters.clj
msgid "Error generating fingerprint for {0}"
msgstr ""

#: metabase/util.clj:81
msgid "Maximum memory available to JVM: {0}"
msgstr ""

#: metabase/util.clj:313
msgid "Timed out after {0}"
msgstr ""

#: metabase/util.clj:504
msgid "Not something with an ID: {0}"
msgstr ""
Overrides

You'll note that we defined a single file "/src/metabase/util.clj" but ended up with an entry from fingerprinters.clj. This is because that usage of trs is inside of a macro which emits a defrecord. The quoting means that the form doesn't actually match our spec since it has more sequence stuff:

user=> '`(trs "foobar")
(clojure.core/seq (clojure.core/concat (clojure.core/list (quote user/trs)) (clojure.core/list "foobar")))

More information in the grasp issue. A quick workaround is just including this manual override.