Skip to content
Snippets Groups Projects
This project is mirrored from https://github.com/metabase/metabase. Pull mirroring updated .
  1. Aug 05, 2022
    • Noah Moss's avatar
      Bump shadow-cljs to 2.19.6 (#24647) · dfd93579
      Noah Moss authored
      * Bump shadow-cljs
      
      This lets us use the template tag functionality
      
      ```
      add experimental support for creating js template strings
      
      used like str but emits native JS `` template
      
        (js-template "foo" (+ 1 2) "bar")
      
      emits the literal
      
        `foo${(1 + 2)}bar`
      ```
      
      But this required a few changes on our side as well.
      
      Required for this:
      
      -- LOGGING
      
      Bumped glogi
      ```diff
         ;; new stuff
      -  [lambdaisland/glogi "1.0.106"]]
      -
      +  [com.lambdaisland/glogi "1.1.144"]]
      ```
      Annoying because it had a group name change due to Clojars policy. This
      had the same error as us below. That's why I'm hopeful that it will
      actually just work.
      
      There were some changes with the Google Closure library with things not
      included or referenceable. So I just removed these imports and the
      typehints that used them. We'll need to verify that this still works but
      both compiling for dev and release (with advanced compilation)
      work. Still possible there will be some runtime errors but i'm hopeful
      there wont be. We can grab a jar from CI and poke around
      
      ```shell
      ❯ npx shadow-cljs compile app
      shadow-cljs - config: /Users/dan/projects/work/metabase/shadow-cljs.edn
      shadow-cljs - connected to server
      [:app] Compiling ...
      [:app] Build completed. (100 files, 0 compiled, 0 warnings, 1.65s)
      
      metabase on  nm-all-options-formatting [$!+?] via :coffee: v17.30 on :cloud:
      
        metabase-query took 2s
      ❯ npx shadow-cljs release app
      shadow-cljs - config: /Users/dan/projects/work/metabase/shadow-cljs.edn
      shadow-cljs - connected to server
      [:app] Compiling ...
      [:app] Build completed. (100 files, 30 compiled, 0 warnings, 28.04s)
      ```
      
      ```diff
                   [lambdaisland.glogi.console :as glogi-console])
      -  (:require-macros metabase.shared.util.log)
      -  (:import goog.debug.Logger
      -           goog.debug.Logger.Level))
      +  (:require-macros metabase.shared.util.log))
      ```
      
      ```diff
      @@ -37,4 +35,4 @@
       (defn is-loggable?
         "Part of the impl for [[metabase.shared.util.log/js-logp]] and [[metabase.shared.util.log/js-logf]]."
         [logger-name level]
      -  (.isLoggable ^Logger (log/logger logger-name) ^Level (log/levels level)))
      +  (.isLoggable (log/logger logger-name) (log/levels level)))
      ```
      
      -- WARNINGS ABOUT shadowing `abs`
      
      -  [medley "1.3.0"]
      +  [medley "1.4.0"]
      
      Similar to what we bumped for the backend. ClojureScript 1.11.x has an
      `abs` function.
      
      * use update-keys and update-vals in cljs
      
      * bump clj-kondo in CI
      
      Co-authored-by: default avatardan sutton <dan@dpsutton.com>
      dfd93579
  2. Aug 03, 2022
  3. Aug 02, 2022
  4. Jul 27, 2022
  5. Jul 20, 2022
    • Noah Moss's avatar
      Include markdown parameters in dashboard subscriptions (#24070) · b30e2e53
      Noah Moss authored
      * basic logic for including params in dashboard subscriptions
      
      * BE date formatting
      
      * redo approach to date formatting locale
      
      * refactor and use the same formatting code for parameters listed in subscription header
      
      * fix tests and lint
      
      * fix more tests
      
      * fix final (?) test and fix namespace lint error
      
      * a couple extra test cases
      
      * fix tests
      
      * new cypress test
      
      * address braden and ngoc's comments
      
      * fix tests
      
      * guard against nil text in substitute_tags
      b30e2e53
  6. Jul 13, 2022
  7. Jul 12, 2022
    • Noah Moss's avatar
      First pass at parameters in Markdown cards (#23641) · e136f430
      Noah Moss authored
      * first pass at parameters in text cards on FE
      
      * trying to get translations working
      
      * relative datetime formatting
      
      * copy changes and 'Text card' header
      
      * default text when no params
      
      * hide header for text cards with height of 1 with params when in param mapping mode
      
      * show UI text in mobile mode
      
      * minor fixes
      
      * enforce that a text card variable can only be mapped to one parameter
      
      * more value formatting
      
      * noop
      
      * fix backend tests
      
      * add back a couple pieces of frontend logic commented out
      
      * misc cleanup
      
      * attempt at adding a FE unit test
      
      * revert unit test, doesn't work
      
      * add a couple of basic cypress tests and fix a couple of bugs
      
      * basic unit tests for cljc
      
      * fix error
      
      * expanded unit tests
      
      * simplify ns
      
      * add cypress test for instance language translation
      
      * basic handling for a couple cases of :date/all-options
      
      * trs docstring clarification
      
      * whitespace tweaks
      
      * fix cypress test
      
      * minor refactor of tag-names
      
      * move cljc file from utils to new parameters dir
      
      * reorder functions
      
      * fix lint
      
      * add test assertion that locale is correctly reset back to english, and add a comment
      
      * fix bug where existing parameter mapping target was not being found
      
      * clojure logic tweaks
      
      * move text card header text to the Text component config
      
      * simplify header logic, and pull out isLoading into a function to reduce complexity
      
      * address alex's css feedback
      
      * fix trs comment
      e136f430
  8. Jun 29, 2022
  9. Jun 20, 2022
    • metamben's avatar
      Fix compilation of temporal arithmetic in between filters (#23292) · 001055df
      metamben authored
      Fix compilation of temporal arithmetic for BigQuery and Mongo 5+
      
      * Mongo 4 doesn't support $dateAdd so the generated filters result in an exception.
      * Support adding field to interval too (time intervals were not allowed in the first place of an addition)
      * Support temporal arithmetic with more than two operands for Mongo
      001055df
  10. Jun 06, 2022
  11. May 12, 2022
  12. May 02, 2022
    • Braden Shepherdson's avatar
      One Clojure lint to rule them all (#22213) · 390d10e5
      Braden Shepherdson authored
      Background:
      
      We had two lint configs: lint-config.edn which was enfored by Github Actions, and
      .clj-kondo/config.edn, the default location used by editors.
      
      
      Problem:
      
      Patchy enforcement of the (larger) config in .clj-kondo/config.edn since not
      everyone uses an editor.
      
      Some new lint, like canonical namespace aliases (#21738, #19930) is not enforced
      automatically and has begun to bit-rot.
      
      Solution:
      
      Combine both configs, fix any outstanding lint, and update our tooling to use
      the unified config.
      
      Caveats:
      
      Anyone who has their local editor configured to use lint-config.edn will have
      their linting broken by this change.
      390d10e5
  13. Apr 28, 2022
  14. Apr 19, 2022
  15. Apr 18, 2022
    • Alexander Kiselev's avatar
      Support boolean literals in custom expression filters (#21342) · 179862b3
      Alexander Kiselev authored
      * adding boolean to custom expr
      
      * added booleans to schema validattion
      
      * changed some mbql to use apply instead of a vector
      
      * update sample-dataset.edn
      
      * undo honeysql mess
      
      * fix sample-dataset.edn
      
      * fix unit tests
      
      * remove console.log
      
      * added `canceled` column to orders table in the sample database
      
      * minor fixes
      
      * bool custom column fixes
      
      * comment out join schema
      
      * rolback custom column changes
      
      * add error on custom column boolean expressions
      
      * unit tests
      
      * fix clj unit tests
      
      * e2e fix
      
      * unit test  fixes
      
      * e2e fixes
      
      * added True/False to suggestions and fixed column formatting
      
      * fix e2e test
      
      * remove bool column from sample-dataset.edn because Oracle doesnt support bool column type (WTF?)
      
      * fix sample dataset tests
      
      * fix sample dataset tests
      
      * couple more fixes
      
      * fix sample-dataset.edn again
      
      * e2e fixes
      
      * fix some unit and e2e tests
      
      * unit test fix
      
      * e2e fixes
      
      * rollback sample database changes
      
      * more rollback
      
      * fix issues and add e2e tests
      
      * remove withAst from boolean pass
      
      * fixed issue with AST node
      179862b3
  16. Apr 13, 2022
  17. Apr 06, 2022
    • Ryan Laurie's avatar
      Add currency symbols to inputs (#21398) · 34378b8d
      Ryan Laurie authored
      * add correct currency symbol to filter inputs
      
      * show currency symbol in between inputs
      
      * lift currency prefix logic up component tree
      
      * remove unused function
      
      * fix existing tests
      
      * test currency prefixes
      
      * fix test nesting
      
      * get currency symbols for columns from our currency map
      
      * use visualization settings for currency inputs
      
      * use keyForColumn() to get visualization settings
      34378b8d
  18. Mar 23, 2022
  19. Mar 16, 2022
  20. Mar 10, 2022
  21. Feb 28, 2022
  22. Feb 23, 2022
  23. Feb 14, 2022
    • Cam Saul's avatar
      Add logic to truncate and uniquely-suffix column alias identifiers (#19659) · f94d5149
      Cam Saul authored
      * Add failing test for #15978
      
      * Improved test
      
      * Add new metabase.driver.query-processor.escape-join-aliases QP middleware
      
      * Test fix :wrench:
      
      * Add reference to #20307
      
      * Add some extra dox
      
      * Test fixes for BigQuery drivers
      
      * revert unneeded change
      
      * Fix :bigquery and :bigquery-cloud-sdk mixup
      
      * Test fixes :wrench:
      
      * Test fix :wrench:
      
      * Remove comment I meant to remove
      f94d5149
  24. Feb 10, 2022
    • Michiel Borkent's avatar
      Linting improvements (#20424) · 8f119d64
      Michiel Borkent authored
      * honeysql var improvements
      
      * Define routes
      
      * Define routes
      
      * define-routes improvement
      
      * honeysql helpers
      
      * honesql
      
      * abs in clojure 1.11
      
      * fix logic unresolved vars
      
      * deprecated
      
      * deprecated var warnings
      
      * undo replace
      8f119d64
  25. Feb 08, 2022
    • Cam Saul's avatar
      Rework how the remappings middleware matches remapped to/from columns for... · e35ecaca
      Cam Saul authored
      Rework how the remappings middleware matches remapped to/from columns for explicit external (FK) remaps (#20009)
      
      * Fix #9236 [WIP]
      
      * Revert unneeded changes
      
      * Only merge namespaced :options into result info
      
      * Revert unneeded changes
      
      * Everything is working :scream_cat:
      
      * Revert unneeded change
      
      * Clean namespaces
      
      * Add `:test` profile to `namespace-checker` to suppress log messages.
      
      * PR feedback
      
      * Fix namespaces
      e35ecaca
  26. Feb 02, 2022
  27. Feb 01, 2022
    • dpsutton's avatar
      Unused bindings part1 (#19995) · dd300077
      dpsutton authored
      * Unused bindings cleanup
      
      `clj-kondo --lint src:shared/src > lint` and then just work my way
      through it. Note that there are warnings about deprecations that we will
      need to get rid of as well. Most likely by turning off that warning
      until we are ready to tackle the project.
      
      * More unused and a sort ns
      
      * Removing unused svg helpers that eastwood is complaining about
      
      * Restore accidentally deleted `begin!` impl for xlsx
      
      * clean ns
      
      * clean ns
      
      * Last few unused bindings
      
      * kondo checks for empty docstrings
      
      * Little more cleanup
      dd300077
    • Cam Saul's avatar
      MBQL :expressions should use strings for keys (#19960) · 5d5b33a1
      Cam Saul authored
      * MBQL :expressions should use strings for keys
      
      * Make some more tests ^:parallel
      
      * Update dox
      
      * Test fix
      
      * Sort namespaces
      5d5b33a1
  28. Jan 26, 2022
  29. Jan 25, 2022
  30. Jan 20, 2022
    • Cam Saul's avatar
      `add-alias-info`: recognize `:field` clauses in join source queries to be the... · 33ac1862
      Cam Saul authored
      `add-alias-info`: recognize `:field` clauses in join source queries to be the same if it has a `:temporal-unit` (#19789)
      
      * Backport Debug QP improvements from #19754
      
      * Enable test
      
      * "Fuzzy" matching when looking for Fields in join source queries
      
      * Add/update tests
      
      * Test fix :wrench:
      
      * Remove trailing whitespace
      
      * Fix a few docstrings
      33ac1862
  31. Jan 19, 2022
  32. Jan 18, 2022
  33. Jan 13, 2022
  34. Dec 15, 2021
    • dpsutton's avatar
      Datasets preserve metadata (#19158) · 5f8bc305
      dpsutton authored
      * Preserve metadata and surface metadata of datasets
      
      Need to handle two cases:
      1. querying the dataset itself
      2. a question that is a nested question of the dataset
      
      1. Querying the dataset itself
      This is a bit of a shift of how metadata works. Previously it was just
      thrown away and saved on each query run. This kind of needs to be this
      way because when you edit a question, we do not ensure that the metadata
      stays in sync! There's some checksum operation that ensures that the
      metadata hasn't been tampered with, but it doesn't ensure that it
      actually matches the query any longer.
      
      So imagine you add a new column to a query. The metadata is not changed,
      but its checksum matches the original query's metadata and the backend
      happily saves this. Then on a subsequent run of the query (or if you hit
      visualize before saving) the metadata is tossed and updated.
      
      So to handle this carelessness, we have to allow the metadata that can
      be edited to persist across just running the dataset query. So when
      hitting the card api, stick the original metadata in the middleware (and
      update the normalize ns not to mangle field_ref -> field-ref among
      others). Once we have this smuggled in, when computing the metadata in
      annotate, we need a way to index the columns. The old and bad way was
      the following:
      
      ```clojure
      ;; old
      (let [field-id->metadata (u/key-by :id source-metadata)] ...)
      ;; new and better
      (let [ref->metadata (u/key-by (comp u/field-ref->key :field_ref) source-metadata)] )
      ```
      
      This change is important because ids are only for fields that map to
      actual database columns. computed columns, case, manipulations, and all
      native fields will lack this. But we can make field references.
      
      Then for each field in the newly computed metadata, allow the non-type
      information to persist. We do not want to override type information as
      this can break a query, but things like description, display name,
      semantic type can survive.
      
      This metadata is then saved in the db as always so we can continue with
      the bit of careless metadata saving that we do.
      
      2. a question that is a nested question of the dataset
      This was a simpler change to grab the source-metadata and ensure that it
      is blended into the result metadata in the same way.
      
      Things i haven't looked at yet: column renaming, if we need to allow
      conversions to carry through or if those necessarily must be opaque (ie,
      once it has been cast forget that it was originally a different type so
      we don't try to cast the already cast value), and i'm sure some other
      things. But it has been quite a pain to figure all of this stuff
      out. Especially the divide between native and mbql since native requires
      the first row of values back before it can detect some types.
      
      * Add in base-type specially
      
      Best to use field_refs to combine metadata from datasets. This means
      that we add this ref before the base-type is known. So we have to update
      this base-type later once they are known from sampling the results
      
      * Allow column information through
      
      I'm not sure how this base-type is set for
      annotate-native-cols. Presumably we don't have and we get it from the
      results but this is not true. I guess we do some analysis on count
      types. I'm not sure why they failed though.
      
      * Correctly infer this stuff
      
      This was annoying. I like :field_ref over :name for indexing, as it has
      a guaranteed unique name. But datasets will have unique names due to a
      restriction*. The problem was that annotating the native results before
      we had type information gave us refs like `[:field "foo" {:base-type
      :type/*}]`, but then this ruined the merge strategy at the end and
      prevented a proper ref being merged on top. Quite annoying. This stuff
      is very whack-a-mole in that you fix one bit and another breaks
      somewhere else**.
      
      * cannot have identical names for a subselect:
          select id from (select 1 as id, 2 as id)
      
      ** in fact, another test broke on this commit
      
      * Revert "Correctly infer this stuff"
      
      This reverts commit 1ffe44e90076b024efd231f84ea8062a281e69ab.
      
      * Annotate but de-annotate in a way
      
      To combine metadata from the db, really, really want to make sure they
      actually match up. Cannot use name as this could collide when there are
      two IDs in the same query. Combining metadata on that gets nasty real
      quick.
      
      For mbql and native, its best to use field_refs. Field_refs offer the
      best of both worlds: if id, we are golden and its by id. If by name,
      they have been uniquified already. So this will run into issues if you
      reorder a query or add a new column in with the same name but i think
      that's the theoretical best we can do.
      
      BUT, we have to do a little cleanup for this stuff. When native adds the
      field_ref, it needs to include some type information. But this isn't
      known until after the query runs for native since its just an opaque
      query until we run it. So annotating will add a `[:field name
      {:base_type :type/*}]` and then our merging doesn't clobber that
      later. So its best to add the field_refs, match up with any db metadata,
      and then remove the field_refs.
      
      * Test that metadata flows through
      
      * Test mbql datasets and questions based on datasets
      
      * Test mbql/native queries and nested queries
      
      * Recognize that native query bubbles into nested
      
      When using a nested query based on a native query, the metadata from the
      underlying dataset is used. Previously we would clobber this with the
      metadata from the expected cols of the wrapping mbql query. This would
      process the display name with `humanization/name->human-readable-name`
      whereas for native it goes through `u/qualified-name`.
      
      I originally piped the native's name through the humanization but that
      leads to lots of test failures, and perhaps correct failures. For
      instance, a csv test asserts the column title is "COUNT(*)" but the
      change would emit "Count(*)", a humanization of count(*) isn't
      necessarily an improvement nor even correct.
      
      It is possible that we could change this in the future but I'd want it
      to be a deliberate change. It should be mechanical, just adjusting
      `annotate-native-cols` in annotate.clj to return a humanized display
      name and then fixing tests.
      
      * Allow computed display name on top of source metadata name
      
      If we have a join, we want the "pretty" name to land on top of the
      underlying table's name. "alias → B Column" vs "B Column".
      
      * Put dataset metadata in info, not middleware
      
      * Move metadata back under dataset key in info
      
      We want to ensure that dataset information is propagated, but card
      information should be computed fresh each time. Including the card
      information each time leads to errors as it erroneously thinks the
      existing card info should shadow the dataset information. This is
      actually a tricky case: figuring out when to care about information at
      arbitrary points in the query processor.
      
      * Update metadata to :info not :middleware in tests
      
      * Make var private and comment about info metadata
      5f8bc305
  35. Dec 10, 2021
    • Cam Saul's avatar
      Big QP parameter refactor; validate param :types for Cards (#19188) · 0c4be936
      Cam Saul authored
      * Refactor: move Card and Dashboard QP code into their own qp.* namespaces
      
      * Disable extra validation for now so a million tests don't fail
      
      * WIP
      
      * Validate template tag :parameters in query in context of a Card
      
      * Fixes
      
      * Disable strict validation for now
      
      * Test fixes [WIP]
      
      * Make the parameter type schema a little more forgiving for now
      
      * Tests & test fixes :wrench:
      
      * More test fixes :wrench:
      
      * 1. Need more tests
      2. Need to actually validate stuff
      
      * More test fixes. :wrench:
      
      * Test fixes (again)
      
      * Test fix :wrench:
      
      * Some test fixes / PR feedback
      
      * Disallow native queries with a tag widget-type of "none"
      
      Template tags with a widget-type that is undefined, null, or "none" now
      cause the query's isRunnable method to return false. Existing questions
      that have this defect won't be runnable until they are resaved with a
      set widget-type.
      
      * Fix prettier error
      
      * add snippet and card types to validation pass
      
      * Make sure template tag map keys + `:names` agree + test fixes
      
      * Have MBQL normalization reconcile template tags map key and :name
      
      * Test fix :wrench:
      
      * Fix tests for Cljs
      
      * Fix Mongo tests.
      
      * Allow passing :category parameters for :text/:number/:date for now.
      
      * Dashboard subscriptions should use qp.dashboard code for executing
      
      * Make sure Dashboard QP parameter resolution code merges in default values
      
      * Add a test for sending a test Dashboard subscription with default params
      
      * Prettier
      
      * If both Dashboard and Card have default param value, prefer Card's default
      
      * Test fix :wrench:
      
      * More tests and more fixes :cry:
      
      
      
      Co-authored-by: default avatarDalton Johnson <daltojohnso@users.noreply.github.com>
      0c4be936
  36. Nov 29, 2021
  37. Nov 23, 2021
Loading