Skip to content
Snippets Groups Projects
This project is mirrored from https://github.com/metabase/metabase. Pull mirroring updated .
  1. Nov 04, 2021
    • Ariya Hidayat's avatar
    • Anton Kulyk's avatar
      Refactor question ViewHeader component (#18827) · 8d8e472b
      Anton Kulyk authored
      * Add basic tests for GUI question header
      
      * Reorganise tests
      
      * Add basic tests for native questions header
      
      * Group saved question tests
      
      * Test filters in question header
      
      * Test "Started from ..." label
      
      * Test refresh button
      
      * Add useToggle hook
      
      * Turn ViewHeader into a functional component
      
      * Reorder imports
      
      * Extract some components
      
      * Extract ViewTitleHeader components
      
      * Fix padding
      
      * Extract visual components
      
      * Fix backticks
      
      * Fix padding
      
      * Fix header items alignment
      
      * Minor fix
      Unverified
      8d8e472b
    • Anton Kulyk's avatar
      Fix notebook editor crash when joining on a custom column (#18845) · 61bfcb85
      Anton Kulyk authored
      * Add repro for #18818
      
      * Fix date time field check in JoinStep
      Unverified
      61bfcb85
    • Alexander Polyankin's avatar
    • Jeff Bruemmer's avatar
      bigquery + java notes (#18846) · 4586f081
      Jeff Bruemmer authored
      Unverified
      4586f081
    • dpsutton's avatar
      Handle api/tiles requests for native queries (#18810) · d55ea823
      dpsutton authored
      * Handle api/tiles requests for native queries
      
      Ordinarily the api takes the source query and adds in a filter clause
      for a region:
      
      ```clojure
      {:database 19
       :query {:source-table 88
               :fields [[:field 562 nil]
                        [:field 574 nil]
                        [:field 576 nil]]
               :limit 2000}
       :type :query}
      
      {:database 19
       :query {:source-table 88
               :fields [[:field 562 nil]
                        [:field 574 nil]
                        [:field 576 nil]]
               :limit 2000
               :filter [:inside
                        [:field 576 nil]
                        [:field 574 nil]
                        -40.97989944013222
                        -179.99999564141046
                        -66.51326189011354
                        -134.99999673105785]}
       :type :query
       :async? false}
      ```
      
      But when native, this would break, for three different reasons:
      - native queries don't necessarily (or possibly) know their field
      ids. So the api request it would construct would have `undefined` in the
      slot where the field-id would normally go. This would cause the route to
      404 as it would fail to match the expected numeric part of the url
      - just passing in the field name isn't suffiencient, because mbql cannot
      natively use an mbql filter clause into a native snippet. We do this to
      a limited extent with filters and substitution but this requires a
      marker like `{{filter}}` placeholder for us to add the filter text.
      - when using `[:field name ...]` type field references we need the
      base-type of a field in order to construct a query.
      
      So the solution:
      - allow the route to take field ids or names
      - if native, rewrite the query to be a nested query using the native
      query as the source query. ie, "select name, lat, long from table" ->
      "select source.* from (select name, lat, long from table) source" but in
      mbql so we can add our fliter clause
      - if a string name is passed in, annotate it with a base-type of
      :type/Float since we are dealing with lats and longs.
      
      A concern was that we need to include the source-metadata. I'm omitting
      this because we are essentially "select * from nested" so we just take
      what we want, and the route already expects the index of the lat and
      long columns and then selects only those from the query results.
      
      As it stands we end up with a working query but a log
      
      ```
      2021-11-02 14:37:25,470 DEBUG middleware.log :: GET /api/tiles/2/1/1/latitude/longitude/1/2/ 200 47.3 ms (5 DB calls) App DB connections: 2/13 Jetty threads: 9/50 (2 idle, 0 queued) (128 total active threads) Queries in flight: 0 (0 queued); postgres DB 19 connections: 3/6 (0 threads blocked)
      2021-11-02 14:37:25,474 WARN middleware.add-implicit-clauses :: Warning: cannot determine fields for an explicit `source-query` unless you also include `source-metadata`.
      2021-11-02 14:37:25,490 WARN middleware.add-implicit-clauses :: Warning: cannot determine fields for an explicit `source-query` unless you also include `source-metadata`.
      2021-11-02 14:37:25,501 DEBUG middleware.log :: GET /api/tiles/2/0/2/latitude/longitude/1/2/ 200 39.4 ms (5 DB calls) App DB connections: 1/13 Jetty threads: 8/50 (2 idle, 0 queued) (128 total active threads) Queries in flight: 0 (0 queued); postgres DB 19 connections: 2/6 (0 threads blocked)
      ```
      
      It might be possible to suppress these logs when we are in a query
      context of `:map-tiles`. Or we might be able to require the frontend to
      send along the metadata to include in the query. I didn't do this yet
      since it seems to work fine and it would just make the urls quite long
      if there are lots of columns (and the metadata can get quite verbose).
      
      ```clojure
      ;; source query of the card
      {:type :native
       :native {:query "select name, latitude, longitude from zomato limit 2000;"
                :template-tags {}}
       :database 19}
      
      ;; nest it into a source query
      {:database 19
       :type :query
       :query {:source-query {:template-tags {}
                              :native "select name, latitude, longitude from zomato limit 2000;"}}}
      
      ;; add the `:inside` filter for the tile
      
      {:database 19
       :type :query
       :query {:source-query {:template-tags {}
                              :native "select name, latitude, longitude from zomato limit 2000;"}
               :filter [:inside
                        [:field
                         "latitude"
                         {:base-type :type/Float}]
                        [:field
                         "longitude"
                         {:base-type :type/Float}]
                        0.0
                        89.99999782070523
                        -66.51326189011354
                        179.99999564141046]}
       :async? false}
      ```
      
      * Add docstring and make function private
      
      * url encode column names to api/tiles
      Unverified
      d55ea823
    • dpsutton's avatar
      Annotate collection tree with datasets and cards (#18833) · cba58286
      dpsutton authored
      * Annotate collection tree with datasets and cards
      
      In order to know which collections have items of interest, they are
      annotated with two new keys: `:here` and `:below`. These keys will have
      a collection (in clojure a set, in js an array) which could contain
      dataset and card, depending if they have those items "here" or in
      collections below this node.
      
      We include information about which subtrees are interesting so we know
      to show or highlight those. We include information about which nodes
      have interesting things versus which contain nodes which have things so
      we can possibly implement drilldown in an intelligent manner.
      
      ie
      
      ```
      .
      ├── root
      ```
      
      could expand automatically to
      
      ```
      .
      ├── root
      │   ├── collection
      │   │   ├── collection that actually has items
      ```
      
      without requiring someone to click on the intermediate collection which
      has no items itself, just a collection with items.
      
      http://localhost:3000/api/collection/tree?tree=true
      
      ```js
      {
          "name": "super",
          "id": 154,
          "location": "/",
          "below": ["dataset"]  ;; indicates collections below have datasets
          "children": [{
              "name": "parent",
              "id": 155,
              "location": "/154/",
              "below": ["dataset"]  ;; indicates the same
              "children": [{
                  "children": [],
                  "slug": "contains",
                  "name": "contains",
                  "here": ["dataset"],  ;; this node has datasets
                  "id": 157,
                  "location": "/154/155/",
              }],
      
          }, {
              "children": [],
              "name": "sibling",
              "id": 156,
              "location": "/154/",
          }],
      }
      ```
      
      * docstring
      
      * Alignment for readability
      Unverified
      cba58286
    • Jeff Evans's avatar
      Change all active TEXT columns in MySQL app DB to LONGTEXT (#18749) · b4610877
      Jeff Evans authored
      
      * Change all active TEXT columns in MySQL app DB to LONGTEXT
      
      Add new text.type Liquibase property to dynamically select the correct text type to use, per DB (which is LONGTEXT for MySQL), very similar to the existing blob.type one
      
      Adding new migrations that update all existing app DB TEXT columns to LONGTEXT columns, only for mariadb/mysql
      
      Update migrations linter to ensure no new text types get added, via a new predicate that searches for text types being added
      
      Do the same logic for "blob" now (should be "${blob.type}"), update rule and test
      
      Add test for migrations, that checks the new types, for all types in scope for this PR, to ensure that they have all been changed to their expected DB-specific type (either LONGTEXT for MySQL or TEXT/CLOB for others)
      
      Updating a couple migrations to remove special casing that was originally done only for MySQL to simply make them universally "${text.type}", in order to unify behavior and reduce surprises going forward
      
      Co-authored-by: default avatarCam Saul <github@camsaul.com>
      Unverified
      b4610877
    • Alexander Polyankin's avatar
    • Ariya Hidayat's avatar
    • Jeff Bruemmer's avatar
      docs - custom expressions (#18836) · fc31d2b8
      Jeff Bruemmer authored
      Unverified
      fc31d2b8
  2. Nov 03, 2021
    • Cam Saul's avatar
      Add NOT NULL constraint to Card.database_id; attempt to set database_id if possible (#18472) · 83de3e86
      Cam Saul authored
      * Drop long-unused Table.entity_name column
      
      * Add NOT NULL constraint to Card.database_id
      
      * Test fix :wrench:
      
      * Test fix :wrench:
      
      * Fix migration
      
      * Add note about H2 shell to deps.edn
      
      * Add new SQL migration to attempt to set database_id when unset
      
      * Remove data migration that is now done in Liquibase land
      
      * Oops, '$.database', not '$.database_id'
      
      * Parse ints as signed rather than unsigned just to be safe (they *might* be -1337 if they're really broken)
      
      * Don't run for H2
      
      * Update comment
      
      * Fix migration indentation
      
      * Clean namespace
      
      * Use new migration number.
      
      * Use the new migration numbers
      
      * Adopt new migration numbering scheme
      
      * Fix comments
      
      * Fix MySQL + MariaDB insanity
      
      * Fix ID range validation
      
      * Actually 382 is the last legacy ID
      
      * Improved validation and tests
      
      * Adopt the new-new migration ID format.
      
      * Test fixes :wrench:
      
      * Fix merge
      
      * Simplify precondition
      Unverified
      83de3e86
    • Cam Saul's avatar
      Add index to ModerationReview moderated_item_type + moderated_item_id (#18799) · 7d614623
      Cam Saul authored
      * Revert changes from Jeff's PR
      
      * Add index to ModerationReview moderated_item_type + moderated_item_id
      
      * Add to 0.41.2 instead
      
      * Require explicit index name for createIndex
      
      * Move 41.2 migrations to after the 41.0 migrations
      
      * Adopt new migration numbering scheme
      
      * Fix comments
      
      * Fix MySQL + MariaDB insanity
      
      * Fix ID range validation
      
      * Actually 382 is the last legacy ID
      
      * Improved validation and tests
      
      * Adopt the new-new migration ID format.
      
      * Test fixes :wrench:
      Unverified
      7d614623
    • Cam Saul's avatar
      Adopt new migration numbering scheme (#18821) · 2fee04b7
      Cam Saul authored
      * Adopt new migration numbering scheme
      
      * Fix comments
      
      * Fix MySQL + MariaDB insanity
      
      * Fix ID range validation
      
      * Actually 382 is the last legacy ID
      
      * Improved validation and tests
      
      * Adopt the new-new migration ID format.
      
      * Test fixes :wrench:
      Unverified
      2fee04b7
    • Anton Kulyk's avatar
      Update query builder data selector for datasets (#18764) · 868f8782
      Anton Kulyk authored
      * Add unit tests for schema utils
      
      * Add data bucket step to DataSelector
      
      * Go to next steps from the data bucket step
      
      * Add different type of AccordionList icons layout
      
      * Add back navigation from 'Raw Data' tab
      
      * Fix selecting a database in 'Raw Data'
      
      * Enable search on data bucket step
      
      * Only render data bucket step if there are datasets
      
      * Prevent opening the data bucket step without datasets
      
      * Update picker search placeholder text
      
      * Update searchable models
      
      * Fix displaying datasets in search results
      
      * Hide "Saved Questions" in "Raw Data" tab
      
      * Encode data in schema str
      
      * Add ability to request dataset virtual tables
      
      * Support datasets in SavedQuestionPicker
      
      * Use saved question picker for datasets too
      
      * Add E2E tests
      
      * Show dataset icon in search results
      
      * Fix E2E selectors
      
      * Fix AccordionList icon rendering
      
      * Fix dataset can't be selected in search
      
      * Fix data selector step hydration
      
      * Fix JoinStep unit tests
      
      * Fix dataset icon size in picker view
      
      * Fix schema picker without datasets
      
      * Fix DataSelector tests
      
      * Fix error about null data selector step
      
      * Move schema utils to `metabase/lib`
      
      * Fix questions don't appear in pickers after created
      
      * Fix JoinStep unit tests
      
      * Test can create a question based on a dataset
      
      * Simplify schema ID parsing
      
      * Fix typo
      
      * Fix search models
      
      * Fix ellipsis character
      
      * Fix "Raw Data" button click area
      
      * Remove temp comment
      
      * Fix URL in personal collection contents request
      Unverified
      868f8782
    • Ariya Hidayat's avatar
    • Anton Kulyk's avatar
      Add visual tests for table visualisation (#18826) · b7cf663b
      Anton Kulyk authored
      * Add visual tests for table visualisation
      
      * Wait until spinner disappears
      Unverified
      b7cf663b
  3. Nov 02, 2021
  4. Nov 01, 2021
  5. Oct 29, 2021
Loading