Skip to content
Snippets Groups Projects
This project is mirrored from https://github.com/metabase/metabase. Pull mirroring updated .
  1. Dec 14, 2022
    • Nick Fitzpatrick's avatar
      Transition InputBlurChange to use core Input component (#27122) · 31999007
      Nick Fitzpatrick authored
      * Convert to TypeScript, handle ColumnItem and MetadataTable
      
      * more progress
      
      * handled numeric inputs
      
      * Transition InputBlurChange to core Input component
      
      * Removing input class from legacy date picker just incase
      
      * Small style adjustments in Admin and Viz Settings
      
      * working on tests
      
      * Tests passing
      
      * Adjusting styling
      
      * PR Cleanup
      
      * add input size=large (#27217)
      
      * Suppress console log when using `with-log-messages-for-level` (#26468)
      
      * disable additivity when using "with-log-level"
      
      * only set when parent is root
      
      * wording
      
      * remove a test that is no longer needed
      
      * fix indents
      
      * one missing indent fix
      
      * [CI] Add `concurrency` to the `drivers` workflow (#27223)
      
      * Fix datetime-diff helper text (#27224)
      
      * Improve datetimeDiff type error message (#27225)
      
      * Update test
      
      * Update implementations
      
      * Relieve db pressure on api/health check (#27192)
      
      * Relieve db pressure on api/health check
      
      https://github.com/metabase/metabase/issues/26266
      
      Servers under heavy load can be slow to respond to the api/health
      check. This can lead to k8s killing healthy instances happily humming
      along serving requests.
      
      One idea floated was to use QoSFilters
      https://www.eclipse.org/jetty/javadoc/jetty-9/org/eclipse/jetty/servlets/QoSFilter.html
      to prioritize those requests in front of others. But I suspect this
      might not be our bottleneck.
      
      Our health endpoint was updated to see if it could acquire an endpoint
      when we were dealing with connection pool issues. We were reporting the
      instance was healthy once it has finished the init process, but would
      report healthy if 60/15 app-db connections were used and no actual
      queries could complete.
      
      The remedy was adding
      `(sql-jdbc.conn/can-connect-with-spec? {:datasource (mdb.connection/data-source)})`
      to the endpoint. But now to get information about the health of the
      system we have to wait in the queue to get a datasource.
      
      The hope is that this change which monitors for recent db
      checkins (query success) and checkouts (query begun) can be a proxy for
      db activity without having to wait for a connection and hit the db ourselves.
      
      Some simple and crude benchmarking:
      - use `siege` to hit `api/database/<app-db>/sync_schema`
      - in a separate tab, use `siege` to hit `api/health`
      
      Three trials with unconditional db access and conditional db
      access (look for recent activity set by the new `ConnectionCustomizer`).
      
      One siege client is synching the app-db's schema with 80 clients each
      sending 60 requests. the other has 1 client sending 60 requests to api/health.
      
      Run             |  Elapsed Time | max tx  | tx rate
       before change  |    7.16s      |  0.79s  |  8.38 tx/s
       before change  |   23.91s      |  1.44s  |  2.51 tx/s
       before change  |   13.00s      |  0.50s  |  4.62 tx/s
      ----------------------------------------------------
       after change   |    4.46s      |  0.27s  |  13.45 tx/s
       after change   |    5.81s      |  0.61s  |  10.33 tx/s
       after change   |    4.54s      |  0.44s  |  13.22 tx/s
      
      Full(er) results below:
      
      ```
      Unconditional db access
      =======================
      
      siege -c80 -r 40 "http://localhost:3000/api/database/2/sync_schema POST" -H "Cookie: $SESSION"
      
      siege -c 1 -r 60 "http://localhost:3000/api/health
      
      "
      
      Elapsed time:		        7.16 secs
      Response time:		        0.12 secs
      Transaction rate:	        8.38 trans/sec
      Longest transaction:	        0.79
      Shortest transaction:	        0.01
      
      Elapsed time:		       23.91 secs
      Response time:		        0.40 secs
      Transaction rate:	        2.51 trans/sec
      Longest transaction:	        1.44
      Shortest transaction:	        0.02
      
      Elapsed time:		       13.00 secs
      Response time:		        0.22 secs
      Transaction rate:	        4.62 trans/sec
      Longest transaction:	        0.50
      Shortest transaction:	        0.06
      
      Conditional db access
      ==============================================================
      
      Elapsed time:		        4.46 secs
      Response time:		        0.07 secs
      Transaction rate:	       13.45 trans/sec
      Longest transaction:	        0.27
      Shortest transaction:	        0.01
      
      Elapsed time:		        5.81 secs
      Response time:		        0.10 secs
      Transaction rate:	       10.33 trans/sec
      Longest transaction:	        0.61
      Shortest transaction:	        0.00
      
      Elapsed time:		        4.54 secs
      Response time:		        0.08 secs
      Transaction rate:	       13.22 trans/sec
      Longest transaction:	        0.44
      Shortest transaction:	        0.01
      ```
      
      * Remove reflection in `.put` call (not the reflections trategy)
      
      also remove the call to `classloader/the-classloader` as it did nothing
      
      * Comment and settle on a single method
      
      * tests
      
      * select from db twice
      
      had a failure in CI. give it time to do its thing with another db call
      
      * block to wait for timestamp update?
      
      * unflake the tests
      
      tasks and events from outside the thread can hit the db. the
      ConnectionCustomizer is also run from c3p0 controlled threads so we
      can't easily isolate everything to our thread
      
      Was running
      
      ```clojure
      (comment
        (dotimes [n 5]
          (dotimes [_ 100]
            (recent-activity-test)
            (CheckinTracker-test))
          (println (* (inc n) 100)))
          )
      ```
      
      to run the tests 500 times and would keep getting flakes at a rate
      ~1/100 to 1/500. Just frustration for the future.
      
      * typehint
      
      * Switch it up a bit
      
      Tests were flaking in h2 and I don't know why. I'm switching to just
      updating recent activity on most methods.
      
      * final touches
      
      Co-authored-by: default avatarAleksandr Lesnenko <alxnddr@users.noreply.github.com>
      Co-authored-by: default avatarNgoc Khuat <qn.khuat@gmail.com>
      Co-authored-by: default avatarNemanja Glumac <31325167+nemanjaglumac@users.noreply.github.com>
      Co-authored-by: default avatarCal Herries <39073188+calherries@users.noreply.github.com>
      Co-authored-by: default avatardpsutton <dan@dpsutton.com>
      Unverified
      31999007
  2. Dec 13, 2022
  3. Dec 09, 2022
    • Cam Saul's avatar
      Make Dimension unique on `field_id` and deduplicate (#27062) · 622558a1
      Cam Saul authored
      * Make Dimension unique on `field_id` and deduplicate
      
      * Fix rollback for new migrations
      
      * Don't require migration comments to contain 'added' anymore since it's part of the version ID now
      
      * Don't require 'Added <version>' in migration comments anymore
      
      * Fix SerDes test that created duplicate dimensions for one Field
      
      * Silly fix to fix MySQL 5.7
      Unverified
      622558a1
  4. Dec 07, 2022
  5. Dec 06, 2022
  6. Dec 05, 2022
  7. Dec 02, 2022
  8. Dec 01, 2022
    • Braden Shepherdson's avatar
      Serdes v2: Rebuild the directory structure to be more human-friendly (#26793) · f0655fc2
      Braden Shepherdson authored
      There are now three top-level trees:
      
      - regular `collections/path/to/collection/...`
      - `:namespace :snippet` collections in `snippets/path/to/collection/...`
      - `databases/mydb/schemas/PUBLIC/tables/customers/fields/name.yaml`
      
      The path for any given entity is determined by the
      `serdes.base/storage-path` multimethod.
      
      On the ingestion side, things are a bit tricky because the paths don't
      map directly to `:serdes/meta` hierarchies anymore. Instead each model
      registers a function to turn a file path into either a `:serdes/meta`
      hierarcy or nil for a bad match.
      
      Ingestion will fetch all these functions once and then try them all in
      arbitrary order until one matches.
      Unverified
      f0655fc2
  9. Nov 30, 2022
    • Anton Kulyk's avatar
      Migrate new collection form to formik (#26820) · 06afa5f1
      Anton Kulyk authored
      * Add basic `CreateCollectionForm`
      
      * Add `CreateCollectionModal`
      
      * Use new form in "New" menu
      
      * Remove no longer needed `onChangeLocation`
      
      * Use new form in collections ellipsis menu
      
      * Add unit tests
      
      * Don't use legacy form lib in collection plugin
      
      * Add `canManageCollectionAuthorityLevel` utility
      
      * Update `FormCollectionAuthorityLevel`
      
      * Add authority level field to new collection form
      
      * Remove old collection form
      
      * Tweak how authority level form field is exposed
      Unverified
      06afa5f1
    • Anton Kulyk's avatar
      Convert metabase-enterprise/collections plugin to TypeScript (#26801) · 681f3bc2
      Anton Kulyk authored
      * Convert authority level plugin to TypeScript
      
      * Don't use `any` too much
      Unverified
      681f3bc2
    • Anton Kulyk's avatar
      Clean up setting selectors usage (#26637) · f1daa097
      Anton Kulyk authored
      * Clean up setting selector wrappers
      
      * Sort `LicenseAndBillingSettings` imports
      
      * Use `getSettings` in `LicenseAndBillingSettings`
      
      * Fix selectors usage in metabase enterprise
      
      * Fix selectors usage in Slack forms
      
      * Fix selectors usage in `GoogleAuthForm`
      
      * Fix selectors usage in `AppBanner`
      
      * Fix selectors usage in `CloudMigrationHelp`
      
      * Unsubscribe query builder from app settings
      
      They don't seem to be used at all
      
      * Fix selectors usage in auth selectors
      
      * Fix selectors usage across the app
      
      * Fix something happened to an import
      Unverified
      f1daa097
  10. Nov 29, 2022
  11. Nov 22, 2022
  12. Nov 18, 2022
  13. Nov 17, 2022
  14. Nov 16, 2022
  15. Nov 15, 2022
  16. Nov 14, 2022
  17. Nov 09, 2022
    • Ngoc Khuat's avatar
      Remove legacy MBQL syntax in tests (#26309) · cb1da495
      Ngoc Khuat authored
      * [:field-id id] => [:field id nil] or $name. Except places where tests
      are meant to check normalization
      
      * [:datetime-field [:field id] unit] -> [:field id {:temporal-unit unit}]
      or !unit.name
      
      * [:fk-> [:field 1] [:field 2]] => [:field 2 {:source-field 1}]
      Unverified
      cb1da495
  18. Nov 07, 2022
    • Bryan Maass's avatar
      Use all text scorers in the final result to increase scoring signal (#26026) · 1c6e8109
      Bryan Maass authored
      * Uses all text scorers in the final result
      
      - instead of just the maximum one
      - add tests
      
      * add prefix scorer test + fix text-score-with
      
      * linter fixes
      
      * pass in number of results to find
      
      * refactor test function
      
      * fix linter by removing unused namespace: metabase.util
      
      * limit arity of serialize to 3
      
      * make oss-score and ee-score different things
      
      - They were defined to be exactly the same, but should be different!
      - Update some tests that broke when a test function was fixed
      
      * remove extra let
      
      * move rseq back out of sorted-take
      
      * improve test feedback
      
      * force weight of text based scorers always weigh 10
      
      * handle 0 score/weights when normalizing scores
      
      * add nil check
      
      * fix more subtle test differences
      
      * more test fiddling
      
      - still test that :offset and :limit respect limits
      
      * reuse bit->boolean from api collection
      
      * clean up some tests
      
      - filter -> remove
      - replace some magic numbers
      - revert to testing entire maps instead of names of sorted items
      
      * add test, docstring, and weight
      
      * sort ns requires
      
      * responding to most of the review comments
      
      * start our zero-score sum check with 0
      
      * do not tokenize / normalize nil raw-search-string
      
      * force equality in basic search test
      
      * modify test to work in dev and test environments
      
      * use display_name in results when appropriate
      
      - This was looking for the _first_ column that had a non-zero score, but
      actually we need to consider all relevant columns.
      - Uses them to figure out if there is a display name, and if there is,
      to use it.
      - Coppied over the logic about showing :context from the prior approach
      Unverified
      1c6e8109
    • Braden Shepherdson's avatar
      Serdes v2: Search for dependencies both on the filesystem and the appdb (#26217) · 1e4fb63b
      Braden Shepherdson authored
      This allows smaller selective exports that (for example) don't need to
      include the data model if you know the other side has it.
      Unverified
      1e4fb63b
  19. Nov 01, 2022
  20. Oct 31, 2022
  21. Oct 26, 2022
  22. Oct 25, 2022
  23. Oct 24, 2022
Loading