Skip to content
Snippets Groups Projects
This project is mirrored from https://github.com/metabase/metabase. Pull mirroring updated .
  1. Aug 14, 2024
  2. Aug 07, 2024
    • dpsutton's avatar
      Exclude javax.annotation/javax.annotation-api (#46546) · e5a3827e
      dpsutton authored
      dep is brought in by google cloud
      
      From `clj -X:deps tree :aliases '[:ee :drivers]'`
      
      ```
        . metabase/bigquery-cloud-sdk metabase/modules/drivers/bigquery-cloud-sdk
          . com.google.cloud/google-cloud-bigquery 2.38.1
            . com.google.cloud/google-cloud-core 2.35.0
            [...]
            . javax.annotation/javax.annotation-api 1.3.2
      ```
      
      Before this change:
      
      ```
      ❯ clj -X:deps tree :aliases '[:ee :drivers]' | grep 'javax.annotation/javax.annotation-api'
            . javax.annotation/javax.annotation-api 1.3.2
      ```
      
      After:
      
      ```
      ❯ clj -X:deps tree :aliases '[:ee :drivers]' > deps-master
      
      ❯ echo $?
      0
      ```
      Unverified
      e5a3827e
  3. Aug 05, 2024
  4. Aug 01, 2024
    • lbrdnk's avatar
      Avoid adding temporal-unit to lhs cols (#46262) · 07648603
      lbrdnk authored
      * Avoid adding temporal-unit to lhs cols
      
      * Use :default temporal-unit
      
      * Simplify logic, update comments
      
      * Update substitute-field-filter-test
      
      * Update align-temporal-unit-with-param-type-test
      
      * Update field-filter-date-test
      
      * Use end-excludding gte lt filter for DateTime fields
      
      * Update substitute-field-filter-test
      
      * Update align-temporal-unit-with-param-type-test
      
      * Update field-filter-date-test
      
      * Update bigquery test + comment
      
      * Update date-str->qp-aware-offset-dt
      
      * Add guard for unexpected date string format
      
      This is just for the completeness, I haven't encountered it.
      
      * Address review remarks
      
      * Update comment
      Unverified
      07648603
  5. Jul 31, 2024
  6. Jul 25, 2024
  7. Jul 23, 2024
  8. Jul 19, 2024
    • lbrdnk's avatar
      Add basic auth params to `dbms-version` call on Druid (#45729) · a5eb647e
      lbrdnk authored
      * Add basic auth to dbms-version
      
      * Add test
      Unverified
      a5eb647e
    • lbrdnk's avatar
      Remove `:foreign-keys` feature or convert to `:metadata/key-constraints` where appropriate (#44894) · 9c708c21
      lbrdnk authored
      
      * Post sync hook stub for implicit joins testing stub
      
      * Add join alias to field lvalues to enable sorting on joined fields
      
      * Disable :foreign-keys on Mongo
      
      * Require :left-join support for implicit joins instead of :foreign-keys
      
      * Update implicit joins tests
      
      * Adjust sync-fields-test
      
      * Update implicit joins feature check test
      
      * Transform post-sync-hook to normal function
      
      * Add foreign key relationships only for dbmses without :foreign-keys feature
      
      * Update test to handle Oracle correctly
      
      * Split convoluted fn
      
      * Avoid unnecessary computations for datasets with no fks
      
      * Update docstring
      
      * Fix driver usage
      
      * Transform :foreign-keys to :metadata/key-constraints in test data loading code
      
      * Update sync_test.clj
      
      * Update driver_test.clj
      
      * Update moviedb.clj
      
      * Update dataset_definition_test.clj
      
      * Update fetch_metadata.clj
      
      * Update fields_test.clj
      
      * Update driver.clj
      
      * Update driver/sql.clj
      
      * Set sql driver join support to true for all joins
      
      Deriving drivers are expected to set to false where applicable.
      
      * Update sqllite.clj
      
      * Remove foreign-keys from spark
      
      * Remove :foreign-keys from presto
      
      * Remove :foreign-keys from Athena
      
      * Remove foreign-keys from big query
      
      Reading docs it seems fk inference should be ok. Let's see the test results. Act based on that.
      
      * Update test_metadata.cljc
      
      * Set key-constraints to false for bigquery
      
      * Add foreign keys to sqlite manually until sync is fixed
      
      * Return driver require to Athena
      
      * Correct typo
      
      * Add naive primary key heuristic
      
      * Update pk fk logic to handle name components correctly
      
      * Add alias escaping to presto
      
      * Add ordering to test
      
      * Add order by to test
      
      * Update test
      
      * Remove use of rewrite-fields-to-force-using-column-aliases in order by fields
      
      * Add exception to alias forcing
      
      * Different approach to exception from alias forcing
      
      * Alternative approach for prefixing idents in bigq
      
      * All seelcted fields by desired alias
      
      * Rewrite only fields not from this source table
      
      * Update test
      
      * Enable breakout-on-fk-field-test for :left-join drivers
      
      * Add feature comment
      
      * Explicit joins tests foreign-keys removal
      
      * Update nested_queries_test.clj
      
      * Update remapping tests
      
      * Update tests
      
      * Update tests to handle sqlite results format
      
      * Disable metadata/key-constraints on sqlite during tests until
      
      * Address remarks
      
      * Remove mt/with-mock-fks-for-drivers-without-fk-constraints
      
      * Update bigquery test
      
      * Update tests
      
      * Adjust row level restrictions
      
      * Add parameterized-sql feature
      
      * Update comment
      
      * Update leftovers
      
      * Order keys
      
      * Remove foreign keys from frontend
      
      * Fix FE unit
      
      * Update driver changelog
      
      * Address review remark
      
      * Update test/metabase/query_processor/test_util.clj
      
      Co-authored-by: default avatarmetamben <103100869+metamben@users.noreply.github.com>
      
      * Update docs/developers-guide/driver-changelog.md
      
      Co-authored-by: default avatarmetamben <103100869+metamben@users.noreply.github.com>
      
      * Update docs/developers-guide/driver-changelog.md
      
      Co-authored-by: default avatarmetamben <103100869+metamben@users.noreply.github.com>
      
      * Update modules/drivers/bigquery-cloud-sdk/src/metabase/driver/bigquery_cloud_sdk/query_processor.clj
      
      Co-authored-by: default avatarmetamben <103100869+metamben@users.noreply.github.com>
      
      * Update docs/developers-guide/driver-changelog.md
      
      Co-authored-by: default avatarmetamben <103100869+metamben@users.noreply.github.com>
      
      * Update test
      
      * Update comment and reduce expression
      
      * Update comment
      
      * Address remarks
      
      * Fix merge
      
      ---------
      
      Co-authored-by: default avatarmetamben <103100869+metamben@users.noreply.github.com>
      Unverified
      9c708c21
  9. Jul 18, 2024
    • Case Nelson's avatar
      test: add database feature to test identifiers with spaces (#45680) · bcd5a002
      Case Nelson authored
      * tests for drivers with spaces in their identifiers
      
      * Add driver feature
      
      * Fix tests
      
      * Remove ambiguous alias for sqlite
      
      * Try to fix bigquery and sqlite tests not seeing left-join support
      
      * Add explicit support for left-join because of foreign-keys during test hacks
      
      * Handle left-join support weirdness
      
      * Fix test
      Unverified
      bcd5a002
    • Cam Saul's avatar
      Use single connection for test data loading & big test data loading improvements (#45268) · bab905c0
      Cam Saul authored
      * Reuse connection for test data loading & JDBC data loading overhaul
      
      * Revert silly change
      
      * Remove unneeded logic for recording which datasets are loaded or not-yet-loaded
      
      * Ok so I guess we do need the code to detect duplicate dataset loading after all
      
      * B I G  improvements
      
      * Appease Kondo
      
      * Test fixes :wrench:
      
      * More test fixes :wrench:
      
      * Fix dataset-already-loaded?-test
      
      * Fix test
      
      * Fix Presto JDBC and greatly improve test data load speed for it
      
      * Postgres test fix :wrench:
      
      * Fix compilation errors
      
      * MySQL fixes :wrench:
      
      * Fix syntax error
      
      * MySQL fixes :wrench:
      
      * Implement `dataset-already-loaded?` for Athena and cache the set of existing databases
      
      * Fix Redshift
      
      * Fix Presto JDBC
      
      * Vertica test fix :wrench:
      
      * Fix Oracle dataset-already-loaded?
      
      * Test fixes and parallelize some API tests.
      
      * Test fixes :wrench:
      
      * Oracle test fix? :wrench:
      
      * Another test fix :wrench:
      
      * Stop putting random spaces in code bodies this is not JavaScript code
      
      * with-temp-vals-in-db should merge in original value of Database settings
      
      * The most test fixes :wrench:
      
      * 3 or 4 test fixes :wrench:
      
      * Add schemas around stuff to catch errors
      
      * Appease Kondo
      
      * Implement dataset-already-loaded? for vertica
      
      * Implement dataset-already-loaded? for SQLite and Snowflake
      Unverified
      bab905c0
  10. Jul 17, 2024
  11. Jul 16, 2024
  12. Jul 12, 2024
    • Braden Shepherdson's avatar
      [perf] Fix remaining thread leak in BigQuery query execution (#45253) · 52d77200
      Braden Shepherdson authored
      This comes in two parts.
      
      **Rebuilt query logic**
      
      I rewrote the query exec logic to use one `future` and one `a/go` block,
      reporting results to a `promise`. This is easier to follow and doesn't leak
      threads like the original implementation with two `future`s.
      
      The single future is guaranteed to exit. It blocks on the BigQuery call.
      That will either complete or throw an exception. Either the exception or
      the BigQuery result is sent to the promise, and then the future is
      complete. The only reference to that future is overwritten after the `promise`
      resolves, so the `future`, its closure, and its thread don't persist longer than needed.
      
      **Return a reducible result**
      
      Previously the results were returned as a lazy sequence, but that brings
      a lot of `seq` overhead, and is a risk of retaining the entire result
      set in memory until the query is all done. Above all, it doesn't support
      `reduced?` to explicitly drop any further results.
      
      The `thunk` function style of `qp.reducible/reducible-rows` doesn't fit
      neatly here, so I implemented the reducible logic directly on top of the
      `Iterable` BigQuery results.
      
      **Verifying**
      
      This is a bit tricky. I've been testing it like this:
      
      - Add a BigQuery database to my instance
      - Create a saved question and note its ID (say, 123)
      - Create an API key in (EE) Metabase and save it in an env var
      - Install the `siege` load testing tool
      - Install VisualVM and monitor my Clojure REPL process's thread count
      - `siege --concurrent=15 --time=30s --header="x-api-key: $API_KEY" "http://localhost:3000/api/card/123/query POST"`
      
      That will create a bunch of new threads while it's getting called, but
      within a minute of the last queries they'll gradually drop out of the
      thread pools and it will return to the baseline number of threads.
  13. Jul 11, 2024
  14. Jul 08, 2024
  15. Jul 03, 2024
  16. Jul 02, 2024
  17. Jun 27, 2024
    • dpsutton's avatar
      Ensure we close the canceled chan in query processor after query (#44860) · 9fb1bd7d
      dpsutton authored
      Customer complaints that thread growth was unbounded. Investigated and
      found bigquery involved and isolated to the cancel thread bits. Gist is
      this:
      
      ```clojure
            (when cancel-chan
              (future                       ; this needs to run in a separate thread, because the <!! operation blocks forever
                (when (a/<!! cancel-chan)
                  (log/debug "Received a message on the cancel channel; attempting to stop the BigQuery query execution")
                  (reset! cancel-requested? true) ; signal the page iteration fn to stop
                  (if-not (or (future-cancelled? res-fut) (future-done? res-fut))
                    ;; somehow, even the FIRST page hasn't come back yet (i.e. the .query call above), so cancel the future to
                    ;; interrupt the thread waiting on that response to come back
                    ;; unfortunately, with this particular overload of .query, we have no access to (nor the ability to control)
                    ;; the jobId, so we have no way to use the BigQuery client to cancel any job that might be running
                    (future-cancel res-fut)
                    (when (future-done? res-fut) ; canceled received after it was finished; may as well return it
                      @res-fut)))))
      ```
      
      The important parts is that we closeover the cancel-chan and start a
      thread that is trying to pull from it. If we never put anything on that
      channel or never close that channel this thread will just remain blocked
      waiting.
      
      In a thread dump, that looks like the following:
      (note, I've made it set the thread names to make it easy to follow run a
      query 30 times)
      
      ```
      Full thread dump OpenJDK 64-Bit Server VM (21.0.2+13-LTS mixed mode):
      ...
      
      "bigquery-cancel-thread" #73 [95747] prio=5 os_prio=31 cpu=2.76ms elapsed=208.53s tid=0x000000029e00f600 nid=95747 waiting on condition  [0x00000002c1266000]
         java.lang.Thread.State: WAITING (parking)
      	at jdk.internal.misc.Unsafe.park(java.base@21.0.2/Native Method)
      --
      "bigquery-cancel-thread" #108 [93211] prio=5 os_prio=31 cpu=154.60ms elapsed=128.72s tid=0x000000029f0dc200 nid=93211 waiting on condition  [0x000000029fa06000]
         java.lang.Thread.State: WAITING (parking)
      	at jdk.internal.misc.Unsafe.park(java.base@21.0.2/Native Method)
      --
      "bigquery-cancel-thread" #116 [116527] prio=5 os_prio=31 cpu=0.16ms elapsed=61.14s tid=0x00000002c1917c00 nid=116527 waiting on condition  [0x000000029fc12000]
         java.lang.Thread.State: WAITING (parking)
      	at jdk.internal.misc.Unsafe.park(java.base@21.0.2/Native Method)
      --
      "bigquery-cancel-thread" #118 [98315] prio=5 os_prio=31 cpu=161.69ms elapsed=60.27s tid=0x00000002b91cca00 nid=98315 waiting on condition  [0x000000029fe1e000]
         java.lang.Thread.State: WAITING (parking)
      	at jdk.internal.misc.Unsafe.park(java.base@21.0.2/Native Method)
      --
      "bigquery-cancel-thread" #119 [121607] prio=5 os_prio=31 cpu=0.13ms elapsed=59.35s tid=0x00000002b9236400 nid=121607 waiting on condition  [0x00000002a002a000]
         java.lang.Thread.State: WAITING (parking)
      	at jdk.internal.misc.Unsafe.park(java.base@21.0.2/Native Method)
      --
      "bigquery-cancel-thread" #120 [99079] prio=5 os_prio=31 cpu=0.13ms elapsed=58.55s tid=0x000000029e295600 nid=99079 waiting on condition  [0x00000002a0506000]
         java.lang.Thread.State: WAITING (parking)
      	at jdk.internal.misc.Unsafe.park(java.base@21.0.2/Native Method)
      --
      "bigquery-cancel-thread" #121 [122375] prio=5 os_prio=31 cpu=0.10ms elapsed=57.75s tid=0x00000002b921f400 nid=122375 waiting on condition  [0x00000002c0e4e000]
         java.lang.Thread.State: WAITING (parking)
      	at jdk.internal.misc.Unsafe.park(java.base@21.0.2/Native Method)
      --
      "bigquery-cancel-thread" #122 [123655] prio=5 os_prio=31 cpu=0.18ms elapsed=56.97s tid=0x0000000134bfa000 nid=123655 waiting on condition  [0x00000002c105a000]
         java.lang.Thread.State: WAITING (parking)
      	at jdk.internal.misc.Unsafe.park(java.base@21.0.2/Native Method)
      --
      "bigquery-cancel-thread" #123 [122631] prio=5 os_prio=31 cpu=0.11ms elapsed=56.28s tid=0x000000013569be00 nid=122631 waiting on condition  [0x00000002c1472000]
         java.lang.Thread.State: WAITING (parking)
      	at jdk.internal.misc.Unsafe.park(java.base@21.0.2/Native Method)
      --
      "bigquery-cancel-thread" #124 [96007] prio=5 os_prio=31 cpu=0.10ms elapsed=55.42s tid=0x0000000133b54e00 nid=96007 waiting on condition  [0x00000002c167e000]
         java.lang.Thread.State: WAITING (parking)
      	at jdk.internal.misc.Unsafe.park(java.base@21.0.2/Native Method)
      --
      "bigquery-cancel-thread" #125 [97799] prio=5 os_prio=31 cpu=0.09ms elapsed=54.73s tid=0x0000000131c19a00 nid=97799 waiting on condition  [0x00000002c2206000]
         java.lang.Thread.State: WAITING (parking)
      	at jdk.internal.misc.Unsafe.park(java.base@21.0.2/Native Method)
      --
      "bigquery-cancel-thread" #126 [96519] prio=5 os_prio=31 cpu=0.10ms elapsed=53.97s tid=0x000000012370f600 nid=96519 waiting on condition  [0x00000002c2412000]
         java.lang.Thread.State: WAITING (parking)
      	at jdk.internal.misc.Unsafe.park(java.base@21.0.2/Native Method)
      --
      "bigquery-cancel-thread" #127 [124167] prio=5 os_prio=31 cpu=0.10ms elapsed=53.33s tid=0x0000000134312200 nid=124167 waiting on condition  [0x00000002c28ea000]
         java.lang.Thread.State: WAITING (parking)
      	at jdk.internal.misc.Unsafe.park(java.base@21.0.2/Native Method)
      --
      "bigquery-cancel-thread" #128 [97287] prio=5 os_prio=31 cpu=0.12ms elapsed=52.57s tid=0x00000001233ae000 nid=97287 waiting on condition  [0x00000002c4706000]
         java.lang.Thread.State: WAITING (parking)
      	at jdk.internal.misc.Unsafe.park(java.base@21.0.2/Native Method)
      --
      "bigquery-cancel-thread" #129 [104715] prio=5 os_prio=31 cpu=0.10ms elapsed=51.83s tid=0x0000000131e47a00 nid=104715 waiting on condition  [0x00000002c4912000]
         java.lang.Thread.State: WAITING (parking)
      	at jdk.internal.misc.Unsafe.park(java.base@21.0.2/Native Method)
      --
      "bigquery-cancel-thread" #130 [102919] prio=5 os_prio=31 cpu=0.08ms elapsed=51.10s tid=0x0000000123540200 nid=102919 waiting on condition  [0x00000002c4b1e000]
         java.lang.Thread.State: WAITING (parking)
      	at jdk.internal.misc.Unsafe.park(java.base@21.0.2/Native Method)
      --
      "bigquery-cancel-thread" #131 [117263] prio=5 os_prio=31 cpu=0.17ms elapsed=50.37s tid=0x00000001233d1200 nid=117263 waiting on condition  [0x00000002c4d2a000]
         java.lang.Thread.State: WAITING (parking)
      	at jdk.internal.misc.Unsafe.park(java.base@21.0.2/Native Method)
      --
      "bigquery-cancel-thread" #132 [114951] prio=5 os_prio=31 cpu=0.09ms elapsed=49.71s tid=0x000000013484dc00 nid=114951 waiting on condition  [0x00000002c4f36000]
         java.lang.Thread.State: WAITING (parking)
      	at jdk.internal.misc.Unsafe.park(java.base@21.0.2/Native Method)
      --
      "bigquery-cancel-thread" #133 [103687] prio=5 os_prio=31 cpu=0.14ms elapsed=49.08s tid=0x0000000123517200 nid=103687 waiting on condition  [0x00000002c5142000]
         java.lang.Thread.State: WAITING (parking)
      	at jdk.internal.misc.Unsafe.park(java.base@21.0.2/Native Method)
      --
      "bigquery-cancel-thread" #134 [91419] prio=5 os_prio=31 cpu=0.08ms elapsed=48.38s tid=0x0000000131b43600 nid=91419 waiting on condition  [0x00000002c534e000]
         java.lang.Thread.State: WAITING (parking)
      	at jdk.internal.misc.Unsafe.park(java.base@21.0.2/Native Method)
      --
      "bigquery-cancel-thread" #135 [127495] prio=5 os_prio=31 cpu=0.14ms elapsed=47.66s tid=0x0000000131ce7000 nid=127495 waiting on condition  [0x00000002c555a000]
         java.lang.Thread.State: WAITING (parking)
      	at jdk.internal.misc.Unsafe.park(java.base@21.0.2/Native Method)
      --
      "bigquery-cancel-thread" #139 [103955] daemon prio=5 os_prio=31 cpu=0.12ms elapsed=23.22s tid=0x0000000121aaca00 nid=103955 waiting on condition  [0x00000002c5766000]
         java.lang.Thread.State: WAITING (parking)
      	at jdk.internal.misc.Unsafe.park(java.base@21.0.2/Native Method)
      --
      "bigquery-cancel-thread" #140 [105219] daemon prio=5 os_prio=31 cpu=0.10ms elapsed=22.54s tid=0x0000000133f47600 nid=105219 waiting on condition  [0x00000002c5972000]
         java.lang.Thread.State: WAITING (parking)
      	at jdk.internal.misc.Unsafe.park(java.base@21.0.2/Native Method)
      --
      "bigquery-cancel-thread" #141 [105731] daemon prio=5 os_prio=31 cpu=0.15ms elapsed=21.69s tid=0x00000001353ba800 nid=105731 waiting on condition  [0x00000002c5b7e000]
         java.lang.Thread.State: WAITING (parking)
      	at jdk.internal.misc.Unsafe.park(java.base@21.0.2/Native Method)
      --
      "bigquery-cancel-thread" #142 [114435] daemon prio=5 os_prio=31 cpu=0.08ms elapsed=20.89s tid=0x0000000131e76400 nid=114435 waiting on condition  [0x00000002c5d8a000]
         java.lang.Thread.State: WAITING (parking)
      	at jdk.internal.misc.Unsafe.park(java.base@21.0.2/Native Method)
      --
      "bigquery-cancel-thread" #143 [106499] daemon prio=5 os_prio=31 cpu=0.10ms elapsed=20.12s tid=0x0000000123107e00 nid=106499 waiting on condition  [0x00000002c5f96000]
         java.lang.Thread.State: WAITING (parking)
      	at jdk.internal.misc.Unsafe.park(java.base@21.0.2/Native Method)
      --
      "bigquery-cancel-thread" #144 [113923] daemon prio=5 os_prio=31 cpu=0.11ms elapsed=19.52s tid=0x00000002c1918400 nid=113923 waiting on condition  [0x00000002c61a2000]
         java.lang.Thread.State: WAITING (parking)
      	at jdk.internal.misc.Unsafe.park(java.base@21.0.2/Native Method)
      --
      "bigquery-cancel-thread" #145 [107011] daemon prio=5 os_prio=31 cpu=0.13ms elapsed=18.93s tid=0x00000002a6ad3e00 nid=107011 waiting on condition  [0x00000002c63ae000]
         java.lang.Thread.State: WAITING (parking)
      	at jdk.internal.misc.Unsafe.park(java.base@21.0.2/Native Method)
      --
      "bigquery-cancel-thread" #146 [113667] daemon prio=5 os_prio=31 cpu=0.12ms elapsed=18.37s tid=0x00000001341a2800 nid=113667 waiting on condition  [0x00000002c65ba000]
         java.lang.Thread.State: WAITING (parking)
      	at jdk.internal.misc.Unsafe.park(java.base@21.0.2/Native Method)
      --
      "bigquery-cancel-thread" #147 [113155] daemon prio=5 os_prio=31 cpu=0.16ms elapsed=17.74s tid=0x000000013445da00 nid=113155 waiting on condition  [0x00000002c67c6000]
         java.lang.Thread.State: WAITING (parking)
      	at jdk.internal.misc.Unsafe.park(java.base@21.0.2/Native Method)
      --
      "bigquery-cancel-thread" #148 [107779] daemon prio=5 os_prio=31 cpu=0.13ms elapsed=17.19s tid=0x0000000134704e00 nid=107779 waiting on condition  [0x00000002c69d2000]
         java.lang.Thread.State: WAITING (parking)
      	at jdk.internal.misc.Unsafe.park(java.base@21.0.2/Native Method)
      --
      "bigquery-cancel-thread" #149 [112643] daemon prio=5 os_prio=31 cpu=0.17ms elapsed=16.54s tid=0x000000011f185000 nid=112643 waiting on condition  [0x00000002c6bde000]
         java.lang.Thread.State: WAITING (parking)
      	at jdk.internal.misc.Unsafe.park(java.base@21.0.2/Native Method)
      --
      "bigquery-cancel-thread" #150 [108035] daemon prio=5 os_prio=31 cpu=0.24ms elapsed=15.95s tid=0x000000011f1be200 nid=108035 waiting on condition  [0x00000002c6dea000]
         java.lang.Thread.State: WAITING (parking)
      	at jdk.internal.misc.Unsafe.park(java.base@21.0.2/Native Method)
      --
      "bigquery-cancel-thread" #151 [108547] daemon prio=5 os_prio=31 cpu=0.11ms elapsed=15.38s tid=0x00000001302cf800 nid=108547 waiting on condition  [0x00000002c6ff6000]
         java.lang.Thread.State: WAITING (parking)
      	at jdk.internal.misc.Unsafe.park(java.base@21.0.2/Native Method)
      --
      "bigquery-cancel-thread" #152 [109059] daemon prio=5 os_prio=31 cpu=0.24ms elapsed=14.85s tid=0x0000000133c4be00 nid=109059 waiting on condition  [0x00000002c7202000]
         java.lang.Thread.State: WAITING (parking)
      	at jdk.internal.misc.Unsafe.park(java.base@21.0.2/Native Method)
      --
      "bigquery-cancel-thread" #153 [109315] daemon prio=5 os_prio=31 cpu=0.10ms elapsed=14.15s tid=0x00000001339dc000 nid=109315 waiting on condition  [0x00000002c740e000]
         java.lang.Thread.State: WAITING (parking)
      	at jdk.internal.misc.Unsafe.park(java.base@21.0.2/Native Method)
      --
      "bigquery-cancel-thread" #154 [109571] daemon prio=5 os_prio=31 cpu=0.17ms elapsed=13.57s tid=0x0000000134ea3600 nid=109571 waiting on condition  [0x00000002c761a000]
         java.lang.Thread.State: WAITING (parking)
      	at jdk.internal.misc.Unsafe.park(java.base@21.0.2/Native Method)
      --
      "bigquery-cancel-thread" #155 [110083] daemon prio=5 os_prio=31 cpu=0.20ms elapsed=12.95s tid=0x0000000121d7aa00 nid=110083 waiting on condition  [0x00000002c7826000]
         java.lang.Thread.State: WAITING (parking)
      	at jdk.internal.misc.Unsafe.park(java.base@21.0.2/Native Method)
      --
      "bigquery-cancel-thread" #156 [110339] daemon prio=5 os_prio=31 cpu=0.12ms elapsed=12.35s tid=0x0000000134ff5c00 nid=110339 waiting on condition  [0x00000002c7a32000]
         java.lang.Thread.State: WAITING (parking)
      	at jdk.internal.misc.Unsafe.park(java.base@21.0.2/Native Method)
      --
      "bigquery-cancel-thread" #157 [110595] daemon prio=5 os_prio=31 cpu=0.09ms elapsed=11.65s tid=0x0000000133e27000 nid=110595 waiting on condition  [0x00000002c7c3e000]
         java.lang.Thread.State: WAITING (parking)
      	at jdk.internal.misc.Unsafe.park(java.base@21.0.2/Native Method)
      --
      "bigquery-cancel-thread" #158 [111107] daemon prio=5 os_prio=31 cpu=0.13ms elapsed=11.04s tid=0x0000000131e8b800 nid=111107 waiting on condition  [0x00000002c7e4a000]
         java.lang.Thread.State: WAITING (parking)
      	at jdk.internal.misc.Unsafe.park(java.base@21.0.2/Native Method)
      --
      "bigquery-cancel-thread" #159 [174339] daemon prio=5 os_prio=31 cpu=0.13ms elapsed=10.51s tid=0x0000000134ff6400 nid=174339 waiting on condition  [0x00000002c8056000]
         java.lang.Thread.State: WAITING (parking)
      	at jdk.internal.misc.Unsafe.park(java.base@21.0.2/Native Method)
      --
      "bigquery-cancel-thread" #160 [173827] daemon prio=5 os_prio=31 cpu=0.12ms elapsed=9.79s tid=0x0000000133ff7c00 nid=173827 waiting on condition  [0x00000002c8262000]
         java.lang.Thread.State: WAITING (parking)
      	at jdk.internal.misc.Unsafe.park(java.base@21.0.2/Native Method)
      --
      "bigquery-cancel-thread" #161 [173315] daemon prio=5 os_prio=31 cpu=0.11ms elapsed=9.18s tid=0x0000000122c9b600 nid=173315 waiting on condition  [0x00000002c846e000]
         java.lang.Thread.State: WAITING (parking)
      	at jdk.internal.misc.Unsafe.park(java.base@21.0.2/Native Method)
      --
      "bigquery-cancel-thread" #162 [131331] daemon prio=5 os_prio=31 cpu=0.12ms elapsed=8.58s tid=0x0000000133b9c400 nid=131331 waiting on condition  [0x00000002c867a000]
         java.lang.Thread.State: WAITING (parking)
      	at jdk.internal.misc.Unsafe.park(java.base@21.0.2/Native Method)
      --
      "bigquery-cancel-thread" #163 [172547] daemon prio=5 os_prio=31 cpu=0.14ms elapsed=8.06s tid=0x0000000134d4cc00 nid=172547 waiting on condition  [0x00000002c8886000]
         java.lang.Thread.State: WAITING (parking)
      	at jdk.internal.misc.Unsafe.park(java.base@21.0.2/Native Method)
      --
      "bigquery-cancel-thread" #164 [131587] daemon prio=5 os_prio=31 cpu=0.15ms elapsed=7.35s tid=0x000000029e17f800 nid=131587 waiting on condition  [0x00000002c8a92000]
         java.lang.Thread.State: WAITING (parking)
      	at jdk.internal.misc.Unsafe.park(java.base@21.0.2/Native Method)
      --
      "bigquery-cancel-thread" #165 [131843] daemon prio=5 os_prio=31 cpu=0.16ms elapsed=6.71s tid=0x00000002b9513a00 nid=131843 waiting on condition  [0x00000002c8c9e000]
         java.lang.Thread.State: WAITING (parking)
      	at jdk.internal.misc.Unsafe.park(java.base@21.0.2/Native Method)
      --
      "bigquery-cancel-thread" #166 [132355] daemon prio=5 os_prio=31 cpu=0.10ms elapsed=6.12s tid=0x0000000122dede00 nid=132355 waiting on condition  [0x00000002c8eaa000]
         java.lang.Thread.State: WAITING (parking)
      	at jdk.internal.misc.Unsafe.park(java.base@21.0.2/Native Method)
      --
      "bigquery-cancel-thread" #167 [132867] daemon prio=5 os_prio=31 cpu=0.11ms elapsed=5.54s tid=0x00000001233b1000 nid=132867 waiting on condition  [0x00000002c90b6000]
         java.lang.Thread.State: WAITING (parking)
      	at jdk.internal.misc.Unsafe.park(java.base@21.0.2/Native Method)
      
      ```
      
      Simple repro:
      
      ```clojure
      bigquery-profile=> (require '[clojure.core.async :as a])
      nil
      bigquery-profile=> (let [c (a/chan)
                               t (Thread. (fn []
                                            (when (a/<!! c)
                                              (println "got something")))
                                          "Forever-running")]
                           (.start t)
                           (a/close! c)
                           (Thread/sleep 4000)
                           (let [thread-names (into #{} (map ^[] Thread/.getName) (.keySet (Thread/getAllStackTraces)))]
                             (contains? thread-names "Forever-running")))
      false
      bigquery-profile=> (let [c (a/chan)
                               t (Thread. (fn []
                                            (when (a/<!! c)
                                              (println "got something")))
                                          "Forever-running")]
                           (.start t)
                           #_(a/close! c) ;; if we don't close it will never stop running
                           (Thread/sleep 4000)
                           (let [thread-names (into #{} (map ^[] Thread/.getName) (.keySet (Thread/getAllStackTraces)))]
                             (contains? thread-names "Forever-running")))
      true
      ```
      
      So this PR just adds a transducer around the rf that closes the
      `cancel-chan` if it's open. Note that multiple closes are fine: from the
      docstring: "Closing a closed channel is a no-op".
      Unverified
      9fb1bd7d
    • lbrdnk's avatar
      [Mongo] Convert bson Decimal128 negative zero to BigInteger zero (#44611) · a4158f13
      lbrdnk authored
      * Convert bson Decimal128 negative zero to BigInteger zero
      Unverified
      a4158f13
  18. Jun 24, 2024
  19. Jun 14, 2024
  20. Jun 10, 2024
  21. Jun 06, 2024
  22. May 30, 2024
  23. May 21, 2024
  24. May 20, 2024
  25. May 16, 2024
  26. May 15, 2024
  27. May 14, 2024
    • Braden Shepherdson's avatar
      [Drivers] New `features` to replace hard-coded driver lists in tests (#42602) · 50a03a19
      Braden Shepherdson authored
      - `:metadata/key-constraints` for those databases which formally track
        PK and FK key constraints. Some databases (eg. Presto/Athena/Starburst
        family) don't support constraints like that, but do support querying
        based on foreign keys (which is the `:foreign-keys` feature).
      - `:connection/multiple-databases` for databases where a single
        connection can connect to many databases. This is the case for Athena,
        since it connects to an S3 bucket and exposes all files in the bucket
        as databases through that single connection.
      - Also removed a troublesome "all DBs except a huge list" case in the
        SSH tunnel tests with a shorter list of supported drivers.
      Unverified
      50a03a19
  28. May 10, 2024
Loading