This project is mirrored from https://github.com/metabase/metabase.
Pull mirroring updated .
- Aug 14, 2024
-
-
Cam Saul authored
* Add Kondo linter to disallow hardcoded driver names in tests * Fix all tests using hardcoded driver names * Test fixes
* Appease Kondo * Remove normal-drivers-except * Test fix * Test fixes * MongoDB test fixes
-
- Aug 07, 2024
-
-
dpsutton authored
dep is brought in by google cloud From `clj -X:deps tree :aliases '[:ee :drivers]'` ``` . metabase/bigquery-cloud-sdk metabase/modules/drivers/bigquery-cloud-sdk . com.google.cloud/google-cloud-bigquery 2.38.1 . com.google.cloud/google-cloud-core 2.35.0 [...] . javax.annotation/javax.annotation-api 1.3.2 ``` Before this change: ``` ❯ clj -X:deps tree :aliases '[:ee :drivers]' | grep 'javax.annotation/javax.annotation-api' . javax.annotation/javax.annotation-api 1.3.2 ``` After: ``` ❯ clj -X:deps tree :aliases '[:ee :drivers]' > deps-master ❯ echo $? 0 ```
-
- Aug 05, 2024
-
-
Case Nelson authored
* fix: mongo ObjectId should be orderable Fixes #46259 Mongo ObjectId derive :type/TextLike but that can't be an orderable type, so we mark the specific Mongo type as orderable for malli schemas. * Fix test
-
- Aug 01, 2024
-
-
lbrdnk authored
* Avoid adding temporal-unit to lhs cols * Use :default temporal-unit * Simplify logic, update comments * Update substitute-field-filter-test * Update align-temporal-unit-with-param-type-test * Update field-filter-date-test * Use end-excludding gte lt filter for DateTime fields * Update substitute-field-filter-test * Update align-temporal-unit-with-param-type-test * Update field-filter-date-test * Update bigquery test + comment * Update date-str->qp-aware-offset-dt * Add guard for unexpected date string format This is just for the completeness, I haven't encountered it. * Address review remarks * Update comment
-
- Jul 31, 2024
-
-
Ngoc Khuat authored
-
- Jul 25, 2024
-
-
bryan authored
* add mu/defn- * use new built-in mu/defn- * mu/defn- preserves metadata --------- Co-authored-by:
Noah Moss <noahbmoss@gmail.com> Co-authored-by:
Noah Moss <32746338+noahmoss@users.noreply.github.com>
-
adam-james authored
* Bigquery Pagination Use next page to get proper results Fixes 45953 WIP * use new-page in values-iterator * disable just the part of the test that is still flaky Want to keep the other part of the test that does prove we're not getting duplicates due to incorrect pagination.
-
- Jul 23, 2024
-
-
Cam Saul authored
* Potential Redshift sync flake fix * Make Redshift sync much more robust
-
Cal Herries authored
-
- Jul 19, 2024
-
-
lbrdnk authored
* Add basic auth to dbms-version * Add test
-
lbrdnk authored
* Post sync hook stub for implicit joins testing stub * Add join alias to field lvalues to enable sorting on joined fields * Disable :foreign-keys on Mongo * Require :left-join support for implicit joins instead of :foreign-keys * Update implicit joins tests * Adjust sync-fields-test * Update implicit joins feature check test * Transform post-sync-hook to normal function * Add foreign key relationships only for dbmses without :foreign-keys feature * Update test to handle Oracle correctly * Split convoluted fn * Avoid unnecessary computations for datasets with no fks * Update docstring * Fix driver usage * Transform :foreign-keys to :metadata/key-constraints in test data loading code * Update sync_test.clj * Update driver_test.clj * Update moviedb.clj * Update dataset_definition_test.clj * Update fetch_metadata.clj * Update fields_test.clj * Update driver.clj * Update driver/sql.clj * Set sql driver join support to true for all joins Deriving drivers are expected to set to false where applicable. * Update sqllite.clj * Remove foreign-keys from spark * Remove :foreign-keys from presto * Remove :foreign-keys from Athena * Remove foreign-keys from big query Reading docs it seems fk inference should be ok. Let's see the test results. Act based on that. * Update test_metadata.cljc * Set key-constraints to false for bigquery * Add foreign keys to sqlite manually until sync is fixed * Return driver require to Athena * Correct typo * Add naive primary key heuristic * Update pk fk logic to handle name components correctly * Add alias escaping to presto * Add ordering to test * Add order by to test * Update test * Remove use of rewrite-fields-to-force-using-column-aliases in order by fields * Add exception to alias forcing * Different approach to exception from alias forcing * Alternative approach for prefixing idents in bigq * All seelcted fields by desired alias * Rewrite only fields not from this source table * Update test * Enable breakout-on-fk-field-test for :left-join drivers * Add feature comment * Explicit joins tests foreign-keys removal * Update nested_queries_test.clj * Update remapping tests * Update tests * Update tests to handle sqlite results format * Disable metadata/key-constraints on sqlite during tests until * Address remarks * Remove mt/with-mock-fks-for-drivers-without-fk-constraints * Update bigquery test * Update tests * Adjust row level restrictions * Add parameterized-sql feature * Update comment * Update leftovers * Order keys * Remove foreign keys from frontend * Fix FE unit * Update driver changelog * Address review remark * Update test/metabase/query_processor/test_util.clj Co-authored-by:
metamben <103100869+metamben@users.noreply.github.com> * Update docs/developers-guide/driver-changelog.md Co-authored-by:
metamben <103100869+metamben@users.noreply.github.com> * Update docs/developers-guide/driver-changelog.md Co-authored-by:
metamben <103100869+metamben@users.noreply.github.com> * Update modules/drivers/bigquery-cloud-sdk/src/metabase/driver/bigquery_cloud_sdk/query_processor.clj Co-authored-by:
metamben <103100869+metamben@users.noreply.github.com> * Update docs/developers-guide/driver-changelog.md Co-authored-by:
metamben <103100869+metamben@users.noreply.github.com> * Update test * Update comment and reduce expression * Update comment * Address remarks * Fix merge --------- Co-authored-by:
metamben <103100869+metamben@users.noreply.github.com>
-
- Jul 18, 2024
-
-
Case Nelson authored
* tests for drivers with spaces in their identifiers * Add driver feature * Fix tests * Remove ambiguous alias for sqlite * Try to fix bigquery and sqlite tests not seeing left-join support * Add explicit support for left-join because of foreign-keys during test hacks * Handle left-join support weirdness * Fix test
-
Cam Saul authored
* Reuse connection for test data loading & JDBC data loading overhaul * Revert silly change * Remove unneeded logic for recording which datasets are loaded or not-yet-loaded * Ok so I guess we do need the code to detect duplicate dataset loading after all * B I G improvements * Appease Kondo * Test fixes
* More test fixes * Fix dataset-already-loaded?-test * Fix test * Fix Presto JDBC and greatly improve test data load speed for it * Postgres test fix * Fix compilation errors * MySQL fixes * Fix syntax error * MySQL fixes * Implement `dataset-already-loaded?` for Athena and cache the set of existing databases * Fix Redshift * Fix Presto JDBC * Vertica test fix * Fix Oracle dataset-already-loaded? * Test fixes and parallelize some API tests. * Test fixes * Oracle test fix? * Another test fix * Stop putting random spaces in code bodies this is not JavaScript code * with-temp-vals-in-db should merge in original value of Database settings * The most test fixes * 3 or 4 test fixes * Add schemas around stuff to catch errors * Appease Kondo * Implement dataset-already-loaded? for vertica * Implement dataset-already-loaded? for SQLite and Snowflake
-
- Jul 17, 2024
-
-
Case Nelson authored
* [WIP] Testing driver filters * Only test returned values in CI * Add :uuid-type for dbs that support that type (test oracle's support) * Try again for oracle test, redshift does not support uuid-type * Add tests, support where available * Add empty/null tests * Adjust test query for athena * Reset logging * Fix comments * Add Athena uuid conversion * Address PR review * Add comments * Address PR comments * Use varchar for portability. Add inline uuid casting for athena --------- Co-authored-by:
Ngoc Khuat <qn.khuat@gmail.com>
-
lbrdnk authored
* Fix deserialization of mongo native query string * Add test
-
Cam Saul authored
* Sync Snowflake `VARIANT` type as :type/SnowflakeVariant` * Test fix
-
- Jul 16, 2024
-
-
bryan authored
* get-sub-props -> ->sub-props because it doesn't 'get' anything * add defendpoint dox * make style consistent * respond to review comment * fix formatting
-
Case Nelson authored
* fix: add timezone to bigquery param substitution Fixes #43597 * Fix indentation and remove report-timezone lookup from loop
-
- Jul 12, 2024
-
-
Braden Shepherdson authored
This comes in two parts. **Rebuilt query logic** I rewrote the query exec logic to use one `future` and one `a/go` block, reporting results to a `promise`. This is easier to follow and doesn't leak threads like the original implementation with two `future`s. The single future is guaranteed to exit. It blocks on the BigQuery call. That will either complete or throw an exception. Either the exception or the BigQuery result is sent to the promise, and then the future is complete. The only reference to that future is overwritten after the `promise` resolves, so the `future`, its closure, and its thread don't persist longer than needed. **Return a reducible result** Previously the results were returned as a lazy sequence, but that brings a lot of `seq` overhead, and is a risk of retaining the entire result set in memory until the query is all done. Above all, it doesn't support `reduced?` to explicitly drop any further results. The `thunk` function style of `qp.reducible/reducible-rows` doesn't fit neatly here, so I implemented the reducible logic directly on top of the `Iterable` BigQuery results. **Verifying** This is a bit tricky. I've been testing it like this: - Add a BigQuery database to my instance - Create a saved question and note its ID (say, 123) - Create an API key in (EE) Metabase and save it in an env var - Install the `siege` load testing tool - Install VisualVM and monitor my Clojure REPL process's thread count - `siege --concurrent=15 --time=30s --header="x-api-key: $API_KEY" "http://localhost:3000/api/card/123/query POST"` That will create a bunch of new threads while it's getting called, but within a minute of the last queries they'll gradually drop out of the thread pools and it will return to the baseline number of threads.
-
- Jul 11, 2024
-
-
lbrdnk authored
-
- Jul 08, 2024
- Jul 03, 2024
-
-
Cam Saul authored
* Support compiling SQL with inlined parameters directly * Test fix * Make `format-honeysql` a multimethod * Add test for #32543 * Test also fixes #44915 * Add explicit test for #44915 * Better test for #44915 * More test fixes
* Fix Kondo error
-
- Jul 02, 2024
- Jun 27, 2024
-
-
dpsutton authored
Customer complaints that thread growth was unbounded. Investigated and found bigquery involved and isolated to the cancel thread bits. Gist is this: ```clojure (when cancel-chan (future ; this needs to run in a separate thread, because the <!! operation blocks forever (when (a/<!! cancel-chan) (log/debug "Received a message on the cancel channel; attempting to stop the BigQuery query execution") (reset! cancel-requested? true) ; signal the page iteration fn to stop (if-not (or (future-cancelled? res-fut) (future-done? res-fut)) ;; somehow, even the FIRST page hasn't come back yet (i.e. the .query call above), so cancel the future to ;; interrupt the thread waiting on that response to come back ;; unfortunately, with this particular overload of .query, we have no access to (nor the ability to control) ;; the jobId, so we have no way to use the BigQuery client to cancel any job that might be running (future-cancel res-fut) (when (future-done? res-fut) ; canceled received after it was finished; may as well return it @res-fut))))) ``` The important parts is that we closeover the cancel-chan and start a thread that is trying to pull from it. If we never put anything on that channel or never close that channel this thread will just remain blocked waiting. In a thread dump, that looks like the following: (note, I've made it set the thread names to make it easy to follow run a query 30 times) ``` Full thread dump OpenJDK 64-Bit Server VM (21.0.2+13-LTS mixed mode): ... "bigquery-cancel-thread" #73 [95747] prio=5 os_prio=31 cpu=2.76ms elapsed=208.53s tid=0x000000029e00f600 nid=95747 waiting on condition [0x00000002c1266000] java.lang.Thread.State: WAITING (parking) at jdk.internal.misc.Unsafe.park(java.base@21.0.2/Native Method) -- "bigquery-cancel-thread" #108 [93211] prio=5 os_prio=31 cpu=154.60ms elapsed=128.72s tid=0x000000029f0dc200 nid=93211 waiting on condition [0x000000029fa06000] java.lang.Thread.State: WAITING (parking) at jdk.internal.misc.Unsafe.park(java.base@21.0.2/Native Method) -- "bigquery-cancel-thread" #116 [116527] prio=5 os_prio=31 cpu=0.16ms elapsed=61.14s tid=0x00000002c1917c00 nid=116527 waiting on condition [0x000000029fc12000] java.lang.Thread.State: WAITING (parking) at jdk.internal.misc.Unsafe.park(java.base@21.0.2/Native Method) -- "bigquery-cancel-thread" #118 [98315] prio=5 os_prio=31 cpu=161.69ms elapsed=60.27s tid=0x00000002b91cca00 nid=98315 waiting on condition [0x000000029fe1e000] java.lang.Thread.State: WAITING (parking) at jdk.internal.misc.Unsafe.park(java.base@21.0.2/Native Method) -- "bigquery-cancel-thread" #119 [121607] prio=5 os_prio=31 cpu=0.13ms elapsed=59.35s tid=0x00000002b9236400 nid=121607 waiting on condition [0x00000002a002a000] java.lang.Thread.State: WAITING (parking) at jdk.internal.misc.Unsafe.park(java.base@21.0.2/Native Method) -- "bigquery-cancel-thread" #120 [99079] prio=5 os_prio=31 cpu=0.13ms elapsed=58.55s tid=0x000000029e295600 nid=99079 waiting on condition [0x00000002a0506000] java.lang.Thread.State: WAITING (parking) at jdk.internal.misc.Unsafe.park(java.base@21.0.2/Native Method) -- "bigquery-cancel-thread" #121 [122375] prio=5 os_prio=31 cpu=0.10ms elapsed=57.75s tid=0x00000002b921f400 nid=122375 waiting on condition [0x00000002c0e4e000] java.lang.Thread.State: WAITING (parking) at jdk.internal.misc.Unsafe.park(java.base@21.0.2/Native Method) -- "bigquery-cancel-thread" #122 [123655] prio=5 os_prio=31 cpu=0.18ms elapsed=56.97s tid=0x0000000134bfa000 nid=123655 waiting on condition [0x00000002c105a000] java.lang.Thread.State: WAITING (parking) at jdk.internal.misc.Unsafe.park(java.base@21.0.2/Native Method) -- "bigquery-cancel-thread" #123 [122631] prio=5 os_prio=31 cpu=0.11ms elapsed=56.28s tid=0x000000013569be00 nid=122631 waiting on condition [0x00000002c1472000] java.lang.Thread.State: WAITING (parking) at jdk.internal.misc.Unsafe.park(java.base@21.0.2/Native Method) -- "bigquery-cancel-thread" #124 [96007] prio=5 os_prio=31 cpu=0.10ms elapsed=55.42s tid=0x0000000133b54e00 nid=96007 waiting on condition [0x00000002c167e000] java.lang.Thread.State: WAITING (parking) at jdk.internal.misc.Unsafe.park(java.base@21.0.2/Native Method) -- "bigquery-cancel-thread" #125 [97799] prio=5 os_prio=31 cpu=0.09ms elapsed=54.73s tid=0x0000000131c19a00 nid=97799 waiting on condition [0x00000002c2206000] java.lang.Thread.State: WAITING (parking) at jdk.internal.misc.Unsafe.park(java.base@21.0.2/Native Method) -- "bigquery-cancel-thread" #126 [96519] prio=5 os_prio=31 cpu=0.10ms elapsed=53.97s tid=0x000000012370f600 nid=96519 waiting on condition [0x00000002c2412000] java.lang.Thread.State: WAITING (parking) at jdk.internal.misc.Unsafe.park(java.base@21.0.2/Native Method) -- "bigquery-cancel-thread" #127 [124167] prio=5 os_prio=31 cpu=0.10ms elapsed=53.33s tid=0x0000000134312200 nid=124167 waiting on condition [0x00000002c28ea000] java.lang.Thread.State: WAITING (parking) at jdk.internal.misc.Unsafe.park(java.base@21.0.2/Native Method) -- "bigquery-cancel-thread" #128 [97287] prio=5 os_prio=31 cpu=0.12ms elapsed=52.57s tid=0x00000001233ae000 nid=97287 waiting on condition [0x00000002c4706000] java.lang.Thread.State: WAITING (parking) at jdk.internal.misc.Unsafe.park(java.base@21.0.2/Native Method) -- "bigquery-cancel-thread" #129 [104715] prio=5 os_prio=31 cpu=0.10ms elapsed=51.83s tid=0x0000000131e47a00 nid=104715 waiting on condition [0x00000002c4912000] java.lang.Thread.State: WAITING (parking) at jdk.internal.misc.Unsafe.park(java.base@21.0.2/Native Method) -- "bigquery-cancel-thread" #130 [102919] prio=5 os_prio=31 cpu=0.08ms elapsed=51.10s tid=0x0000000123540200 nid=102919 waiting on condition [0x00000002c4b1e000] java.lang.Thread.State: WAITING (parking) at jdk.internal.misc.Unsafe.park(java.base@21.0.2/Native Method) -- "bigquery-cancel-thread" #131 [117263] prio=5 os_prio=31 cpu=0.17ms elapsed=50.37s tid=0x00000001233d1200 nid=117263 waiting on condition [0x00000002c4d2a000] java.lang.Thread.State: WAITING (parking) at jdk.internal.misc.Unsafe.park(java.base@21.0.2/Native Method) -- "bigquery-cancel-thread" #132 [114951] prio=5 os_prio=31 cpu=0.09ms elapsed=49.71s tid=0x000000013484dc00 nid=114951 waiting on condition [0x00000002c4f36000] java.lang.Thread.State: WAITING (parking) at jdk.internal.misc.Unsafe.park(java.base@21.0.2/Native Method) -- "bigquery-cancel-thread" #133 [103687] prio=5 os_prio=31 cpu=0.14ms elapsed=49.08s tid=0x0000000123517200 nid=103687 waiting on condition [0x00000002c5142000] java.lang.Thread.State: WAITING (parking) at jdk.internal.misc.Unsafe.park(java.base@21.0.2/Native Method) -- "bigquery-cancel-thread" #134 [91419] prio=5 os_prio=31 cpu=0.08ms elapsed=48.38s tid=0x0000000131b43600 nid=91419 waiting on condition [0x00000002c534e000] java.lang.Thread.State: WAITING (parking) at jdk.internal.misc.Unsafe.park(java.base@21.0.2/Native Method) -- "bigquery-cancel-thread" #135 [127495] prio=5 os_prio=31 cpu=0.14ms elapsed=47.66s tid=0x0000000131ce7000 nid=127495 waiting on condition [0x00000002c555a000] java.lang.Thread.State: WAITING (parking) at jdk.internal.misc.Unsafe.park(java.base@21.0.2/Native Method) -- "bigquery-cancel-thread" #139 [103955] daemon prio=5 os_prio=31 cpu=0.12ms elapsed=23.22s tid=0x0000000121aaca00 nid=103955 waiting on condition [0x00000002c5766000] java.lang.Thread.State: WAITING (parking) at jdk.internal.misc.Unsafe.park(java.base@21.0.2/Native Method) -- "bigquery-cancel-thread" #140 [105219] daemon prio=5 os_prio=31 cpu=0.10ms elapsed=22.54s tid=0x0000000133f47600 nid=105219 waiting on condition [0x00000002c5972000] java.lang.Thread.State: WAITING (parking) at jdk.internal.misc.Unsafe.park(java.base@21.0.2/Native Method) -- "bigquery-cancel-thread" #141 [105731] daemon prio=5 os_prio=31 cpu=0.15ms elapsed=21.69s tid=0x00000001353ba800 nid=105731 waiting on condition [0x00000002c5b7e000] java.lang.Thread.State: WAITING (parking) at jdk.internal.misc.Unsafe.park(java.base@21.0.2/Native Method) -- "bigquery-cancel-thread" #142 [114435] daemon prio=5 os_prio=31 cpu=0.08ms elapsed=20.89s tid=0x0000000131e76400 nid=114435 waiting on condition [0x00000002c5d8a000] java.lang.Thread.State: WAITING (parking) at jdk.internal.misc.Unsafe.park(java.base@21.0.2/Native Method) -- "bigquery-cancel-thread" #143 [106499] daemon prio=5 os_prio=31 cpu=0.10ms elapsed=20.12s tid=0x0000000123107e00 nid=106499 waiting on condition [0x00000002c5f96000] java.lang.Thread.State: WAITING (parking) at jdk.internal.misc.Unsafe.park(java.base@21.0.2/Native Method) -- "bigquery-cancel-thread" #144 [113923] daemon prio=5 os_prio=31 cpu=0.11ms elapsed=19.52s tid=0x00000002c1918400 nid=113923 waiting on condition [0x00000002c61a2000] java.lang.Thread.State: WAITING (parking) at jdk.internal.misc.Unsafe.park(java.base@21.0.2/Native Method) -- "bigquery-cancel-thread" #145 [107011] daemon prio=5 os_prio=31 cpu=0.13ms elapsed=18.93s tid=0x00000002a6ad3e00 nid=107011 waiting on condition [0x00000002c63ae000] java.lang.Thread.State: WAITING (parking) at jdk.internal.misc.Unsafe.park(java.base@21.0.2/Native Method) -- "bigquery-cancel-thread" #146 [113667] daemon prio=5 os_prio=31 cpu=0.12ms elapsed=18.37s tid=0x00000001341a2800 nid=113667 waiting on condition [0x00000002c65ba000] java.lang.Thread.State: WAITING (parking) at jdk.internal.misc.Unsafe.park(java.base@21.0.2/Native Method) -- "bigquery-cancel-thread" #147 [113155] daemon prio=5 os_prio=31 cpu=0.16ms elapsed=17.74s tid=0x000000013445da00 nid=113155 waiting on condition [0x00000002c67c6000] java.lang.Thread.State: WAITING (parking) at jdk.internal.misc.Unsafe.park(java.base@21.0.2/Native Method) -- "bigquery-cancel-thread" #148 [107779] daemon prio=5 os_prio=31 cpu=0.13ms elapsed=17.19s tid=0x0000000134704e00 nid=107779 waiting on condition [0x00000002c69d2000] java.lang.Thread.State: WAITING (parking) at jdk.internal.misc.Unsafe.park(java.base@21.0.2/Native Method) -- "bigquery-cancel-thread" #149 [112643] daemon prio=5 os_prio=31 cpu=0.17ms elapsed=16.54s tid=0x000000011f185000 nid=112643 waiting on condition [0x00000002c6bde000] java.lang.Thread.State: WAITING (parking) at jdk.internal.misc.Unsafe.park(java.base@21.0.2/Native Method) -- "bigquery-cancel-thread" #150 [108035] daemon prio=5 os_prio=31 cpu=0.24ms elapsed=15.95s tid=0x000000011f1be200 nid=108035 waiting on condition [0x00000002c6dea000] java.lang.Thread.State: WAITING (parking) at jdk.internal.misc.Unsafe.park(java.base@21.0.2/Native Method) -- "bigquery-cancel-thread" #151 [108547] daemon prio=5 os_prio=31 cpu=0.11ms elapsed=15.38s tid=0x00000001302cf800 nid=108547 waiting on condition [0x00000002c6ff6000] java.lang.Thread.State: WAITING (parking) at jdk.internal.misc.Unsafe.park(java.base@21.0.2/Native Method) -- "bigquery-cancel-thread" #152 [109059] daemon prio=5 os_prio=31 cpu=0.24ms elapsed=14.85s tid=0x0000000133c4be00 nid=109059 waiting on condition [0x00000002c7202000] java.lang.Thread.State: WAITING (parking) at jdk.internal.misc.Unsafe.park(java.base@21.0.2/Native Method) -- "bigquery-cancel-thread" #153 [109315] daemon prio=5 os_prio=31 cpu=0.10ms elapsed=14.15s tid=0x00000001339dc000 nid=109315 waiting on condition [0x00000002c740e000] java.lang.Thread.State: WAITING (parking) at jdk.internal.misc.Unsafe.park(java.base@21.0.2/Native Method) -- "bigquery-cancel-thread" #154 [109571] daemon prio=5 os_prio=31 cpu=0.17ms elapsed=13.57s tid=0x0000000134ea3600 nid=109571 waiting on condition [0x00000002c761a000] java.lang.Thread.State: WAITING (parking) at jdk.internal.misc.Unsafe.park(java.base@21.0.2/Native Method) -- "bigquery-cancel-thread" #155 [110083] daemon prio=5 os_prio=31 cpu=0.20ms elapsed=12.95s tid=0x0000000121d7aa00 nid=110083 waiting on condition [0x00000002c7826000] java.lang.Thread.State: WAITING (parking) at jdk.internal.misc.Unsafe.park(java.base@21.0.2/Native Method) -- "bigquery-cancel-thread" #156 [110339] daemon prio=5 os_prio=31 cpu=0.12ms elapsed=12.35s tid=0x0000000134ff5c00 nid=110339 waiting on condition [0x00000002c7a32000] java.lang.Thread.State: WAITING (parking) at jdk.internal.misc.Unsafe.park(java.base@21.0.2/Native Method) -- "bigquery-cancel-thread" #157 [110595] daemon prio=5 os_prio=31 cpu=0.09ms elapsed=11.65s tid=0x0000000133e27000 nid=110595 waiting on condition [0x00000002c7c3e000] java.lang.Thread.State: WAITING (parking) at jdk.internal.misc.Unsafe.park(java.base@21.0.2/Native Method) -- "bigquery-cancel-thread" #158 [111107] daemon prio=5 os_prio=31 cpu=0.13ms elapsed=11.04s tid=0x0000000131e8b800 nid=111107 waiting on condition [0x00000002c7e4a000] java.lang.Thread.State: WAITING (parking) at jdk.internal.misc.Unsafe.park(java.base@21.0.2/Native Method) -- "bigquery-cancel-thread" #159 [174339] daemon prio=5 os_prio=31 cpu=0.13ms elapsed=10.51s tid=0x0000000134ff6400 nid=174339 waiting on condition [0x00000002c8056000] java.lang.Thread.State: WAITING (parking) at jdk.internal.misc.Unsafe.park(java.base@21.0.2/Native Method) -- "bigquery-cancel-thread" #160 [173827] daemon prio=5 os_prio=31 cpu=0.12ms elapsed=9.79s tid=0x0000000133ff7c00 nid=173827 waiting on condition [0x00000002c8262000] java.lang.Thread.State: WAITING (parking) at jdk.internal.misc.Unsafe.park(java.base@21.0.2/Native Method) -- "bigquery-cancel-thread" #161 [173315] daemon prio=5 os_prio=31 cpu=0.11ms elapsed=9.18s tid=0x0000000122c9b600 nid=173315 waiting on condition [0x00000002c846e000] java.lang.Thread.State: WAITING (parking) at jdk.internal.misc.Unsafe.park(java.base@21.0.2/Native Method) -- "bigquery-cancel-thread" #162 [131331] daemon prio=5 os_prio=31 cpu=0.12ms elapsed=8.58s tid=0x0000000133b9c400 nid=131331 waiting on condition [0x00000002c867a000] java.lang.Thread.State: WAITING (parking) at jdk.internal.misc.Unsafe.park(java.base@21.0.2/Native Method) -- "bigquery-cancel-thread" #163 [172547] daemon prio=5 os_prio=31 cpu=0.14ms elapsed=8.06s tid=0x0000000134d4cc00 nid=172547 waiting on condition [0x00000002c8886000] java.lang.Thread.State: WAITING (parking) at jdk.internal.misc.Unsafe.park(java.base@21.0.2/Native Method) -- "bigquery-cancel-thread" #164 [131587] daemon prio=5 os_prio=31 cpu=0.15ms elapsed=7.35s tid=0x000000029e17f800 nid=131587 waiting on condition [0x00000002c8a92000] java.lang.Thread.State: WAITING (parking) at jdk.internal.misc.Unsafe.park(java.base@21.0.2/Native Method) -- "bigquery-cancel-thread" #165 [131843] daemon prio=5 os_prio=31 cpu=0.16ms elapsed=6.71s tid=0x00000002b9513a00 nid=131843 waiting on condition [0x00000002c8c9e000] java.lang.Thread.State: WAITING (parking) at jdk.internal.misc.Unsafe.park(java.base@21.0.2/Native Method) -- "bigquery-cancel-thread" #166 [132355] daemon prio=5 os_prio=31 cpu=0.10ms elapsed=6.12s tid=0x0000000122dede00 nid=132355 waiting on condition [0x00000002c8eaa000] java.lang.Thread.State: WAITING (parking) at jdk.internal.misc.Unsafe.park(java.base@21.0.2/Native Method) -- "bigquery-cancel-thread" #167 [132867] daemon prio=5 os_prio=31 cpu=0.11ms elapsed=5.54s tid=0x00000001233b1000 nid=132867 waiting on condition [0x00000002c90b6000] java.lang.Thread.State: WAITING (parking) at jdk.internal.misc.Unsafe.park(java.base@21.0.2/Native Method) ``` Simple repro: ```clojure bigquery-profile=> (require '[clojure.core.async :as a]) nil bigquery-profile=> (let [c (a/chan) t (Thread. (fn [] (when (a/<!! c) (println "got something"))) "Forever-running")] (.start t) (a/close! c) (Thread/sleep 4000) (let [thread-names (into #{} (map ^[] Thread/.getName) (.keySet (Thread/getAllStackTraces)))] (contains? thread-names "Forever-running"))) false bigquery-profile=> (let [c (a/chan) t (Thread. (fn [] (when (a/<!! c) (println "got something"))) "Forever-running")] (.start t) #_(a/close! c) ;; if we don't close it will never stop running (Thread/sleep 4000) (let [thread-names (into #{} (map ^[] Thread/.getName) (.keySet (Thread/getAllStackTraces)))] (contains? thread-names "Forever-running"))) true ``` So this PR just adds a transducer around the rf that closes the `cancel-chan` if it's open. Note that multiple closes are fine: from the docstring: "Closing a closed channel is a no-op".
-
lbrdnk authored
* Convert bson Decimal128 negative zero to BigInteger zero
-
- Jun 24, 2024
-
-
Cal Herries authored
-
lbrdnk authored
This PR removes `:role` removal from spec, even if it was added in `:connection-uri`. It seems that `:connection-uri` is ignored on `do-with-connection-options` code path. Hence use of non-default roles is again available on Snowflake.
-
lbrdnk authored
This commit adds `$size` operator wrapping to `$addToSet`. That enables use of `:distinct` clause in expressions on Mongo.
-
- Jun 14, 2024
-
-
Cal Herries authored
Fix fields from Amazon Athena Tables failing to sync if there is not an underscore in the Table Name (#44032)
-
- Jun 10, 2024
-
-
lbrdnk authored
* Add connection string parsing * Add test case with role suffixed key
-
- Jun 06, 2024
-
-
lbrdnk authored
* Move role to url parameters from connection properties * Fix test
-
- May 30, 2024
-
-
Cam Saul authored
* Remove Google Analytics driver * Remove more GA-related tests * Un-remove tests that aren't related to GA
-
- May 21, 2024
-
-
metamben authored
* Migrate druid metrics-inside-aggregation-clauses-test to druid-jdbc * Fix test-runner: druid should not exclude druid-jdbc * Implement set-parameter and unprepare-value for LocalDateTime for druid-jdbc
-
- May 20, 2024
-
-
Kamil Mielnik authored
-
- May 16, 2024
-
-
Mahatthana (Kelvin) Nomsawadi authored
-
- May 15, 2024
-
-
Luis Paolini authored
same compatibility of the previous one but this is compatible with JDK 21 https://learn.microsoft.com/en-us/sql/connect/jdbc/release-notes-for-the-jdbc-driver?view=sql-server-ver16 Co-authored-by:
Cam Saul <1455846+camsaul@users.noreply.github.com>
-
- May 14, 2024
-
-
Braden Shepherdson authored
- `:metadata/key-constraints` for those databases which formally track PK and FK key constraints. Some databases (eg. Presto/Athena/Starburst family) don't support constraints like that, but do support querying based on foreign keys (which is the `:foreign-keys` feature). - `:connection/multiple-databases` for databases where a single connection can connect to many databases. This is the case for Athena, since it connects to an S3 bucket and exposes all files in the bucket as databases through that single connection. - Also removed a troublesome "all DBs except a huge list" case in the SSH tunnel tests with a shorter list of supported drivers.
-
- May 10, 2024
-
-
Cam Saul authored
* Remove unused `metabase.api.transform` * Copy the deps graph improvements from my other PR * New combined X-Rays API namespace * Fix kondo error
-