Skip to content
Snippets Groups Projects
Unverified Commit 31999007 authored by Nick Fitzpatrick's avatar Nick Fitzpatrick Committed by GitHub
Browse files

Transition InputBlurChange to use core Input component (#27122)

* Convert to TypeScript, handle ColumnItem and MetadataTable

* more progress

* handled numeric inputs

* Transition InputBlurChange to core Input component

* Removing input class from legacy date picker just incase

* Small style adjustments in Admin and Viz Settings

* working on tests

* Tests passing

* Adjusting styling

* PR Cleanup

* add input size=large (#27217)

* Suppress console log when using `with-log-messages-for-level` (#26468)

* disable additivity when using "with-log-level"

* only set when parent is root

* wording

* remove a test that is no longer needed

* fix indents

* one missing indent fix

* [CI] Add `concurrency` to the `drivers` workflow (#27223)

* Fix datetime-diff helper text (#27224)

* Improve datetimeDiff type error message (#27225)

* Update test

* Update implementations

* Relieve db pressure on api/health check (#27192)

* Relieve db pressure on api/health check

https://github.com/metabase/metabase/issues/26266

Servers under heavy load can be slow to respond to the api/health
check. This can lead to k8s killing healthy instances happily humming
along serving requests.

One idea floated was to use QoSFilters
https://www.eclipse.org/jetty/javadoc/jetty-9/org/eclipse/jetty/servlets/QoSFilter.html
to prioritize those requests in front of others. But I suspect this
might not be our bottleneck.

Our health endpoint was updated to see if it could acquire an endpoint
when we were dealing with connection pool issues. We were reporting the
instance was healthy once it has finished the init process, but would
report healthy if 60/15 app-db connections were used and no actual
queries could complete.

The remedy was adding
`(sql-jdbc.conn/can-connect-with-spec? {:datasource (mdb.connection/data-source)})`
to the endpoint. But now to get information about the health of the
system we have to wait in the queue to get a datasource.

The hope is that this change which monitors for recent db
checkins (query success) and checkouts (query begun) can be a proxy for
db activity without having to wait for a connection and hit the db ourselves.

Some simple and crude benchmarking:
- use `siege` to hit `api/database/<app-db>/sync_schema`
- in a separate tab, use `siege` to hit `api/health`

Three trials with unconditional db access and conditional db
access (look for recent activity set by the new `ConnectionCustomizer`).

One siege client is synching the app-db's schema with 80 clients each
sending 60 requests. the other has 1 client sending 60 requests to api/health.

Run             |  Elapsed Time | max tx  | tx rate
 before change  |    7.16s      |  0.79s  |  8.38 tx/s
 before change  |   23.91s      |  1.44s  |  2.51 tx/s
 before change  |   13.00s      |  0.50s  |  4.62 tx/s
----------------------------------------------------
 after change   |    4.46s      |  0.27s  |  13.45 tx/s
 after change   |    5.81s      |  0.61s  |  10.33 tx/s
 after change   |    4.54s      |  0.44s  |  13.22 tx/s

Full(er) results below:

```
Unconditional db access
=======================

siege -c80 -r 40 "http://localhost:3000/api/database/2/sync_schema POST" -H "Cookie: $SESSION"

siege -c 1 -r 60 "http://localhost:3000/api/health

"

Elapsed time:		        7.16 secs
Response time:		        0.12 secs
Transaction rate:	        8.38 trans/sec
Longest transaction:	        0.79
Shortest transaction:	        0.01

Elapsed time:		       23.91 secs
Response time:		        0.40 secs
Transaction rate:	        2.51 trans/sec
Longest transaction:	        1.44
Shortest transaction:	        0.02

Elapsed time:		       13.00 secs
Response time:		        0.22 secs
Transaction rate:	        4.62 trans/sec
Longest transaction:	        0.50
Shortest transaction:	        0.06

Conditional db access
==============================================================

Elapsed time:		        4.46 secs
Response time:		        0.07 secs
Transaction rate:	       13.45 trans/sec
Longest transaction:	        0.27
Shortest transaction:	        0.01

Elapsed time:		        5.81 secs
Response time:		        0.10 secs
Transaction rate:	       10.33 trans/sec
Longest transaction:	        0.61
Shortest transaction:	        0.00

Elapsed time:		        4.54 secs
Response time:		        0.08 secs
Transaction rate:	       13.22 trans/sec
Longest transaction:	        0.44
Shortest transaction:	        0.01
```

* Remove reflection in `.put` call (not the reflections trategy)

also remove the call to `classloader/the-classloader` as it did nothing

* Comment and settle on a single method

* tests

* select from db twice

had a failure in CI. give it time to do its thing with another db call

* block to wait for timestamp update?

* unflake the tests

tasks and events from outside the thread can hit the db. the
ConnectionCustomizer is also run from c3p0 controlled threads so we
can't easily isolate everything to our thread

Was running

```clojure
(comment
  (dotimes [n 5]
    (dotimes [_ 100]
      (recent-activity-test)
      (CheckinTracker-test))
    (println (* (inc n) 100)))
    )
```

to run the tests 500 times and would keep getting flakes at a rate
~1/100 to 1/500. Just frustration for the future.

* typehint

* Switch it up a bit

Tests were flaking in h2 and I don't know why. I'm switching to just
updating recent activity on most methods.

* final touches

Co-authored-by: default avatarAleksandr Lesnenko <alxnddr@users.noreply.github.com>
Co-authored-by: default avatarNgoc Khuat <qn.khuat@gmail.com>
Co-authored-by: default avatarNemanja Glumac <31325167+nemanjaglumac@users.noreply.github.com>
Co-authored-by: default avatarCal Herries <39073188+calherries@users.noreply.github.com>
Co-authored-by: default avatardpsutton <dan@dpsutton.com>
parent 0975f5d9
No related branches found
No related tags found
No related merge requests found
Showing
with 121 additions and 161 deletions
Loading
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment