* improved support for Google Cloud SQL

* upgrade `postgres-socket-factory` and add license overrides

* fix license check

Co-authored-by: Cam Saul <1455846+camsaul@users.noreply.github.com>

We have already had support for server authentication based on custom
certificates. This change adds support for authenticating the client
based on custom client key and certificate.

* bumps outdated deps versions to be current

* un-upgrade h2 and jetty

* un-upgrade joda-time and kixi/stats

* drop Java 8 support in circle CI config

- things that used to rely on be-tests-java-8-ee now rely on be-tests-java-11-ee

* remove java 8 from github health check matrix

* revert toucan to 1.17.0

* revert mariadb java client to 2.7.5

* Back to 18, and handle new behavior

toucan used to just look in *.models.<model-name> for models and just
give up apparently. I made a feature that toucan will look in a model
registry to create models rather than using the convention
https://github.com/metabase/toucan/commit/762ad69defc1477423fa9423e9320ed318f7cfe7


but now we're getting errors in these tests about maps vs models.

```clojure
revision_test.clj:154
Check that revisions+details pulls in user info and adds description
expected: [#metabase.models.revision.RevisionInstance{:is_reversion false,
                                                      :is_creation false,
                                                      :message nil,
                                                      :user
                                                      {:id 1,
                                                       :common_name "Rasta Toucan",
                                                       :first_name "Rasta",
                                                       :last_name "Toucan"},
                                                      :diff
                                                      {:o1 nil, :o2 {:name "Tips Created by Day", :serialized true}},
                                                      :description nil}]
  actual: (#metabase.models.revision.RevisionInstance{:description nil,
                                                      :is_creation false,
                                                      :is_reversion false,
                                                      :user
                                                      {:id 1,
                                                       :first_name "Rasta",
                                                       :last_name "Toucan",
                                                       :common_name "Rasta Toucan"},
                                                      :message nil,
                                                      :diff
                                                      {:o1 nil,
                                                       :o2
                                                       #metabase.models.revision_test.FakedCardInstance{:name
                                                                                                        "Tips Created by Day",
                                                                                                        :serialized
                                                                                                        true}}})
```

The only difference here is `:o2` is a
`metabase.models.revision_test.FakedCardInstance` but still has the same
keys, `:name`, and `:serialized`. So all is well, we're just able to
make the model.

So a few different fixes. Some are use `partial=` which doesn't care
about record/map distinction. Some are just make the model, and some are
turning them into maps for revision strings (which more closely mimics
what the real revision stuff does):

```clojure
(defn default-diff-map
  "Default implementation of `diff-map` which simply uses clojures `data/diff` function and sets the keys `:before` and `:after`."
  [_ o1 o2]
  (when o1
    (let [[before after] (data/diff o1 o2)]
      {:before before
       :after  after})))

(defn default-diff-str
  "Default implementation of `diff-str` which simply uses clojures `data/diff` function and passes that on to `diff-string`."
  [entity o1 o2]
  (when-let [[before after] (data/diff o1 o2)]
    (diff-string (:name entity) before after)))
```

So all in all this change impacts nothing in the app itself, because
those models follow convention and are correct in
`metabase.models.<model-name>` and are thus "modelified":

```clojure
revision-test=> (revision/revisions Card 1)
[#metabase.models.revision.RevisionInstance{:is_creation true,
                                            :model_id 1,
                                            :id 1,
                                            :is_reversion false,
                                            :user_id 2,
                                            :timestamp #object[java.time.OffsetDateTime
                                                               "0x77e037f"
                                                               "2021-10-28T15:10:19.828539Z"],
                                            :object #metabase.models.card.CardInstance
                                            {:description nil,
                                             :archived false,
                                             :collection_position nil,
                                             :table_id 5,
                                             :database_id 2,
                                             :enable_embedding false,
                                             :collection_id nil,
                                             :query_type :query,
                                             :name "ECVYUHSWQJYMSOCIFHQC",
                                             :creator_id 2,
                                             :made_public_by_id nil,
                                             :embedding_params nil,
                                             :cache_ttl 1234,
                                             :dataset_query {:database 2,
                                                             :type :query,
                                                             :query {:source-table 5,
                                                                     :aggregation [[:count]]}},
                                             :id 1,
                                             :display :scalar,
                                             :visualization_settings {:global {:title nil}},
                                             :dataset false,
                                             :public_uuid nil},
                                            :message nil,
                                            :model "Card"}]
```

so the model/no-model is just arbitrary distinction in the test. All of
them in the actual app are turned into models:

```clojure
(defn- do-post-select-for-object
  "Call the appropriate `post-select` methods (including the type functions) on the `:object` this Revision recorded.
  This is important for things like Card revisions, where the `:dataset_query` property needs to be normalized when
  coming out of the DB."
  [{:keys [model], :as revision}]
  ;; in some cases (such as tests) we have 'fake' models that cannot be resolved normally; don't fail entirely in
  ;; those cases
  (let [model (u/ignore-exceptions (db/resolve-model (symbol model)))]
    (cond-> revision
    ;; this line would not find a model previously for FakedCard and
    ;; just return the map. But now the registry in toucan _finds_ the
    ;; model defintion and returns the model'd map
      model (update :object (partial models/do-post-select model)))))

(u/strict-extend (class Revision)
  models/IModel
  (merge models/IModelDefaults
         {:types       (constantly {:object :json})
          :pre-insert  pre-insert
          :pre-update  (fn [& _] (throw (Exception. (tru "You cannot update a Revision!"))))
          :post-select do-post-select-for-object}))
```

* try using mssql-jdbc 10.2.1.jre11

- Important that we get off the jre8 version

* various fixes that needn't be reverted

* Revert "various fixes that needn't be reverted"

This reverts commit 2a820db0743d0062eff63366ebe7bc78b852e81f.

* go back to using circle ci's java 11 docker image

* java-16 (?) -> java-17

* Revert "go back to using circle ci's java 11 docker image"

This reverts commit b9b14c535a689f701d7e2541081164288c988c4e.

Co-authored-by: dan sutton <dan@dpsutton.com>

Retry sending notifications (pulses/alerts) for about one minute

The parameters of the exponential backoff can be configured via settings.

* bump snowplow

* bufferSize -> batchSize

* pin jackson-databind to 2.13.2.2

* Store test failure reports

* Just set Java option instead of fancy CI config.

This is to prevent test failures on machines with non-US locales.  See
https://github.com/metabase/metabase/pull/20023#issuecomment-1024872051 for
details.

* Remove :google driver

* Remove unneeded stuff

* Remove another reference to :google driver

* Remove another reference to :google

Upgrade Liquibase to latest version; remove final Java source file and need for `clojure -X:deps prep` (#20611)

* Upgrade Liquibase to latest version

* Try adjusting log

* Fix checksums for the TWO migrations with ID = 32

* FINALLY get Liquibase to use Log4j2

* Set Liquibase ConsoleUIService OutputStream to null OutputStream

* Manually define a package for our H2 proxy class so Java 8 works

* Fix package-name determination code

* Update migrations file spec

* `databasechangelog` shouldn't be upper-case

* Lower-case quartz table names

* More MySQL fixes 

* Properties for all the Quartz tables 

* Formatting tweaks [ci skip]

* Revert a few more busted changes

* Fix more busted changes

* Bump Liquibase version to 4.8.0 to fix MySQL defaultValueBoolean bug

* OMG I think I finally fixed MySQL

* Remove Java source file and prep-deps code

* Remove two more references to bin/prep.sh

* Minor cleanup

* Revert unneeded changes

* Fix busted indentation

* Don't search inside java/ anymore since it's G-O-N-E

* Appease the namespace linter

* Update src/metabase/db/liquibase/h2.clj

* Bump SAML lib version

* Add entry to recognize metabase/saml20-clj as being released under EPL v2

* Replace AOT Spark SQL deps with a `proxy` and a `DataSource`

* Support `connection-details->spec` returning a `DataSource`

* Remove target/classes

* Don't need to deps prep drivers anymore

* Fix duplicate `this` params in `proxy` methods; add `:test` alias for Eastwood to make it be a little quieter

* Make sure to call `pool/map->properties`

* Upgrade Hive JDBC driver version from 1.2.2 -> 3.1.2; Bump Spark SQL from 2.1.1 to 3.2.1

* Clean the namespaces

* Don't need to register the a proxy JDBC driver since we're not even using it anymore

* Fix Spark SQL :schema sync for newer versions

* Remove unneeded override

* Fix day-of-week extract for new :sparksql

* Hive/Spark SQL needs to escape question marks inside QUOTED identifiers now 

* Some minor SQL generation improvements to avoid duplicate casts

* Revert change to debug test

* Add aliases for H2 and Liquibase CLIs

* Use reflection to invoke Liquibase method so it doesn't bust stuff

* Add org.quartz-scheduler/quartz as an explict dep and bump version from 2.17 -> 2.3.2

* Add no-op impl for `initialize` for our Quartz `ConnectionProvider`

* Bump backend dependencies (Jan 2022)

* Revert java-time version upgrade for now until https://github.com/dm3/clojure.java-time/issues/77 is fixed

* Add license overrides

* Bump a few more deps (again). Revert Google/BigQuery and Vertica version bumps

* Revert MariaDB and Redshift version changes

* Turn defendpoint config macro into analyze-call hook for better linting

* Simplify hook

* Remove previous macro config

* Fix all errors concerning third party libs

* revert whitespace change:

We probably never want to use message lookups or JNDI integration

* Bump log4j from 2.14.1 to 2.15.0

* Disable failing logging tests when bumping log4j

0day in log4j requires bump in dependency. These tests look for logs in
testing but our test logger doesn't seem to have levels set
correctly. The disease is certainly worse than the remedy in this case
and each instance is annotated with the reason it is disabled, and we
can reenable them in calmer waters

* Fix unused ns

Co-authored-by: Youngho Kim <miku@korea.ac.kr>
Co-authored-by: dan sutton <dan@dpsutton.com>

Bump metabase/connection-pool version from 1.1.1 to 1.2.0 to get the updated c3p0 version (0.9.5.5)

* Drop long-unused Table.entity_name column

* Add NOT NULL constraint to Card.database_id

* Test fix 

* Test fix 

* Fix migration

* Add note about H2 shell to deps.edn

* Add new SQL migration to attempt to set database_id when unset

* Remove data migration that is now done in Liquibase land

* Oops, '$.database', not '$.database_id'

* Parse ints as signed rather than unsigned just to be safe (they *might* be -1337 if they're really broken)

* Don't run for H2

* Update comment

* Fix migration indentation

* Clean namespace

* Use new migration number.

* Use the new migration numbers

* Adopt new migration numbering scheme

* Fix comments

* Fix MySQL + MariaDB insanity

* Fix ID range validation

* Actually 382 is the last legacy ID

* Improved validation and tests

* Adopt the new-new migration ID format.

* Test fixes 

* Fix merge

* Simplify precondition

`org.apache.sshd` requires `net.i2p.crypto/eddsa` dependency
to work with ED25519 keys.

This worked in 0.40 because `eddsa` was included as a transitive
dependency of another unrelated dependency (`buddy`) but stopped
working in 0.41 because the new version of `buddy` no longer depends
on `eddsa`. Thus, we must explicitly include the dependency.

To prevent this from breaking again, switched one of the test
keys to an ED25519 key.

Version 2.6.0 until 2.7.0 of org.apache.sshd has a bug that
will fail to connect to SSH servers configured to use
`-oKexAlgorithms=diffie-hellman-group-exchange-sha256`.

Reverting to the known working version. We can upgrade
to 2.7.1 once it is released.

Reference: https://issues.apache.org/jira/browse/SSHD-1163

Fixes #18316

I'm not sure we can fix this. We need the graal compiler which doesn't
appear to be jdk 8 compatible but I can't actually find javadocs for
it. Its not clear from
https://github.com/oracle/graaljs/blob/master/docs/user/RunOnJDK.md
how you can actually run it.

I tried adding the graal compiler but that seems to assume you are in
a graal vm and fails not finding a class.

To get a list of all the options available for a context's engine,

```clojure
(doto 'metabase.pulse.render.js-engine require in-ns)
(map (comp (juxt :name :help) bean)
     (.getOptions (.getEngine (context))))
```

I've done some profiling

```clojure
(dotimes [_ 5]
  (time
   (do (let [rows [["apples" 2]
                   ["bananas" 3]]
             colors {"apples" "red" "bananas" "yellow"}]
         (js-svg/categorical-donut rows colors))
       nil)))

"Elapsed time: 210.32659 msecs"
"Elapsed time: 193.211736 msecs"
"Elapsed time: 190.97775 msecs"
"Elapsed time: 195.843254 msecs"
"Elapsed time: 188.077405 msecs"
nil
```

For interpreted and in another language, this doesn't seem the end of
the world.

And adding more quantities
```clojure
(letfn [(rand-string [] (str/join "-" (repeatedly 4 #(rand-nth ["banana" "horse" "battery" "cloak"]))))]
  (dotimes [_ 5]
    (let [rows (repeatedly 20 (fn [] [(rand-string) (rand-int 100)]))
          colors (zipmap (map first rows) (cycle ["red" "green" "blue" "yellow"]))]
      (time
       (js-svg/categorical-donut rows colors)))))

"Elapsed time: 288.659887 msecs"
"Elapsed time: 334.733071 msecs"
"Elapsed time: 317.369294 msecs"
"Elapsed time: 272.084918 msecs"
"Elapsed time: 272.7203 msecs"
nil
```

For a timeline line chart, rendering one datapoint per day for a whole
year:

```clojure
(dotimes [_ 5]
  (let [dates (take 365 (iterate #(.plusDays % 1) #t "2020"))
        rows   (map (fn [d] [d (rand-int 200)]) dates)
        labels {:left "count" :bottom "year"}]
    (time
     (js-svg/timelineseries-line rows labels))))

"Elapsed time: 569.691124 msecs"
"Elapsed time: 544.415676 msecs"
"Elapsed time: 539.131092 msecs"
"Elapsed time: 492.60486 msecs"
"Elapsed time: 523.765691 msecs"
nil
```

New BigQuery Driver

Create new :bigquery-cloud-sdk driver using the google-cloud-bigquery library instead, and whose source is adapted from the :bigquery driver

https://cloud.google.com/bigquery/docs/reference/libraries

Marking existing :bigquery driver as deprecated, and superseded-by the new one (bigquery-cloud-sdk)

Update new driver and query processor code to use newer Google SDK

Switch test data loading over to use new API

Add project-id connection property to override the value from the service account JSON, and use it as part of qualified names in the query processor if set

Updating google driver so its libraries are compatible with the newer ones used in BigQuery

Update date bucketing tests to skip :bigquery-cloud-sdk (new driver) where :bigquery is skipped

Update `with-bigquery-fks` to take in the driver, since there are now multiple ones

Adding test to confirm that overriding project-id for a public BQ project works (sync and query)

Fixing a bunch of whitespace alignment errors in tests

* Backend SVG rendering proof of concept [ci skip]

* Update cssbox to 5.0.0

* Render bar, line, and pie charts in js to svg

sparkline is now done in js, bar is now recognized and done in js, new
:categorical/donut as well

* Remove api route for render

* pass along render-type, not hardcoded to :inline

* Move bar chart above sparkline and remove line check

In order to introduce the bar chart type need it above the sparkline
check since it is otherwise the same except for display property of
the card. But lots of tests assume that this will get hit with a nil
display type set in testing so remove checking for `:line` allows all
the testing cases to hit the right type

* Fix tests now that bar graphs aren't html but images

* Include attachments for bar charts

* Move over to in-tree bundle

* Force everything [ci noskip]

trying to ensure that the built jar includes the newer
"resources/frontend_client/app/dist/lib-static-viz.bundle.js"

* Run `yarn build-static-viz` in backend-deps in CI

this js file is now a hard dependency of the backend so it fits in
this tsk. All such things that depend on the backend sources will need
it. Makes me think perhaps we want a checked in version but i'm not
sure yet.

* Look on classpath not filesystem for js bundle [ci noskip]

* Move yarn build-static-viz into the checkout step

* License information for antlr4-runtime

* create attachment for categorical donuts

* add ordinal legend to donuts (#17177)

* set widths of html image and svg image to 1200

* Revert "add ordinal legend to donuts (#17177)"

This reverts commit 1eb81d2e.

* Helper functions to render html easily

* readme in dev

* readme ensure that static viz bundle exists

* Cleanup ns after removing proxy

* Donut chart colors and legend (#17251)

* use external color map for fill per dimension

* Add support new color legend for donut

* Ensure text doesn't appear as link

entire thing is actually the body of a link tag for emails but we want
a decent text color rather than a default link color

* use chart colors from https://stats.metabase.com/_internal/colors



* Make checkers happy

- remove unused imports
- add a docstring
- don't shadow fn with a local

* cleanup ns import

* Remove reflective call

* Cleanup ns on correct branch

Co-authored-by: dan sutton <dan@dpsutton.com>

* X-axis: just use (approx) 5 ticks to avoid overlapping labels (#17287)

* increase gap between arcs (#17271)

* Set rendering hints on html->image

* ignore width for now and make them larger

* Ns deprecation and some cleanup

* make namespace checker happy

* Simple tests for detecting chart type

* Rename from poc

* Tests for scalar/smartscalar

* cleanup js svg namespace a bit

* Tests of svg engine

* ns sorting after renaming

* Unify our two different js engine usages

settled on the js context. Has typed returns `(.asString ^Value ...)`
instead of perhaps capturing std out?
https://www.graalvm.org/sdk/javadoc/org/graalvm/polyglot/Value.html



Context is a bit more friendly for getting source into it. One
downside is that the invocable bit isn't quite as nice. The old way
would return a java.util.functionFunction but the difference is

(.apply function (object-array args))

vs

(.execute fn-ref (object-array args))

* Don't io/resource the io/resource

* js engine tests

* Ns cleanup in js-svg

type hints in the js-engine ns mean we don't need as many classes from
polyglot here

* Cleanup of text, ns docstrings, alignment

* Fix fill->fill-opacity with parsed doc, not regex

* Make a single helper that loads a static viz bundle context

* Docstrings and make private in js-svg

* Sort imported classes in js-svg

* Make width passed down through rendering aparatus

- svgs are always rendered at 1200 for quality
- slack images of html are rendered at 1200 so that they can be zoomed
  in in the ui but slack automatically scales down
- email sends html and the full svg but includes img width tags so
  that is handled appropriately

* docstring

Co-authored-by: dan sutton <dan@dpsutton.com>
Co-authored-by: Kyle Doherty <5248953+kdoh@users.noreply.github.com>
Co-authored-by: Ariya Hidayat <ariya@metabase.com>

* Add the whitespace linter

* Fix whitespace linter errors [except for one file]

* Add a line that will intentionally break stuff to verify the linter is working.

* Ok, remove the line that caused the linter to fail.

* Use latest version of the linter

* Fix missing newline

* Bump some more backend dependencies

* Add license info for com.sun.activation/jakarta.activation

* Fix reflection warnings

* Revert bump of MariaDB for now

* Bump some backend deps.

* Update i18n/locale to work with new version of LocaleUtils