Co-authored-by: bryan <bryan.maass@gmail.com>

See: https://security.snyk.io/vuln/SNYK-JAVA-ORGBOUNCYCASTLE-6612984



Co-authored-by: Chris Truter <crisptrutski@users.noreply.github.com>

Co-authored-by: Noah Moss <32746338+noahmoss@users.noreply.github.com>

Co-authored-by: Cam Saul <1455846+camsaul@users.noreply.github.com>

Co-authored-by: Noah Moss <32746338+noahmoss@users.noreply.github.com>

Co-authored-by: Cal Herries <39073188+calherries@users.noreply.github.com>

* wip

* SLO works with auth slo handler route

* move slo handling endpoint to /auth/sso/handle_slo

* fix slo redirect url

* SLO works, and the sso-handle-slo for saml is where it belongs

- a ton of cleanup

* fix api/session namespace + add docstrings

* cleaning up logout action

* add slo logout test along with slo response xml

* whitespace + linter

* add docstring

* update exclusions in deps.edn

* un-require metabase-enterprise ns from oss ns

* add docs for how to setup SLO to metabase docs

* docs: clarify that setting up SLO is optional

* move slo logout endpoint into ee code

- removes sso-info defenterprise since it is no longer needed

* use current version of saml20-clj

---------

Co-authored-by: bryan <bryan.maass@gmail.com>
Co-authored-by: Nick Fitzpatrick <nickfitz.582@gmail.com>

Co-authored-by: Alexander Solovyov <alexander@solovyov.net>

Co-authored-by: Alexander Solovyov <alexander@solovyov.net>

* Redshift: drop session schema when test suite finishes [ci skip]

* Update Hawk version

* PR feedback

* Bump Hawk version

* Skip after-run for drivers that don't have test extensions

---------

Co-authored-by: Cal Herries <39073188+calherries@users.noreply.github.com>

pulling in https://github.com/camsaul/toucan2/pull/162 to fix #38516

* Remove references to shared/ directory which no longer exists

* Update shadow-cljs and pinned cider/nrepl versions

* Support legacy metadata and raw Dates for brush filter helper functions

* Fix fn name

* Apply suggestions from code review

Co-authored-by: metamben <103100869+metamben@users.noreply.github.com>

---------

Co-authored-by: metamben <103100869+metamben@users.noreply.github.com>

Co-authored-by: metamben <103100869+metamben@users.noreply.github.com>

* Add dev fn to render dashboards with static-viz renderers

Run `(dev.render-png/render-dashboard-to-html 1)` to render the dashboard with id 1 to a handy html file.

This file will render each dashcard in the dashboard 3 ways:
 - as a png, like you would see in Slack or in an email body (if it's a chart)
 - as html/svg, like you'd see in the email body (for tables).. might also be helpful to inspect the svg output from
 the graaljs interpreter
 - 10 row table representing what the .csv attachment for the dashcard would look like

Note: to get this branch working properly, there are a couple lines commented out to pass the linter... it's a work in
progress and will be cleaned up for better use.

Todo:
 - [ ] avoid with-redefs to pass the linter
 - [ ] add a preview endpoint that can be used so that you don't need to run a Clojure repl to get the same result.

* Eliminate with-redefs to allow csv 'render'

* Make a 'preview' ns to power `api/pulse/preview_dashboard/:id`

* Make dev.render-png dashboard preview fns the same as the preview ns

* Dashboard Subscription preview ns now also uses csv attachment code

This change now uses the csv attachment code to include an html table representing what we expect to see from csv exports.

* Add dev fn to render dashboards with static-viz renderers

Run `(dev.render-png/render-dashboard-to-html 1)` to render the dashboard with id 1 to a handy html file.

This file will render each dashcard in the dashboard 3 ways:
 - as a png, like you would see in Slack or in an email body (if it's a chart)
 - as html/svg, like you'd see in the email body (for tables).. might also be helpful to inspect the svg output from
 the graaljs interpreter
 - 10 row table representing what the .csv attachment for the dashcard would look like

Note: to get this branch working properly, there are a couple lines commented out to pass the linter... it's a work in
progress and will be cleaned up for better use.

Todo:
 - [ ] avoid with-redefs to pass the linter
 - [ ] add a preview endpoint that can be used so that you don't need to run a Clojure repl to get the same result.

* Eliminate with-redefs to allow csv 'render'

* Make a 'preview' ns to power `api/pulse/preview_dashboard/:id`

* Make dev.render-png dashboard preview fns the same as the preview ns

* Dashboard Subscription preview ns now also uses csv attachment code

This change now uses the csv attachment code to include an html table representing what we expect to see from csv exports.

* Collect all element styles into a single style tag to use with the server's nonce

This allows the dashboard_preview endpoint to render properly by gathering all of the element styles into a single
style tag, which can then be given the server's nonce value so that CSP doesn't strip the style out.

This is useful for the preview endpoint so that we get an accurate picture of what our tables will look like in
emails, plus it makes the presentation look just a bit more readable.

* dev render_png fns match output used on the endpoint now too.

* Hickory dependency out of dev into regular deps

* Bump dependencies based on vulnerabilities

* More deps bumped

* Bump more deps

* Rollback H2 dependency bump

* Add comments and make guava match elsewhere.

* Adding results_metadata to qp middleware

This adds two middlewares to `metabase.query-processor.middleware.results-metadata`:
- `inject-result-metadata`: Adds `result_metadata` from the context, if present, into the query map during preprocessing.
- `merge-existing-metadata`: A post-processing middleware that merges `results_metadata` (if present) from the query data into the metadata.

This has the effect of preserving existing metadata, which is particularly important for user-curated metadata, such as semantic type overrides.

This may also facilitate addressing the following TODO in the same ns:

```
;; 1. Is there some way we could avoid doing this every single time a Card is ran? Perhaps by passing the current Card
;;    metadata as part of the query context so we can compare for changes
```

* Preserving result_metadata in query processor

This PR uses `qp.util/combine-metadata` to simplify the combination of provided `:result_metadata` with computed metadata. It also updates pulse rendering code to use this new code path and adds and updates tests to vet this logic.

* fmt

* Modifying PR to document and use existing fns

The current qp pipeline will correctly propagate custom metadata if a question is derived from a model or if the model is processed like so:

```clojure
(qp/process-query
  (assoc-in dataset_query [:info :metadata/dataset-metadata] result_metadata))
```

This PR fixes rendering unit tests to clarify this and provides a much better working example in the `dev.render-png` ns.

* Updating test

* Adding Pulse w/Metadata Tests

This pr adds a unit test that constructs a dashboard and simulates a pulse email, then checks the resulting HTML for correctness.

A dependency on the [hickory](https://github.com/clj-commons/hickory) library was added to dev. This allows us to easily parse HTML into data and navigate the parse tree.

Note that it does NOT check the attachments for formatting correctness.

Spelling and import consistency changes were also made.

This reverts commit 10cdb49d.

CI fails because of commit that is being reverted.

* Adding clojure goes fast dependencies and configuration for debugging slow queries.

* Adding OTEL Span Instrumentation

This PR adds a Clojure library for adding opentelemetry spans for
code instrumetation along with a couple examples of instrumented
areas that could be hot spots.

* Adding span around run-query-async

* Adding `publish-event-async!`

The evidence for the slow loads of dashboards such as https://stats.metabase.com/dashboard/2117
as described in https://github.com/metabase/metabase/issues/33499
point to long synchronous calls in `metabase.events/publish-event!`.

This change does the following:

- Adds a span to `publish-event!` for instrumentation
- Adds an async version of the function
- Changes the GET api/dashboard/:id call to use the async function

Based on the discussion in [this slack thread](https://metaboat.slack.com/archives/CKZEMT1MJ/p1697604232592589)
we may not want to retain the async call ATM, but we could use it as
a stopgap for now.

As an aside, we might want to introduce a queue such as activemq if
we really need asynchronous behavior.

* Rebasing and reconciling with master

* Modifying ws to be like master in deps.edn

* sort nses

* Removed dead code

* Removed unneeded check-404

* Added TODO comment wrt goals for db performance

* Whitespace changes

* Removing async publication call

* Adding linter and formatter to staged BE files.

Note that `.cljfmt/indents.clj` tries to capture our indent rules
and, so far, seems to get things right. There are still some inconsistencies
in some of our nses in how we indent assoc(-in) and some t2 functions
when it comes to key-value pairs on following lines. Should the keys
align with the function call or be indented. Aesthetically, people
seem to like indents, but these are functions, so should align with the
first argument. IDK that it really matters as long as we have agreement.

We may, as we adopt this, have some files be reformatted in unexpected ways
for items that have been missed. If so, the developer can simply update
the .cljfmt/indents.clj file, rerun the hook, and commit that along with
their other changes.

* Added whitespace linter.

* Updating husky commit hook

* Updated hook scripts to be exclusively for staged files.

Updated cljfmt to use the latest and greatest along with the new format of config file.

* Commenting out the actual formatting hook in .husky/pre-commit until we come up with a globally happy solution to forms that don't have good formatting rules, like defprotocol+ in a reader conditional.

* Reverting formatting

* Moving pre-commit hooks from .husky/pre-commit to the package.json's lint-staged section.

Note that we're still not calling the cljfmt file (.bin/cljfmt_staged.sh)
until we get agreement on formatting.

One thing we might do since we're using lint-staged is we can probably
exclude the "bad" files that go sideways with linting and format everything
else automatically. This might be a good follow-on PR.

Co-authored-by: Ryan Laurie <iethree@gmail.com>
Co-authored-by: Luiz Arakaki <luiz.arakaki@metabase.com>
Co-authored-by: bryan <bryan.maass@gmail.com>
Co-authored-by: Noah Moss <32746338+noahmoss@users.noreply.github.com>
Co-authored-by: Ryan Laurie <30528226+iethree@users.noreply.github.com>
Co-authored-by: Nick Fitzpatrick <nick@metabase.com>
Co-authored-by: Nick Fitzpatrick <nickfitz.582@gmail.com>
Co-authored-by: Noah Moss <noahbmoss@gmail.com>
Co-authored-by: Case Nelson <case@metabase.com>
Co-authored-by: Jerry Huang <jhuang37050@gmail.com>
Co-authored-by: Bryan Maass <bryan.maass@gmail.com>
Co-authored-by: Jerry Huang <34140255+qwef@users.noreply.github.com>
Co-authored-by: adam-james <21064735+adam-james-v@users.noreply.github.com>
Co-authored-by: Adam James <adam.vermeer2@gmail.com>
Co-authored-by: John Swanson <john.swanson@metabase.com>

```diff
-  info.sunng/ring-jetty9-adapter            {:mvn/version "0.22.1"}             ; Drop-in replacement for official Ring Jetty adapter. Supports Jetty 11 webserver.
+  info.sunng/ring-jetty9-adapter            {:mvn/version "0.22.3"}             ; Drop-in replacement for official Ring Jetty adapter. Supports Jetty 11 webserver.
```

Simple change, and quite possibly a no-op.

We have previously been overriding the jetty dep to be

```clojure
org.eclipse.jetty/jetty-server            {:mvn/version "11.0.17"}
```

And the dep tree before this change was:

```
info.sunng/ring-jetty9-adapter 0.22.1
  X ring/ring-core 1.10.0 :use-top
  X org.eclipse.jetty/jetty-server 11.0.15 :use-top
```

This is saying it declared a dep on 11.0.15 but was ignoring that in
favor of "use-top", aka the top level dependency we declared (11.0.17).

But synk is a scanning tool that doesn't check the artifact but the
manifests involved and flags issues in 11.0.15 and therefore Metabase
despite us not having that version.

Now the deps tree:

```
clj -X:deps tree :aliases '[:ee :drivers]'
info.sunng/ring-jetty9-adapter 0.22.3
  X org.eclipse.jetty/jetty-server 11.0.17 :use-top
```

Doesn't specify an older version of jetty and we're using the same jetty
version as we have for the last bit.