* Druid Docker image [WIP]

* More Druid Docker stuff [WIP]

* Test fixes 

* Remove maxRowsQueuedForOrdering from query, don't think we need it

* Force test Druid host -> localhost for now

* Update sync tests for updated test data

* Revert unused changes

Whacks in search changes for backend needed in #15768 and #15738. Also whacks in the changes for frontend needed in #15738 but not for #15768.

How does Lisp work, again?

* Provide `:last-edit-info` information on collection items

only provided for cards and dashboards. At the moment, collections can
have:
- cards ()
- dashboards ()
- snippets ()(not in UI)
- pulses ()(deprecated)
- other collection ( to be done. needs a table for these
changes. they are revisioned)

* Send edit info along with cards

* Last-edit-info on dashboards

* Move last-edit-info functions into a bespoke namespace under revision

Under revision as the source of changes all come from revisions at the
moment. This might be ill-considered once we add collections into the
mix since they are not good candidates for revisions.

* Renames and docstring update on the last-edit namespace

* Add types

* Add `:last-edit-info` on card and dashboard creation

* Expect last-edit-info in creation repsonse for cards/dashboard

* Docstring on last-edit types

* Make the namespace checker happy

* namespace checker

* Remove moved function

* Add edit info to api/card/ and api/dashboard/

* Move event emission outside of transaction

Co-authored-by: Raimon Grau <raimonster@gmail.com>

Add new namespace to manage diagnostic info about the DW DB connection pools

Adding new atom/fn/macro to wrap the capture of diagnostic info, modeled after the call counting functionality of toucan.db

Capturing diagnostic info from a new `datasource-with-diagnostic-info!` fn in the execute namespace, and calling that from the various drivers (instead of `datasource`)

Adding to log message generated for the "stats" portion of the line logged via middleware to output this information

* limit and offset work upon hand exercise

* limit added

* add the test

* docstring is wrong make it not wrong

* fix namespace decl

* they were test bugs only

* stick that limit and offset in there

* Permissions imports

* make it work in dbs that are not just h2

* add limit test get

* needs integer

* juice the coverity lol

* just stick querying in the overall dealie

* tests pass

* add counter

* rip out the clauses

* not just if limited

* docstring misplaced

* more linting

* cool tests

* tests pass but lacking test lol

* group id works

* test works

* fix jeff nits

* break that api

* add test for count

* cypress works for some reason lol

* fix capitalizations

* admin people pagination frontend (#15821)

* admin people pagination frontend

* admin people pagination and search specs

* address review comments

* fix rebase

* address review comments

Co-authored-by: Alexander Lesnenko <alxnddr@users.noreply.github.com>

Set c3p0 dataSourceName property to include Metabase specific info

Change data-warehouse-connection-pool-properties multimethod to include database as 2nd param

Setting the dataSourceName c3p0 property to have the format db-<N>-<D>-<DB> where <N> is the DW database ID, <D> is the driver name, and <DB> is the database name from the db details

Adding test

Setting c3p0 dataSourceName for the app DB pool as well

* Table collection_revision -> collection_permission_graph_revision

in anticipation of getting collection revisions in, need the table
name as the previous table named this was tracking changes to the
permission graph

* rename collection permission graph revision alias

* Remove unnecessary or for precondition

* Rename sequence in sql

liquibase has a renameSequence changeset but it doesn't work for h2
obviously. However, it doesn't allow this constraint to be honored
with a precondition. Just having this changeset on h2 blows up

* Run tests on Java 8 again (fix #15707); run tests against Java 16

* Switch from Nashorn to GraalVM JS engine

* Remove unused var

* Change PG app name to version + process uuid e.g. `Metabase v0.39.1-SNAPSHOT [04d518eb-df2d-4781-93b3-8701286fd45e]`

* Code cleanup

* Test fixes 

* Watch for munge collisions

* Always have to check for munge collisions

was thinking only in the case where munging made a difference did we
have to check, but if the first one defined is foo? -> foo as the
munge, defining foo later will cause a collision

* Prettier fix

* #15080 Repro: Archived items not shown in archive (#15198)

* /search returns archived items again [Fixes #15080]

Co-authored-by: Nemanja Glumac <31325167+nemanjaglumac@users.noreply.github.com>

* Much simpler impl

* Remove unused var

* Java 11+ test fixes 

* Remove type/UNIX* and type/ISO8601* from frontend

* Set effective-type and coercion strategy when syncing

these values will most likely only be available when using tests, as
metadata is very unlikely to have this. As far as I can tell the
toucannery apparatus is the only bit that has this. Its quite
artificial. I'm not sure if this is a good idea.

* :type/UNIXTimestampSeconds and type/ISO8601DateTimeString out of type

remove the coercions from the type hierarchy. This brought up a
strange issue that has been present for a while: all liquidbase
migrations run and then all data migrations run. They are not
interleaved. This allows for the following scenario:

- data migration migrates all X values to Y in version 26
- liquibase migration migrates all Y values to Z in version 28

But these are not run in version order, but all liquibase migrations
are run and then all data migrations are run. If you were on an old
version for a while, you could end up with Y values even though the
liquibase migration which comes "after" the data migration turns all Y
values into Z values.

This impacts this change because a data migration in this way:
- liquibase changesets 287, 288, 289, and 290 remove all 'type/UNIX*'
and 'type/ISO8601*' semantic types. These were in 0.39
- data migration `migrate-field-types` added in 0.20.0 was looking for
special_type's of "timestamp_milliseconds" and migrating them to
"type/UNIXTimestampMilliseconds".

Since the liquibase runs before the migrate-field-types, it's possible
for a 'type/UNIX*' semantic type to reappear. And since these were
removed from the type system it would fail validation and blow up. In
this case it actually can't happen since the field its attempting to
migrate (special_type) no longer exists. But since the migrations are
not interleaved this problem still exists.

* whatever prettier

* Appease the machines

* Restore the unix and iso semantic types to hierarchy

migration tests still use these
`(impl/test-migrations [283 296] [migrate!] ...)`

A few things in tension: the field requires valid semantic_types,
these _were_ valid semantic_types until v39.

* Remove old "coercion" semantic types

* Migrate the v20 version semantic types for unix epoch

* Time travelling migrations

these target v20 and v29 which at the time they existed had a column
special type not semantic type. But these run after all of the schema
migrations, not interleaved, so they will have a semantic_type not
special_type when they actually run even though they target v20 or v29
data. strange world

* add migration's new checksum

* Backend feature flag for new field filters

* Feature flag new parameter options

When the "field-filter-operators-enabled?" flag is disabled we do the following:
1. Replace new operator options with old category and location/city, etc., options
   in the PARAMETER_OPTIONS list found in metabase/meta/Parameter.js
2. Hide numbers section in the PARAMETER_SECTIONS list found in
   metabase/meta/Dashboard.js
3. Return args as-is in the mapUIParameterToQueryParameter function
   found in metabase/meta/Parameter.js

React/UI code handles both old options and new options so doesn't need
to change. Old parameter types like "category" and "location/city" are
treated like "string/=" in the UI but retain their own parameter type
when used to send a new query.

* Fix FE issues caused by meta/Parameter refactor

* mock the field operator param flag to make tests pass

* add/fix cypress tests

* fix import in ParametersPopover

* update widget tag type

* Enable field filter operators for cypress tests

* Question marks are questionable

* Conditionally use category or string/= if field filters are enabled

* rmv mocks where we don't need them

* rmv mock from chained-filters test

* env vars as string in project.clj, alignment

Co-authored-by: dan sutton <dan@dpsutton.com>

* datetime fields in nested queries should get auto-bucketed the same way as top-level ones

* Test fix 

* Revert name case changes

* Test fix 

* Test fix 

* (#15460) Correct field names in native field filters

```sql
select p.created_at, products.category
from products
left join products p on p.id=products.id
where {{category}}
```

field filter is a string/= on category, a column in this twice due to
the self join.

Just mimic how the rest of this emits a column name

* Unskip tests added in #15468

* Include fully qualified names in tests

* Backend for dashboard sub filters, including EE implementation

* Fix pulse/update-notification! to respect parameters

* Add some tests

* add FE EE parameter section to dashboard subscriptions

* fix improper filter counting for label

* hide parameters section when dashboard has none

* fix thrown errorin pulses list sidebar

* fix linting error

* add cypress tests

* rmv describe.only from test

* refactor FE code a little

* Merge pulse/dashboard parameters correctly for dashboard subs

* fix issues with getting list of active pulse parameters

* update formatting of multi value parameters

* fix cypress test

* tweak pulse details styling

* move ParametersSection to enterprise tree

Co-authored-by: Dalton Johnson <daltojohnso@users.noreply.github.com>

* Repro for #15118

* Revert demo tests

* Send email when a user logs in from a new device for the first time

* Test fixes 

* Remove stray printlns

* Test fix 

* Don't send email on the very first login.

* Test fix 



* Update src/metabase/email/login_from_new_device.mustache

Co-authored-by: Walter Leibbrandt <23798+walterl@users.noreply.github.com>

* Remove unused u/prog1

* PR feedback

* Add IP address geocode timeout

* Geocode all IP addresses at once

* Update tests for updated email template

* Use built-in localized DateTimeFormatters per @walterl suggestion

* Test fixes 



* Remove extra is

Co-authored-by: Walter Leibbrandt <23798+walterl@users.noreply.github.com>

* Throw an error if a setting already exists

```clojure
setting-test=> (defsetting site-name (deferred-tru "A testing setting") :visibility :public)
Execution error (ExceptionInfo) at metabase.models.setting/register-setting! (setting.clj:464).
Setting :site-name already registered in metabase.public-settings
setting-test=> (pprint (ex-data *e))
{:existing-setting
 {:description "The name used for this instance of Metabase.",
  :cache? true,
  :default "Metabase",
  :name :site-name,
  :type :string,
  :sensitive? false,
  :tag java.lang.String,
  :namespace metabase.public-settings,
  :visibility :admin}}
nil
```

* Tests in ci are run from user

* Appease eastwood; don't shadow function `setting-name`

this pattern is exactly what the helpful protocol does. i actually
wanted to call that protocol but it throws an error if it doesn't
exist which is a bit much for our purposes

* Move namespace name into setting information

* Explicitly specify %throwable in Log4j logging pattern so the ex-data gets printed

This tweaks means Exceptions get rendered for logging with .toString instead .getMessage.

See https://github.com/clojure/tools.logging/tree/31073fe1fbb0533f3c775fa667de70809f24b127#log4j2
and https://logging.apache.org/log4j/2.x/manual/layouts.html#Patterns

* Remove ad-hoc query perms check on save logic from metabase.models.card.

We already do this in metabase.api.card, so in practice this is redundant code.

* Better permissions exceptions when missing ad-hoc perms for creating/updating a Card

* Fix #11719

* Return more info with most API exception responses

* Add tests for #11719 and #14931

* Fix namespace lint errors

* Fix error message grammar (thanks @jeff303)

* Fix SAML redirect to /collection/root/

* Test util HTTP client should automatically URL-encode query parameters.

* Simplify HTTP client

* Minor tweaks

* Remove *url-encode-query-parameters?*

* experimental CI fix: cache build script deps

* Fix typo

* Fix Circle config again

* Fetch BE deps if deps.edn files change

* Use CI executor for be-deps, so we have the Clojure CLI

* Bump a few dep versions and add missing exclusions

* Fix export timezones; fix Excel time values

* Text fixes & other improvements

* Test fix 

* Move some XLSX/CSV specific tests into appropriate test namespaces

* Fix SQLite parsing of date-only values

* address PR feedback

* Add missing "is" assertions to various tests

* Fixing save-card-with-empty-result-metadata-test by simply asserting on API response

* Fix more failing tests

* Switch row-type-agnostic-test back to strings for expected

* Modify check for convert-id-to-string to also include keys whose :base_type is :type/Number, in addition to :type/Integer to fix failures in Oracle and Snowflake

Fix Oracle SSL tests

Define new test for Oracle SSL connectivity, in oracle_test.clj, similar to how things work in mysql_test.clj

Add new test util macro, with-env-keys-renamed-by, to support running tests with environ keys temporarily renamed

Using new test macro from both MySQL and Oracle SSL connectivity tests

Removing now unneeded be-tests-oracle-ssl-ee CircleCI job

Removing now unneeded test-selector parameter for test-driver orb in CircleCI config.yml

Updating JVM_OPTS to use a trust store that starts with cacerts and adds the RDS root CA, rather than one only
containing the RDS root CA

* Support empty date field filters

[Fixes #15067]

* Add a test

* No more old-style tests

* Presto test fix 

* Remove location sub-categories

These sub-categories are only for filtering the list of options
when mapping a parameter filter to a field. Since we are introducing
operator types as a sub-category of location, city/zip/etc. just get in
the way.

* add number section + number/string operator subtypes

Light refactor of meta/Dashboard changes

rmv 'all-options' options (for now)

* add/update parameter type icons

* pass operator to ParameterFieldWidget + show input per operator field

* Add operator helper fns that aren't dependent on fields/tables

* Make operator prop optional

fix date filter err

* add combined  name for native question filter widget type list

Otherwise, a field that matches both "Location" and "Category"
options will show duplicate "Starts with" options, etc. Now,
that'll look like "Category - Starts with" and "Location - Starts with"

* correct some unused prop/arg passing

* Convert location/category parameter types to string for query

location/category don't mean anything to BE but we use them for
"reasons" on the FE. Reasons are legacy reliance on unique-ness of
the parameter.type value, primarily.

* operators in backend

* Remove errant tap>

* Docstrings and differing numbers in tests in some dbs

* Make unary private so docstring checker ~shuts-up~ is satisfied

* Don't parse arguments to operators params

they were coming in just fine from the FE as numeric or string
types. no need to ensure strings everywhere and parse here

* add max-width to PopoverPicker

* rmv unused value

* use combinedName on dashboard parameters

* fix parameter to mbql code

* Ensure = operator filter popovers have no label

This is to match "old" style of parameter popover

* Update Cypress tests to reflect new parameter flow

fix cypress dashboard parameter tests

Fix more cy tests

* Don't call fk/joinAlias on ExpressionDimension

The methods don't exist on ExpressionDimension class.

This doesn't make them work (yet), but it prevents the app
from crashing.

* Namespace doc and remove unnecessary comment

* tap>-spy in dev

* first pass at substitution of new operators in native

* Docstring on wrap-value-literals-in-mbql to appease the gods

* variadic equality operators (string/= number/=)

* move functions out of component file

* Pass parameter object to tag editor for use in default input

We should inline this input eventually because it looks ugly.

* map parameters in Questions to correct type

* continue to pass janky fake parameter for text/number tags

* mongo native substitution

* variadic not-equals for string and number

* Docstring and use correct function to make errors

* add number/between dash param cy test

* Update function name to better reflect behavior

* Add unit tests for paramer/operator util fns

add unit tests for parameter util functions

add unit test for operator util fns

* add variadic string 'is not' param operator option

* Modify operator parameter display labels

don't append 'matches exactly' to location/category parameters

label tweaks

Update cy tests to reference correct label name

rmv it.only

* Desugar mongo parameters mbql

desugaring makes for a bit more verbose query but that's ok.This
change was done to ensure that we negated regexes in a correct way,
and to do so we always return the string version. This ensures that it
can be json/generate-string'd for native parameters or left as
datastructures and sent to monger

* Cleanup stale comments and fixup docstring for consistency

* Arglists metadata on defmulti and denude some threaded forms

* add single arity number tag predicate to variable filter

* add Location operators to fix parameter<->filter mapping

For question filters to work we need for the new parameter operators
to be supported by "location" fields in all areas of the app.

* Don't show coords for param number widgets

I don't think we want to support all the various number operators when
dealing with coordinates, so in order to avoid that I'm preventing the
mapping of number parameter operator to coord fields.

* prevent mapping of tags to non-equal operators

while possibly useful to end users, this needs more UI work on the
native question side of things.

* Ensure parameter values are wrapped in an array

When an = operator is mapped to a field AND a tag, it ends up not being
wrapped in an array due to the TextWidget (I think).

ensure parameter value is an array

ensure number params have an array value

* Sort imports correctly

clojure-lsp used to do this incorrectly (sorting `[` before `j`) and
that has now been fixed

Co-authored-by: dan sutton <dan@dpsutton.com>

Adding ssl-cert config field to MySQL DB details map, to hold the server cert chain in PEM format (similar to what is done in MongoDB driver)

Updating MySQL driver init to map :ssl-cert into :serverSslCert for the JDBC url, when ssl is in use and cert is provided (the MariaDB driver we are using accepts PEM format certificates inline directly for the param value, so no need to shepherd into a temp file)

Adding new test to mysql_test.clj to run a single test while connecting via SSL with PEM cert

Update CircleCI config:
 - use extra-env to set all the MySQL SSL instance DB related vars (for an RDS instance, currently)
 - adding the rds-combined-ca-bundle.pem certificate to resources/certificates
 - loading that cert bundle from resources directory via env var

Adding to/fixing assertion in connection-spec-test for :ssl