* Test fix 

 [ci mysql] [ci mariadb] [ci postgres]

* object-defaults should always remove created_at/updated_at
[ci mysql] [ci mariadb] [ci postgres]

Co-authored-by: Cam Saul <github@camsaul.com>

* Remove Table.rows entirely

* Lots of test updates/modernization

* Test fixes 

Co-authored-by: Cam Saul <github@camsaul.com>
Co-authored-by: Maz Ameli <maz@metabase.com>

* Support substituting SQL snippets in queries

* Rename SQL snippets to native query snippets

* Add migration and model for native query snippet table

* Add API endpoints for native query snippets CRUD

Tests are incomplete. Consequently the API is not well tested.

* Cosmetic: Docstring typo and indentation

* Remove pre-delete handler that prevents deletion

It breaks in cases where deletion is reasonable, like during tests. No
DELETE API endpoint will be added, so no it shouldn't be necessary to
add any additional safeguards.

* Fix `:database_id` field reference

And remove unused import.

* Test snippets listing via API

* Add more API test stubs

* Refactor snippets listing endpoint for improved readability

* Align NativeQuerySnippet model behavior with Segment model's

* Clean ns

* Fix and test individual snippet read via API

* Fix snippet name schema

* Test creation of snippets via API

* Test update API endpoint for native query snippets

* Handle query snippet model in H2 dump/load commands

* Port load-from-h2 tests to clojure.test

* Deduplicate models list used to dump/load H2 dbs

The failing load-from-h2 test that checks that all models are accounted
for alerted me that I need to handle the new `NativeQuerySnippet` model.
However, there was no such test for dump-to-h2. Instead of adding such a
test, I opted to simply use load-from-h2's list in dump-to-h2 to
eliminate the problem, rather than testing for it.

* Dump/load collections before models that depend on it

* Linter appeasement

* Use snippet ID and content in query parameter record

* Implement snippet lookup for template tags with ID

* Basic FE support for snippet template tags

* Use shorter "snippet" prefix instead of "native-query-snippet"

* Use simpler `tru` instead of `str` with `deferred-tru`

* Allow whitespace between snippet prefix and name

Also accept any sequence of characters after prefix.

* Don't allow snippet names to start with whitespace

Whitespace is allowed between the `snippet:` template tag prefix and the
snippet name. That whitespace is trimmed out when extracting the snippet
name.

* Add database ID to snippet template tags before expanding parameters

* Ensure that snippet db matches query db

* Avoid setting `:template-tags` for native queries without it

* Factor out snippet database ID validation

* Test snippet database ID validation

* Avoid validating nil snippets

* First pass at nested template tags in snippets

* Pass parent query parameters on to nested sub-queries

* Remove unused let-binding

* Added disabled test for reading query param from template tag

* Temporarily disable failing test

* Remove half baked nested snippet support

* Snippet model perms = native query perms

* Rename param k→ks for clarity

* Remove superfluous `some?` call

* Use utility functions to reduce cruft

* Test expansion of multiple snippets in (sub-)query

* Rather use testing utils from metabase.test

* Check for snippet changes only in provided fields

* Deduplicate field checks, with added test case output

* remove unused import from bad merge

Co-authored-by: Paul Rosenzweig <paul.a.rosenzweig@gmail.com>
Co-authored-by: Paul Rosenzweig <paulrosenzweig@users.noreply.github.com>

* Backported fixes

* Explicitly include data.xml as a dep

* Use OffsetDateTime instead of org.h2.api.TimestampWithTimeZone [ci mysql]

* Revert JUnit tweaks -- don't seem to be working correctly

Co-authored-by: Marcelo Gimenes de Oliveira <marcelo.gimenes.oliveira@gmail.com>

* Add Database auto_run_queries setting; code cleanup

* More code cleanup & test fixes

* Test fixes ?

[ci mysql] [ci postgres]

[ci drivers]

[ci drivers]

* Default wrapper  padding left & right

For show cards padding on mobile view

* restructure visualization settings markup to fix label wrapping (#8816)

* Change query result hashing to convert integers to floats first

When the FE returns the query result metadata, any integer values will
be converted to a floating point number. Those doubles hash different
than the original integers which causes the hashes not to match and
the query to be reran when saving the question. This commit just
converts the integer to a double before computing the hash.

Fixes #8824

* Ensure queries to compute metadata include user information

When saving a question and the included metadata is incorrect or
different we will execute another query to compute that metadata. This
commit adds the `:info` map needed to include the user and query hash
information we use in the SQL remark.

Fixes #8825

* Metastore code SHOULD NOT barf it API response includes unrecognized keys

* Fix minification

* Update copyright year (#8711)

* Update copyright year

* 2018 year

* Could the admin nav bar use a little color? (#8819)

* purple admin nav bar

* decouple admin navbar color from accent7

* filter out hidden tables in table entity (#8831)

* Dropped tables should appear in a schema's data browse page

When the database had only a single schema, the dropped tables were
being hidden correctly. When multiple schemas were present, viewing
those tables their the schema route caused the inactive tables to
appear in the result.

Fixes #8822 fixes #8363

* Ensure consistent ordering of maps/sets when hashing metadata

This commit will ensure all hashes and sets are ordered before
serializing them to JSON and hashing that string. This will prevent
surprising behavior when the number of items in a map reach 9 or if
the order of elements changes between machines or JDK versions.

Fixes #8826

* Far-future cache header for JS/CSS files with cache-busting query strings

* Fix import on TableSettings page (#8829)

* fix import

* clean up nav

* Fix test checksum test failure on MySQL [ci drivers]

The newly added checksum tests fail when running on MySQL in
Circle. The MySQL tests include a secret key for encryption which are
used when computing the checksum. Because of that encryption, the
checksum string is different every time (it must be decrypted before
compared). The test isn't covering the encryption code but is only
concerned with ensure the datastructure hashes consitently. This
commit uses a `with-redefs` to set that secret key to `nil` so that
encryption won't be used on those two tests.

* Add better error messages to loading GeoJSON files

Previously the only error message was a schema validation failure. The
same error would be given for an unreachable host, unparsable JSON,
bad URL etc. This commit attempts to separate that out and provide a
better message in the failure case.

Fixes #8793

* Only include a CSV attachment for pulses when XLS is not selected

Previously we would always include a CSV attachment when there were
more rows/columns than would be included in the table. This ended up
being redundant for users that would prefer an XLS attachment and have
indicated that the pulse should include an XLS attachment. This commit
will only not attach a CSV if the user has select an XLS attachment
for that pulse.

Fixes #7302

* Add a C3P0 exclusion to Quartzite [ci drivers]

* Merge release 0.31.0 into master (#8858)

* Dash card display fixes (#8777)

* fix scalar long title ellipsification and description

* fix table simple in night mode

* check for description

* v0.31.0-RC1

* add scroll to outer container for database / schema selector (#8782)

add scroll to outer container for database / schema selector

* Fix minification

* Far-future cache header for JS/CSS files with cache-busting query strings

* Add Snowflake to list of supported databases in dox [ci skip]

* scroll table selector (#8857)

* Use compressed bytes for caching size comparison

Previously we would serialize rows as strings and use that as an
estimate for how big the cache would be. This commit changes that to
use actual compressed bytes to determine size.

* Switch to a stream based cache serialization/compression

This commit switches to a stream based nippy
compression/serialization. This has the benefit of being able to stop
serialization once the threshold for cache size has been reached.

Fixes #7479

* Bump the `java.classpath` and `tools.namespace` versions

Bumping `java.classpath` to `0.3.0` fixes the `classpath` function on
JDK 9+.

* Don't strip insights from response in public links

* update tests

* Bump Jetty server version

* Better-organized HTTP client used for tests and add full-response option.

* Use middleware impl for test util fn bind-current-user

* Add some i18n tags and wrap in str where missing

* Namespace form & indentation cleanup; docstring improvments

* Remove dead code in tests 

* MBQL schema tweaks; make SQL QP nested-query-level available to public

* Ignore results_metadata in revision tests

* Add test for updating permissions graph from API 

* Add u/hexadecimal-string? util fn

* Add query-perms/can-run-query? helper fn 

* Record database_id in QueryExecution; record query in Query 

* Add a few util fns; other minor code tweaks

* Change query perms-set fn to take kwargs for opts 

* Support backslashes in DB/schema/table names for permissions checks

* Bump docstring checker -> 1.0.3

* Fix bug where Snowflake driver used wrong DB name to filter tables [ci snowflake]

* Add new bird-count dataset; fix division by zero in MBQL expressions

* Add test profiling info & don't suppress dataset load output [ci drivers]

* Remove unused default-schema; share Presto catalog for all test data [ci drivers]

* Add format-name for SparkSQL test extensions [ci drivers]

* Fix presto tests  [ci presto]

* handle smaller smart scalar cards better (#8897)

* handle smaller cards better

* handle automatic dashboard sidebar case

* make more specific

* Fix presto test [ci presto] (#8900)

Previously we would serialize rows as strings and use that as an
estimate for how big the cache would be. This commit changes that to
use actual compressed bytes to determine size.

Previously there was nothing to uniquely identify a row in the
databasechangelog table as the uniqueness constraint was removed by
Liquibase (see https://liquibase.jira.com/browse/CORE-1909). Not
having something to uniquely identify a row can cause issues for
database replication software. Since we only support PostgreSQL, MySQL
and H2, the composite key referenced in the above issue (ID, AUTHOR,
FILENAME) will work for us. This commit puts that uniqueness
constraint back in for users wanting to replicate the Metabase
application database.

Fixes #8266

This adds a new table for storing sync metrics and a quartz task for
cleaning up old entries in the table.

For now this only stores sync metrics and metadata for the task
history cleanup task. Going forward our other background tasks would
store information as well to help with debugging/troubleshooting those
tasks.

Fixes #7611

Quoting all objects works for MySQL/MariaDB but that same setting
causes H2 and PostgreSQL to fail. H2 appears to be quoting table names
in a create table statement but not quoting that same table name when
referenced via a foreign key which fails.

This commit sets up the quote all objects strategy for MySQL/Maria but
leaves the `LEGACY` quoting strategy for H2/PostgreSQL.

We have several spots in our migrations that we have slightly
different behavior for MySQL than we do for H2 or PostgreSQL. This
just extends those same changes to MariaDB.

MySQL 8+ includes many new reserved words including words that collide
with some of our column names. This commit adds quoting to migrations
that include column names that are reserved words.

This column name and logic matches what is done currently for other
entities in the model. This commit also adds a `archived` column for
pulses/alerts to make it more consistent with the other entities. The
`archived` column for Pulses is just included in responses currently,
there's no plumbing to populate it, filter by it etc.