Static viz waterfall needs BE bits. Here are the BE bits. Color and total summation coming...

* Change query_cache.results column to correct blob type

Add Liquibase changeSet to modifyDataType of query_cache.results to ${blob.type} (an idempotent operation on most DBs)

Add schema migrations test to confirm, which simulates the broken app DB state, then runs the migration, then confirms the fixed type

* Nest qbnewb under /modal to allow for other modals

move the api from api/user/:id/qbnewb -> api/user/:id/modal/qbnewb to
allow for the upcoming new dismissable modal

* Add additional datasetnewb route

staying with the same naming scheme as qbnewb. the db field is
is_datasetnewb, the route just takes datasetnewb.

I'm not wild about the default is true, set to false, and the naming
scheme. But will be easier to migrate and easier for FE to use if we
continue the defaults/logic etc.

* Remark on db change in migration file

* Change all active TEXT columns in MySQL app DB to LONGTEXT

Add new text.type Liquibase property to dynamically select the correct text type to use, per DB (which is LONGTEXT for MySQL), very similar to the existing blob.type one

Adding new migrations that update all existing app DB TEXT columns to LONGTEXT columns, only for mariadb/mysql

Update migrations linter to ensure no new text types get added, via a new predicate that searches for text types being added

Do the same logic for "blob" now (should be "${blob.type}"), update rule and test

Add test for migrations, that checks the new types, for all types in scope for this PR, to ensure that they have all been changed to their expected DB-specific type (either LONGTEXT for MySQL or TEXT/CLOB for others)

Updating a couple migrations to remove special casing that was originally done only for MySQL to simply make them universally "${text.type}", in order to unify behavior and reduce surprises going forward

Co-authored-by: Cam Saul <github@camsaul.com>

* Drop long-unused Table.entity_name column

* Add NOT NULL constraint to Card.database_id

* Test fix 

* Test fix 

* Fix migration

* Add note about H2 shell to deps.edn

* Add new SQL migration to attempt to set database_id when unset

* Remove data migration that is now done in Liquibase land

* Oops, '$.database', not '$.database_id'

* Parse ints as signed rather than unsigned just to be safe (they *might* be -1337 if they're really broken)

* Don't run for H2

* Update comment

* Fix migration indentation

* Clean namespace

* Use new migration number.

* Use the new migration numbers

* Adopt new migration numbering scheme

* Fix comments

* Fix MySQL + MariaDB insanity

* Fix ID range validation

* Actually 382 is the last legacy ID

* Improved validation and tests

* Adopt the new-new migration ID format.

* Test fixes 

* Fix merge

* Simplify precondition

* Revert changes from Jeff's PR

* Add index to ModerationReview moderated_item_type + moderated_item_id

* Add to 0.41.2 instead

* Require explicit index name for createIndex

* Move 41.2 migrations to after the 41.0 migrations

* Adopt new migration numbering scheme

* Fix comments

* Fix MySQL + MariaDB insanity

* Fix ID range validation

* Actually 382 is the last legacy ID

* Improved validation and tests

* Adopt the new-new migration ID format.

* Test fixes 

* Adopt new migration numbering scheme

* Fix comments

* Fix MySQL + MariaDB insanity

* Fix ID range validation

* Actually 382 is the last legacy ID

* Improved validation and tests

* Adopt the new-new migration ID format.

* Test fixes 

* Add dataset flag to Card

* Set dataset viz type to table

* Include datasets in collection children

* Ensure can write as dataset, include more tests

Add secret model

Add Liquibase migration for secret table

Implement bare bones model namespace for secret

Add simple test to ensure secret values are stored and retrieved under an encryption key (or not)

Add :creator_id to secret model to capture the user who created this secret instance/version

Support updating secret values as part of encryption key rotation

Add test assertions for secret values in the `rotate-encryption-key!-test`

* Fix precondition of change set 97

Without the `type` and with the space Liquibase is unable to parse this
precondition.

During `lein test` it outputs:
```
[clojure-agent-send-off-pool-0] DEBUG liquibase.changelog - Running Changeset:migrations/000_migrations.yaml::97::senior
[clojure-agent-send-off-pool-0] DEBUG liquibase.executor - Changeset migrations/000_migrations.yaml::97::senior
[clojure-agent-send-off-pool-0] DEBUG liquibase.executor - Added 0.32.0
[clojure-agent-send-off-pool-0] INFO  liquibase.changelog - Marking ChangeSet: migrations/000_migrations.yaml::97::senior ran despite precondition failure due to onFail='MARK_RAN':
          liquibase.yaml : DBMS Precondition failed: expected null, got h2

[clojure-agent-send-off-pool-0] DEBUG liquibase.changelog - Skipping ChangeSet: migrations/000_migrations.yaml::97::senior
[clojure-agent-send-off-pool-0] DEBUG liquibase.executor - Executing with the 'jdbc' executor
[clojure-agent-send-off-pool-0] DEBUG liquibase.executor - 1 row(s) affected
```

After this change the output changes to:
```
[clojure-agent-send-off-pool-0] DEBUG liquibase.changelog - Running Changeset:migrations/000_migrations.yaml::97::senior
[clojure-agent-send-off-pool-0] DEBUG liquibase.executor - Changeset migrations/000_migrations.yaml::97::senior
[clojure-agent-send-off-pool-0] DEBUG liquibase.executor - Added 0.32.0
[clojure-agent-send-off-pool-0] INFO  liquibase.changelog - Marking ChangeSet: migrations/000_migrations.yaml::97::senior ran despite precondition failure due to onFail='MARK_RAN':
          liquibase.yaml : DBMS Precondition failed: expected mysql,mariadb, got h2

[clojure-agent-send-off-pool-0] DEBUG liquibase.changelog - Skipping ChangeSet: migrations/000_migrations.yaml::97::senior
[clojure-agent-send-off-pool-0] DEBUG liquibase.executor - Executing with the 'jdbc' executor
[clojure-agent-send-off-pool-0] DEBUG liquibase.executor - 1 row(s) affected
```

For documentation of the syntax cf.
 https://docs.liquibase.com/concepts/advanced/preconditions.html



* Extend migration linter to check dbms preconditions

* Also validate the `type` field of the `dbms` precondition

Co-authored-by: dpsutton <dan@dpsutton.com>

* Static viz: respect custom X and Y axis labels

* Static viz: allow rendering non-timeseries line charts

* Test fixes 

Cache controls all landed but is lacking in the audit capacity wanted for in the notion doc. This PR adds that audit capability and by the by changes the ViewLog model in order to be able to deliver on the question of whether ViewLogs were cache hits or not.

There's more to come WRT the audits, but anton and I agreed to merge something faster and work on that stuff after

* Pass along date formatting

* extract and rename some things

* Number formatting

* Update tests

* Improve test validation of svg labels

* Clean up tests

* Move static-viz interface file to custom file

* Determine MB project root directory relative to source directory of build scripts

* Support calling build-driver! as a -X fn, and support dirs outside of modules/drivers

* Support loading 3rd-party driver manifest files at (dev) launch

* Fix log4j not working on latest version of CIDER (unrelated)

* Fix dissoc

* project-root-directory is supposed to be a string

* Add the whitespace linter

* Fix whitespace linter errors [except for one file]

* Add a line that will intentionally break stuff to verify the linter is working.

* Ok, remove the line that caused the linter to fail.

* Use latest version of the linter

* Fix missing newline

* rmv old bucm icons and remove verified fill color

* add moderation action section to sidebar

* add moderation review icon to the saved question header button

* hide moderation section when is not a moderator

* add UI for ModerationReviewBanner

* Backend for moderation-review

- create table moderation_review. Same as before but also has a
  "most_recent" boolean flag for the most recent moderation for easy
  lookup
- POST /moderation-review/ . Status can be "verified" or nil
- must be an admin to post
- No PUT or edit route yet. Not sure if this is even
  necessary. _MAYBE_ to edit the text, but certainly not for the
  status, ids, etc. If there's to be history, let's build some history
- Ensure we never have more than 10 reviews. Adding a new review will
  delete the older ones, mark all old ones as not `most_recent`, and
  add the newest one as `most_recent true`
- Ensure the card actually exists before creating the mod review
- Since admin only at this time, don't need to check moderate
  permission or view permission
- When hydrating ensure reviews are ordered by id desc. Should mimic
  the created_at desc

* fix moderation review banner tooltip offset

* disable verification button when already verified

* rmv iconOnly prop because it seems to do nothing

* update getLatestModerationReview to rely on most_recent boolean

* Return 400 on invalid status to post /moderation-review

the schema was using keywords on the left hand side rather than the
symbols. Required a change to the docstring generator, when it made a
docstring for enums, it would call (sort (:vs enum)) and need to
string em.

* Add ModerationReview model to models.clj and copy infra

* hydrate moderation reviews on cards

* clean up + wire up to BE + ensure mod buttons don't show for normal users

* rmv unused moderation redux logic from QuestionDetailsSidebarPanel

* finish writing unit tests for FE

* ensure getIconForReview returns an object

* enable/disable verify button tooltip when unverified/verified

* add e2e tests

* fix tests

* styling tweaks

* more styling on moderationReviewBanner

* add function for abbreviated timestamp

* increase fontsize of timestamp back to 12

* fix tooltip offset

* ensure custom locale is separate from 'en' and not used for other languages

* Deletion moderation reviews when deleting cards

i had actually thought this was a much larger problem. But it turns
out we almost never delete cards (thanks comment!). And so we won't
really generate a lot of garbage.

I was worried that since we aren't using actual foreign keys but just
`moderated_item_type "card"` and `moderated_item_id 2` we would have
deleted cards with these moderation reviews but that is not the case
as the cards aren't deleted.

* hide verify disabled button when a question is verified

* update test to use queryByTestId

* Hydrate moderation reviews on cards on ordered cards

* Handle mysql's lack of offset functionality

mysql cannot handle just a `offset` clause, it also needs a limit

clause grammar from
https://dev.mysql.com/doc/refman/8.0/en/select.html:

[LIMIT {[offset,] row_count | row_count OFFSET offset}]

select id, name from metabase_field offset 5;         -- errors
select id, name from metabase_field limit 2 offset 5; -- works

Since our numbers are so small here there is no worry and just do the
offset in memory rather than jump through hoops for different dbs.

* Batch hydrate moderation reviews

* Don't let /api/user/:userId failure conceal moderation banner

* fix moderation cy tests

* work around possible bug in toucan hydration

dashboards hydrate ordered cards
(hydrate [:ordered_cards [:card :moderation_reviews] :series])

Ordered_cards are dashboard_cards which have an optional card_id. But
toucan hydration doesn't filter out the nils as they go down. It seems
toucan returns a nil card, and then when hydrating the
moderation_review, passes the collection of all "cards" including the
nil ones into the hydration function for moderation_reviews

This feels like a bug to me

* Cleanup moderation warnings

* Docstring in moderation review

* include hoisted moderated_status on cards in collections api

* Expect unverified in test 



Co-authored-by: dan sutton <dan@dpsutton.com>
Co-authored-by: Maz Ameli <maz@metabase.com>
Co-authored-by: alxnddr <alxnddr@gmail.com>

Co-authored-by: Mário Valney <mariovalney@gmail.com>

Friendly names bugfix 1 was a noop because of a typo in the WHERE clause of the DB migration. This one has been manually tested to go and sync on all 4 supported MB db's.

There was no DB migration for previous friendly field name defriendlification. This adds DB migration.

Peeps want to make questions faster, but we don't give the data about query execution in a very user-friendly way. Here's it in a user-friendly way.

Co-authored-by: Alexander Lesnenko <alxnddr@users.noreply.github.com>

* Add type on collection

* Search with collection type

* Cleanup bounded heap accumulator

* Make search conditionally bump official collection items

* collection api and tests

* Put collection type onto hydrated cards on dashboards

* Move official collection type scoring into ee codebase

* ensure ee and oss agree when not official collection

* Mark Collections tree with type

* Remove unnecessary `and`s when no check for enhancements

* Tests for setting collection tree type

* Include hydration for collection type on dashboards

* Make sure created test collections are cleaned up

* Cleanup tests, don't search on collection_type

looks for all text fields and searches the term against that. official
would bring back all official types. no bueno

* Docstring on protocol and don't shadow comparator

* update to new ee impl var passing style

* Collection model ensures no type change on personal collection

* Check for personal collection when updating type

model checks for personal collection and rejects in the update but
doesn't check for children of personal collection.

* Update checking for unchangeable properties of a personal collection

* Cleanup: type collection tree, batched hydration, combine error checks

* Cleanup

* move bounded-heap accumulator to utils

* switch to test.check testing

* Bad test just to see what our CI will report

* remove purposeful failing test (was for CI)

* collection.type -> collection.authority_level

* Test the actual ee scoring impl, not the wrapped one

locally i had enhancements enabled so the tests were passing, but in
CI it was failing as it did not have enhancements enabled

* Delete obsolete Safari favicon

* Set the default favicon path to the existing icon

* Delete fallback png favicons

* Fix placeholder in related Cypress test

* Port legacy data type migrations -> Liquibase

* Fix migration IDs

* Fix migrations

* Add deny list of common passwords

* replace filtered pw list with full list

* switch to using a FileReader to check password line-by-line

* bypass deny list if complexity is set to easy

* fix tests

* more test fixes and remove unneeded ns

* fix typos

* fix reflection warning

* needed to import File

* fix final tests, and update docs to reflect change

* address initial review comments

* fix build after removing import statement

* address more review comments

* use InputStreamReader to read password file in JARs

* Table collection_revision -> collection_permission_graph_revision

in anticipation of getting collection revisions in, need the table
name as the previous table named this was tracking changes to the
permission graph

* rename collection permission graph revision alias

* Remove unnecessary or for precondition

* Rename sequence in sql

liquibase has a renameSequence changeset but it doesn't work for h2
obviously. However, it doesn't allow this constraint to be honored
with a precondition. Just having this changeset on h2 blows up

* Remove type/UNIX* and type/ISO8601* from frontend

* Set effective-type and coercion strategy when syncing

these values will most likely only be available when using tests, as
metadata is very unlikely to have this. As far as I can tell the
toucannery apparatus is the only bit that has this. Its quite
artificial. I'm not sure if this is a good idea.

* :type/UNIXTimestampSeconds and type/ISO8601DateTimeString out of type

remove the coercions from the type hierarchy. This brought up a
strange issue that has been present for a while: all liquidbase
migrations run and then all data migrations run. They are not
interleaved. This allows for the following scenario:

- data migration migrates all X values to Y in version 26
- liquibase migration migrates all Y values to Z in version 28

But these are not run in version order, but all liquibase migrations
are run and then all data migrations are run. If you were on an old
version for a while, you could end up with Y values even though the
liquibase migration which comes "after" the data migration turns all Y
values into Z values.

This impacts this change because a data migration in this way:
- liquibase changesets 287, 288, 289, and 290 remove all 'type/UNIX*'
and 'type/ISO8601*' semantic types. These were in 0.39
- data migration `migrate-field-types` added in 0.20.0 was looking for
special_type's of "timestamp_milliseconds" and migrating them to
"type/UNIXTimestampMilliseconds".

Since the liquibase runs before the migrate-field-types, it's possible
for a 'type/UNIX*' semantic type to reappear. And since these were
removed from the type system it would fail validation and blow up. In
this case it actually can't happen since the field its attempting to
migrate (special_type) no longer exists. But since the migrations are
not interleaved this problem still exists.

* whatever prettier

* Appease the machines

* Restore the unix and iso semantic types to hierarchy

migration tests still use these
`(impl/test-migrations [283 296] [migrate!] ...)`

A few things in tension: the field requires valid semantic_types,
these _were_ valid semantic_types until v39.

* Remove old "coercion" semantic types

* Migrate the v20 version semantic types for unix epoch

* Time travelling migrations

these target v20 and v29 which at the time they existed had a column
special type not semantic type. But these run after all of the schema
migrations, not interleaved, so they will have a semantic_type not
special_type when they actually run even though they target v20 or v29
data. strange world

* add migration's new checksum

* Revert name case changes

* Test fix 

* Test fix 

* Backend for dashboard sub filters, including EE implementation

* Fix pulse/update-notification! to respect parameters

* Add some tests

* add FE EE parameter section to dashboard subscriptions

* fix improper filter counting for label

* hide parameters section when dashboard has none

* fix thrown errorin pulses list sidebar

* fix linting error

* add cypress tests

* rmv describe.only from test

* refactor FE code a little

* Merge pulse/dashboard parameters correctly for dashboard subs

* fix issues with getting list of active pulse parameters

* update formatting of multi value parameters

* fix cypress test

* tweak pulse details styling

* move ParametersSection to enterprise tree

Co-authored-by: Dalton Johnson <daltojohnso@users.noreply.github.com>

* Explicitly specify %throwable in Log4j logging pattern so the ex-data gets printed

This tweaks means Exceptions get rendered for logging with .toString instead .getMessage.

See https://github.com/clojure/tools.logging/tree/31073fe1fbb0533f3c775fa667de70809f24b127#log4j2
and https://logging.apache.org/log4j/2.x/manual/layouts.html#Patterns

* Remove ad-hoc query perms check on save logic from metabase.models.card.

We already do this in metabase.api.card, so in practice this is redundant code.

* Better permissions exceptions when missing ad-hoc perms for creating/updating a Card

* Fix #11719

* Return more info with most API exception responses

* Add tests for #11719 and #14931

* Fix namespace lint errors

* Fix error message grammar (thanks @jeff303)