That is: collection, dimension, metric, native_query_snippet, pulse,
report_card, report_dashboard, report_dashcard, segment, timeline

Notably that doesn't include database, table, or field, since those all
have external unique IDs that are used instead.

remove param_values hydration keys

Limit the length of the row for JSON parsing. This was the simplest way to make another attack at #22636.

* Add can-manage to databases to help front-end

* Only return persisted true when the state agrees

* Add event for persisted-info to catch new models and turn on peristence when appropriate

* Fix bad threading

* Add comment about the case being detected to add the persisted-info

* Fix pre-existing bug in expand-db-details where a function was used as the condition

* Indicate field responsible for database error

* Implement new interface

* Make error showing logic a little easier to understand

* Show individual field error in database form

* Add backend tests for connection errors when adding a database

* Add new form error component

* Make database form error message pretty

* Make main database form error message field bold

* Change create account database form according to feedback

* Fix failed E2E tests from UI changed.

* Make it easier to tree-shake "lodash"

* Change according to PR review

* Cleanup + remove FormError (to be included in a separate PR)

Co-authored-by: Mahatthana Nomsawadi <mahatthana.n@gmail.com>

* Add anchor time to persistence schedule

Introduces a new public-setting persisted-model-refresh-anchor-time,
representing the time to begin the refresh job. Defaults to midnight
"00:00".

Also peg the cron jobs to run in the first of: reporting timezone, system timezone, or
UTC. This means that if a user needs the refresh to be anchored consistently to a certain time in a certain place, they need to update reporting timezone to match.

* Add tests for anchor time

* Force anchor-time to midnight if refresh hours is less than 6

* Need to check that hours was passed in

* Fix ns lint

* Add tests to ensure timezone is being set on trigger

we assert that the schedule gets randomized and it is no longer one of
the default schedules. We need to make sure that it cannot return one of
the default schedules in that case :). Low probability (1 in 58) but
with 28 test suites run for each commit it starts to add up and becomes
Flaky™

Settings require db access so cannot be called at read/require time. Not
until we have initialized the db are setting values possible to be read.

* `defsetting` setter functions should end in `!`

* Fix typo

* Update .clj-kondo/hooks/metabase/models/setting.clj

* Fix clj-kondo for Toucan defmodel not emitting a docstring

* Remove `^:private` metadata on a couple of Settings since it makes Eastwood fussy

* bumps outdated deps versions to be current

* un-upgrade h2 and jetty

* un-upgrade joda-time and kixi/stats

* drop Java 8 support in circle CI config

- things that used to rely on be-tests-java-8-ee now rely on be-tests-java-11-ee

* remove java 8 from github health check matrix

* revert toucan to 1.17.0

* revert mariadb java client to 2.7.5

* Back to 18, and handle new behavior

toucan used to just look in *.models.<model-name> for models and just
give up apparently. I made a feature that toucan will look in a model
registry to create models rather than using the convention
https://github.com/metabase/toucan/commit/762ad69defc1477423fa9423e9320ed318f7cfe7


but now we're getting errors in these tests about maps vs models.

```clojure
revision_test.clj:154
Check that revisions+details pulls in user info and adds description
expected: [#metabase.models.revision.RevisionInstance{:is_reversion false,
                                                      :is_creation false,
                                                      :message nil,
                                                      :user
                                                      {:id 1,
                                                       :common_name "Rasta Toucan",
                                                       :first_name "Rasta",
                                                       :last_name "Toucan"},
                                                      :diff
                                                      {:o1 nil, :o2 {:name "Tips Created by Day", :serialized true}},
                                                      :description nil}]
  actual: (#metabase.models.revision.RevisionInstance{:description nil,
                                                      :is_creation false,
                                                      :is_reversion false,
                                                      :user
                                                      {:id 1,
                                                       :first_name "Rasta",
                                                       :last_name "Toucan",
                                                       :common_name "Rasta Toucan"},
                                                      :message nil,
                                                      :diff
                                                      {:o1 nil,
                                                       :o2
                                                       #metabase.models.revision_test.FakedCardInstance{:name
                                                                                                        "Tips Created by Day",
                                                                                                        :serialized
                                                                                                        true}}})
```

The only difference here is `:o2` is a
`metabase.models.revision_test.FakedCardInstance` but still has the same
keys, `:name`, and `:serialized`. So all is well, we're just able to
make the model.

So a few different fixes. Some are use `partial=` which doesn't care
about record/map distinction. Some are just make the model, and some are
turning them into maps for revision strings (which more closely mimics
what the real revision stuff does):

```clojure
(defn default-diff-map
  "Default implementation of `diff-map` which simply uses clojures `data/diff` function and sets the keys `:before` and `:after`."
  [_ o1 o2]
  (when o1
    (let [[before after] (data/diff o1 o2)]
      {:before before
       :after  after})))

(defn default-diff-str
  "Default implementation of `diff-str` which simply uses clojures `data/diff` function and passes that on to `diff-string`."
  [entity o1 o2]
  (when-let [[before after] (data/diff o1 o2)]
    (diff-string (:name entity) before after)))
```

So all in all this change impacts nothing in the app itself, because
those models follow convention and are correct in
`metabase.models.<model-name>` and are thus "modelified":

```clojure
revision-test=> (revision/revisions Card 1)
[#metabase.models.revision.RevisionInstance{:is_creation true,
                                            :model_id 1,
                                            :id 1,
                                            :is_reversion false,
                                            :user_id 2,
                                            :timestamp #object[java.time.OffsetDateTime
                                                               "0x77e037f"
                                                               "2021-10-28T15:10:19.828539Z"],
                                            :object #metabase.models.card.CardInstance
                                            {:description nil,
                                             :archived false,
                                             :collection_position nil,
                                             :table_id 5,
                                             :database_id 2,
                                             :enable_embedding false,
                                             :collection_id nil,
                                             :query_type :query,
                                             :name "ECVYUHSWQJYMSOCIFHQC",
                                             :creator_id 2,
                                             :made_public_by_id nil,
                                             :embedding_params nil,
                                             :cache_ttl 1234,
                                             :dataset_query {:database 2,
                                                             :type :query,
                                                             :query {:source-table 5,
                                                                     :aggregation [[:count]]}},
                                             :id 1,
                                             :display :scalar,
                                             :visualization_settings {:global {:title nil}},
                                             :dataset false,
                                             :public_uuid nil},
                                            :message nil,
                                            :model "Card"}]
```

so the model/no-model is just arbitrary distinction in the test. All of
them in the actual app are turned into models:

```clojure
(defn- do-post-select-for-object
  "Call the appropriate `post-select` methods (including the type functions) on the `:object` this Revision recorded.
  This is important for things like Card revisions, where the `:dataset_query` property needs to be normalized when
  coming out of the DB."
  [{:keys [model], :as revision}]
  ;; in some cases (such as tests) we have 'fake' models that cannot be resolved normally; don't fail entirely in
  ;; those cases
  (let [model (u/ignore-exceptions (db/resolve-model (symbol model)))]
    (cond-> revision
    ;; this line would not find a model previously for FakedCard and
    ;; just return the map. But now the registry in toucan _finds_ the
    ;; model defintion and returns the model'd map
      model (update :object (partial models/do-post-select model)))))

(u/strict-extend (class Revision)
  models/IModel
  (merge models/IModelDefaults
         {:types       (constantly {:object :json})
          :pre-insert  pre-insert
          :pre-update  (fn [& _] (throw (Exception. (tru "You cannot update a Revision!"))))
          :post-select do-post-select-for-object}))
```

* try using mssql-jdbc 10.2.1.jre11

- Important that we get off the jre8 version

* various fixes that needn't be reverted

* Revert "various fixes that needn't be reverted"

This reverts commit 2a820db0743d0062eff63366ebe7bc78b852e81f.

* go back to using circle ci's java 11 docker image

* java-16 (?) -> java-17

* Revert "go back to using circle ci's java 11 docker image"

This reverts commit b9b14c535a689f701d7e2541081164288c988c4e.

Co-authored-by: dan sutton <dan@dpsutton.com>

* Add font utilities to derive list of available fonts from font dirs

This PR adds two defsettings to facilitate Enterprise users' ability to set the font for their instance. The
'available-fonts' can be grabbed by the frontend to populate the dropdown menu. The 'application-font' setting can be
written by the frontend to save the chosen font.

* Adding some fonts and font selector

* Send  (("dirname", "font name"),) to frontend

Instead of only sending a list of names, send a tuple of `[directory-name, Display Name]`

* Adding half of the fonts

* Font list derived from font directory/font file names

This is an attempt to derive the 'Display Name' of a font by comparing the directory name to the file names. '_' is
replaced with spaces, but '-' exists in the 'Lato' font directory name, and we may have naming confusion in the
future. Basically, I want to be liberal with what font directory/file names we can receive.

Not certain this is the right approach yet, but it will work and correctly provides names based on the fonts available.

* Adding remaining fonts

* Using Default Font family in visual components

* Small change to fix some lint warnings

* Removing Noto Sans JP

* Move logging

* Alter hardcoded font path

* Added cypress test

* Change Cypress Test to look for correct text

* Added unit tests for fonts.clj, and simplified creating display name

The display name approach is simple -> underscores become spaces, and names are split on dashes/first word is taken
from the split.

This might not be robust to future font installations, but will be a good v1

* adding visual test

* fixing cypress test

* Actually add the unit tests

* Fix lint problems

* Add setter to application-font to prevent invalid font setting

* Addressing PR review points

- fixed 'normalize-font-dirname' description
- 'contains-font-file?' returns true/false
- 'available-fonts-test' now properly uses 'is'

* Make font-path private

* Lint error.

* Modified fonts to use java.io/resource to hopefully work in a jar

* Simplified Lato folder name

* PR Cleanup

* Address some backend feedback

* Lint error from reflection with .toString on a path

I removed the explicit .toString inside the lamdba functino and str/includes still works.

* no longer need u.files in test ns

Co-authored-by: Nick Fitzpatrick <nick@metabase.com>

* Revert "Revert new precondition"

This reverts commit 1f97a4d6ccfdaa92ff82fd0256f08f3512c3c5e7.

* Revert "plug in and alter tests to use with-actions-test-data"

This reverts commit 2f33cb47b700b1da106adf54d41928c5956978d0.

* Revert "Add Database-local Setting for enabling Actions for a specific Database"

This reverts commit a52f8d0a478a31bb1f1b4a6693abf8f8e658d9cf.

* Revert "Make sure api.actions is loaded"

This reverts commit 7d7b913a7a0d64cbe2b6907f389d994a1130c0f5.

* Revert "Add `experimental-enable-actions` feature flag (#22559)"

This reverts commit 9b313752.

* Revert "SQL JDBC Delete Row Action (#22608)"

This reverts commit d6864f38.

* Remove actions api docs

* Revert "Destructible version of `test-data` dataset for testing writeback actions (#22692)"

This reverts commit 15c57d9a.

* keep around a few misc improvements

* removes precondition

- {:pre [(nil? (namespace symb))]}

* various fixes that needn't be reverted

This reverts commit f106ebfd.

* Auto enable persistence for models

When persistence is turned on for a db, we want to enable persistence
caching for all models in the db.

We do this by finding any models without a PersistentInfo at the top of
the scheduled refresh task and creating one that will get picked up by
the refresh.

This necessitated introducing another "off" state on PersistedInfo that
will get set from the front end, manually disabling persistence on a
model.  This turns PersistedInfo into a marker so that when the refresh
task runs again, these models will not be turned back on.

The prune job will prune "off" or "deletable" PersistedInfo. Since we don't
have a second "off-ing" state; the prune job will "drop if exists" the
cache table each time. This may need to change.

* Cherry-pick persist-refresh changes from persist-refresh-fail-email

* Ready models when enabling persistence on db

* Handle automatic model persistence in Tools table

* Address review: insert-many instead of  doseq insert

Co-authored-by: Anton Kulyk <kuliks.anton@gmail.com>

* Add font utilities to derive list of available fonts from font dirs

This PR adds two defsettings to facilitate Enterprise users' ability to set the font for their instance. The
'available-fonts' can be grabbed by the frontend to populate the dropdown menu. The 'application-font' setting can be
written by the frontend to save the chosen font.

* Adding some fonts and font selector

* Send  (("dirname", "font name"),) to frontend

Instead of only sending a list of names, send a tuple of `[directory-name, Display Name]`

* Adding half of the fonts

* Font list derived from font directory/font file names

This is an attempt to derive the 'Display Name' of a font by comparing the directory name to the file names. '_' is
replaced with spaces, but '-' exists in the 'Lato' font directory name, and we may have naming confusion in the
future. Basically, I want to be liberal with what font directory/file names we can receive.

Not certain this is the right approach yet, but it will work and correctly provides names based on the fonts available.

* Adding remaining fonts

* Using Default Font family in visual components

* Removing Noto Sans JP

* Small change to fix some lint warnings

* Move logging

* Alter hardcoded font path

* Added cypress test

* Change Cypress Test to look for correct text

* adding visual test

* fixing cypress test

* Added unit tests for fonts.clj, and simplified creating display name

The display name approach is simple -> underscores become spaces, and names are split on dashes/first word is taken
from the split.

This might not be robust to future font installations, but will be a good v1

* Actually add the unit tests

* Fix lint problems

* Add setter to application-font to prevent invalid font setting

* Addressing PR review points

- fixed 'normalize-font-dirname' description
- 'contains-font-file?' returns true/false
- 'available-fonts-test' now properly uses 'is'

* Make font-path private

* Lint error.

* PR Cleanup

Co-authored-by: Nick Fitzpatrick <nick@metabase.com>

UUIDs are treated as strings and when checking is-empty/not-empty, we compare
them against the empty string.
But an empty string is not a valid UUID, so this comparison is useless for
Postgres where a UUID field cannot have such a value. Assuming that a UUID
field is only ever compared to the empty string in the context of
is-empty/non-empty checks, replacing it with nil is OK.

* Record startup times

updated `dev/start!`. `mb.core/init!` itself sets up the db, loads
plugins, and sets the init status so those were superfluous.

* clean up dev namespace

* testing doesn't call init so set a value

* SQL JDBC Delete Action [ci skip]

* Add note about parsing values [ci skip]

* transition to using mbql for actions/row/:action

- TODO: how to setup a qp store?
- TODO: better way to keywordize filters?

* some cleanup [ci skip]

* better message on delet statement conflicts

- and a test

* mt/with-everything-store -> seed store manually [ci skip]

* fix some test input

* get more tests passing

* add more validation tests

* privatize catch-throw

* plug in and alter tests to use with-actions-test-data

Co-authored-by: Bryan Maass <bryan.maass@gmail.com>

* Don't reschedule syncs on startup

Previously when we started up, for each database we compare its current
schedule to its schedule in Quartz, and update the triggers if they were
different. But we handle this change on db inserts and db updates so
this is completely unnecessary. We did this back when quartz was in
memory only and it remained when we persisted.

We can remove this complexity. On an app-db with 400 postgres databases,
this takes 46 seconds in the old world and appears as just another log
statement now.

Reports from the wild are even more distressing: the quartz tables got
full of dead tuples causing queries to the triggers to take a long
time. They also had 500+ databases. We query all triggers and linear
search for each database and this process exploded up to as much as 8
hours. Now we are not doing all of this extra work and startup should be
quick again.

* clj-kondo ignore unused private var

* appease ns linter

* remove clojure.core/binding and unused bindings

* Rename `default-schedules` -> `default-randomized-schedule`

makes it a bit more clear what's going on

* Put randomizer in transducer over reducible query; tests

basically a `run!` over a reducible query that also does a count for
logging. Tests now check the three options:
- default schedules, updated
- non-default schedules, not updated timestamp verified that it is
identical.
- default schedules but user controls schedule: not updated, timestamp
verified that it is identical.

* Prevent randomized schedules from returning a default value

it actually doesn't matter in the application but the tests are
asserting that the newly randomized schedule is not one of the defaults
it was changed from and that can fail sometimes. Doesn't matter
logically but no biggie to simplify the tests.

Otherwise they fail in (1/60)+(1/24) test runs which is actually
decently frequently with 26 test suites per commit

* Destructible version of `test-data` dataset for testing the actions stuff

* Revert new precondition

Pursuant to https://github.com/metabase/metabase/issues/22174.

Make the JSON columns work on MySQL in exactly the same way as they do in Postgres. Bit of a refactor about some bits of Postgres implementation.

Importantly, MariaDB JSON columns work significantly more like SQL Server JSON columns. That is, they aren't JSON columns at all with respect to type, they're text columns - and we don't detect them very well as JSON columns qua JSON columns. So MariaDB is not supported at this time.

See https://github.com/metabase/metabase/issues/22561 for context.

Previously identifier isn't fully specified (with schema and table) in nested field column description in syncing. Now it is!

Co-authored-by: Noah Moss <32746338+noahmoss@users.noreply.github.com>

Co-authored-by: Noah Moss <32746338+noahmoss@users.noreply.github.com>

* Fix automagic dashboards for queries with source queries

* Replace ->entity with a multimethod

* Remove unused namespace

* Fix Cypress test

* Add `experimental-enable-actions` feature flag

* Simplify Ring middleware

* Sort namespaces

* Make sure api.actions is loaded

* fix ldap requires uid to do group sync

* fix ee test

Retry sending notifications (pulses/alerts) for about one minute

The parameters of the exponential backoff can be configured via settings.

* Fix :Postgres :day-of-week extracts coming back as 0..6 instead of 1..7

* Test Tuesday-Saturday per @dpsutton suggestion and fix issues with those as well 

* Fix MongoDB for Tuesday

Audit usages of `memoize` and `memoize/ttl` and use app-DB-specific memoization where appropriate (#21605)

* More app-DB-specific memoization

* Appease Eastwood

* overrides for certain :unit values

* tests

* fix test & formatting tweaks

* BE: user with view collection perms can create Alerts

* FE: don't update qs when create Alert