* Adjust JWT and SAML fetch-and-update user to save new attributes

Before this change, JWT/SAML logins would attempt to update attributes, but never considered the first-name or
last-name attributes.

* Attempts to fix tests to prevent pulluting test users with "Unknown"

* No deleting users.

* Unit tests checking that first/last names are updated for SSO users

When an SSO user is first logged in, they might not have first_name and/or last_name keys. This is allowed, but the
names will be "Unknown" in the app-db. Subsequently, a User may log in again with SSO but have fisrt/last name
attributes, which should update the Metabase user data in the app-db.

These unit tests set up such a scenario to check that the :first_name and :last_name keys are indeed updated.

* Adjust Enterprise LDAP to also use SSO-UTILS

Trying to unify the LDAP implementation with JWT/SAML a bit here.

* Lint error

* Reverting LDAP ns changes to get the PR unstuck

This is to keep the ball rolling on SSO fixes. I'll add LDAP as an item in the Epic to address this separately.

remove param_values hydration keys

Since this information is used in the model cache logging screen
accessible by monitoring users, :monitoring rather than :setting
permission is the correct one.

* `defsetting` setter functions should end in `!`

* Fix typo

* Update .clj-kondo/hooks/metabase/models/setting.clj

* Fix clj-kondo for Toucan defmodel not emitting a docstring

* Remove `^:private` metadata on a couple of Settings since it makes Eastwood fussy

* Auto enable persistence for models

When persistence is turned on for a db, we want to enable persistence
caching for all models in the db.

We do this by finding any models without a PersistentInfo at the top of
the scheduled refresh task and creating one that will get picked up by
the refresh.

This necessitated introducing another "off" state on PersistedInfo that
will get set from the front end, manually disabling persistence on a
model.  This turns PersistedInfo into a marker so that when the refresh
task runs again, these models will not be turned back on.

The prune job will prune "off" or "deletable" PersistedInfo. Since we don't
have a second "off-ing" state; the prune job will "drop if exists" the
cache table each time. This may need to change.

* Cherry-pick persist-refresh changes from persist-refresh-fail-email

* Ready models when enabling persistence on db

* Handle automatic model persistence in Tools table

* Address review: insert-many instead of  doseq insert

Co-authored-by: Anton Kulyk <kuliks.anton@gmail.com>

* Update persist model api permissions for EE

On OSS, only Admins can enable model cache in `Settings/Cache`,
enable database model cache in `Settings/Databases` and cache individual models.

On EE/Pro, users with `Settings access` application permissions can enable model
cache in `Settings/Cache` and users with `Manage database` can enable model
cache for a database.

* Add tests for application permissions

Open redirects means doing some sso with a built-in redirect, and redirecting into an unhappy place (aka, a non-MB place) afterwards so that someone gets phished or other bad things happen. This is already prevented for OSS sso's but not EE - prevents this for EE sso's by forcing redirects to be in MB `site-url` set domain.

* Only apply persistence substitution when there's no sandboxing

* Added test for sandboxed persistence

* Recursively apply persisted-info/native queries

* Move test into permissions test that has ee code

* fix ldap requires uid to do group sync

* fix ee test

* clear download perms when setting block perms

* test

* fix function call

* Repro #22408: Block permissions obstructing download permissions set to "No" (#22435)

Co-authored-by: Nemanja Glumac <31325167+nemanjaglumac@users.noreply.github.com>

* ignore exceptions when updating perms for tests

* ignore exceptions in finally block as well

* restore correct graph

* migrate scoring to defenterprise

* migrate parameters to defenterprise

* migrate snippet perms to defenterprise

* migrate gtap deletion to defenterprise

* migrate params.field-values to defenterprise

* remove ee-strategy-impl code

* MVP fix

* make parser more robust

* adding tests

* scaffolding, docstring & arg parsing

* tweaks

* registry and basic oss->ee dispatch

* stubbed defenterprise-ee

* full defenterprise-ee impl

* fix build and tweak docstring

* remove debug code

* basic tests

* use defonce

* support for :any feature

* schema support

* fix schema test

* switch to using format

* support for schema on return val

* tweak macro

* move schema validation to submacros

* do ee fn resolution at runtime

* do conditionals at macro-expansion time

* adjust semantics & refactor

* fix test

* add memoization for ee resolution

* fix tests

* emit register-mapping! call to avoid eval

* call starts-with on symbol

* remove schema support

* use spec for arg parsing, generate fns, and remove :error fallback option

* clean ns

* change schema alias to schema

* add :arglists meta and small refactor

* refactor

* move EE logic to fn

* validate that the correct args are passed in EE/OSS namespaces, and make :feature required

* remove logic for handling nil :feature

* fix lint errors

* fix ee macros

* defenterprise-schema macro & tests

* clean ns

* propogate metadata

* don't memoize ee resolution in dev to make debugging easier

* dont do ee fn resolution during macroexpansion

* minor refactor and cleanup

* major refactor

* update docstrings

* more docstring tweaks

* try to fix lint error on ldap PR

* use dan's suggestion

* Make /api/user works with Group manager

* add tests

* fix linting

* hope clj-kondo is happpy now

* update docs

* nit

* GET /api/user/:id returns user_group_memberships and allow group manager

* return additional fields if caller is Group Manager

* fix ns

* use set instead of list in tests

* POST /api/user/:id takes user_group_memberships too

* address Noah's comments

* typo

* sort by superuser -> group managers -> normal user when filter by group-id

* FE: Group managers  (#21111)

* group managers ui

* update specs

* review

* review

* fix merge

Co-authored-by: Alexander Lesnenko <alxnddr@users.noreply.github.com>
Co-authored-by: Aleksandr Lesnenko <alxnddr@gmail.com>

Allow users with block perms to still use the `rescan_values` APIs if they have DB details or data model perms (#21808)

* Make namespace aliasing consistent everywhere; enforce with clj-kondo

See the table of aliases in .clj-kondo/config.edn

Notable patterns:
- `[metabase.api.foo :as api.foo]`
- `[metabase.models.foo :as foo]`
- `[metabase.query-processor.foo :as qp.foo]`
- `[metabase.server.middleware.foo :as mw.foo]`
- `[metabase.util.foo :as u.foo]`
- `[clj-http.client :as http]` and `[metabase.http-client :as client]`

Fixes #19930.

* fix DB perm enforcement for users without data perms

* update table perm checks

* fix output of helper fn

* revise approach

* fix table read path

* remove blank line

* refactor and cleanup

* syntax fix

* add tests

* fix test

* another test

* typo

* fix more tests

* make perm test helper more robust to fix final test failures

* clean ns

* grant read perms for a DB if a user has only data model perms, using a special :data-model perms object set

* Revert "grant read perms for a DB if a user has only data model perms, using a special :data-model perms object set"

This reverts commit f6cb724e513f6be5a26bc7252ffea30c192dfc8b.

* change exclude-uneditable-data-model? to include-editable-data-model?

* also adjust behavior & flag on /api/database/:id/metadata

* update FE data model flags

* one more datamodel fe query param

* also add include_editable_data_model flag to /api/database/:id

* a couple of more places for new query params

* add include_editable_data_model query parameter to data model page requests

* fix variable name

* docstring fixes

* fix test helper

* fix perm check in /api/database/:id

* try to fix cypress test

Co-authored-by: Aleksandr Lesnenko <alxnddr@gmail.com>

* change the General permissions to Application

* rename general permissions to application permissions

* BE: Rename General Perms to Application Perms (#21709)

* BE: Change General Perms to Application Perms

* lint migration file

* add migration to update seq name

* update application perms graph endpoint in fe

Co-authored-by: Aleksandr Lesnenko <alxnddr@gmail.com>
Co-authored-by: Ngoc Khuat <qn.khuat@gmail.com>

* add is_group_manager and hydrate it

* update test title

* fix namespaces and 1 test

* update docstring and make sure the is-group-manager? is converted to boolean in any db

* even more docstring

* fix ns

* Fix a test appeared when merge with master

* add tests and more api enforcement

* appease linter and make sure the test can run in ee

* update by membership id

* one dot

* update docs

* make is_group_manager optional

* hydrate is_group_manager when get single group

* - Split the `check-group-manager` into 2 functions
- Address Noah's comments

* Update enterprise/backend/test/metabase_enterprise/advanced_permissions/api/group_manager_test.clj

Co-authored-by: Noah Moss <32746338+noahmoss@users.noreply.github.com>

* change is_group_manager in to bool instead of boolstring

* remove debug code

Co-authored-by: Noah Moss <32746338+noahmoss@users.noreply.github.com>

* disallow adding recipients by non-admins with monitoring permission

* enforce sharing and embedding apis

* enforce update monitoring perms for alert, pulse and tasks API

* add minor test cases

* remove debug code

* linting

* fix  not able to remove users from pulse and make sure our tests cover that case

* address Noah's comments

* one space

* add is_group_manager and hydrate it

* update docstring and make sure the is-group-manager? is converted to boolean in any db

* disallow adding recipients by non-admins with monitoring permission

* enforce sharing and embedding apis

Co-authored-by: Aleksandr Lesnenko <alxnddr@gmail.com>

Add `exclude_uneditable_data_model` and `exclude_uneditable_details` flags to database list API (#21556)

* add exclude-uneditable-data-model and exclude-uneditable-details flags to GET /api/database

* remove debugging code

* settings global permission

* fix specs

* Enforce Setting permissions (cont) (#21464)

* settings global permission

* more api permissions enforcement

* only admin could call token checks

* address Noah's comments

* clean ns

* clean ns 



Co-authored-by: Aleksandr Lesnenko <alxnddr@gmail.com>

* hide subscriptions buttons for users with no permissions

* fix specs

* review fixes

* update spec

Co-authored-by: Ngoc Khuat <qn.khuat@gmail.com>

* first pass

* a few more endpoints

* add can_access_db_details to /api/user/common

* add can_access_data_model key to api/user/current

* add exclude_uneditable flag to /api/database/:id/metadata

* clean ns

* WIP figuring out how to update perm checks for Field model

* fix errors

* fix more errors

* tests for field APIs

* table perms changes

* tests for table API

* fix function call

* clean ns

* perm enforcement for other table APIs

* perm enforcement for other field APIs

* address comments

* fix failed to sync admin group

* address noah's comments and add migration script

* document for run-with-data-migration-index

* update comments

* fix name space

* adding data_migrations tests

* add docg

* make sure we don't remove admin group if sso and ldap are not configured

* fix tests for be-ee

* fix tests for oss

* misc docs updates

* remove data-migration-index

* return some newlines

* is it failling here?

* update data_migration docs

* update data_migration docs

* fix all styling comments

* make migration to run both in oss and enterprise and make sure the tests are accounted for that

* fix failed namespace checks

* Add a comment to the test

* fix per comments

* Update permissions.clj

* tweaking with the  icon

* refactor with-temporary-raw-setting-values

* update comments

* Add extension for cert file

* address Noah's comments

* Clear out existing DashboardCards when running load with --mode update

* Namespace cleanup

* Fix circular references

* Another test fix

* Settings cache needs to be per-app-DB

* (Experimental -- new application DB dynamic var)

* Create truly rebindable `metabase.db.connection/*application-db*`

* call-on-change-fn -> call-on-change

* Revert unneeded commit

* Namespace cleanup

* Add missing docstrings

* Appease linters again

* Fix handler stats logging middleware

* PR feedback

* PR feedback: slight optimization

* Remove NOCOMMIT

* PR feedback

* Clean namespace

* add API to fetch general permisisons graph

* add API to update general permissionns

* change author of migration

* update documents

* misc fixes to applease the CIs

* Add tests for general permission APIs and models

* linting and fix a failed test case

* fix some failed tests

* update docs and change /subscription/ to /general/subscription/ for consistency

* Hook and migration to make sure subscription are created for new groups by default

* add schema migrations tests

* set for the win

* address noah's comments

* Parse number as is in http-client for tests

* address Cam's comments

* revert the last commit about parsing API response in tests

* change fk name

* delete a comment

* Changes:
- Rename `changes` column to `after` to keep things consistent
- If a group  has no General Permisions, it'll not be included in the graph
- Update tests and some docs

* fix failing tests in ee

* add some tests and make docstring completes

* polishing comments

* namespaces

* fix namespaces

* Add general permisison flags to `GET /api/user/current` (#21250)

* return general permisison flags for /api/current

* namespaces

* move permission flags to under

* Enforce Subscription permissions (#21285)

* return general permisison flags for /api/current

* namespaces

* enforce general permissions for subscription and tests

* update check-has-general-permisison to add option to require superuser

* adding arity

* add tests for permissions helper function

* Move advanced permissions check funcs to ee namespace

* unpushed changes

* namespaces

* ignore exception when load namespaces

* change helper fn name

* Enforce Monitoring Permissions (#21321)

* return general permisison flags for /api/current

* namespaces

* enforce general permissions for subscription and tests

* update check-has-general-permisison to add option to require superuser

* adding arity

* enforce permissions to call /api/dataset for internal queries

* enforce monitoring permissions for api/task and api/util

* add tests for OSS

* add tests for db-connection-info endpoint

* change test schema

* update name func and fix ns

* whydon't CI run ?

* Enforce Setting Permissions (#21386)

* return general permisison flags for /api/current

* namespaces

* enforce general permissions for subscription and tests

* update check-has-general-permisison to add option to require superuser

* adding arity

* enforce permissions to call /api/dataset for internal queries

* enforce monitoring permissions for api/task and api/util

* add tests for OSS

* add tests for db-connection-info endpoint

* change test schema

* update name func and fix ns

* whydon't CI run ?

* Enforce Setting permissions

* fix failing test

* make sure we could run slack test twice

* make the mock consistent

* address Noah's comments

* shorter permissions check

* add API to fetch general permisisons graph

* add API to update general permissionns

* change author of migration

* update documents

* misc fixes to applease the CIs

* Add tests for general permission APIs and models

* linting and fix a failed test case

* fix some failed tests

* update docs and change /subscription/ to /general/subscription/ for consistency

* address noah's comments

* Parse number as is in http-client for tests

* revert the last commit about parsing API response in tests

* change fk name

* Changes:
- Rename `changes` column to `after` to keep things consistent
- If a group  has no General Permisions, it'll not be included in the graph
- Update tests and some docs

* fix failing tests in ee

* add some tests and make docstring completes

* fix namespaces

Better key parsing for json in test HTTP responses, and new test for data perms graph GET endpoint (#21216)