* adjustments to CSP header in dev environment to make react dev tools work on Firefox

* add test for script-src directive

* fix position of doc string

* fix error in common_test

There was no DB migration for previous friendly field name defriendlification. This adds DB migration.

Signify this is for internal use for the Metabase team, and remove ETA from the template (since that's tracked outside of GitHub).

#16350 Repro: "Invite someone" button overlaps activate/deactivate radio buttons on small screen widths (#16355)

Peeps want to make questions faster, but we don't give the data about query execution in a very user-friendly way. Here's it in a user-friendly way.

Co-authored-by: Alexander Lesnenko <alxnddr@users.noreply.github.com>

* Add the last-edit-info to the child query

just for cards right now, need to extend to dashboards and then make
sure we clean it up into the proper map.

Have a set of all columns that are required and ensure those are in
each query. Nulls need to be cast to the correct type if they aren't a
text column (at least on postgres, need to investigate h2 and mysql).

* Correct find last-edit-info for cards and dashboards

* Move from group-by max id to self join

* Paginate and search on collection items

- include the metadata information in the query
  - to accomplish this, make a backstop to add all columns into each
    query
  - need types to union the null. in postgres it assumes all nulls are
    text. Haven't checked what mysql and h2 do here yet
- sort_column: name, last_edited, model
- sort_direction: asc, desc

* Add default sort column and direction

* handle mysql

mysql doesn't need types on nulls to unify them

* Correct batch post processing for cards and dashboards

* Correct order of expectations

they were alphabetical inside of models
previous order:
- acme products           | pulse
- electro-magnetic pulse  | pulse
- birthday card           | card
- dine & Dashboard        | dashboard

and is now ordered solely by name

* Tests and sort on lower name after timestamp

When sorting by last edited, lots of things don't have this, so don't
want arbitrary sorting, but sort by name afterwards

* Ensure nulls go to the end of the list

* Tests

* trying to make h2, mysql, and pg happy

sorting timestamps and always having nulls last is quite difficult

* Make diff bigger

this was going in as a string and losing the timezone information, so
was actually advancing time.

Now: (java.time.OffsetDateTime/now)
"2021-06-01T13:59:33.165483Z"

Minus 2 hours: (.minusHours (java.time.OffsetDateTime/now) 2)
"2021-06-01T11:59:27.528500Z"

As seen in the code:
;; unaltered "now"
"2021-06-01T08:55:28"
;; altered "two hours ago"
"2021-06-01T11:55:29" <- UTC shenanigans. so just bump back 24 hours

* Sort by model ranking

* Remove clj-time and use ZonedDateTime instead of OffsetDateTime

...since the back-end is already capable of handling those.

* Handle PK/FK of the type STRING or STRING_LIKE

* Warn users about sandboxing and public sharing

* Update docs/enterprise-guide/data-sandboxes.md

Co-authored-by: Jeff Bruemmer <jeff@metabase.com>

Co-authored-by: Jeff Bruemmer <jeff@metabase.com>

* Add type on collection

* Search with collection type

* Cleanup bounded heap accumulator

* Make search conditionally bump official collection items

* collection api and tests

* Put collection type onto hydrated cards on dashboards

* Move official collection type scoring into ee codebase

* ensure ee and oss agree when not official collection

* Mark Collections tree with type

* Remove unnecessary `and`s when no check for enhancements

* Tests for setting collection tree type

* Include hydration for collection type on dashboards

* Make sure created test collections are cleaned up

* Cleanup tests, don't search on collection_type

looks for all text fields and searches the term against that. official
would bring back all official types. no bueno

* Docstring on protocol and don't shadow comparator

* update to new ee impl var passing style

* Collection model ensures no type change on personal collection

* Check for personal collection when updating type

model checks for personal collection and rejects in the update but
doesn't check for children of personal collection.

* Update checking for unchangeable properties of a personal collection

* Cleanup: type collection tree, batched hydration, combine error checks

* Cleanup

* move bounded-heap accumulator to utils

* switch to test.check testing

* Bad test just to see what our CI will report

* remove purposeful failing test (was for CI)

* collection.type -> collection.authority_level

* Test the actual ee scoring impl, not the wrapped one

locally i had enhancements enabled so the tests were passing, but in
CI it was failing as it did not have enhancements enabled

* add useDebounceValue hook

* add AddCardSidebar components

* extract DashboardSidebars component

* use AddCardSidebar component

* remove the old modal for adding questions

* fix specs, unify dashboard toolbar buttons

* fix enterprise specs

* lint

* fix more specs

* maz empty state tweaks

* review

* specs

* add role attributes

* `Add question` => `Add questions`

* add missing onkeydown handler to the QuestionPickerItem

Co-authored-by: Maz Ameli <maz@metabase.com>

Serialization refactoring and fixes

*********************************************************
* Support loading linked cards in different collections *
*********************************************************

Architectural changes:
Lay foundation for more generic retry mechanism
Establish a keyword to represent a failed name->id lookup that can happen during load, and map that to information about which part of the card resolution failed (for logging purposes)
Similar to dashboards, add a separate fn for performing a load of cards
Invoking the load-cards fn from the load multimethod implementation when some cards failed name resolution

Test enhancements:
Add new multimethod to perform more specific assertions on the loaded entities
For cards - check that the native SQL of the loaded card matches that of the original (pre-dump) instance
For dashboards - check that the linked card for each dashboard card series matches

Bug fixes:
Apply the mbql-id->fully-qualified-name fn on serialize to filter, breakout, and aggregation clauses
Updating "My Test" card to include filter clause so that fix is exercised
Instead of upserting users with a "changeme" password, simply do not attempt to change the password
Adding :engine to the identity condition of Database (so unrelated test-data instances don't get trashed on load)

Cleanup:
Remove unused specs namespace

****************************************
* Support dashboard card click actions *
****************************************

Architectural changes:
Add visualization settings shared model namespace with utility functions

Adding tests for viz settings

Changes to serialization:
Update visualization_settings in dump to convert column_settings keys from field ID to fully qualified name, and update the click action target with the fully qualified name
Also converting targetId in top level click action to fully qualified name

Doing the reverse in load, using the same utility code

Changes to serialization tests:
Updating the existing dashcard in the "world" roundtrip test to include the click action

Adding assertion for the click action in the assert-loaded-entity implementation for Dashboard

******************************************************************
* Stop nesting cards and check for unresolved query dependencies *
******************************************************************

In names.clj, stop qualifying a card's name by its dataset query's source card ID (this is what caused the "nested" directory structure)

In load.clj, start checking that either the source query [:dataset_query :query] is resolved properly as a card ID or table ID, and if neither, then propagate an ::unresolved-names entry up into the card (causing it to be reloaded on second pass)

Fix logging of unresolved name info

Test changes:
Add two new cards to the "world", one that lives in the root collection but depends on a card within a collection for its query, and one opposite (i.e. lives in a collection but depends on a root card for its query)

Adding new check that collection names match after loading

************************
* Logging improvements *
************************

Include causes when logging errors to aid with diagnosis

Add BEGIN and END messages for both dump and load commands

Add dynamic var to suppress printing of name lookup error on first pass load

*************************************************
* Handle collection namespaces and retry pulses *
*************************************************

Architectural changes:
Considering a collection namespace to be part of its fully qualified name (putting in a subdirectory called :namespace when writing to disk)
Accounting for that change when dumping and loading, and upsert (i.e. namespace is part of unique columns)
Bumping serialization format version to 2 because of this

Changes to load:
Add load-pulses fn that works similarly to others, to facilitate reloading
Add similar logic as other entities to reload a pulse if the :card_id is not found

Model changes:
Add hack fn to compare namespace values for the "namespace does not change" assertion with appropriate comment

Test changes:
Added test for pulse with reload

***********************************************
* Handle dimension entry within template-tags *
***********************************************

Add :field to the recognized clauses under mbql-entity-reference? (since that is how field refs are now represented in MBQL)

Add temp-field helper fn in test_util.clj to reduce some of the "world" verbosity

Adding new test card to world that uses template-tags with a field reference under dimension

************************
* Handle textbox cards *
************************

Add support for passing textbox (virtual) cards through visualization_settings.cljc

Updating load test to add a textbox type card to the dashboard and ensuring it is persisted

***************************************
* Handle dimension parameter mappings *
***************************************

In visualization_settings.cljc:
Adding handling for parameterMapping
Adding new roundtrip test for parameter mappings
Some refactoring to split up larger fns

In serialize.clj, convert all parts of the map that contain the dimension key to use fully qualified names for the respective fields

In load.clj, do the opposite (convert back to new IDs)

Updating test to add parameter mapping to an existing click action, and adding new assertions for that to ensure it is preserved

Fixing serialize test now that the "ids" can themselves be maps (i.e within dimension vector)

In visualization_settings_test.cljc
Adding new roundtrip test for problem dashboard

***********************************
* Handle field literals correctly *
***********************************

On load, only attempt to translate field name to ID if it's actually a fully qualified field name

Adding new fn to names.clj to provide that logic

Adding new test card that uses such a field reference

**************************************
* Accept only-db-ids option for dump *
**************************************

Accept options map for the dump command.  Currently, the only supported option is `:only-db-ids`, which is expected to be a set of DB IDs to dump.  If set, then only these databases (and their respective tables, fields, and segments) will be dumped to the dump-dir.  If not set, then the previous behavior takes effect (all DBs).

Update the load_test.clj to set this to be only the new temporary DB to avoid tramping on existing `sample-dataset` and other instances

*****************************************************
* Don't include personal collections of other users *
*****************************************************

Use the `select-collections` fn to filter only collections that are public, or owned bythe given user (by way of email), plus all collections with those as an ancestor

Updating test to assert a non-owned personal collection is not persisted (including nested), but all nested personal collections are

Adding new wrapped macro to also bind collection/*allow-deleting-personal-collections* true around the with-temp* call so that personal collections can legitimately be deleted (since they're temporary); the name of this macro was chosen to preserve the same amount of spacing within `with-world`

*********************************
* Support native query snippets *
*********************************

Adding support for NativeQuerySnippet:

- in names
- in dump (putting into subdirectory of collection, as cards)
- in upsert (identity condition of :name :collection_id)
- in load (no retry fn necessary since they should not have any dependencies themselves)

Adding a snippet and a card referencing it to roundtrip test (including a deeply nested snippet)

Fixing up handling of snippet namespaced collections and adding relevant name tests

*********************************
* Handle source-table for joins *
*********************************

Adding `mbql-fully-qualified-names->ids*` for recursive calls, which does not normalize its args, keeping mbql-fully-qualified-names->ids` as a wrapper to that

Adding clause to `mbql-fully-qualified-names->ids*` to resolve `:source-table` when it's a fully qualified table name

Adding util fn to names.clj to support that scenario

Updating "My Card" test card to include a join

Skipping two more drivers from `dump-load-entities-test` - SQLite and Spark because they don't support FKs

***************************
* Cleanup and refactoring *
***************************

Responding to PR feedback

Updating upsert_test.clj to check that the identity-condition keys actually exist for the models

Removing legacy MBQL syntax in a couple places

Adding ordering by `:id` to serialization load command for predictability

Lots of fixes and refactorings in visualization_settings.cljc:
 - added lots of specs and fdefs
 - added test fixture that instruments all the speced fns and required it from both visualization_settings.cljc and load_test.clj
 - renamed a bunch of fns for more consistency
 - added and fixed some docstrings
 - fix implementation of `parse-json-string` for cljs

*************************************************
* Preserve temporal-unit in source-field clause *
*************************************************

Update serialize and match macros to keep the rest of the match, when modifying the source field ID

Adding a bunch of new tables in order to perform a "bucketing on implicit join" type query

Updating test_util.clj to initialize the query processor with all the new temp tables so that queries actually work

**************************
* Misc cleanup and fixes *
**************************

Fixing the implementation of `fully-qualified-name->context` to work properly when the entity name itself is also a model name

***************************
* Handle joining to cards *
***************************

Recognize :source-table pointing to a fully qualified card name in `mbql-fully-qualified-names->ids*`, and adding the unresolved name capture to that lookup

Adding new util fns to capture the unresolved named entries from anywhere within the MBQL query tree: `paths-to-key-in` and `gather-all-unresolved-names`

Updating `resolve-card-dataset-query` to call this new `gather-all-unresolved-names` to pull all unresolved names from the MBQL query tree into card itself

Renaming `fully-qualified-name->id-rec` to `resolve-source-query` for clarity, and removing now unneeded clause

To the test case, adding a card that joins to another card

#16246 Repro: Incorrect values on data points for waterfall charts when one of the values is 0 (#16265)

* Failing test

* Fix expansion of :metric macros inside nested queries

* Unskip Cypress test

* Test fix 



* Rewrite (fix) repro for #12507

Co-authored-by: Nemanja <31325167+nemanjaglumac@users.noreply.github.com>

* Add repros for #16249

* Use "month" breakout for all questions

* Add assertions for the legend and y-axis

* Basic `react-gird-layout` usage for Dashboards (#16167)

* Install react-grid-layout

* Add basic NewGridLayout implementation

* Wrap DashboardGrid with WidthProvider

* Bind DashboardGrid's onDrag methods

* Coerce layout item IDs to string

* Add method rendering NewDashboardGrid

* Refactor `onLayoutChange` method to handle new grid

* Use react-grid-layout's CSS classes

* Fix lag while dragging grid items

* Switch back to ExplicitSize HOC

* Fix grid in editing state is too short

* Import missing `react-grid-layout` CSS

* Fix RGL overwrites dragging placeholder color

* Hide default RGL's resize handle

* Fix random dashboard layout rearrangements

Before dashboard cards' parameters (x, y, width, height)
were updated individually for each card
It worked fine because there was only one card moving at a time

Now when cards are automatically moving when a user drags another card,
a race conditon appears, cards' parameters get overwritten, and layout breaks

Fixed by using bulk updates for dashboard cards drag and resizing events

* Fix dashboard background grid

* Align RGL with parent component edges

* Fix grid offset

* Fix dashboard grid height

* Fix no automatic page scroll while dragging

* Fix navbar is stuck when scrolling dashboard

* Remove previous grid component

* Fix grid doesn't respect item min size

* Extract grid background into a separate utils file

* Fix dashboards layout issues (#16236)

* Fix RGL cards are higher than the previous ones

* Return the fixed header in editing mode back

* Add support for responsive dashboard layouts (#16213)

* Fix `gridItemWidth` calculation

* Memoize grid children

* Add utility to adapt layout for breakpoints

* Use responsive react-grid-layout for dashboards

* Don't overwrite main layout with smaller ones

* Pass correct `isMobile` prop to DashCard component

* Extract `renderGridItem` method

* Increase dashboard cards vertical margin for mobile

* Remove dashboard's old renderMobile method

* Add isMobileBreakpoint utility function

* Disable dashboard drag and resize for mobile

* Remove old method disabling mobile drag and resize

* Fix grid background rendering

* Switch to desktop and mobile breakpoints

* Update card heights for dashboard's mobile view

* Add a comment about `!important` in CSS

* Minor code style fix

* Remove not used functions

* Fix #15998: Tooltip only shows first Y-axis value when X-axis is numeric and style is Ordinal/Histogram

* Fix #15998: Tooltip only shows first Y-axis value when X-axis is numeric and style is Ordinal/Histogram

* prettier

Co-authored-by: Roman Abdulmanov <roman.abdulmanov@gmail.com>

When attempting to convert a filter, check first if the filter is a
comparison against a constant. Otherwise, preserve the filter is in its
custom expression (MBQL) form.

Each in the following list can be converted into a standard filter
(since the simple UI for the standard filter only allows literal
comparison):

  [Total] < 300
  between([Discount], 5, 15)
  [Name] = "Joe"

but none of these should be converted

  [Discount] <= [Tax]
  between([Rating], 0, [MaxRating])
  [Reviewer] != UPPER([Name])

* Allow standard filter to be invalid

This is necessary to allow an incomplete filter, i.e. the user hasn't
filled Y in "X < Y" form yet (hence Y is undefined). The filter is not
valid but it is still considered in the standard form.

* Check whether string comparison can be considered a standard filter

* Idiomatic with array destructuring

When attempting to convert a filter, check first if the filter is a
comparison against a constant. Otherwise, preserve the filter is in its
custom expression (MBQL) form.

Each in the following list can be converted into a standard filter
(since the simple UI for the standard filter only allows literal
comparison):

  [Total] < 300
  between([Discount], 5, 15)
  [Name] = "Joe"

but none of these should be converted

  [Discount] <= [Tax]
  between([Rating], 0, [MaxRating])
  [Reviewer] != UPPER([Name])

* Allow standard filter to be invalid

This is necessary to allow an incomplete filter, i.e. the user hasn't
filled Y in "X < Y" form yet (hence Y is undefined). The filter is not
valid but it is still considered in the standard form.

* Check whether string comparison can be considered a standard filter

* Idiomatic with array destructuring