Skip to content
Snippets Groups Projects
This project is mirrored from https://github.com/metabase/metabase. Pull mirroring updated .
  1. Oct 26, 2022
  2. Oct 18, 2022
  3. Oct 17, 2022
  4. Oct 14, 2022
    • Dalton's avatar
      Move more parameters utils to metabase-lib (#25900) · 0b118bd2
      Dalton authored
      * Move subset of parameter-values utils to metabase-lib
      
      * Move parameter mbql utils to metabase-lib
      
      * Create some small target utils
      
      * Move template tag utils to metabase-lib
      
      * Move target/field logic to metabase-lib
      
      * Remove eslint-enable
      Unverified
      0b118bd2
  5. Oct 13, 2022
  6. Oct 12, 2022
  7. Oct 10, 2022
  8. Oct 07, 2022
  9. Oct 05, 2022
  10. Oct 04, 2022
  11. Oct 03, 2022
  12. Sep 29, 2022
  13. Sep 22, 2022
    • Dalton's avatar
      Remove Question's `update` method & related code (#25531) · 2b11cfdf
      Dalton authored
      * Remove _update from Question
      
      * Update TimeseriesGroupingWidget
      
      * Update ExtendedOptions
      
      * Update SummarizeSidebar
      
      * Update AddAggregationButton and AggreationItem
      
      * Remove update method from MBQLClause
      
      * Update ChartTypeSidebar
      
      * Update GuiQueryEditor
      
      * Update NativeQueryEditor
      
      * Update BulkFilterModal
      
      * Update Notebook
      
      * Update NotebookSteps
      
      * Update NotebookStep
      
      * Update AggregateStep
      
      * Update BreakoutStep
      
      * Update DataStep
      
      * Update ExpressionStep
      
      * Update FilterStep
      
      * Update JoinStep
      
      * Update LimitStep
      
      * Update SortStep
      
      * Update NativeQueryButton
      
      * Update QuestionDataSelector
      
      * Update QuestionFilters
      
      * Update QuestionRowCount
      
      * Update View
      
      * Update random doc
      
      * Ensure we have a 'datasetQuery' in the setDatasetQuery function
      
      * Ensure we have a 'datasetQuery' in the setDatasetQuery action
      Unverified
      2b11cfdf
  14. Sep 19, 2022
  15. Sep 16, 2022
    • Cal Herries's avatar
      Use slugs for card template tags in the native query editor + autocomplete (#25319) · 2c7178ea
      Cal Herries authored
      * Allow slugs in question template tags, like `{{#123-a-question}}`
      
      * Update CardTagEditor to add slug names to the query text, instead of IDs
      
      * Fix dispatchApiErrorEvent prop for entity object loader
      
      * Ignore errors loading names of referenced questions
      
      * Fix e2e tests
      
      * Implement autocomplete for model slugs
      
      * Prioritize models above questions
      
      * Make the ace_autocomplete popup slightly wider to account for longer question names
      
      * Reorder requires
      
      * Remove unnecessary ? from endpoint string
      
      * Add e2e test for card tag autocomplete
      
      * Fix NativeQueryEditor not swapping back to standard completers
      
      * Refactor replaceCardSlug
      
      * Add e2e test for updating tags when card name is changed
      
      * Refactor updateReferencedQuestionNames
      
      * Add slug method to Question class
      
      * Use map for autocomplete API results
      
      * Rename endpoint path
      
      * Rename for consistency
      
      * Add docstrings
      
      * Update slug() to guard against undefined names
      
      * Fix read permissions for the endpoint
      
      * Add backend test
      
      * Fix unit tests
      
      * Fix permissions error
      
      * Make regex simpler
      
      * Undo change of value on QuestionPicker
      
      * Fix tests
      
      * Fix models e2e test
      
      * Simplify updateReferencedQuestionNames
      
      * Fix bugs in updateReferencedQuestionNames
      
      * Rename variable
      
      * Move NativeQuery methods to new native-query utils module
      
      * Fix typescript errors
      
      * Fix e2e tests
      
      * Fix e2e test
      
      * Fix BE tests
      
      * Actually fix BE test
      
      * Refactor updateQuestionTagNames
      
      * Align comments
      
      * Reorder autocomplete results
      
      * Refactor: move all query text tag utils in one place
      
      * Add tests for regex
      
      * Add unit tests for cardIdFromTagName
      
      * Add schema to autocomplete_suggestions endpoints
      
      * Add perms checks to autocomplete tests
      
      * Add unit test for updateCardTagNames
      
      * Rename updateQuestionTagNames -> updateCardTagNames
      
      * Tidy card-autocomplete-suggestions-test
      
      * Rename questionReference-> cardTag for consistency
      
      * Rename card-reference -> card-tag for consistency
      
      * Use ternary operator for comprehension
      
      * Fix broken swapCompleters
      
      * Use prefix match on name for 123-foo case
      Unverified
      2c7178ea
  16. Sep 12, 2022
    • Ryan Laurie's avatar
      Enforce import order in ESLint/import (#25123) · d9b0b652
      Ryan Laurie authored
      * enforce import order
      
      * reorder all imports
      
      * more import updates
      Unverified
      d9b0b652
    • Dalton's avatar
      Normalize virtual card table fields using its `table_id` and `id` (#25267) · 223708bd
      Dalton authored
      * Update FieldSchema to store virtual card table fields using both table_id and field id
      
      * Update the metadata selector to look use field's uniqueId property
      
      * Add tableId arg to Metadata.prototype.field method
      
      * Normalize fields in sample_database_fixture
      
      * Fix incorrect e2e test
      
      * Make the second arg of field optional
      
      * Pull out small fn
      
      * Refactoring + clean up + mocks & tests
      
      * Update metadata.field(...) usage in a few places
      
      * Revert FieldDimension change
      
      * Move complex FieldDimension `field` logic to the instantiation of "virtual" fields by the associated query's `table` method (#25109)
      
      * Add merge method to Field
      
      * add new virtual table utils + tests
      
      * Use new utils in implementation of Structured/NativeQuery table methods + remove unneeded Dimension logic
      
      * let all Questions return dependentMetadata
      
      * Fix types related to query.table() potentially returning null
      
      * Fix updateQuestion tests
      
      * Remove unused Dimension imports
      
      * Fix issue resolving rootTable on queries based on virtual tables
      
      * Fallback to the nested card table stored in the Metadata instance
      
      * Make the nested card Table the main driver in returning fields for nested cards + other misc fixes
      
      * more tweaks related to cloned Tables, source queries
      
      * Don't clobber metadata if something is already there (like a method named 'metadata')
      
      * Fix for nested native query
      
      * Don't use real table for source query
      
      * Update table code to use the actual nested card table fields instead of doing merge logic
      
      * Skip an e2e test
      
      * Workaround for getting Dimensions from joins
      
      * Update arg name
      
      * Try removing the Table fetch from Question's dependentMetadata method
      
      * Don't rerun query when converting question to model
      
      * Fix for source query not showing 'previous results'
      
      * Fix unit tests
      
      * Update models e2e tests not to wait on query
      
      * Unskip repro for #23449
      
      * make sure a join's joinedQuery is not a dataset/model
      
      * use createQuestion in e2e test
      
      * don't use Question redux dispatch method for updating
      
      * split test cases
      
      * Add comment
      Unverified
      223708bd
  17. Aug 30, 2022
  18. Aug 12, 2022
    • Dalton's avatar
      Retrieve field metadata from cached card object when a dimension has a virtual card table (#24645) · 0646a3e5
      Dalton authored
      * Add cards to the metadata godhead
      
      * Use source query result_metadata when calling fieldDimension.field()
      
      * Various unit test fixes
      
      * First pass at attempting to refactor fields method
      
      * Round 2: more aggressive refactor...
      
      * Third pass, improved comments + minor tweaks
      
      * Add unit tests for more scenarios
      
      * Remove metadata from cards in normalization fn
      
      * Fix broken tests
      
      * Separate field logic for int id fields and non-int id fields
      
      * Remove usage of old metadata.question method
      
      * Limit some logic to datsets/models
      
      * Don't default objects arg of copyObjects
      
      * Add instantiateQuestion unit test
      
      * revert unit test that no longer needs fixing
      
      * Dimension tweaks
      
      * fix unit test fixture
      Unverified
      0646a3e5
  19. Aug 02, 2022
  20. Jul 08, 2022
  21. Jul 06, 2022
  22. Jul 04, 2022
  23. Jun 23, 2022
  24. Jun 21, 2022
  25. Jun 20, 2022
  26. Jun 16, 2022
  27. May 31, 2022
    • Benoit Vinay's avatar
      Alert to typescript (#22832) · e8ba7d53
      Benoit Vinay authored
      * Types added to Alert
      
      * Alert package created
      
      * utils added to Alert
      
      * Clean up Question
      
      * alertType back in
      
      * Query import added back in
      
      * Types imports updated
      
      * Fix Dashboard types
      Unverified
      e8ba7d53
  28. May 19, 2022
    • Anton Kulyk's avatar
      Fix model level persistence UI states (#22795) · e5da4423
      Anton Kulyk authored
      * Add `PLUGIN_MODEL_PERSISTENCE` plugin
      
      * Remove persist/unpersist actions
      
      * Move model cache control to enterprise folder
      
      * Move model cache management to enterprise folder
      
      * Add `setPersisted` method to `Question`
      
      * Refresh state when toggling model level persistence
      Unverified
      e5da4423
  29. May 17, 2022
  30. May 12, 2022
  31. May 11, 2022
  32. May 02, 2022
    • dpsutton's avatar
      Persisted models schema (#21109) · c504a12e
      dpsutton authored
      * dir locals for api/let-404
      
      * Driver supports persisted model
      
      * PersistedInfo model
      
      far easier to develop this model with the following sql:
      
      ```sql
      create table persisted_info (
         id serial primary key not null
        ,db_id int references metabase_database(id) not null
        ,card_id int references report_card(id) not null
        ,question_slug text not null
        ,query_hash text not null
        ,table_name text not null
        ,active bool not null
        ,state text not null
        ,UNIQUE (db_id, card_id)
      )
      
      ```
      and i'll make the migration later. Way easier to just dorp table, \i
      persist.sql and keep developing without worrying about the migration
      having changed so it can't rollback, SHAs, etc
      
      * Persisting api (not making/deleting tables yet)
      
      http POST "localhost:3000/api/card/4075/persist" Cookie:$COOKIE -pb
      http DELETE "localhost:3000/api/card/4075/persist" Cookie:$COOKIE -pb
      
      useful from commandline (this is httpie)
      
      * Pull format-name into ddl.i
      
      * Postgres ddl
      
      * Hook up endpoints
      
      * move schema-name into interface
      
      * better jdbc connection management
      
      * Hotswap peristed tables into qp
      
      * clj-kondo fixes
      
      * docstrings
      
      * bad alias in test infra
      
      * goodbye testing format-name function
      
      left over. everything uses ddl.i/format-name and this rump was left
      
      * keep columns in persisted info
      
      columns that are in the persisted query. I thought about a tuple of
      [col-name type] instead of just the col-name. I didn't do this this type
      because I want to ensure that we compute the db-type in ONLY ONE WAY
      ever and i wasn't ready to commit to that yet. I'm not sure this is
      necessary in the future so it remains out now.
      
      Context: we hot-swap the persisted table in for the original
      query. Match up on query hash remaining the same. It continues to use
      the metadata from the original query and just `select cols from table`
      
      * Add migration for persisted_info table
      
      also removes the db_id. Don't know why i was thinking that was
      necessary. also means we don't need another unique constraint on (db_id,
      card_id) since we can just mark the card_id as unique. no idea what i
      was thinking.
      
      * fix ns in a sad manner :(
      
      far better to just have no alias to indicate it is required for side
      effects.
      
      * Dont hardcode a card-id :(:(:( my B
      
      * copy the PersistedInfo
      
      * ns cleanup, wrong alias, reflection warning
      
      * Check that state of persisted_info is persisted
      
      * api to enable persistence on a db
      
      i'm not wild about POST /api/database/:id/persist and POST
      /api/database/:id/unpersist but carrying on. left a note about it.
      
      So now you can enable persistence on a db, enable persistence on a model
      by posting to api/card/:id/persist and everything works.
      
      What does not work yet is the unpersisting or re-persisting of models
      when using the db toggle.
      
      * Add refresh_begin and refresh_end to persisted_info
      
      This information helps us with two bits:
      - when we need to chunk refreshing models, this lets us order by
      staleness so we can refresh a few models and pick up later
      - if we desire, we can look at the previous elapsed time of refreshes
      and try to gauge amount of work we want. This gives us a bit of
      look-ahead. We can of course track our progress as we go but there's no
      way to know if the next refresh might take an hour. This gives us a bit
      of insight.
      
      * Refresh tables every 8 hours ("0 0 0/8 * * ? *")
      
      Tables are refreshed every 8 hours. There is one single job doing this
      named "metabase.task.PersistenceRefresh.job" but it has 0 triggers by
      default. Each database with persisted models will add a trigger to this
      to refresh those models every 8 hours.
      
      When you unpersist a model, it will immediately remove the table and
      then delete the persisted_info record.
      
      When you mark a database as persist false, it will immediately mark all
      persisted_info rows as inactive and deleteable, and unschedule its
      trigger. A background thread will then start removing the tables.
      
      * Schedule refreshing on startup, watching for already scheduled
      
      does not allow for schedule changes but that's a future endeavor
      
      * appease our linter overlords
      
      * Dynamic var to inbhit persistence when refreshing
      
      also, it checked the state against "active" instead of "persisted" which
      is really freaky. how has this worked in the past if thats the case?
      
      * api docstrings on card persist
      
      * docstring
      
      * Don't sync the persisted schemas
      
      * Fix bad sql when no deleteable rows
      
      getting error with bad sql when there were no ids
      
      * TaskHistory for refreshing
      
      * Add created_at to persist_info table
      
      helpful if this ever ends up in the audit section
      
      * works on redshift
      
      hooked up the hierarchy and redshift is close enought that it just works
      
      * Remove persist_info record after deleting "deleteable"
      
      * Better way to check that something exists
      
      * POST /api/<card-id>/refresh
      
      api to refresh a model's persisted record
      
      * return a 204 from refreshing
      
      * Add buttons to persist/unpersist a database and a model for PoC (#21344)
      
      * Redshift and postgres report true for persist-models
      
      there are separate notions of persistence is possible vs persistence is
      enabled. Seems like we're just gonna check details for enabled and rely
      on the driver multimethod for whether it is possible.
      
      * feature for enabled, hydrate card with persisted
      
      two features: :persist-models for which dbs support it, and
      :persist-models-enabled for when that option is enabled.
      
      POST to api/<card-id>/unpersist
      
      hydrate persisted on cards so FE can display persist/unpersist for
      models
      
      * adjust migration number
      
      * remove deferred-tru :shrug:
      
      
      
      * conditionally hydrate persisted on models only
      
      * Look in right spot for persist-models-enabled
      
      * Move persist enabled into options not details
      
      changing details recomposes the pool, which is especially bad now that
      we have refresh tasks going on reusing the same connection
      
      * outdated comment
      
      * Clean up source queries from persisted models
      
      their metadata might have had [:field 19 nil] field_refs and we should
      substitute just [:field "the-name" {:base-type :type/Whatever-type}
      since it will be a select from a native query.
      
      Otherwise you get the following:
      
      ```
      2022-03-31 15:52:11,579 INFO api.dataset :: Source query for this query is Card 4,088
      2022-03-31 15:52:11,595 WARN middleware.fix-bad-references :: Bad :field clause [:field 4070 nil] for field "category.catid" at [:fields]: clause should have a :join-alias. Unable to infer an appropriate join. Query may not work as expected.
      2022-03-31 15:52:11,596 WARN middleware.fix-bad-references :: Bad :field clause [:field 4068 nil] for field "category.catgroup" at [:fields]: clause should have a :join-alias. Unable to infer an appropriate join. Query may not work as expected.
      2022-03-31 15:52:11,596 WARN middleware.fix-bad-references :: Bad :field clause [:field 4071 nil] for field "category.catname" at [:fields]: clause should have a :join-alias. Unable to infer an appropriate join. Query may not work as expected.
      2022-03-31 15:52:11,596 WARN middleware.fix-bad-references :: Bad :field clause [:field 4069 nil] for field "category.catdesc" at [:fields]: clause should have a :join-alias. Unable to infer an appropriate join. Query may not work as expected.
      2022-03-31 15:52:11,611 WARN middleware.fix-bad-references :: Bad :field clause [:field 4070 nil] for field "category.catid" at [:fields]: clause should have a :join-alias. Unable to infer an appropriate join. Query may not work as expected.
      2022-03-31 15:52:11,611 WARN middleware.fix-bad-references :: Bad :field clause [:field 4068 nil] for field "category.catgroup" at [:fields]: clause should have a :join-alias. Unable to infer an appropriate join. Query may not work as expected.
      2022-03-31 15:52:11,611 WARN middleware.fix-bad-references :: Bad :field clause [:field 4071 nil] for field "category.catname" at [:fields]: clause should have a :join-alias. Unable to infer an appropriate join. Query may not work as expected.
      2022-03-31 15:52:11,611 WARN middleware.fix-bad-references :: Bad :field clause [:field 4069 nil] for field "category.catdesc" at [:fields]: clause should have a :join-alias. Unable to infer an appropriate join. Query may not work as expected.
      2022-03-31 15:52:11,622 WARN middleware.fix-bad-references :: Bad :field clause [:field 4070 nil] for field "category.catid" at [:fields]: clause should have a :join-alias. Unable to infer an appropriate join. Query may not work as expected.
      2022-03-31 15:52:11,622 WARN middleware.fix-bad-references :: Bad :field clause [:field 4068 nil] for field "category.catgroup" at [:fields]: clause should have a :join-alias. Unable to infer an appropriate join. Query may not work as expected.
      2022-03-31 15:52:11,622 WARN middleware.fix-bad-references :: Bad :field clause [:field 4071 nil] for field "category.catname" at [:fields]: clause should have a :join-alias. Unable to infer an appropriate join. Query may not work as expected.
      2022-03-31 15:52:11,623 WARN middleware.fix-bad-references :: Bad :field clause [:field 4069 nil] for field "category.catdesc" at [:fields]: clause should have a :join-alias. Unable to infer an appropriate join. Query may not work as expected.
      ```
      I think its complaining that that table is not joined in the query and
      giving up.
      
      While doing this i see we are hitting the database a lot:
      
      ```
      2022-03-31 22:52:18,838 INFO api.dataset :: Source query for this query is Card 4,111
      2022-03-31 22:52:18,887 INFO middleware.fetch-source-query :: Substituting cached query for card 4,111 from metabase_cache_1e483_229.model_4111_redshift_c
      2022-03-31 22:52:18,918 INFO middleware.fetch-source-query :: Substituting cached query for card 4,111 from metabase_cache_1e483_229.model_4111_redshift_c
      2022-03-31 22:52:18,930 INFO middleware.fetch-source-query :: Substituting cached query for card 4,111 from metabase_cache_1e483_229.model_4111_redshift_c
      ```
      
      I tried to track down why we are doing this so much but couldn't get
      there.
      
      I think I need to ensure that we are using the query store annoyingly :(
      
      * Handle native queries
      
      didn't nest the vector in the `or` clause correctly. that was truthy
      only when the mbql-query local was truthy. Can't put the vector `[false
      mbql-query]` there and rely on that behavior
      
      * handle datetimetz in persisting
      
      * Errors saved into persisted_info
      
      * Reorder migrations to put v43.00-047 before 048
      
      * correct arity mismatch in tests
      
      * comment in refresh task
      
      * GET localhost:3000/api/persist
      
      Returns persisting information:
      - most information from the `persist_info` table. Excludes a few
      columns (query_hash, question_slug, created_at)
      - adds database name and card name
      - adds next fire time from quartz scheduling
      
      ```shell
      ❯ http GET "localhost:3000/api/persist" Cookie:$COOKIE -pb
      [
          {
              "active": false,
              "card_name": "hooking reviews to events",
              "columns": [
                  "issue__number",
                  "actor__login",
                  "user__login",
                  "submitted_at",
                  "state"
              ],
              "database_id": 19,
              "database_name": "pg-testing",
              "error": "No method in multimethod 'field-base-type->sql-type' for dispatch value: [:postgres :type/DateTimeWithLocalTZ]",
              "id": 4,
              "next-fire-time": "2022-04-06T08:00:00.000Z",
              "refresh_begin": "2022-04-05T20:16:54.654283Z",
              "refresh_end": "2022-04-05T20:16:54.687377Z",
              "schema_name": "metabase_cache_1e483_19",
              "state": "error",
              "table_name": "model_4077_hooking_re"
          },
          {
              "active": true,
              "card_name": "redshift Categories",
              "columns": [
                  "catid",
                  "catgroup",
                  "catname",
                  "catdesc"
              ],
              "database_id": 229,
              "database_name": "redshift",
              "error": null,
              "id": 3,
              "next-fire-time": "2022-04-06T08:00:00.000Z",
              "refresh_begin": "2022-04-06T00:00:01.242505Z",
              "refresh_end": "2022-04-06T00:00:01.825512Z",
              "schema_name": "metabase_cache_1e483_229",
              "state": "persisted",
              "table_name": "model_4088_redshift_c"
          }
      ]
      
      ```
      
      * include card_id in /api/persist
      
      * drop table if exists
      
      * Handle rescheduling refresh intervals
      
      There is a single global value for the refresh interval. The API
      requires it to be 1<=value<=23. There is no validation if someone
      changes the value in the db or with an env variable. Setting this to a
      nonsensical value could cause enormous load on the db so they shouldn't
      do that.
      
      On startup, unschedule all tasks and then reschedule them to make sure
      that they have the latest value.
      
      One thing to note: there is a single global value but i'm making a task
      for each database. Seems like an obvious future enhancement so I don't
      want to deal with migrations. Figure this gives us the current spec
      behavior to have a trigger for each db with the same value and lets us
      get more interesting using the `:options` on the database in the
      future.
      
      * Mark as admin not internal
      
      lets it show up in `api/setting/` . I'm torn on how special this value
      is. Is it the setting code's requirement to invoke the reschedule
      refresh triggers or should that be on the setting itself.
      
      It feels "special" and can do a lot of work from such just setting an
      integer. There's a special endpoint to set it which is aware, and thus
      would be a bit of an error to set this setting through the more
      traditional setting endpoint
      
      * Allow for "once a day" refresh interval
      
      * Global setting to enable/disable
      
      post api/persist/enable
      post api/persist/disable
      
      enable allows for other scheduling operations (enabling on a db, and
      then on a model).
      
      Disable will
      - update each enabled database and disable in options
      - update each persisted_info record and set it inactive and state
      deleteable
      - unschedule triggers to refresh
      - schedule task to unpersist each model (deleting table and associated
      pesisted_info row)
      
      * offset and limits on persisted info list
      
      ```shell
      http get "localhost:3000/api/persist?limit=1&offset=1" Cookie:$COOKIE -pb
      {
          "data": [
              {
                  "active": true,
                  "card_id": 4114,
                  "card_name": "Categories from redshift",
                  "columns": [
                      "catid",
                      "catgroup",
                      "catname",
                      "catdesc"
                  ],
                  "database_id": 229,
                  "database_name": "redshift",
                  "error": null,
                  "id": 12,
                  "next-fire-time": "2022-04-08T00:00:00.000Z",
                  "refresh_begin": "2022-04-07T22:12:49.209997Z",
                  "refresh_end": "2022-04-07T22:12:49.720232Z",
                  "schema_name": "metabase_cache_1e483_229",
                  "state": "persisted",
                  "table_name": "model_4114_categories"
              }
          ],
          "limit": 1,
          "offset": 1,
          "total": 2
      }
      ```
      
      * Include collection id, name, and authority level
      
      * Include creator on persisted-info records
      
      * Add settings to manage model persistence globally (#21546)
      
      * Common machinery for running steps
      
      * Add model cache refreshes monitoring page (#21551)
      
      * don't do shenanigans
      
      * Refresh persisted and error persisted_info rows
      
      * Remarks on migration column
      
      * Lint nits (sorted-ns and docstrings)
      
      * Clean up unused function, docstring
      
      * Use `onChanged` prop to call extra endpoints (#21593)
      
      * Tests for persist-refresh
      
      * Reorder requires
      
      * Use quartz for individual refreshing for safety
      
      switch to using one-off jobs to refresh individual tables. Required
      adding some job context so we know which type to run.
      
      Also, cleaned up the interface between ddl.interface and the
      implementations. The common behaviors of advancing persisted-info state,
      setting active, duration, etc are in a public `persist!` function which
      then calls to the multimethod `persist!*` function for just the
      individual action on the cached table.
      
      Still more work to be done:
      - do we want creating and deleting to be put into this type of system?
      Quite possible
      - we still don't know if a query is running against the cached table
      that can prevent dropping the table. Perhaps using some delay to give
      time for any running query to finish. I don't think we can easily solve
      this in general because another instance in the cluster could be
      querying against it and we don't have any quick pub/sub type of
      information sharing. DB writes would be quite heavy.
      - clean up the ddl.i/unpersist method in the same way we did refresh and
      persist. Not quite clear what to do about errors, return values, etc.
      
      * Update tests with more job-info in context
      
      * Fix URL type conflicts
      
      * Whoops get rid of our Thread/sleep test :)
      
      * Some tests for the new job-data, clean up task history saving
      
      * Fix database model persistence button states (#21636)
      
      * Use plain database instance on form
      
      * Fix DB model persistence toggle button state
      
      * Add common `getSetting` selector
      
      * Don't show caching button when turned off globally
      
      * Fix text issue
      
      * Move button from "Danger zone"
      
      * Fix unit test
      
      * Skip default setting update request for model persistence settings (#21669)
      
      * Add a way to skip default setting update request
      
      * Skip default setting update for persistence
      
      * Add changes for front end persistence
      
      - Order by refresh_begin descending
      - Add endpoint /persist/:persisted-info-id for fetching a single entry.
      
      * Move PersistInfo creation into interface function
      
      * Hide model cache monitoring page when caching is turned off (#21729)
      
      * Add persistence setting keys to `SettingName` type
      
      * Conditionally hide "Tools" from admin navigation
      
      * Conditionally hide caching Tools tab
      
      * Add route guard for Tools
      
      * Handle missing settings during init
      
      * Add route for fetching persistence by card-id
      
      * Wrangling persisted-info states
      
      Make quartz jobs handle any changes to database.
      Routes mark persisted-info state and potentially trigger jobs.
      Job read persisted-info state.
      
      Jobs
      
      - Prune
      -- deletes PersistedInfo `deleteable`
      -- deletes cache table
      
      - Refresh
      -- ignores `deletable`
      -- update PersistedInfo `refreshing`
      -- drop/create/populate cache table
      
      Routes
      
      card/x/persist
      - creates the PersistedInfo `creating`
      - trigger individual refresh
      
      card/x/unpersist
      - marks the PersistedInfo `deletable`
      
      database/x/unpersist
      - marks the PersistedInfos `deletable`
      - stops refresh job
      
      database/x/persist
      - starts refresh job
      
      /persist/enable
      - starts prune job
      
      /persist/disable
      - stops prune job
      - stops refresh jobs
      - trigger prune once
      
      * Save the definition on persist info
      
      This removes the columns and query_hash columns in favor of definition.
      
      This means, that if the persisted understanding of the model is
      different than the actual model during fetch source query we won't
      substitute.
      
      This makes sure we keep columns and datatypes in line.
      
      * Remove columns from api call
      
      * Add a cache section to model details sidebar (#21771)
      
      * Extract `ModelCacheRefreshJob` type
      
      * Add model cache section to sidebar
      
      * Use `ModelCacheRefreshStatus` type name
      
      * Add endpoint to fetch persistence info by model ID
      
      * Use new endpoint at QB
      
      * Use `CardId` from `metabase-types/api`
      
      * Remove console.log
      
      * Fix `getPersistedModelInfoByModelId` selector
      
      * Use `t` instead of `jt`
      
      * Provide seam for prune testing
      
      - Fix spelling of deletable
      
      * Include query hash on persisted_info
      
      we thought we could get away with just checking the definition but that
      is schema shaped. So if you changed a where clause we should invalidate
      but the definition would be the same (same table name, columns with
      types).
      
      * Put random hash in PersistedInfo test defaults
      
      * Fixing linters
      
      * Use new endpoint for model cache refresh modal (#21742)
      
      * Use new endpoint for cache status modal
      
      * Update refresh timestamps on refresh
      
      * Move migration to 44
      
      * Dispatch on initialized driver
      
      * Side effects get bangs!
      
      * batch hydrate :persisted on cards
      
      * bang on `models.persisted-info/make-ready!`
      
      * Clean up a doc string
      
      * Random fixes: docstrings, make private, etc
      
      * Bangs on side effects
      
      * Rename global setting to `persisted-models-enabled`
      
      felt awkward (enabled-persisted-models) and renamed to make it a bit
      more natural. If you are developing you need to set the new value to
      true and then your state will stay the same
      
      * Rename parameter for site-uuid-str for clarity
      
      * Lint cleanups
      
      interesting that the compojure one is needed for clj-kondo. But i guess
      it makes sense since there is a raw `GET` in `defendpoint`.
      
      * Docstring help
      
      * Unify type :type/DateTimeWithTZ and :type/DateTimeWithLocalTZ
      
      both are "TIMESTAMP WITH TIME ZONE". I had got an error and saw that the
      type was timestamptz so i used that. They are synonyms although it might
      require an extension.
      
      * Make our old ns linter happy
      
      Co-authored-by: default avatarAlexander Polyankin <alexander.polyankin@metabase.com>
      Co-authored-by: default avatarAnton Kulyk <kuliks.anton@gmail.com>
      Co-authored-by: default avatarCase Nelson <case@metabase.com>
      Unverified
      c504a12e
Loading