Skip to content
Snippets Groups Projects
This project is mirrored from https://github.com/metabase/metabase. Pull mirroring updated .
  1. Jul 13, 2022
    • Cal Herries's avatar
      Session activity timeout (#23349) · 0defe7f8
      Cal Herries authored
      
      * logout when session expires, login when session appears
      
      * add setting UI
      
      * Add last_activity column to session table
      
      * Start implementing session middleware to check for expired sessions
      
      * Change last_activity field to include timezone offset
      
      * Update session middleware to check user activity timeout
      
      * Update last_activity after checking the timeout, or not at all if the setting is nil
      
      * Move session-timeout settings to server.middleware.session
      
      * Handcode timeout for testing
      
      * Fix migrations validation error
      
      * Fix whitespace
      
      * Change session timeout to use metabase.TIMEOUT cookie with expiry
      
      * Remove migration for last_activity column on session table
      
      * Revert changes to logout endpoint
      
      * Revert change to Session model pre-update
      
      * Remove tap>
      
      * Fix tests to include cookie value
      
      * Fix timeout when user is logged out. Timeout loop should only start when a user is logged in
      
      * Update comment and date format
      
      * Store the session-timeout setting as json and convert it to seconds on the fly
      
      * Set zoned date time to use GMT instead of default time zone
      
      * Refactor for testing
      
      * refactor session listener (#23686)
      
      * remove old session listener
      
      * Clear the timeout cookie when user signs out
      
      * Clear session cookie if the timeout cookie expires
      
      * fe tweaks
      
      * Update expires attribute for session and timeout cookies together
      
      * Reapply minimum limit on session-timeout
      
      * Rename functions and fix lint warnings
      
      * Fix resetting session-timeout
      
      * Fix sign out
      
      * Fix tests
      
      * Whitespace
      
      * Get full-app-embeds working
      
      * Add test for embedded session
      
      * session timeout ui tweaks
      
      * fix security issue
      
      * Fix test
      
      * Fix tests
      
      * Do not redirect to "/" if there isn't any redirect URL
      
      * Add test for session-cookies setting
      
      * Fix bug when toggling off timeout and adjust tests
      
      Co-authored-by: default avatarAleksandr Lesnenko <alxnddr@gmail.com>
      Co-authored-by: default avatarAleksandr Lesnenko <alxnddr@users.noreply.github.com>
      Unverified
      0defe7f8
    • Gustavo Saiani's avatar
      Remove unused data-test-id attr (#23910) · ed1497d0
      Gustavo Saiani authored
      Unverified
      ed1497d0
    • Cam Saul's avatar
      Remove `u/optional` (#23877) · b603c439
      Cam Saul authored
      * Remove u/optional
      
      * Spec => `s/` and Schema => `schema/`
      
      * Fix docstring generation.
      
      * Another test fix :wrench:
      
      * Remove unused namespaces
      
      * Parallelize some of the defendpoint tests
      Unverified
      b603c439
    • Alexander Polyankin's avatar
    • Nick Fitzpatrick's avatar
    • Nick Fitzpatrick's avatar
    • Braden Shepherdson's avatar
      Serdes v2 for Metrics (#23868) · 4e32716f
      Braden Shepherdson authored
      Unverified
      4e32716f
    • Gustavo Saiani's avatar
    • Alexander Polyankin's avatar
    • dpsutton's avatar
      Handle archived and models reverted to regular questions (#23738) · 7caca2ec
      dpsutton authored
      * Handle archived and models reverted to regular questions
      
      Should not refresh these and should unpersist them
      
      Updated the signature of `refresh!` in the `Refresher` protocol to take
      a card instead of a dataset query so that the tests could use the card's
      id for checks. The `dispatching-refresher` then calls the
      `ddl.i/refresh!` multimethod with the dataset_query from the card so its
      a very surface level refactoring.
      
      Updated the stats from refreshing to be a bit smarter. It used to have a
      `when` block for its check against the model's state. this meant on a
      state change that prevented refreshing it would throw away accumulated
      stats. Whoops. Now it records that as a skip. The check itself has grown
      to check archived and dataset status of the underlying card.
      
      * Block api/card/card-id/refresh from refreshing archived/unmodeled
      
      From page http://localhost:3000/admin/tools/model-caching we have a list
      of PersistedInfo records and a way to refresh them. We've already
      prevented the scheduled tasks from refreshing archived/unmodeled
      questions and now the api won't let a user manually do it either.
      
      * ns linter demands allegiance
      
      * Include card archived and dataset information on api/persist
      
      Include information so that the frontend can disallow refreshing models
      based on cards that are archived or no longer archived. The response
      from `http://localhost:3000/api/persist?limit=20&offset=0`
      
       now includes
      two new attributes on the items in the data array: `"card_archived"` and
      `"card_dataset"`. I'm happy to rename if desired.
      
      These are present so that if `card_archived=true` or
      `card_dataset=false` we should not allow refreshing the persisted
      model. The API will already reject this request with a 400 so we should
      let the user know why this is no longer valid.
      
      I'm happy to leave them as the regular property names `archived` and
      `dataset` but put them with the `card_` prefix so it is clear that these
      are not properties of a PersistedInfo but the card that the persisted
      info references
      
      ```
      {
        "data": [
          {
            "definition": {
              "table-name": "model_3_model_from",
              "field-definitions": [
                { "field-name": "id", "base-type": "type/Integer" },
                { "field-name": "total", "base-type": "type/Float" },
                { "field-name": "quantity", "base-type": "type/Integer" }
              ]
            },
            "creator": null,
            "schema_name": "metabase_cache_944b9_2",
            "database_id": 2,
            "collection_id": null,
            "database_name": "sample dataset pg",
            "state": "persisted",
            "refresh_begin": "2022-07-07T14:22:51.512016Z",
            "refresh_end": "2022-07-07T14:22:51.52032Z",
            "collection_name": null,
            "collection_authority_level": null,
            "creator_id": null,
            "card_archived": true, <---- new
            "active": true,
            "id": 3,
            "card_dataset": true,  <---- new
            "card_id": 3,
            "error": null,
            "next-fire-time": null,
            "table_name": "model_3_model_from",
            "card_name": "model from orders archived"
          },
          {
            "definition": {
              "table-name": "model_2_model_from",
              "field-definitions": [
                { "field-name": "id", "base-type": "type/Integer" },
                { "field-name": "total", "base-type": "type/Float" },
                { "field-name": "quantity", "base-type": "type/Integer" }
              ]
            },
            "creator": null,
            "schema_name": "metabase_cache_944b9_2",
            "database_id": 2,
            "collection_id": null,
            "database_name": "sample dataset pg",
            "state": "persisted",
            "refresh_begin": "2022-07-07T14:22:33.509687Z",
            "refresh_end": "2022-07-07T14:22:33.517815Z",
            "collection_name": null,
            "collection_authority_level": null,
            "creator_id": null,
            "card_archived": false,  <---- new
            "active": true,
            "id": 2,
            "card_dataset": false,   <---- new
            "card_id": 2,
            "error": null,
            "next-fire-time": null,
            "table_name": "model_2_model_from",
            "card_name": "model from orders no longer model"
          },
          {
            "definition": {
              "table-name": "model_1_model_from",
              "field-definitions": [
                { "field-name": "id", "base-type": "type/Integer" },
                { "field-name": "total", "base-type": "type/Float" },
                { "field-name": "quantity", "base-type": "type/Integer" }
              ]
            },
            "creator": null,
            "schema_name": "metabase_cache_944b9_2",
            "database_id": 2,
            "collection_id": null,
            "database_name": "sample dataset pg",
            "state": "persisted",
            "refresh_begin": "2022-07-07T14:22:33.476072Z",
            "refresh_end": "2022-07-07T14:22:33.504025Z",
            "collection_name": null,
            "collection_authority_level": null,
            "creator_id": null,
            "card_archived": false, <---- new
            "active": true,
            "id": 1,
            "card_dataset": true,   <---- new
            "card_id": 1,
            "error": null,
            "next-fire-time": null,
            "table_name": "model_1_model_from",
            "card_name": "model from orders"
          }
        ],
        "total": 3,
        "limit": 20,
        "offset": 0
      }
      ```
      
      * Add conditions to checkCanRefreshModelCache (#23778)
      
      Co-authored-by: default avatarGustavo Saiani <gustavo@poe.ma>
      Unverified
      7caca2ec
    • Jeff Bruemmer's avatar
    • Braden Shepherdson's avatar
    • Nick Fitzpatrick's avatar
    • Nemanja Glumac's avatar
    • Nemanja Glumac's avatar
    • Alexander Polyankin's avatar
    • Howon Lee's avatar
      Deal with SAML responses having whitespace (#23451) (#23633) · a9ca102c
      Howon Lee authored
      Pursuant to #23451.
      
      The end effect of whitespace existing in a SAML response is us choking on it as reported in #23451. Two possible interpretations of causes of this bug:
      
      There was an upstream change in our fork of the clojure SAML lib as flamber noted,
      The decoding of base64 in our SAML endpoint (which uses the SAML lib) chokes on whitespace.
      The proximate cause is the second one and ultimate cause is the first. However, I tend to believe that fixing the second one would be the better fix. For comparison, onelogin's first party SAML thing for java decodes base64 (https://github.com/onelogin/java-saml/blob/master/core/src/main/java/com/onelogin/saml2/util/Util.java) via apache's lib, which seems to do the thing that a lot of base64 decoders do of skipping whitespace.
      Unverified
      a9ca102c
    • Ngoc Khuat's avatar
  2. Jul 12, 2022
    • Cal Herries's avatar
      Fix for #21984: Viewing tables in Data Model is registered in "Recently viewed" (#23780) · 2c1404f5
      Cal Herries authored
      
      * Add explicit ignore_true parameter to Tables.load requests in admin panel to avoid counting the requests as a table view
      
      * Remove .skip for repro
      
      * Fix formatting
      
      * Adapt repro to the fix
      
      Repro was written before I know what the fix would look like.
      
      We're conditionally choosing whether to display `HomePopularSection` or `HomeRecentSection`.
      With this fix applied, data model table visit is not registered any more so there is nothing else really
      that the test user visited. Hence, "Pick where you left off" is never displayed on the page and we
      display `HomePopularSection` instead.
      
      This adaption reflects those changes.
      
      * Revert "Add explicit ignore_true parameter to Tables.load requests in admin panel to avoid counting the requests as a table view"
      
      This reverts commit f887d0d2a6076c7b5073bf4356f3aa2361372055.
      
      * Remove :table-read event from GET api/table/:id endpoint
      
      * Remove unused ns
      
      Co-authored-by: default avatarNemanja <31325167+nemanjaglumac@users.noreply.github.com>
      Unverified
      2c1404f5
    • dpsutton's avatar
      Log bad token only once every five minutes (#23831) · 7b2f4be5
      dpsutton authored
      Bad tokens will constantly log that they cannot be validated. They will
      do this for each feature it attempts to check. So when you call
      `(count (setting/admin-writable-settings))` (something that happens on
      each page load) you get the following output: (stacktrace and response
      condensed for brevity)
      
      ```
      2022-07-11 15:05:44,172 INFO public-settings.premium-features :: Checking with the MetaStore to see whether ffff...ffff is valid...
      2022-07-11 15:05:45,085 ERROR public-settings.premium-features :: Error fetching token status:
      clojure.lang.ExceptionInfo: clj-http: status 404 {:body "{\"valid\":false,\"status\":\"Token does not exist.\"}"}
          ...
      	at metabase.public_settings.premium_features$eval118094$fetch_token_status_STAR___118099$fn__118100$fn__118101.invoke(premium_features.clj:72)
          ...
      2022-07-11 15:05:45,096 ERROR public-settings.premium-features :: Error validating token : Token does not exist.
      2022-07-11 15:05:45,097 ERROR public-settings.premium-features :: Error validating token : Token does not exist.
      2022-07-11 15:05:45,098 ERROR public-settings.premium-features :: Error validating token : Token does not exist.
      2022-07-11 15:05:45,099 ERROR public-settings.premium-features :: Error validating token : Token does not exist.
      2022-07-11 15:05:45,100 ERROR public-settings.premium-features :: Error validating token : Token does not exist.
      2022-07-11 15:05:45,100 ERROR public-settings.premium-features :: Error validating token : Token does not exist.
      2022-07-11 15:05:45,102 INFO util.fonts :: Reading available fonts from /Users/dan/projects/work/metabase/resources/frontend_client/app/fonts
      2022-07-11 15:05:45,107 INFO i18n.impl :: Reading available locales from locales.clj...
      2022-07-11 15:05:45,123 ERROR public-settings.premium-features :: Error validating token : Token does not exist.
      2022-07-11 15:05:45,126 ERROR public-settings.premium-features :: Error validating token : Token does not exist.
      2022-07-11 15:05:45,128 ERROR public-settings.premium-features :: Error validating token : Token does not exist.
      2022-07-11 15:05:45,131 ERROR public-settings.premium-features :: Error validating token : Token does not exist.
      2022-07-11 15:05:45,131 ERROR public-settings.premium-features :: Error validating token : Token does not exist.
      2022-07-11 15:05:45,131 ERROR public-settings.premium-features :: Error validating token : Token does not exist.
      2022-07-11 15:05:45,133 ERROR public-settings.premium-features :: Error validating token : Token does not exist.
      2022-07-11 15:05:45,133 ERROR public-settings.premium-features :: Error validating token : Token does not exist.
      2022-07-11 15:05:45,133 ERROR public-settings.premium-features :: Error validating token : Token does not exist.
      2022-07-11 15:05:45,140 ERROR public-settings.premium-features :: Error validating token : Token does not exist.
      2022-07-11 15:05:45,142 ERROR public-settings.premium-features :: Error validating token : Token does not exist.
      2022-07-11 15:05:45,143 ERROR public-settings.premium-features :: Error validating token : Token does not exist.
      2022-07-11 15:05:45,143 ERROR public-settings.premium-features :: Error validating token : Token does not exist.
      2022-07-11 15:05:45,152 ERROR public-settings.premium-features :: Error validating token : Token does not exist.
      2022-07-11 15:05:45,152 ERROR public-settings.premium-features :: Error validating token : Token does not exist.
      2022-07-11 15:05:45,156 ERROR public-settings.premium-features :: Error validating token : Token does not exist.
      2022-07-11 15:05:45,157 ERROR public-settings.premium-features :: Error validating token : Token does not exist.
      2022-07-11 15:05:45,157 ERROR public-settings.premium-features :: Error validating token : Token does not exist.
      2022-07-11 15:05:45,157 ERROR public-settings.premium-features :: Error validating token : Token does not exist.
      2022-07-11 15:05:45,160 ERROR public-settings.premium-features :: Error validating token : Token does not exist.
      2022-07-11 15:05:45,161 ERROR public-settings.premium-features :: Error validating token : Token does not exist.
      2022-07-11 15:05:45,162 ERROR public-settings.premium-features :: Error validating token : Token does not exist.
      2022-07-11 15:05:45,168 ERROR public-settings.premium-features :: Error validating token : Token does not exist.
      2022-07-11 15:05:45,171 ERROR public-settings.premium-features :: Error validating token : Token does not exist.
      2022-07-11 15:05:45,174 ERROR public-settings.premium-features :: Error validating token : Token does not exist.
      2022-07-11 15:05:45,175 ERROR public-settings.premium-features :: Error validating token : Token does not exist.
      2022-07-11 15:05:45,176 ERROR public-settings.premium-features :: Error validating token : Token does not exist.
      2022-07-11 15:05:45,177 ERROR public-settings.premium-features :: Error validating token : Token does not exist.
      2022-07-11 15:05:45,182 ERROR public-settings.premium-features :: Error validating token : Token does not exist.
      2022-07-11 15:05:45,184 ERROR public-settings.premium-features :: Error validating token : Token does not exist.
      2022-07-11 15:05:45,197 ERROR public-settings.premium-features :: Error validating token : Token does not exist.
      2022-07-11 15:05:45,220 ERROR public-settings.premium-features :: Error validating token : Token does not exist.
      2022-07-11 15:05:45,229 ERROR public-settings.premium-features :: Error validating token : Token does not exist.
      2022-07-11 15:05:45,267 ERROR public-settings.premium-features :: Error validating token : Token does not exist.
      2022-07-11 15:05:45,285 ERROR public-settings.premium-features :: Error validating token : Token does not exist.
      2022-07-11 15:05:45,286 ERROR public-settings.premium-features :: Error validating token : Token does not exist.
      2022-07-11 15:05:45,286 ERROR public-settings.premium-features :: Error validating token : Token does not exist.
      2022-07-11 15:05:45,286 ERROR public-settings.premium-features :: Error validating token : Token does not exist.
      2022-07-11 15:05:45,286 ERROR public-settings.premium-features :: Error validating token : Token does not exist.
      2022-07-11 15:05:45,287 ERROR public-settings.premium-features :: Error validating token : Token does not exist.
      2022-07-11 15:05:45,287 ERROR public-settings.premium-features :: Error validating token : Token does not exist.
      2022-07-11 15:05:45,288 ERROR public-settings.premium-features :: Error validating token : Token does not exist.
      2022-07-11 15:05:45,288 ERROR public-settings.premium-features :: Error validating token : Token does not exist.
      ```
      
      After this memoization:
      
      ```
      2022-07-11 15:29:57,052 INFO public-settings.premium-features :: Checking with the MetaStore to see whether ffff...ffff is valid...
      2022-07-11 15:29:57,290 ERROR public-settings.premium-features :: Error fetching token status:
      clojure.lang.ExceptionInfo: clj-http: status 404 {:body "{\"valid\":false,\"status\":\"Token does not exist.\"}", :trace-redirects []}
      	...
      	at metabase.public_settings.premium_features$eval121393$fetch_token_status_STAR___121398$fn__121399$fn__121400.invoke(premium_features.clj:72)
      	...
      2022-07-11 15:29:57,306 ERROR public-settings.premium-features :: Error validating token : Token does not exist.
      ```
      
      I simulated this by setting an invalid token:
      
      ```
      (setting/set-value-of-type! :string :premium-embedding-token
                                  (apply str (repeat 64 "f")))
      ```
      
      and then calling the function to enumerate all of the settings:
      
      ```
      (count (setting/admin-writable-settings))
      ```
      
      Another case this comes up is when a server doesn't have internet
      access. It will keep trying to hit the token check endpoint because the
      check doesn't catch `Exception` but `clojure.lang.ExceptionInfo`.
      
      I think this was done on purpose because we don't want to cache a
      transient network issue and rather have it succeed on a subsequent
      request. But this is a poor strategy if it can never succeed.
      
      My thinking is we should catch the error and cache the "not valid"
      response rather than try each time. But we'll have to expose a way for
      the UI to "force recheck" by clearing the caches (there are two we'd
      have to clear but no big technical hurddle i don't believe)
      
      ```clojure
      => (doc memoize/memo-clear!)
      -------------------------
      clojure.core.memoize/memo-clear!
      ([f] [f args])
        Reaches into an core.memo-memoized function and clears the cache.  This is a
         destructive operation and should be used with care.
      
         When the second argument is a vector of input arguments, clears cache only
         for argument vector.
      
         Keep in mind that depending on what other threads or doing, an
         immediate call to `snapshot` may not yield an empty cache.  That's
         cool though, we've learned to deal with that stuff in Clojure by
         now.
      ```
      Unverified
      7b2f4be5
    • Noah Moss's avatar
      First pass at parameters in Markdown cards (#23641) · e136f430
      Noah Moss authored
      * first pass at parameters in text cards on FE
      
      * trying to get translations working
      
      * relative datetime formatting
      
      * copy changes and 'Text card' header
      
      * default text when no params
      
      * hide header for text cards with height of 1 with params when in param mapping mode
      
      * show UI text in mobile mode
      
      * minor fixes
      
      * enforce that a text card variable can only be mapped to one parameter
      
      * more value formatting
      
      * noop
      
      * fix backend tests
      
      * add back a couple pieces of frontend logic commented out
      
      * misc cleanup
      
      * attempt at adding a FE unit test
      
      * revert unit test, doesn't work
      
      * add a couple of basic cypress tests and fix a couple of bugs
      
      * basic unit tests for cljc
      
      * fix error
      
      * expanded unit tests
      
      * simplify ns
      
      * add cypress test for instance language translation
      
      * basic handling for a couple cases of :date/all-options
      
      * trs docstring clarification
      
      * whitespace tweaks
      
      * fix cypress test
      
      * minor refactor of tag-names
      
      * move cljc file from utils to new parameters dir
      
      * reorder functions
      
      * fix lint
      
      * add test assertion that locale is correctly reset back to english, and add a comment
      
      * fix bug where existing parameter mapping target was not being found
      
      * clojure logic tweaks
      
      * move text card header text to the Text component config
      
      * simplify header logic, and pull out isLoading into a function to reduce complexity
      
      * address alex's css feedback
      
      * fix trs comment
      Unverified
      e136f430
    • Ryan Laurie's avatar
      Field Search in Bulk Filter Modal (#23813) · a74e5de8
      Ryan Laurie authored
      * add field search to filter modal
      * remove close button
      * show table name for segments in search
      * update segment tests
      Unverified
      a74e5de8
    • adam-james's avatar
      Remove `update_collection_tree_authority_level` from PUT collection/:id (#23718) · 61ee8a35
      adam-james authored
      * Rmv `update_collection_tree_authority_level` from PUT collection/:id
      
      * Remove test that was only testing the removed feature anyway
      Unverified
      61ee8a35
    • Nick Fitzpatrick's avatar
      Mobile View Info Panel (#23850) · 727b9e2d
      Nick Fitzpatrick authored
      * Mobile View Info Panel
      
      * PR Feedback
      Unverified
      727b9e2d
    • Jeff Bruemmer's avatar
      docs - embed params (#23844) · ae1b445b
      Jeff Bruemmer authored
      Unverified
      ae1b445b
    • Alexander Polyankin's avatar
    • Ryan Laurie's avatar
      Bulk Filter Layout v2 (#23746) · 26ce1911
      Ryan Laurie authored
      * update bulk filter layout and operator selectors
      * improved testing helpers
      Unverified
      26ce1911
    • Dalton's avatar
      Handle field values endpoint has_more_values state (#23709) · bc74316c
      Dalton authored
      * Handle field values endpoint has_more_values state
      
      * Fix isSearchable fn
      
      * make shouldSearch evaluation lazy
      
      * Prettier nonsense
      
      * Remove default has_more_values value
      
      * Introduce concept of valuesMode to handle has_more_values (#23795)
      
      * Introduce concept of valuesMode to handle has_more_values
      
      * use hasList in render method
      
      * Pass explicit args to getValuesMode
      
      * Fix override logic
    • Gustavo Saiani's avatar
      Fix console warning in PieChart (#23838) · 351189fe
      Gustavo Saiani authored
      Unverified
      351189fe
    • Anton Kulyk's avatar
    • Alexander Polyankin's avatar
    • Jeff Bruemmer's avatar
      bastion note (#23855) · e7e1d59b
      Jeff Bruemmer authored
      Unverified
      e7e1d59b
    • Howon Lee's avatar
      Port fixes for Postgresql JSON issues to MySQL (#23758) · a26cba2a
      Howon Lee authored
      #22967 and boat#161 also apply to MySQL JSON implementation. This PR ports the fixes for those (in #23278 and #22997 respectively) to MySQL.
      
      There are many other JSON implementation fixes which don't need a separate port because we moved the describe-table mechanism to the SQL JDBC driver instead of just the Postgres driver.
      Unverified
      a26cba2a
    • Gustavo Saiani's avatar
      Unverified
      a29e1877
    • Cal Herries's avatar
  3. Jul 11, 2022
Loading