Skip to content
Snippets Groups Projects
Unverified Commit a1a66521 authored by adam-james's avatar adam-james Committed by GitHub
Browse files

Start page be simplified popular items (#21553)


* Add `:is_installer` key to users returned from api/user/current

* Add `:has_question_and_dashboard` Key to current users via hydration

This is also part of the starting page project, it's a key that helps the frontend serve more specific data to the
user.

* Moved `permissions-set` function up so I can use it in hydration

* Adjust recents query and first pass at popular items query

Recents:

Before, the recent query would look at the ViewLog, which records a view on cards even when it was only 'viewed' via a
dashboard.

For dashboards and tables, we don't need to make a change.

For cards, we now query the QueryExecution table and use started_at as a proxy for the last viewed timestamp, and can
then only grab cards with an execution context of 'question'.

Popular:

We now want to have a notion of 'popular' items as well, which can look different than 'recents'. This is the first
attempt at a reasonable query with scoring system.

The score takes into account:
 - recency of view
 - count of views total
 - if the card/dashboard is 'verified' (enterprise)
 - if the cad/dashboard is inside an 'official' Collection (enterprise)

The popular score currently uses only the *current-user-id* to fetch items, so popularity is (in this commit, at
least) a per-user concept.

* Fixed mistake and renamed endpoint

* Typo fix

* Clean up QueryExecution in tests

* Indent, naming, and some simple defs

* try to reduce db calls for :has_question_and_dashboard

I've moved the fn out of the hydration system to guarantee that it only runs for a single user, and is only used in
`GET /api/user/current` (and not accidentally used in other spots via hydration mechanism).

Still don't know how to check Card and Dashboard tables more efficiently

* ViewLog and QueryExecution have different timestamps. Can't compare

Pushing this for some review, but might want to build a proper way to compare, so that recent cards and recent
dashboards/tables are sorted equally

* fix namespace sorting

* Fix the endpoint name

* Sorting mixed date time formats 'works' but not ideal

This formats the timestamps into strings and uses those for comparison. Not perfect.

Pushing in case people are trying the branch

* Use simpler db function

* Let views and runs query work with one user or all users

Popular_items are an all-users notion, but recent_views applies only to the current user.

* Unify view_log.timestamp to query_execution.started_at type

these used to both be `DATETIME` in the migration file. Then migration
168 bumped up the type of the view_log.timestamp column:

```

  - changeSet:
      id: 168
      author: camsaul
      comment: Added 0.36.0
      changes:
        - modifyDataType:
            tableName: query_execution
            columnName: started_at
            newDataType: ${timestamp_type}
```

So these were no longer the same on h2 (and possibly mysql). But sorting
viewlogs and query_executions would fail.

```clojure
activity=> (->> (ViewLog) first :timestamp #_type)
        "0x6a33e42e"
        "2022-04-04T21:57:07.471"]
activity=> (->> (ViewLog) first :timestamp type)
java.time.LocalDateTime
activity=> (->> (QueryExecution) first :started_at #_type)
        "0x7af249ac"
        "2022-04-04T21:57:07.625738Z"]
activity=> (->> (QueryExecution) first :started_at type)
java.time.OffsetDateTime
```

The LocalDateTime and OffsetDateTime were not comparable. So make them
identical types again.

* Bookmarked results should not show up in recents/ popular.

This is done in a db query to still try fetch enough items from the db to present to the user.

Filtering bookmarked items later may result in an empty list. It's possible that the firt N results are all
bookmarked, and then the frontend would have no items to show. Filtering bookmarked results out from the beginning
increases the chances of a non-empty result.

* :first_login populated with the earliest login timestamp.

If there is a first login timestamp, that is used. If one does not exist, we assume it's the first time logging in and
use an OffsetDateTime (now). This is possible since the login_history is done off thread, so on a real first login, we
might hit the db before the login is logged, resulting in an empty list returned.

On subsequent checks, we should see a proper timestamp from the db.

Since this is used to check if a user is 'new' (within 7 days of first logging in), the accuracy of the timestamp
matters on the order of days, not milliseconds, so this should be ok.

* Passing test

* Popular_items test

Tweak the create-views! function to more consistently order items. And creates views for dashboards/tables (ViewLog)
and cards (QueryExecution) in a unified way, meaning we can reliably order views, and write tests more easily.

Note that the popular_items test may have to change if the scoring math changes.

* Fix e2e test

* Fix nit and bug

- forgot to remove '0' from the and clause, so we didn't get the expected boolean
- popular_items not WIP

* another nit

* Fix popular items on the frontend

Co-authored-by: default avatarAlexander Polyankin <alexander.polyankin@metabase.com>
Co-authored-by: default avatardan sutton <dan@dpsutton.com>
parent 1422fa7a
No related branches found
No related tags found
No related merge requests found
Showing
with 311 additions and 114 deletions
Loading
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment