Skip to content
Snippets Groups Projects
Unverified Commit d55ea823 authored by dpsutton's avatar dpsutton Committed by GitHub
Browse files

Handle api/tiles requests for native queries (#18810)

* Handle api/tiles requests for native queries

Ordinarily the api takes the source query and adds in a filter clause
for a region:

```clojure
{:database 19
 :query {:source-table 88
         :fields [[:field 562 nil]
                  [:field 574 nil]
                  [:field 576 nil]]
         :limit 2000}
 :type :query}

{:database 19
 :query {:source-table 88
         :fields [[:field 562 nil]
                  [:field 574 nil]
                  [:field 576 nil]]
         :limit 2000
         :filter [:inside
                  [:field 576 nil]
                  [:field 574 nil]
                  -40.97989944013222
                  -179.99999564141046
                  -66.51326189011354
                  -134.99999673105785]}
 :type :query
 :async? false}
```

But when native, this would break, for three different reasons:
- native queries don't necessarily (or possibly) know their field
ids. So the api request it would construct would have `undefined` in the
slot where the field-id would normally go. This would cause the route to
404 as it would fail to match the expected numeric part of the url
- just passing in the field name isn't suffiencient, because mbql cannot
natively use an mbql filter clause into a native snippet. We do this to
a limited extent with filters and substitution but this requires a
marker like `{{filter}}` placeholder for us to add the filter text.
- when using `[:field name ...]` type field references we need the
base-type of a field in order to construct a query.

So the solution:
- allow the route to take field ids or names
- if native, rewrite the query to be a nested query using the native
query as the source query. ie, "select name, lat, long from table" ->
"select source.* from (select name, lat, long from table) source" but in
mbql so we can add our fliter clause
- if a string name is passed in, annotate it with a base-type of
:type/Float since we are dealing with lats and longs.

A concern was that we need to include the source-metadata. I'm omitting
this because we are essentially "select * from nested" so we just take
what we want, and the route already expects the index of the lat and
long columns and then selects only those from the query results.

As it stands we end up with a working query but a log

```
2021-11-02 14:37:25,470 DEBUG middleware.log :: GET /api/tiles/2/1/1/latitude/longitude/1/2/ 200 47.3 ms (5 DB calls) App DB connections: 2/13 Jetty threads: 9/50 (2 idle, 0 queued) (128 total active threads) Queries in flight: 0 (0 queued); postgres DB 19 connections: 3/6 (0 threads blocked)
2021-11-02 14:37:25,474 WARN middleware.add-implicit-clauses :: Warning: cannot determine fields for an explicit `source-query` unless you also include `source-metadata`.
2021-11-02 14:37:25,490 WARN middleware.add-implicit-clauses :: Warning: cannot determine fields for an explicit `source-query` unless you also include `source-metadata`.
2021-11-02 14:37:25,501 DEBUG middleware.log :: GET /api/tiles/2/0/2/latitude/longitude/1/2/ 200 39.4 ms (5 DB calls) App DB connections: 1/13 Jetty threads: 8/50 (2 idle, 0 queued) (128 total active threads) Queries in flight: 0 (0 queued); postgres DB 19 connections: 2/6 (0 threads blocked)
```

It might be possible to suppress these logs when we are in a query
context of `:map-tiles`. Or we might be able to require the frontend to
send along the metadata to include in the query. I didn't do this yet
since it seems to work fine and it would just make the urls quite long
if there are lots of columns (and the metadata can get quite verbose).

```clojure
;; source query of the card
{:type :native
 :native {:query "select name, latitude, longitude from zomato limit 2000;"
          :template-tags {}}
 :database 19}

;; nest it into a source query
{:database 19
 :type :query
 :query {:source-query {:template-tags {}
                        :native "select name, latitude, longitude from zomato limit 2000;"}}}

;; add the `:inside` filter for the tile

{:database 19
 :type :query
 :query {:source-query {:template-tags {}
                        :native "select name, latitude, longitude from zomato limit 2000;"}
         :filter [:inside
                  [:field
                   "latitude"
                   {:base-type :type/Float}]
                  [:field
                   "longitude"
                   {:base-type :type/Float}]
                  0.0
                  89.99999782070523
                  -66.51326189011354
                  179.99999564141046]}
 :async? false}
```

* Add docstring and make function private

* url encode column names to api/tiles
parent cba58286
No related branches found
No related tags found
No related merge requests found
Loading
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment