Skip to content
Snippets Groups Projects
Unverified Commit 2a8e19b2 authored by dpsutton's avatar dpsutton Committed by GitHub
Browse files

Nested queries have limits which can defeat their purpose (#41051)

* Nested queries have limits which can defeat their purpose

Consider a query

```sql
select count(*) from {{#199}}
```

This query should return the number of distinct rows in the query
defined by 199. But it's actually limited by the excel limit of
1048575. And that's because when the value of `{{#199}}` is expanded it
has that limit applied as normal.

```clojure
qp=> (let [card-id 199] ;; use a valid card id for you
       (-> {:database 1,
            :type :native,
            :native {:query "select count(*) from {{ref}}"
                     :template-tags {:ref {:card-id card-id
                                           :type :card
                                           :name "ref"
                                           :display-name "ref"}}}
            :middleware {:disable-max-results? true}}
           qp.compile/compile
           :query
           (metabase.db.query/format-sql )
           println))
select
  count(*)
from
  (
    SELECT
      "PUBLIC"."ORDERS"."ID" AS "ID",
      "PUBLIC"."ORDERS"."TOTAL" AS "TOTAL"
    FROM
      "PUBLIC"."ORDERS"
    LIMIT
      1048575
  )
```

But we can suppress this limit when substituting a query inside yielding

```sql
select
  count(*)
from
  (
    SELECT
      "PUBLIC"."ORDERS"."ID" AS "ID",
      "PUBLIC"."ORDERS"."TOTAL" AS "TOTAL"
    FROM
      "PUBLIC"."ORDERS"
  )
```

And this is proper because we want to limit the _outer_ query, not
internal queries.

* Remove limit from test expectation

* stupid trailing space

* another subquery test

* Use helper function to disable limit middleware
parent 51142efc
Branches
Tags
No related merge requests found
......@@ -18,6 +18,7 @@
[metabase.models.native-query-snippet :refer [NativeQuerySnippet]]
[metabase.query-processor.compile :as qp.compile]
[metabase.query-processor.error-type :as qp.error-type]
[metabase.query-processor.middleware.limit :as limit]
[metabase.query-processor.store :as qp.store]
[metabase.query-processor.util.persisted-cache :as qp.persistence]
[metabase.util :as u]
......@@ -201,7 +202,7 @@
{:query (qp.persistence/persisted-info-native-query
(u/the-id (lib.metadata/database (qp.store/metadata-provider)))
persisted-info)})
(qp.compile/compile query)))))
(qp.compile/compile (limit/disable-max-results query))))))
(catch ExceptionInfo e
(throw (ex-info
(tru "The sub-query from referenced question #{0} failed with the following error: {1}"
......
......@@ -366,8 +366,7 @@
"\"PUBLIC\".\"VENUES\".\"LONGITUDE\" AS \"LONGITUDE\", "
"\"PUBLIC\".\"VENUES\".\"PRICE\" AS \"PRICE\" "
"FROM \"PUBLIC\".\"VENUES\" "
"WHERE \"PUBLIC\".\"VENUES\".\"PRICE\" < 3 "
"LIMIT 1048575")]
"WHERE \"PUBLIC\".\"VENUES\".\"PRICE\" < 3")]
(qp.store/with-metadata-provider (lib.tu/metadata-provider-with-cards-for-queries
meta/metadata-provider
[mbql-query])
......
......@@ -268,8 +268,7 @@
"\"PUBLIC\".\"VENUES\".\"LATITUDE\" AS \"LATITUDE\", "
"\"PUBLIC\".\"VENUES\".\"LONGITUDE\" AS \"LONGITUDE\", "
"\"PUBLIC\".\"VENUES\".\"PRICE\" AS \"PRICE\" "
"FROM \"PUBLIC\".\"VENUES\" "
"LIMIT 1048575")]
"FROM \"PUBLIC\".\"VENUES\"")]
(is (= (native-query
{:query (str "SELECT COUNT(*) FROM (SELECT * FROM (" card-1-subquery ") AS c1) AS c2") :params []})
(substitute-params
......
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Please register or to comment