Only select type information on type inference of coalesce (#19008)
* Only select type information on type inference of coalesce
This previously just selected hte inferred information from the second
argument to the coalesce. For fields this would return the entire field
object which includes an id.
Later, in `source-metadata->fields` this assumes that since an id is
present, we should select `[:field id nil]`, but this confused that the
type of the coalesce column is the same as that field-id, not that we
want to select that field id.
```clojure
add-source-metadata=> (pprint
(mbql-source-query->metadata {:source-table 2
:joins [{:fields [[:field
22
{:join-alias "People - User"}]
[:field
15
{:join-alias "People - User"}]]
:source-table 3
:condition [:=
[:field 3 nil]
[:field
22
{:join-alias "People - User"}]]
:alias "People - User"}]
:expressions {:coalesce [:coalesce
[:field 3 nil]
[:field
22
{:join-alias "People - User"}]]}
:aggregation [[:count]]
:breakout [[:expression "coalesce"]]}))
({:semantic_type :type/FK,
:table_id 2,
:coercion_strategy nil,
:name "coalesce", ;; merging maintains the correct name
:settings nil,
:field_ref [:expression "coalesce"], ;; merging maintains the field ref
:effective_type :type/Integer,
:parent_id nil,
:id 3, ;; the type inference selected
;; the entire field, including its id
:display_name "coalesce",
:fingerprint {:global {:distinct-count 929, :nil% 0.0}},
:base_type :type/Integer}
{:name "count",
:display_name "Count",
:base_type :type/BigInteger,
:semantic_type :type/Quantity,
:field_ref [:aggregation 0]})
nil
```
updating to only select the type information:
```clojure
add-source-metadata=> (pprint
(mbql-source-query->metadata {:source-table 2
:joins [{:fields [[:field
22
{:join-alias "People - User"}]
[:field
15
{:join-alias "People - User"}]]
:source-table 3
:condition [:=
[:field 3 nil]
[:field
22
{:join-alias "People - User"}]]
:alias "People - User"}]
:expressions {:coalesce [:coalesce
[:field 3 nil]
[:field
22
{:join-alias "People - User"}]]}
:aggregation [[:count]]
:breakout [[:expression "coalesce"]]}))
({:name "coalesce",
:display_name "coalesce",
:base_type :type/Integer,
:effective_type :type/Integer, ;; no more field information beyond
;; the types
:coercion_strategy nil,
:semantic_type :type/FK,
:field_ref [:expression "coalesce"]}
{:name "count",
:display_name "Count",
:base_type :type/BigInteger,
:semantic_type :type/Quantity,
:field_ref [:aggregation 0]})
```
* Update question in repro now that it doesn't fail on the backend
previously this question would actually fail on the backend and
independently the frontend would go into a stack overflow killing the
page. Fixing the bug actually broke the test as it asserted that the
query would break and look for the error message. Now that the query
returns assert we can find the name of the question "test question"
* Correct same bug for case
When inferring type for a case, we were returning the type information
for the first field we could find in the case statement. And that
information included the field id. We always merge the current column's
name on top of that to remove the analyzed field's name and other
things, but the field-id would remain. `source-metadata->fields` in
add_implicit_clauses will add a field reference if it finds a field id.
Before, the following was returned
```clojure
({:description "The date and time an order was submitted."
:semantic_type :type/CreationTimestamp
:table_id 2
:coercion_strategy nil
:unit :month
:name "CREATED_AT"
:settings nil
:source :breakout
:field_ref [:field
4
{:temporal-unit :month}]
:effective_type :type/DateTime
:parent_id nil
:id 4
:visibility_type :normal
:display_name "Created At"
:fingerprint {:global {:distinct-count 9998
:nil% 0.0}
:type {:type/DateTime {:earliest "2016-04-30T18:56:13.352Z"
:latest "2020-04-19T14:07:15.657Z"}}}
:base_type :type/DateTime}
{:description "The raw, pre-tax cost of the order. Note that this might be different in the future from the product price due to promotions, credits, etc."
:semantic_type :type/Quantity
:table_id 2
:coercion_strategy nil
:name "distinct case"
:settings nil
:source :aggregation
:field_ref [:aggregation 0]
:effective_type :type/Float
:parent_id nil
:id 6
:visibility_type :normal
:display_name "distinct case"
:fingerprint {:global {:distinct-count 340
:nil% 0.0}
:type {:type/Number {:min 15.691943673970439
:q1 49.74894519060184
:q3 105.42965746993103
:max 148.22900526552291
:sd 32.53705013056317
:avg 77.01295465356547}}}
:base_type :type/BigInteger})
```
The important bit above is that the metadata for the "distinct case" has
the correct field_ref but also has a field id.
After, we now return the following metadata:
```clojure
({:description "The date and time an order was submitted."
:semantic_type :type/CreationTimestamp
:table_id 2
:coercion_strategy nil
:unit :month
:name "CREATED_AT"
:settings nil
:source :fields
:field_ref [:field
4
{:temporal-unit :default}]
:effective_type :type/DateTime
:parent_id nil
:id 4
:visibility_type :normal
:display_name "Created At"
:fingerprint {:global {:distinct-count 9998
:nil% 0.0}
:type {:type/DateTime {:earliest "2016-04-30T18:56:13.352Z"
:latest "2020-04-19T14:07:15.657Z"}}}
:base_type :type/DateTime}
{:base_type :type/BigInteger
:effective_type :type/Float
:coercion_strategy nil
:semantic_type :type/Quantity
:name "distinct case"
:display_name "distinct case"
:source :fields
:field_ref [:field
"distinct case"
{:base-type :type/BigInteger}]}
{:semantic_type :type/Quantity
:coercion_strategy nil
:name "cc"
:expression_name "cc"
:source :fields
:field_ref [:expression "cc"]
:effective_type :type/Float
:display_name "cc"
:base_type :type/Float})
```
* Add tests looking for full field analysis without id
Showing
- frontend/test/metabase/scenarios/question/reproductions/18630-field-literals-in-joins.cy.spec.js 4 additions, 3 deletions...on/reproductions/18630-field-literals-in-joins.cy.spec.js
- src/metabase/query_processor/middleware/annotate.clj 7 additions, 2 deletionssrc/metabase/query_processor/middleware/annotate.clj
- test/metabase/query_processor/middleware/annotate_test.clj 62 additions, 14 deletionstest/metabase/query_processor/middleware/annotate_test.clj
Please register or sign in to comment