Skip to content
Snippets Groups Projects
Commit 58ba5d4e authored by Cam Saul's avatar Cam Saul
Browse files

limit field distinct count query when syncing

parent ef643009
No related branches found
No related tags found
No related merge requests found
......@@ -297,7 +297,7 @@
"If FIELD doesn't yet have a `special_type`, and has low cardinality, mark it as a category."
[field]
(when-not (:special_type field)
(let [cardinality (queries/field-distinct-count field)]
(let [cardinality (queries/field-distinct-count field low-cardinality-threshold)]
(when (and (> cardinality 0)
(< cardinality low-cardinality-threshold))
(log/info (format "Field '%s.%s' has %d unique values. Marking it as a category." (:name @(:table field)) (:name field) cardinality))
......
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment