-
- Downloads
Allow text strings to special type to date [ci drivers] (#13860)
* Allow text strings to special type to date [ci drivers] * Allow for timestamps/time/date for iso8601 strings [ci drivers] * First pass at tests for :type/ISO8601DateTimeString [ci drivers] * Only run ISO8601datetimestring tests on sql dbs * Make the default a timestamp and mysql the exception [ci drivers] * Fixes for sqlite and sqlserver i'm not happy with sqlite yet. they are returning strings not java.dates. But this is the same with a field who's base type is a datetime. However, the UI allows for ordering on the date times but not on the :type/ISO8601 fields and i'm not sure why * Handle oracle [ci drivers] oracle doesn't have a time type so use a similar edn dataset that doesn't have the time column * Move dataset definitions inline [ci drivers] * correct oracle datetime format [ci drivers] sparksql seems to have issues with time so just punt on that for now by running in the test that doesn't have the time column * wack-a-mole [ci drivers] * Fix tests for redshift, oracle, and sparksql [ci drivers] * Oracle is funky [ci drivers] * need to investigate what's going on with oracle here [ci drivers] text fields with special_type :type/ISO8601DateTimeString are queryable as dates a datetime field :oracle expected: (= 1 (count (mt/rows (mt/dataset string-times (mt/run-mbql-query times {:filter [:= [:datetime-field $ts :day] "2008-10-19"]}))))) actual: java.lang.NullPointerException: null at oracle.jdbc.driver.T4CTTIdcb.fillupAccessors (T4CTTIdcb.java:1041) oracle.jdbc.driver.T4CTTIdcb.receiveCommon (T4CTTIdcb.java:221) oracle.jdbc.driver.T4CTTIdcb.receive (T4CTTIdcb.java:161) oracle.jdbc.driver.T4C8Oall.readDCB (T4C8Oall.java:991) oracle.jdbc.driver.T4CTTIfun.receive (T4CTTIfun.java:560) oracle.jdbc.driver.T4CTTIfun.doRPC (T4CTTIfun.java:252) oracle.jdbc.driver.T4C8Oall.doOALL (T4C8Oall.java:612) oracle.jdbc.driver.T4CPreparedStatement.doOall8 (T4CPreparedStatement.java:226) oracle.jdbc.driver.T4CPreparedStatement.doOall8 (T4CPreparedStatement.java:59) oracle.jdbc.driver.T4CPreparedStatement.executeForDescribe (T4CPreparedStatement.java:747) oracle.jdbc.driver.OracleStatement.executeMaybeDescribe (OracleStatement.java:904) oracle.jdbc.driver.OracleStatement.doExecuteWithTimeout (OracleStatement.java:1082) oracle.jdbc.driver.OraclePreparedStatement.executeInternal (OraclePreparedStatement.java:3780) oracle.jdbc.driver.T4CPreparedStatement.executeInternal (T4CPreparedStatement.java:1343) oracle.jdbc.driver.OraclePreparedStatement.executeQuery (OraclePreparedStatement.java:3822) oracle.jdbc.driver.OraclePreparedStatementWrapper.executeQuery (OraclePreparedStatementWrapper.java:1165) com.mchange.v2.c3p0.impl.NewProxyPreparedStatement.executeQuery (NewProxyPreparedStatement.java:431) metabase.driver.sql_jdbc.execute$eval71770$fn__71771.invoke (execute.clj:267) clojure.lang.MultiFn.invoke (MultiFn.java:234) metabase.driver.sql_jdbc.execute$execute_reducible_query.invokeStatic (execute.clj:392) metabase.driver.sql_jdbc.execute$execute_reducible_query.invoke (execute.clj:377) metabase.driver.sql_jdbc.execute$execute_reducible_query.invokeStatic (execute.clj:386) metabase.driver.sql_jdbc.execute$execute_reducible_query.invoke (execute.clj:377) metabase.driver.sql_jdbc$eval102870$fn__102871.invoke (sql_jdbc.clj:49) metabase.driver.oracle$eval187218$fn__187219.invoke (oracle.clj:268) clojure.lang.MultiFn.invoke (MultiFn.java:244) metabase.query_processor.context$executef.invokeStatic (context.clj:59) metabase.query_processor.context$executef.invoke (context.clj:48) metabase.query_processor.context.default$default_runf.invokeStatic (default.clj:69) metabase.query_processor.context.default$default_runf.invoke (default.clj:67) metabase.query_processor.context$runf.invokeStatic (context.clj:45) metabase.query_processor.context$runf.invoke (context.clj:39) metabase.query_processor.reducible$pivot.invokeStatic (reducible.clj:34) metabase.query_processor.reducible$pivot.invoke (reducible.clj:31) * sparksql has some issues querying on datetime [ci drivers] text fields with special_type :type/ISO8601DateTimeString are queryable as dates a datetime field :sparksql: org.apache.spark.sql.catalyst.parser.ParseException: DataType time() is not supported.(line 2, pos 178) == SQL == -- Metabase SELECT `t1`.`id` AS `id`, `t1`.`name` AS `name`, CAST(CAST(`t1`.`ts` AS timestamp) AS timestamp) AS `ts`, CAST(CAST(`t1`.`d` AS date) AS timestamp) AS `d`, CAST(CAST(`t1`.`t` AS time) AS timestamp) AS `t` FROM `string_times`.`times` `t1` WHERE (CAST(CAST(`t1`.`ts` AS timestamp) AS timestamp) >= CAST(to_utc_timestamp('2008-10-19 00:00:00', 'UTC') AS timestamp) AND CAST(CAST(`t1`.`ts` AS timestamp) AS timestamp) < CAST(to_utc_timestamp('2008-10-20 00:00:00', 'UTC') AS timestamp)) LIMIT 1048576 ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------^^^ expected: (= 1 (count (mt/rows (mt/dataset string-times (mt/run-mbql-query times {:filter [:= [:datetime-field $ts :day] "2008-10-19"]}))))) actual: java.sql.SQLException: org.apache.spark.sql.catalyst.parser.ParseException: DataType time() is not supported.(line 2, pos 178) == SQL == -- Metabase SELECT `t1`.`id` AS `id`, `t1`.`name` AS `name`, CAST(CAST(`t1`.`ts` AS timestamp) AS timestamp) AS `ts`, CAST(CAST(`t1`.`d` AS date) AS timestamp) AS `d`, CAST(CAST(`t1`.`t` AS time) AS timestamp) AS `t` FROM `string_times`.`times` `t1` WHERE (CAST(CAST(`t1`.`ts` AS timestamp) AS timestamp) >= CAST(to_utc_timestamp('2008-10-19 00:00:00', 'UTC') AS timestamp) AND CAST(CAST(`t1`.`ts` AS timestamp) AS timestamp) < CAST(to_utc_timestamp('2008-10-20 00:00:00', 'UTC') AS timestamp)) LIMIT 1048576 ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------^^^ at org.apache.hive.jdbc.HiveStatement.execute (HiveStatement.java:297) org.apache.hive.jdbc.HiveStatement.executeQuery (HiveStatement.java:392) org.apache.hive.jdbc.HivePreparedStatement.executeQuery (HivePreparedStatement.java:109) com.mchange.v2.c3p0.impl.NewProxyPreparedStatement.executeQuery (NewProxyPreparedStatement.java:431) metabase.driver.sql_jdbc.execute$eval71770$fn__71771.invoke (execute.clj:267) clojure.lang.MultiFn.invoke (MultiFn.java:234) metabase.driver.sql_jdbc.execute$execute_reducible_query.invokeStatic (execute.clj:392) metabase.driver.sql_jdbc.execute$execute_reducible_query.invoke (execute.clj:377) metabase.driver.sql_jdbc.execute$execute_reducible_query.invokeStatic (execute.clj:386) metabase.driver.sql_jdbc.execute$execute_reducible_query.invoke (execute.clj:377) metabase.driver.sql_jdbc$eval102870$fn__102871.invoke (sql_jdbc.clj:49) metabase.driver.sparksql$eval187174$fn__187176.invoke (sparksql.clj:142) clojure.lang.MultiFn.invoke (MultiFn.java:244) metabase.query_processor.context$executef.invokeStatic (context.clj:59) metabase.query_processor.context$executef.invoke (context.clj:48) metabase.query_processor.context.default$default_runf.invokeStatic (default.clj:69) metabase.query_processor.context.default$default_runf.invoke (default.clj:67) metabase.query_processor.context$runf.invokeStatic (context.clj:45) metabase.query_processor.context$runf.invoke (context.clj:39) metabase.query_processor.reducible$pivot.invokeStatic (reducible.clj:34) metabase.query_processor.reducible$pivot.invoke (reducible.clj:31) metabase.query_processor.middleware.mbql_to_native$mbql__GT_native$fn__57878.invoke (mbql_to_native.clj:26) metabase.query_processor.middleware.check_features$check_features$fn__57065.invoke (check_features.clj:42) metabase.query_processor.middleware.optimize_datetime_filters$optimize_datetime_filters$fn__58077.invoke (optimize_datetime_filters.clj:133) metabase.query_processor.middleware.auto_parse_filter_values$auto_parse_filter_values$fn__55756.invoke (auto_parse_filter_values.clj:44) metabase.query_processor.middleware.wrap_value_literals$wrap_value_literals$fn__59813.invoke (wrap_value_literals.clj:149) metabase.query_processor.middleware.annotate$add_column_info$fn__55502.invoke (annotate.clj:575) metabase.query_processor.middleware.permissions$check_query_permissions$fn__56924.invoke (permissions.clj:70) metabase.query_processor.middleware.pre_alias_aggregations$pre_alias_aggregations$fn__58633.invoke (pre_alias_aggregations.clj:40) metabase.query_processor.middleware.cumulative_aggregations$handle_cumulative_aggregations$fn__57154.invoke (cumulative_aggregations.clj:61) metabase_enterprise.sandbox.query_processor.middleware.row_level_restrictions$apply_row_level_permissions$fn__60343.invoke (row_level_restrictions.clj:292) metabase.query_processor.middleware.resolve_joined_fields$resolve_joined_fields$fn__58912.invoke (resolve_joined_fields.clj:36) metabase.query_processor.middleware.resolve_joins$resolve_joins$fn__59239.invoke (resolve_joins.clj:183) metabase.query_processor.middleware.add_implicit_joins$add_implicit_joins$fn__50719.invoke (add_implicit_joins.clj:254) metabase.query_processor.middleware.large_int_id$convert_id_to_string$fn__57823.invoke (large_int_id.clj:44) metabase.query_processor.middleware.limit$limit$fn__57856.invoke (limit.clj:38) metabase.query_processor.middleware.format_rows$format_rows$fn__57795.invoke (format_rows.clj:84) metabase.query_processor.middleware.desugar$desugar$fn__57226.invoke (desugar.clj:22) metabase.query_processor.middleware.binning$update_binning_strategy$fn__56123.invoke (binning.clj:229) metabase.query_processor.middleware.resolve_fields$resolve_fields$fn__56714.invoke (resolve_fields.clj:24) metabase.query_processor.middleware.add_dimension_projections$add_remapping$fn__50239.invoke (add_dimension_projections.clj:318) metabase.query_processor.middleware.add_implicit_clauses$add_implicit_clauses$fn__50462.invoke (add_implicit_clauses.clj:141) metabase_enterprise.sandbox.query_processor.middleware.row_level_restrictions$apply_row_level_permissions$fn__60343.invoke (row_level_restrictions.clj:292) metabase.query_processor.middleware.add_source_metadata$add_source_metadata_for_source_queries$fn__50882.invoke (add_source_metadata.clj:105) metabase_enterprise.sandbox.query_processor.middleware.column_level_perms_check$maybe_apply_column_level_perms_check$fn__60025.invoke (column_level_perms_check.clj:24) metabase.query_processor.middleware.reconcile_breakout_and_order_by_bucketing$reconcile_breakout_and_order_by_bucketing$fn__58858.invoke (reconcile_breakout_and_order_by_bucketing.clj:98) * Remove sparksql can't query on date [ci drivers]
Showing
- backend/mbql/src/metabase/mbql/util/match.clj 7 additions, 0 deletionsbackend/mbql/src/metabase/mbql/util/match.clj
- frontend/src/metabase/lib/core.js 15 additions, 0 deletionsfrontend/src/metabase/lib/core.js
- frontend/test/metabase-bootstrap.js 4 additions, 0 deletionsfrontend/test/metabase-bootstrap.js
- modules/drivers/oracle/src/metabase/driver/oracle.clj 8 additions, 0 deletionsmodules/drivers/oracle/src/metabase/driver/oracle.clj
- modules/drivers/redshift/src/metabase/driver/redshift.clj 5 additions, 0 deletionsmodules/drivers/redshift/src/metabase/driver/redshift.clj
- modules/drivers/sqlite/src/metabase/driver/sqlite.clj 13 additions, 0 deletionsmodules/drivers/sqlite/src/metabase/driver/sqlite.clj
- modules/drivers/sqlserver/src/metabase/driver/sqlserver.clj 4 additions, 0 deletionsmodules/drivers/sqlserver/src/metabase/driver/sqlserver.clj
- src/metabase/driver/mysql.clj 4 additions, 0 deletionssrc/metabase/driver/mysql.clj
- src/metabase/driver/sql/query_processor.clj 19 additions, 7 deletionssrc/metabase/driver/sql/query_processor.clj
- src/metabase/driver/sql_jdbc.clj 12 additions, 0 deletionssrc/metabase/driver/sql_jdbc.clj
- src/metabase/types.clj 6 additions, 0 deletionssrc/metabase/types.clj
- test/metabase/query_processor_test/alternative_date_test.clj 193 additions, 0 deletionstest/metabase/query_processor_test/alternative_date_test.clj
Loading
Please register or sign in to comment