Skip to content

Problem with Enum type #35

New issue
Jump to bottom
New issue
Jump to bottom
Closed
Closed
Problem with Enum type#35
@mgilbir

Description

@mgilbir
mgilbir
opened on Sep 30, 2019

Hi,

I'm encountering problems with enums when the clickhouse definition is longer than 255 chars as it cannot be written into the DATABASE_TYPE column of the database.

The redacted error message for metabase.sync.util is:

Sep 30 13:18:01 ERROR metabase.sync.util :: Error checking if Fields (<REDACTED_LIST_OF_FIELDS>) need to be created or reactivated: Value too long for column "DATABASE_TYPE VARCHAR(255) NOT NULL COMMENT 'The actual type of this column in the database. e.g. VARCHAR or TEXT.'": "'Enum8(''unknown'' = 0, ''A'' = 1, ''B'' = 2, ''C'' = 3, ''D'' = 4, ''E'' = ... (275)"; SQL statement:
INSERT INTO "METABASE_FIELD" ("DESCRIPTION", "DATABASE_TYPE", "TABLE_ID", "SPECIAL_TYPE", "NAME", "UPDATED_AT", "PARENT_ID", "DISPLAY_NAME", "CREATED_AT", "BASE_TYPE") VALUES (NULL, ?, 26, NULL, ?, ?, NULL, ?, ?, ?) [22001-197]
("org.h2.message.DbException.getJdbcSQLException(DbException.java:357)"
"org.h2.message.DbException.get(DbException.java:179)"
"org.h2.table.Column.validateConvertUpdateSequence(Column.java:398)"
"org.h2.table.Table.validateConvertUpdateSequence(Table.java:798)"
"org.h2.command.dml.Insert.insertRows(Insert.java:177)"
"org.h2.command.dml.Insert.update(Insert.java:134)"
"org.h2.command.CommandContainer.update(CommandContainer.java:102)"
"org.h2.command.Command.executeUpdate(Command.java:261)"
"org.h2.jdbc.JdbcPreparedStatement.executeUpdateInternal(JdbcPreparedStatement.java:199)"
"org.h2.jdbc.JdbcPreparedStatement.executeUpdate(JdbcPreparedStatement.java:153)"
"com.mchange.v2.c3p0.impl.NewProxyPreparedStatement.executeUpdate(NewProxyPreparedStatement.java:410)"
"clojure.java.jdbc$db_do_execute_prepared_return_keys$exec_and_return_keys__14493.invoke(jdbc.clj:961)"
"clojure.java.jdbc$db_do_execute_prepared_return_keys.invokeStatic(jdbc.clj:989)"
"clojure.java.jdbc$db_do_execute_prepared_return_keys.invoke(jdbc.clj:945)"
"clojure.java.jdbc$db_do_prepared_return_keys.invokeStatic(jdbc.clj:1020)"
"clojure.java.jdbc$db_do_prepared_return_keys.invoke(jdbc.clj:997)"
"clojure.java.jdbc$db_do_prepared_return_keys.invokeStatic(jdbc.clj:1022)"
"clojure.java.jdbc$db_do_prepared_return_keys.invoke(jdbc.clj:997)"
"toucan.db$simple_insert_many_BANG_$iter__17393__17397$fn__17398.invoke(db.clj:547)"
"clojure.lang.LazySeq.sval(LazySeq.java:42)"
"clojure.lang.LazySeq.seq(LazySeq.java:51)"
"clojure.lang.Cons.next(Cons.java:39)"
"clojure.lang.RT.next(RT.java:709)"
"clojure.core$next__5371.invokeStatic(core.clj:64)"
"clojure.core$dorun.invokeStatic(core.clj:3142)"
"clojure.core$doall.invokeStatic(core.clj:3148)"
"clojure.core$doall.invoke(core.clj:3148)"
"toucan.db$simple_insert_many_BANG_.invokeStatic(db.clj:544)"
"toucan.db$simple_insert_many_BANG_.invoke(db.clj:530)"
"toucan.db$insert_many_BANG_.invokeStatic(db.clj:563)"
"toucan.db$insert_many_BANG_.invoke(db.clj:550)"
"--> sync.sync_metadata.fields.sync_instances$fn__48056$insert_new_fields_BANG___48061$fn__48062.invoke(sync_instances.clj:45)"
"sync.sync_metadata.fields.sync_instances$fn__48056$insert_new_fields_BANG___48061.invoke(sync_instances.clj:41)"
"sync.sync_metadata.fields.sync_instances$fn__48099$create_or_reactivate_fields_BANG___48104$fn__48105.invoke(sync_instances.clj:68)"
"sync.sync_metadata.fields.sync_instances$fn__48099$create_or_reactivate_fields_BANG___48104.invoke(sync_instances.clj:56)"
"sync.sync_metadata.fields.sync_instances$fn__48126$sync_active_instances_BANG___48131$fn__48132$iter__48133__48137$fn__48138$fn__48143.invoke(sync_instances.clj:104)"
"sync.util$do_with_error_handling.invokeStatic(util.clj:149)"
"sync.util$do_with_error_handling.invoke(util.clj:144)"
"sync.sync_metadata.fields.sync_instances$fn__48126$sync_active_instances_BANG___48131$fn__48132$iter__48133__48137$fn__48138.invoke(sync_instances.clj:99)"
"sync.util$sum_for_STAR_.invokeStatic(util.clj:462)"
"sync.util$sum_for_STAR_.invoke(util.clj:459)"
"sync.sync_metadata.fields.sync_instances$fn__48126$sync_active_instances_BANG___48131$fn__48132.invoke(sync_instances.clj:98)"
"sync.sync_metadata.fields.sync_instances$fn__48126$sync_active_instances_BANG___48131.invoke(sync_instances.clj:85)"
"sync.sync_metadata.fields.sync_instances$fn__48310$sync_instances_BANG___48319$fn__48322.invoke(sync_instances.clj:193)"
"sync.sync_metadata.fields.sync_instances$fn__48310$sync_instances_BANG___48319.invoke(sync_instances.clj:178)"
"sync.sync_metadata.fields.sync_instances$fn__48310$sync_instances_BANG___48319$fn__48320.invoke(sync_instances.clj:184)"
"sync.sync_metadata.fields.sync_instances$fn__48310$sync_instances_BANG___48319.invoke(sync_instances.clj:178)"
"sync.sync_metadata.fields$fn__48478$sync_and_update_BANG___48483$fn__48484.invoke(fields.clj:76)"
"sync.sync_metadata.fields$fn__48478$sync_and_update_BANG___48483.invoke(fields.clj:72)"
"sync.sync_metadata.fields$fn__48506$sync_fields_for_table_BANG___48515$fn__48521$fn__48523.invoke(fields.clj:102)"
"sync.util$do_with_error_handling.invokeStatic(util.clj:149)"
"sync.util$do_with_error_handling.invoke(util.clj:144)"
"sync.sync_metadata.fields$fn__48506$sync_fields_for_table_BANG___48515$fn__48521.invoke(fields.clj:90)"
"sync.sync_metadata.fields$fn__48506$sync_fields_for_table_BANG___48515.invoke(fields.clj:83)"
"sync.sync_metadata.fields$fn__48551$sync_fields_BANG___48556$fn__48557$iter__48558__48562$fn__48563$fn__48564.invoke(fields.clj:117)"
"sync.sync_metadata.fields$fn__48551$sync_fields_BANG___48556$fn__48557$iter__48558__48562$fn__48563.invoke(fields.clj:116)"
"sync.sync_metadata.fields$fn__48551$sync_fields_BANG___48556$fn__48557.invoke(fields.clj:116)"
"sync.sync_metadata.fields$fn__48551$sync_fields_BANG___48556.invoke(fields.clj:111)"
"sync.util$fn__39397$run_step_with_metadata__39402$fn__39406$fn__39408.invoke(util.clj:356)"
"sync.util$with_start_and_finish_logging_SINGLEQUOTE_.invokeStatic(util.clj:104)"
"sync.util$with_start_and_finish_logging_SINGLEQUOTE_.invoke(util.clj:98)"
"sync.util$with_start_and_finish_debug_logging.invokeStatic(util.clj:121)"
"sync.util$with_start_and_finish_debug_logging.invoke(util.clj:118)"
"sync.util$fn__39397$run_step_with_metadata__39402$fn__39406.invoke(util.clj:353)"
"sync.util$fn__39397$run_step_with_metadata__39402.invoke(util.clj:348)"
"sync.util$fn__39589$run_sync_operation__39594$fn__39595$fn__39596.invoke(util.clj:441)"
"sync.util$fn__39589$run_sync_operation__39594$fn__39595.invoke(util.clj:441)"
"sync.util$fn__39589$run_sync_operation__39594.invoke(util.clj:435)"
"sync.sync_metadata$fn__49238$sync_db_metadata_BANG___49243$fn__49244$fn__49245.invoke(sync_metadata.clj:51)"
"sync.util$do_with_error_handling.invokeStatic(util.clj:149)"
"sync.util$do_with_error_handling.invoke(util.clj:144)"
"sync.util$do_with_error_handling.invokeStatic(util.clj:147)"
"sync.util$do_with_error_handling.invoke(util.clj:144)"
"driver$fn__18264.invokeStatic(driver.clj:646)"
"driver$fn__18264.invoke(driver.clj:646)"
"sync.util$sync_in_context$fn__39303.invoke(util.clj:140)"
"sync.util$with_db_logging_disabled$fn__39300.invoke(util.clj:131)"
"sync.util$with_start_and_finish_logging_SINGLEQUOTE_.invokeStatic(util.clj:104)"
"sync.util$with_start_and_finish_logging_SINGLEQUOTE_.invoke(util.clj:98)"
"sync.util$with_start_and_finish_logging$fn__39289.invoke(util.clj:116)"
"sync.util$with_sync_events$fn__39284.invoke(util.clj:90)"
"sync.util$with_duplicate_ops_prevented$fn__39275.invoke(util.clj:69)"
"sync.util$do_sync_operation.invokeStatic(util.clj:168)"
"sync.util$do_sync_operation.invoke(util.clj:165)"
"sync.sync_metadata$fn__49238$sync_db_metadata_BANG___49243$fn__49244.invoke(sync_metadata.clj:50)"
"sync.sync_metadata$fn__49238$sync_db_metadata_BANG___49243.invoke(sync_metadata.clj:47)"
"sync$fn__49288$sync_database_BANG___49293$fn__49294$fn__49295.invoke(sync.clj:28)"
"sync.util$do_with_error_handling.invokeStatic(util.clj:149)"
"sync.util$do_with_error_handling.invoke(util.clj:144)"
"sync.util$do_with_error_handling.invokeStatic(util.clj:147)"
"sync.util$do_with_error_handling.invoke(util.clj:144)"
"driver$fn__18264.invokeStatic(driver.clj:646)"
"driver$fn__18264.invoke(driver.clj:646)"
"sync.util$sync_in_context$fn__39303.invoke(util.clj:140)"
"sync.util$with_db_logging_disabled$fn__39300.invoke(util.clj:131)"
"sync.util$with_start_and_finish_logging_SINGLEQUOTE_.invokeStatic(util.clj:104)"
"sync.util$with_start_and_finish_logging_SINGLEQUOTE_.invoke(util.clj:98)"
"sync.util$with_start_and_finish_logging$fn__39289.invoke(util.clj:116)"
"sync.util$with_sync_events$fn__39284.invoke(util.clj:90)"
"sync.util$with_duplicate_ops_prevented$fn__39275.invoke(util.clj:69)"
"sync.util$do_sync_operation.invokeStatic(util.clj:168)"
"sync.util$do_sync_operation.invoke(util.clj:165)"
"sync$fn__49288$sync_database_BANG___49293$fn__49294.invoke(sync.clj:26)"
"sync$fn__49288$sync_database_BANG___49293.invoke(sync.clj:19)"
"events.sync_database$process_sync_database_event$fn__64936.invoke(sync_database.clj:36)")

I've seen a bug reported on the metabase project metabase/metabase#11002 but I'm wondering if the full Enum definition is really needed. If not, could we mitigate the problem directly at the driver by just setting the type as Enum8 or Enum16?

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions