nifi-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Nicolas Delsaux (Jira)" <j...@apache.org>
Subject [jira] [Updated] (NIFI-6422) Handling of field type in BigQuery table schema definition is incorrect
Date Thu, 03 Oct 2019 09:49:00 GMT

     [ https://issues.apache.org/jira/browse/NIFI-6422?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Nicolas Delsaux updated NIFI-6422:
----------------------------------
    Description: 
When setting the field type to "record" in BigQuery schema definition, the following exception
is thrown

{{}}{{2019-07-03 08:35:24,964 ERROR [Timer-Driven Process Thread-8] o.a.n.p.gcp.bigquery.PutBigQueryBatch
PutBigQueryBatch[id=b2b1c6bf-016b-1000-e8c9-b3f9fb5b417e] null: java.lang.NullPointerException}}
 {{java.lang.NullPointerException: null}}
 {{    at org.apache.nifi.processors.gcp.bigquery.BigQueryUtils.mapToField(BigQueryUtils.java:42)}}
 {{    at org.apache.nifi.processors.gcp.bigquery.BigQueryUtils.listToFields(BigQueryUtils.java:68)}}
 {{    at org.apache.nifi.processors.gcp.bigquery.BigQueryUtils.schemaFromString(BigQueryUtils.java:80)}}
 {{    at org.apache.nifi.processors.gcp.bigquery.PutBigQueryBatch.onTrigger(PutBigQueryBatch.java:277)}}
 {{    at org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27)}}
 {{    at org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1162)}}
 {{    at org.apache.nifi.controller.tasks.ConnectableTask.invoke(ConnectableTask.java:209)}}
 {{    at org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:117)}}
 {{    at org.apache.nifi.engine.FlowEngine$2.run(FlowEngine.java:110)}}
 {{    at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)}}
 {{    at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)}}
 {{    at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)}}
 {{    at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)}}
 {{    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)}}
 {{    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)}}
 {{    at java.lang.Thread.run(Thread.java:748)}}

 

This seems to happen because the type handling is ... not so good

 

Beside, it seems like the TYPE case is not handled correctly, and RECORD type is absolutly
not handled.

  was:
When setting the field type to "record" in BigQuery schema definition, the following exception
is thrown

{{}}{{2019-07-03 08:35:24,964 ERROR [Timer-Driven Process Thread-8] o.a.n.p.gcp.bigquery.PutBigQueryBatch
PutBigQueryBatch[id=b2b1c6bf-016b-1000-e8c9-b3f9fb5b417e] null: java.lang.NullPointerException}}
{{java.lang.NullPointerException: null}}
{{    at org.apache.nifi.processors.gcp.bigquery.BigQueryUtils.mapToField(BigQueryUtils.java:42)}}
{{    at org.apache.nifi.processors.gcp.bigquery.BigQueryUtils.listToFields(BigQueryUtils.java:68)}}
{{    at org.apache.nifi.processors.gcp.bigquery.BigQueryUtils.schemaFromString(BigQueryUtils.java:80)}}
{{    at org.apache.nifi.processors.gcp.bigquery.PutBigQueryBatch.onTrigger(PutBigQueryBatch.java:277)}}
{{    at org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27)}}
{{    at org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1162)}}
{{    at org.apache.nifi.controller.tasks.ConnectableTask.invoke(ConnectableTask.java:209)}}
{{    at org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:117)}}
{{    at org.apache.nifi.engine.FlowEngine$2.run(FlowEngine.java:110)}}
{{    at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)}}
{{    at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)}}
{{    at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)}}
{{    at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)}}
{{    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)}}
{{    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)}}
{{    at java.lang.Thread.run(Thread.java:748)}}

 

This seems to happen because the type handling is ... not so good


> Handling of field type in BigQuery table schema definition is incorrect
> -----------------------------------------------------------------------
>
>                 Key: NIFI-6422
>                 URL: https://issues.apache.org/jira/browse/NIFI-6422
>             Project: Apache NiFi
>          Issue Type: Bug
>          Components: Extensions
>    Affects Versions: 1.9.2
>            Reporter: Nicolas Delsaux
>            Priority: Major
>              Labels: easyfix, pull-request-available
>          Time Spent: 3h 40m
>  Remaining Estimate: 0h
>
> When setting the field type to "record" in BigQuery schema definition, the following
exception is thrown
> {{}}{{2019-07-03 08:35:24,964 ERROR [Timer-Driven Process Thread-8] o.a.n.p.gcp.bigquery.PutBigQueryBatch
PutBigQueryBatch[id=b2b1c6bf-016b-1000-e8c9-b3f9fb5b417e] null: java.lang.NullPointerException}}
>  {{java.lang.NullPointerException: null}}
>  {{    at org.apache.nifi.processors.gcp.bigquery.BigQueryUtils.mapToField(BigQueryUtils.java:42)}}
>  {{    at org.apache.nifi.processors.gcp.bigquery.BigQueryUtils.listToFields(BigQueryUtils.java:68)}}
>  {{    at org.apache.nifi.processors.gcp.bigquery.BigQueryUtils.schemaFromString(BigQueryUtils.java:80)}}
>  {{    at org.apache.nifi.processors.gcp.bigquery.PutBigQueryBatch.onTrigger(PutBigQueryBatch.java:277)}}
>  {{    at org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27)}}
>  {{    at org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1162)}}
>  {{    at org.apache.nifi.controller.tasks.ConnectableTask.invoke(ConnectableTask.java:209)}}
>  {{    at org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:117)}}
>  {{    at org.apache.nifi.engine.FlowEngine$2.run(FlowEngine.java:110)}}
>  {{    at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)}}
>  {{    at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)}}
>  {{    at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)}}
>  {{    at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)}}
>  {{    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)}}
>  {{    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)}}
>  {{    at java.lang.Thread.run(Thread.java:748)}}
>  
> This seems to happen because the type handling is ... not so good
>  
> Beside, it seems like the TYPE case is not handled correctly, and RECORD type is absolutly
not handled.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Mime
View raw message