nifi-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mark Payne <marka...@hotmail.com>
Subject Re: QueryRecord error
Date Mon, 02 Oct 2017 16:29:16 GMT
Joe,

Thanks. I was going to suggest we spawn off a new thread if this continued with more back-and-forth.

I did notice that in the JIRA you indicated that
you are using a CSVReader service to read the data. In the message, it indicates that the
FlowFIle's
filename is "billing-detailed-line-items-with-resources-and-tags-2017-10.csv.avro" -- which
would imply
to me that it is Avro (perhaps converted from CSV to avro upstream?) Is it possible that you
just selected
the wrong reader?

Thanks
-Mark

> On Oct 2, 2017, at 12:24 PM, Joe Gresock <jgresock@gmail.com> wrote:
> 
> I'm changing the subject name so as not to muddy the [VOTE] thread.
> 
> I finally found the error in the logs.  This was also not happening prior
> to the upgrade.  I thinkI'll debug the server to determine the root cause:
> 
> 2017-10-02 16:09:47,004 ERROR [Timer-Driven Process Thread-81]
> o.a.nifi.processors.standard.QueryRecord
> QueryRecord[id=21570e18-015c-1000-0000-000058832883] Unable to query
> StandardFlowFileRecord[uuid=2da6d0f9-3856-420e-99bd-9589f7f74911,claim=StandardContentClaim
> [resourceClaim=StandardResourceClaim[id=1506960462126-919,
> container=default, section=919], offset=55116,
> length=616757],offset=0,name=billing-detailed-line-items-with-resources-and-tags-2017-10.csv.avro,size=616757]
> due to org.apache.nifi.processor.exception.ProcessException: Failed to read
> next record in stream for
> StandardFlowFileRecord[uuid=2da6d0f9-3856-420e-99bd-9589f7f74911,claim=StandardContentClaim
> [resourceClaim=StandardResourceClaim[id=1506960462126-919,
> container=default, section=919], offset=55116,
> length=616757],offset=0,name=billing-detailed-line-items-with-resources-and-tags-2017-10.csv.avro,size=616757]:
> {}
> org.apache.nifi.processor.exception.ProcessException: Failed to read next
> record in stream for
> StandardFlowFileRecord[uuid=2da6d0f9-3856-420e-99bd-9589f7f74911,claim=StandardContentClaim
> [resourceClaim=StandardResourceClaim[id=1506960462126-919,
> container=default, section=919], offset=55116,
> length=616757],offset=0,name=billing-detailed-line-items-with-resources-and-tags-2017-10.csv.avro,size=616757]
>        at
> org.apache.nifi.queryrecord.FlowFileEnumerator.moveNext(FlowFileEnumerator.java:65)
>        at Baz$1$1.moveNext(Unknown Source)
>        at
> org.apache.calcite.linq4j.Linq4j$EnumeratorIterator.<init>(Linq4j.java:664)
>        at
> org.apache.calcite.linq4j.Linq4j.enumeratorIterator(Linq4j.java:98)
>        at
> org.apache.calcite.linq4j.AbstractEnumerable.iterator(AbstractEnumerable.java:33)
>        at
> org.apache.calcite.avatica.MetaImpl.createCursor(MetaImpl.java:89)
>        at
> org.apache.calcite.avatica.AvaticaResultSet.execute(AvaticaResultSet.java:196)
>        at
> org.apache.calcite.jdbc.CalciteResultSet.execute(CalciteResultSet.java:67)
>        at
> org.apache.calcite.jdbc.CalciteResultSet.execute(CalciteResultSet.java:44)
>        at
> org.apache.calcite.avatica.AvaticaConnection.executeQueryInternal(AvaticaConnection.java:513)
>        at
> org.apache.calcite.avatica.AvaticaPreparedStatement.executeQuery(AvaticaPreparedStatement.java:132)
>        at
> org.apache.nifi.processors.standard.QueryRecord.queryWithCache(QueryRecord.java:445)
>        at
> org.apache.nifi.processors.standard.QueryRecord.onTrigger(QueryRecord.java:276)
>        at
> org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27)
>        at
> org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1119)
>        at
> org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:147)
>        at
> org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:47)
>        at
> org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:128)
>        at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
>        at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
>        at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
>        at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
>        at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>        at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>        at java.lang.Thread.run(Thread.java:748)
> 
> 
> On Mon, Oct 2, 2017 at 4:03 PM, Mark Payne <markap14@hotmail.com> wrote:
> 
>> Hey Joe,
>> 
>> I've seen this error crop up a time or two before. The issue is a bug in
>> how we are handling a failure
>> condition. Haven't yet figured out the best way to tackle it, but the
>> thing to note is that this error occurs
>> when there is some other sort of failure -- typically, a failure to obtain
>> the schema for the Record Reader
>> or Record Writer. Can you check your logs for any other errors that may be
>> occurring?
>> 
>> Thanks
>> -Mark
>> 
>> 
>>> On Oct 2, 2017, at 11:32 AM, Joe Gresock <jgresock@gmail.com> wrote:
>>> 
>>> I have to change my non-binding +1 vote to a 0, unless there is a
>>> workaround for the bug [1] I just submitted, regarding the QueryRecord
>>> processor being broken.
>>> 
>>> 
>>> [1] https://issues.apache.org/jira/browse/NIFI-4452
>>> 
>>> 
>>> On Mon, Oct 2, 2017 at 3:23 AM, Kevin Doran <kdoran.apache@gmail.com>
>> wrote:
>>> 
>>>> +1, non-binding
>>>> 
>>>> - Verified hashes
>>>> - Full build with contrib-check passes
>>>> - Was able to run nifi server and send flow files to remote input port
>>>> over s2s protocol. Web UI and API working as expected.
>>>> 
>>>> Platform:
>>>> Apache Maven 3.5.0 (ff8f5e7444045639af65f6095c62210b5713f426;
>>>> 2017-04-03T15:39:06-04:00)
>>>> Java version: 1.8.0_131, vendor: Oracle Corporation
>>>> OS name: "mac os x", version: "10.12.6", arch: "x86_64", family: "mac"
>>>> 
>>>> Looks good! Nice work everyone!
>>>> 
>>>> Kevin
>>>> 
>>>> On 9/28/17, 15:54, "Jeff" <jstorck@apache.org> wrote:
>>>> 
>>>>   Hello,
>>>> 
>>>>   I am pleased to be calling this vote for the source release of Apache
>>>> NiFi
>>>>   nifi-1.4.0.
>>>> 
>>>>   The source zip, including signatures, digests, etc. can be found at:
>>>>   https://repository.apache.org/content/repositories/
>> orgapachenifi-1111
>>>> 
>>>>   The Git tag is nifi-1.4.0-RC2
>>>>   The Git commit ID is e6508ba7d3da5bba54abd6233a7a8f9dd4c32151
>>>>   https://git-wip-us.apache.org/repos/asf?p=nifi.git;a=commit;h=
>>>> e6508ba7d3da5bba54abd6233a7a8f9dd4c32151
>>>> 
>>>>   Checksums of nifi-1.4.0-source-release.zip:
>>>>   MD5: 41e4083e602883a3e180032f32913414
>>>>   SHA1: 26770625138126f45bed4989adb0a6b65a767aa2
>>>> 
>>>>   Release artifacts are signed with the following key:
>>>>   https://people.apache.org/keys/committer/jstorck.asc
>>>> 
>>>>   KEYS file available here:
>>>>   https://dist.apache.org/repos/dist/release/nifi/KEYS
>>>> 
>>>>   199 issues were closed/resolved for this release:
>>>>   https://issues.apache.org/jira/secure/ReleaseNote.jspa?
>>>> projectId=12316020&version=12340589
>>>> 
>>>>   Release note highlights can be found here:
>>>>   https://cwiki.apache.org/confluence/display/NIFI/
>>>> Release+Notes#ReleaseNotes-Version-1.4.0
>>>> 
>>>>   The vote will be open for 72 hours.
>>>>   Please download the release candidate and evaluate the necessary
>> items
>>>>   including checking hashes, signatures, build
>>>>   from source, and test.  The please vote:
>>>> 
>>>>   [ ] +1 Release this package as nifi-1.4.0
>>>>   [ ] +0 no opinion
>>>>   [ ] -1 Do not release this package because...
>>>> 
>>>> 
>>>> 
>>>> 
>>> 
>>> 
>>> --
>>> I know what it is to be in need, and I know what it is to have plenty.  I
>>> have learned the secret of being content in any and every situation,
>>> whether well fed or hungry, whether living in plenty or in want.  I can
>> do
>>> all this through him who gives me strength.    *-Philippians 4:12-13*
>> 
>> 
> 
> 
> -- 
> I know what it is to be in need, and I know what it is to have plenty.  I
> have learned the secret of being content in any and every situation,
> whether well fed or hungry, whether living in plenty or in want.  I can do
> all this through him who gives me strength.    *-Philippians 4:12-13*


Mime
View raw message