Mailing-List: contact drill-user-help@incubator.apache.org; run by ezmlm
Precedence: bulk
Reply-To: drill-user@incubator.apache.org
Received-SPF: pass (athena.apache.org: domain of backmeupone@gmail.com
 designates 74.125.82.174 as permitted sender)
MIME-Version: 1.0
In-Reply-To: 
 <CAK55V=++0dddexhYudqUKVKn18s=VNQ6K9nvyDMj0evWGq1Ebw@mail.gmail.com>
References: 
 <CAK55V=++0dddexhYudqUKVKn18s=VNQ6K9nvyDMj0evWGq1Ebw@mail.gmail.com>
Date: Mon, 2 Dec 2013 15:15:56 -0800
Message-ID: 
 <CAFUp1YB3Z53Pa0j6qZHd+TSP=Wj1RnxXgm0EUGaM14Rn4aMxVg@mail.gmail.com>
Subject: Re: How to load data in Drill
From: Madhu Borkar <backmeupone@gmail.com>
To: drill-user@incubator.apache.org
Content-Type: multipart/alternative; boundary=047d7bfcfc98ae16ba04ec955c3b

--047d7bfcfc98ae16ba04ec955c3b
Content-Type: text/plain; charset=ISO-8859-1

Hi Jinfeng,

Here is the part of the log you are looking for
18:08:12.905 [WorkManager-2] DEBUG o.a.d.e.work.batch.IncomingBuffers -
Came up with a list of 0 required fragments.  Fragments {}
18:08:16.181 [WorkManager Event Thread] DEBUG
o.apache.drill.exec.work.WorkManager - Starting pending task
org.apache.drill.exec.work.FragmentRunner@2122d9d0
18:08:16.184 [WorkManager-3] DEBUG o.a.drill.exec.work.FragmentRunner -
Starting fragment runner. 0:0
18:08:16.188 [WorkManager-3] DEBUG o.a.d.e.w.f.RunningFragmentManager - New
fragment status was provided to Foreman of memory_use: 0
batches_completed: 0
records_completed: 0
state: RUNNING
data_processed: 0
handle {
  query_id {
    part1: -3386430666417617918
    part2: -5241641154650077119
  }
  major_fragment_id: 0
  minor_fragment_id: 0
}
running_time: 429655087179513

18:08:16.237 [WorkManager-3] DEBUG o.a.d.e.s.p.ParquetRecordReader -
records to read in this pass: 4000
18:08:16.339 [WorkManager-3] DEBUG o.a.drill.exec.work.FragmentRunner -
Caught exception while running fragment
java.lang.NullPointerException: null
        at
org.apache.drill.exec.store.parquet.PageReadStatus.next(PageReadStatus.java:92)
~[java-exec-1.0.0-m1-rebuffed.jar:1.0.0-m1]
        at
org.apache.drill.exec.store.parquet.VarLenBinaryReader.readFields(VarLenBinaryReader.java:124)
~[java-exec-1.0.0-m1-rebuffed.jar:1.0.0-m1]
        at
org.apache.drill.exec.store.parquet.ParquetRecordReader.next(ParquetRecordReader.java:386)
~[java-exec-1.0.0-m1-rebuffed.jar:1.0.0-m1]
        at
org.apache.drill.exec.physical.impl.ScanBatch.next(ScanBatch.java:95)
~[java-exec-1.0.0-m1-rebuffed.jar:1.0.0-m1]
        at
org.apache.drill.exec.physical.impl.ScreenCreator$ScreenRoot.next(ScreenCreator.java:77)
~[java-exec-1.0.0-m1-rebuffed.jar:1.0.0-m1]
        at
org.apache.drill.exec.work.FragmentRunner.run(FragmentRunner.java:79)
~[java-exec-1.0.0-m1-rebuffed.jar:1.0.0-m1]
        at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
[na:1.7.0_45]
        at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
[na:1.7.0_45]
        at java.lang.Thread.run(Thread.java:744) [na:1.7.0_45]
18:08:16.341 [WorkManager-3] ERROR o.a.d.e.w.AbstractFragmentRunnerListener
- Error b7fa738a-1d3a-4b06-acb6-226a9744dbb7: Failure while running
fragment.
java.lang.NullPointerException: null
        at
org.apache.drill.exec.store.parquet.PageReadStatus.next(PageReadStatus.java:92)
~[java-exec-1.0.0-m1-rebuffed.jar:1.0.0-m1]
        at
org.apache.drill.exec.store.parquet.VarLenBinaryReader.readFields(VarLenBinaryReader.java:124)
~[java-exec-1.0.0-m1-rebuffed.jar:1.0.0-m1]
        at
org.apache.drill.exec.store.parquet.ParquetRecordReader.next(ParquetRecordReader.java:386)
~[java-exec-1.0.0-m1-rebuffed.jar:1.0.0-m1]
        at
org.apache.drill.exec.physical.impl.ScanBatch.next(ScanBatch.java:95)
~[java-exec-1.0.0-m1-rebuffed.jar:1.0.0-m1]
        at
org.apache.drill.exec.physical.impl.ScreenCreator$ScreenRoot.next(ScreenCreator.java:77)
~[java-exec-1.0.0-m1-rebuffed.jar:1.0.0-m1]
        at
org.apache.drill.exec.work.FragmentRunner.run(FragmentRunner.java:79)
~[java-exec-1.0.0-m1-rebuffed.jar:1.0.0-m1]
        at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
[na:1.7.0_45]
        at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
[na:1.7.0_45]
        at java.lang.Thread.run(Thread.java:744) [na:1.7.0_45]
18:08:16.344 [WorkManager-3] DEBUG o.a.d.e.w.f.RunningFragmentManager - New
fragment status was provided to Foreman of memory_use: 0
batches_completed:
records_completed: 0
state: FAILED
data_processed: 0
handle {
  query_id {
    part1: -3386430666417617918
    part2: -5241641154650077119
  }
  major_fragment_id: 0
  minor_fragment_id: 0
}
error {
  error_id: "b7fa738a-1d3a-4b06-acb6-226a9744dbb7"
  endpoint {
    address: "Node-0.etouch.net"
    user_port: 31010
    bit_port: 32011
  }
  error_type: 0
  message: "Failure while running fragment. < NullPointerException"
}
running_time: 155084239

18:08:16.346 [WorkManager-3] DEBUG o.a.drill.exec.rpc.user.UserServer -
Sending result to client with QueryWritableBatch [header=query_state: FAILED
query_id {
  part1: -3386430666417617918
  part2: -5241641154650077119
}
error {
  error_id: "b7fa738a-1d3a-4b06-acb6-226a9744dbb7"
  endpoint {
    address: "Node-0.etouch.net"
    user_port: 31010
    bit_port: 32011
  }
  error_type: 0
  message: "Failure while running fragment. < NullPointerException"
}
, buffers=[]]
18:08:16.351 [WorkManager-3] DEBUG o.a.drill.exec.work.FragmentRunner -
Fragment runner complete. 0:

Please, let me know if this one helps!


On Sun, Dec 1, 2013 at 10:34 PM, Jinfeng Ni <jinfengni99@gmail.com> wrote:

> Hi Tom and Madhu,
>
> Regarding the NullPointerException you encountered when you run the query
> in sqlline:
>
> SELECT * FROM some_parquet_file;
>
> Could you please post some debug log in the sqlline's log?  If you use
> sqlline in drill's binary distribution, sqlline's log would be in
> /var/log/drill/sqlline.log.  Please search keyword "FragmentRunner" and
> "Caught exception".  The sqlline.log should contain a call stack when the
> NullPointerException is threw.  For instance, here is the log for an
> IndexOutBoundaryException in my sqlline.log:
>
> 21:44:40.984 [WorkManager-4] DEBUG o.a.drill.exec.work.FragmentRunner -
> Caught exception while running fragment
> java.lang.IndexOutOfBoundsException: index: 31999268, length: 4 (expected:
> range(0, 4194244))
>         at
> io.netty.buffer.AbstractByteBuf.checkIndex(AbstractByteBuf.java:1130)
> ~[netty-buffer-4.0.7.Final.jar:na]
>         at io.netty.buffer.AbstractByteBuf.getInt(AbstractByteBuf.java:378)
> ~[netty-buffer-4.0.7.Final.jar:na]
>         at
> org.apache.drill.exec.vector.UInt4Vector$Accessor.get(UInt4Vector.java:188)
>
> ~[drill-java-exec-1.0.0-m2-incubating-SNAPSHOT-rebuffed.jar:1.0.0-m2-incubating-SNAPSHOT]
>         at
>
> org.apache.drill.exec.vector.VarBinaryVector$Mutator.setValueCount(VarBinaryVector.java:355)
>
> ~[drill-java-exec-1.0.0-m2-incubating-SNAPSHOT-rebuffed.jar:1.0.0-m2-incubating-SNAPSHOT]
>         at
>
> org.apache.drill.exec.physical.impl.svremover.RemovingRecordBatch.doWork(RemovingRecordBatch.java:92)
>
> ~[drill-java-exec-1.0.0-m2-incubating-SNAPSHOT-rebuffed.jar:1.0.0-m2-incubating-SNAPSHOT]
>         at
>
> org.apache.drill.exec.record.AbstractSingleRecordBatch.next(AbstractSingleRecordBatch.java:63)
>
> ~[drill-java-exec-1.0.0-m2-incubating-SNAPSHOT-rebuffed.jar:1.0.0-m2-incubating-SNAPSHOT]
>         at
>
> org.apache.drill.exec.record.AbstractSingleRecordBatch.next(AbstractSingleRecordBatch.java:42)
>
> ~[drill-java-exec-1.0.0-m2-incubating-SNAPSHOT-rebuffed.jar:1.0.0-m2-incubating-SNAPSHOT]
>         at
>
> org.apache.drill.exec.record.AbstractSingleRecordBatch.next(AbstractSingleRecordBatch.java:42)
>
> ~[drill-java-exec-1.0.0-m2-incubating-SNAPSHOT-rebuffed.jar:1.0.0-m2-incubating-SNAPSHOT]
>         at
>
> org.apache.drill.exec.physical.impl.limit.LimitRecordBatch.next(LimitRecordBatch.java:89)
>
> ~[drill-java-exec-1.0.0-m2-incubating-SNAPSHOT-rebuffed.jar:1.0.0-m2-incubating-SNAPSHOT]
>         at
>
> org.apache.drill.exec.record.AbstractSingleRecordBatch.next(AbstractSingleRecordBatch.java:42)
>
> ~[drill-java-exec-1.0.0-m2-incubating-SNAPSHOT-rebuffed.jar:1.0.0-m2-incubating-SNAPSHOT]
>         at
>
> org.apache.drill.exec.physical.impl.ScreenCreator$ScreenRoot.next(ScreenCreator.java:77)
>
> ~[drill-java-exec-1.0.0-m2-incubating-SNAPSHOT-rebuffed.jar:1.0.0-m2-incubating-SNAPSHOT]
>         at
> org.apache.drill.exec.work.FragmentRunner.run(FragmentRunner.java:79)
>
> ~[drill-java-exec-1.0.0-m2-incubating-SNAPSHOT-rebuffed.jar:1.0.0-m2-incubating-SNAPSHOT]
>         at
>
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> [na:1.7.0_45]
>         at
>
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> [na:1.7.0_45]
>         at java.lang.Thread.run(Thread.java:744) [na:1.7.0_45]
> 21:44:40.990 [WorkManager-4] ERROR o.a.d.e.w.AbstractFragmentRunnerListener
> - Error c8efdbf1-9a6f-427c-ab90-ce16002904af: Failure while running
> fragment.
>
> I need the call stack when the NPE is threw, to see what went wrong for
> your query.
>
> The call stack that you posted (started from
>
>
> org.apache.drill.exec.rpc.user.QueryResultHandler.batchArrived(QueryResultHandler.java:72)
> ) is when the Query Result Lister detects
>
> an exception has been threw.
>
> Thanks!
>
> Jinfeng
>

--047d7bfcfc98ae16ba04ec955c3b--