asterixdb-notifications mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Wenhai (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (ASTERIXDB-1412) Batch import errors
Date Mon, 25 Apr 2016 07:44:12 GMT

     [ https://issues.apache.org/jira/browse/ASTERIXDB-1412?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Wenhai updated ASTERIXDB-1412:
------------------------------
    Description: 
When we importing about 2 million ACM datasets with the following schema, we got a data fields
mismatching error. While we splitting the records into 6 parts, the error disappears. Always,
we put the dataset onto the following site for verifying.
Dataset
{noformat}
http://pan.baidu.com/s/1c26Y9LQ
{noformat}
Schema
{noformat}
drop dataverse fuzzytest if exists;

create dataverse fuzzytest;

use dataverse fuzzytest;

create type PaperType as open {
  tid:uuid,
  title: string,
  authors: string?,
  year: int?,
  conf: string?,
  idx: string,
  abstract: string?
}
{noformat}
Splitting import successfully
{noformat}

use dataverse test;
create dataset ACM(PaperType) primary key tid autogenerated;
load dataset ACM
using localfs
(("path"="127.0.0.1:///home/hadoop/Downloads/doccorpus/acm_split.aa,127.0.0.1:///home/hadoop/Downloads/doccorpus/acm_split.ab,127.0.0.1:///home/hadoop/Downloads/doccorpus/acm_split.ac,127.0.0.1:///home/hadoop/Downloads/doccorpus/acm_split.ad,127.0.0.1:///home/hadoop/Downloads/doccorpus/acm_split.ae"),("format"="delimited-text"),("delimiter"="#"),("quote"="\u0000"));
{noformat}

Batch Import with failure
{noformat}
use dataverse test;
drop dataset ACM if exists;
create dataset ACM(PaperType) primary key tid autogenerated;
load dataset ACM
using localfs
(("path"="127.0.0.1:///home/hadoop/Downloads/doccorpus/reproduce/acm_raw.txt"),("format"="delimited-text"),("delimiter"="#"),("quote"="\u0000"));
{noformat}

Error message
{noformat}
Long Parser - a digit is expected. But, encountered this character: V in the incoming input:
VLSID 05 Proceedings of the 18th International Conference on VLSI Design held jointly with
4th International Conference on Embedded Systems Design [HyracksDataException]
{noformat}

Error trace
{noformat}
org.apache.hyracks.api.exceptions.HyracksDataException: java.util.concurrent.ExecutionException:
org.apache.hyracks.api.exceptions.HyracksDataException: org.apache.hyracks.api.exceptions.HyracksDataException:
org.apache.hyracks.api.exceptions.HyracksDataException: Long Parser - a digit is expected.
But, encountered this character: V in the incoming input: VLSID 05 Proceedings of the 18th
International Conference on VLSI Design held jointly with 4th International Conference on
Embedded Systems Design
        at org.apache.hyracks.api.rewriter.runtime.SuperActivityOperatorNodePushable.runInParallel(SuperActivityOperatorNodePushable.java:218)
        at org.apache.hyracks.api.rewriter.runtime.SuperActivityOperatorNodePushable.initialize(SuperActivityOperatorNodePushable.java:83)
        at org.apache.hyracks.control.nc.Task.run(Task.java:263)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)
Caused by: java.util.concurrent.ExecutionException: org.apache.hyracks.api.exceptions.HyracksDataException:
org.apache.hyracks.api.exceptions.HyracksDataException: org.apache.hyracks.api.exceptions.HyracksDataException:
Long Parser - a digit is expected. But, encountered this character: V in the incoming input:
VLSID 05 Proceedings of the 18th International Conference on VLSI Design held jointly with
4th International Conference on Embedded Systems Design
        at java.util.concurrent.FutureTask.report(FutureTask.java:122)
        at java.util.concurrent.FutureTask.get(FutureTask.java:192)
        at org.apache.hyracks.api.rewriter.runtime.SuperActivityOperatorNodePushable.runInParallel(SuperActivityOperatorNodePushable.java:212)
        ... 5 more
Caused by: org.apache.hyracks.api.exceptions.HyracksDataException: org.apache.hyracks.api.exceptions.HyracksDataException:
org.apache.hyracks.api.exceptions.HyracksDataException: Long Parser - a digit is expected.
But, encountered this character: V in the incoming input: VLSID 05 Proceedings of the 18th
International Conference on VLSI Design held jointly with 4th International Conference on
Embedded Systems Design
        at org.apache.asterix.external.operators.ExternalDataScanOperatorDescriptor$1.initialize(ExternalDataScanOperatorDescriptor.java:65)
        at org.apache.hyracks.api.rewriter.runtime.SuperActivityOperatorNodePushable.lambda$initialize$0(SuperActivityOperatorNodePushable.java:83)
        at org.apache.hyracks.api.rewriter.runtime.SuperActivityOperatorNodePushable$1.call(SuperActivityOperatorNodePushable.java:205)
        at org.apache.hyracks.api.rewriter.runtime.SuperActivityOperatorNodePushable$1.call(SuperActivityOperatorNodePushable.java:202)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        ... 3 more
Caused by: org.apache.hyracks.api.exceptions.HyracksDataException: org.apache.hyracks.api.exceptions.HyracksDataException:
Long Parser - a digit is expected. But, encountered this character: V in the incoming input:
VLSID 05 Proceedings of the 18th International Conference on VLSI Design held jointly with
4th International Conference on Embedded Systems Design
        at org.apache.asterix.external.dataflow.RecordDataFlowController.start(RecordDataFlowController.java:65)
        at org.apache.asterix.external.dataset.adapter.GenericAdapter.start(GenericAdapter.java:37)
        at org.apache.asterix.external.operators.ExternalDataScanOperatorDescriptor$1.initialize(ExternalDataScanOperatorDescriptor.java:62)
        ... 7 more
Caused by: org.apache.hyracks.api.exceptions.HyracksDataException: Long Parser - a digit is
expected. But, encountered this character: V in the incoming input: VLSID 05 Proceedings of
the 18th International Conference on VLSI Design held jointly with 4th International Conference
on Embedded Systems Design
        at org.apache.hyracks.dataflow.common.data.parsers.LongParserFactory$1.parse(LongParserFactory.java:74)
        at org.apache.asterix.external.parser.DelimitedDataParser.parseRecord(DelimitedDataParser.java:144)
        at org.apache.asterix.external.parser.DelimitedDataParser.parse(DelimitedDataParser.java:159)
        at org.apache.asterix.external.dataflow.RecordDataFlowController.start(RecordDataFlowController.java:57)
        ... 9 more
Apr 24, 2016 11:38:19 AM org.apache.hyracks.control.common.work.WorkQueue$WorkerThread run
INFO: Executing: NotifyTaskFailure
Apr 24, 2016 11:38:19 AM org.apache.hyracks.control.common.work.WorkQueue$WorkerThread run
INFO: Executing: AbortTasks
Apr 24, 2016 11:38:19 AM org.apache.hyracks.control.nc.work.AbortTasksWork run
INFO: Aborting Tasks: JID:19:[TAID:TID:ANID:ODID:1:0:12:0, TAID:TID:ANID:ODID:1:0:13:0, TAID:TID:ANID:ODID:1:0:14:0,
TAID:TID:ANID:ODID:1:0:15:0, TAID:TID:ANID:ODID:1:0:16:0, TAID:TID:ANID:ODID:1:0:17:0, TAID:TID:ANID:ODID:1:0:18:0,
TAID:TID:ANID:ODID:1:0:19:0, TAID:TID:ANID:ODID:1:0:20:0, TAID:TID:ANID:ODID:1:0:21:0, TAID:TID:ANID:ODID:1:0:22:0,
TAID:TID:ANID:ODID:1:0:23:0]
java.lang.InterruptedException
        at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1302)
        at java.util.concurrent.Semaphore.acquire(Semaphore.java:467)
        at org.apache.hyracks.control.nc.Task.run(Task.java:299)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)
java.lang.InterruptedException
        at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1302)
        at java.util.concurrent.Semaphore.acquire(Semaphore.java:467)
        at org.apache.hyracks.control.nc.Task.run(Task.java:299)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)
java.lang.InterruptedException
        at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1302)
        at java.util.concurrent.Semaphore.acquire(Semaphore.java:467)
        at org.apache.hyracks.control.nc.Task.run(Task.java:299)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)
java.lang.InterruptedException
        at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1302)
        at java.util.concurrent.Semaphore.acquire(Semaphore.java:467)
        at org.apache.hyracks.control.nc.Task.run(Task.java:299)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)
java.lang.InterruptedException
        at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1302)
        at java.util.concurrent.Semaphore.acquire(Semaphore.java:467)
        at org.apache.hyracks.control.nc.Task.run(Task.java:299)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)
java.lang.InterruptedException
        at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1302)
        at java.util.concurrent.Semaphore.acquire(Semaphore.java:467)
        at org.apache.hyracks.control.nc.Task.run(Task.java:299)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)
java.lang.InterruptedException
        at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1302)
        at java.util.concurrent.Semaphore.acquire(Semaphore.java:467)
        at org.apache.hyracks.control.nc.Task.run(Task.java:299)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)
java.lang.InterruptedException
        at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1302)
        at java.util.concurrent.Semaphore.acquire(Semaphore.java:467)
        at org.apache.hyracks.control.nc.Task.run(Task.java:299)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)
java.lang.InterruptedException
        at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1302)
        at java.util.concurrent.Semaphore.acquire(Semaphore.java:467)
        at org.apache.hyracks.control.nc.Task.run(Task.java:299)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)
java.lang.InterruptedException
        at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1302)
        at java.util.concurrent.Semaphore.acquire(Semaphore.java:467)
        at org.apache.hyracks.control.nc.Task.run(Task.java:299)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)
java.lang.InterruptedException
        at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1302)
        at java.util.concurrent.Semaphore.acquire(Semaphore.java:467)
        at org.apache.hyracks.control.nc.Task.run(Task.java:299)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
 java.lang.InterruptedException
        at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1302)
        at java.util.concurrent.Semaphore.acquire(Semaphore.java:467)
        at org.apache.hyracks.control.nc.Task.run(Task.java:299)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)
Apr 24, 2016 11:38:19 AM org.apache.hyracks.control.common.work.WorkQueue$WorkerThread run
INFO: Executing: CleanupJoblet

{noformat}

  was:
When we importing about 2 million ACM datasets with the following schema, we got a data fields
mismatching error. While we splitting the records into 6 parts, the error disappears. Always,
we put the dataset onto the following site for verifying.
Dataset
{noformat}
http://yun.baidu.com/share/link?shareid=2678954841&uk=4030601168
{noformat}
Schema
{noformat}
drop dataverse fuzzytest if exists;

create dataverse fuzzytest;

use dataverse fuzzytest;

create type PaperType as open {
  tid:uuid,
  title: string,
  authors: string?,
  year: int?,
  conf: string?,
  idx: string,
  abstract: string?
}
{noformat}
Splitting import successfully
{noformat}

use dataverse test;
create dataset ACM(PaperType) primary key tid autogenerated;
load dataset ACM
using localfs
(("path"="127.0.0.1:///home/hadoop/Downloads/doccorpus/acm_split.aa,127.0.0.1:///home/hadoop/Downloads/doccorpus/acm_split.ab,127.0.0.1:///home/hadoop/Downloads/doccorpus/acm_split.ac,127.0.0.1:///home/hadoop/Downloads/doccorpus/acm_split.ad,127.0.0.1:///home/hadoop/Downloads/doccorpus/acm_split.ae"),("format"="delimited-text"),("delimiter"="#"),("quote"="\u0000"));
{noformat}

Batch Import with failure
{noformat}
use dataverse test;
drop dataset ACM if exists;
create dataset ACM(PaperType) primary key tid autogenerated;
load dataset ACM
using localfs
(("path"="127.0.0.1:///home/hadoop/Downloads/doccorpus/reproduce/acm_raw.txt"),("format"="delimited-text"),("delimiter"="#"),("quote"="\u0000"));
{noformat}

Error message
{noformat}
Long Parser - a digit is expected. But, encountered this character: V in the incoming input:
VLSID 05 Proceedings of the 18th International Conference on VLSI Design held jointly with
4th International Conference on Embedded Systems Design [HyracksDataException]
{noformat}

Error trace
{noformat}
org.apache.hyracks.api.exceptions.HyracksDataException: java.util.concurrent.ExecutionException:
org.apache.hyracks.api.exceptions.HyracksDataException: org.apache.hyracks.api.exceptions.HyracksDataException:
org.apache.hyracks.api.exceptions.HyracksDataException: Long Parser - a digit is expected.
But, encountered this character: V in the incoming input: VLSID 05 Proceedings of the 18th
International Conference on VLSI Design held jointly with 4th International Conference on
Embedded Systems Design
        at org.apache.hyracks.api.rewriter.runtime.SuperActivityOperatorNodePushable.runInParallel(SuperActivityOperatorNodePushable.java:218)
        at org.apache.hyracks.api.rewriter.runtime.SuperActivityOperatorNodePushable.initialize(SuperActivityOperatorNodePushable.java:83)
        at org.apache.hyracks.control.nc.Task.run(Task.java:263)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)
Caused by: java.util.concurrent.ExecutionException: org.apache.hyracks.api.exceptions.HyracksDataException:
org.apache.hyracks.api.exceptions.HyracksDataException: org.apache.hyracks.api.exceptions.HyracksDataException:
Long Parser - a digit is expected. But, encountered this character: V in the incoming input:
VLSID 05 Proceedings of the 18th International Conference on VLSI Design held jointly with
4th International Conference on Embedded Systems Design
        at java.util.concurrent.FutureTask.report(FutureTask.java:122)
        at java.util.concurrent.FutureTask.get(FutureTask.java:192)
        at org.apache.hyracks.api.rewriter.runtime.SuperActivityOperatorNodePushable.runInParallel(SuperActivityOperatorNodePushable.java:212)
        ... 5 more
Caused by: org.apache.hyracks.api.exceptions.HyracksDataException: org.apache.hyracks.api.exceptions.HyracksDataException:
org.apache.hyracks.api.exceptions.HyracksDataException: Long Parser - a digit is expected.
But, encountered this character: V in the incoming input: VLSID 05 Proceedings of the 18th
International Conference on VLSI Design held jointly with 4th International Conference on
Embedded Systems Design
        at org.apache.asterix.external.operators.ExternalDataScanOperatorDescriptor$1.initialize(ExternalDataScanOperatorDescriptor.java:65)
        at org.apache.hyracks.api.rewriter.runtime.SuperActivityOperatorNodePushable.lambda$initialize$0(SuperActivityOperatorNodePushable.java:83)
        at org.apache.hyracks.api.rewriter.runtime.SuperActivityOperatorNodePushable$1.call(SuperActivityOperatorNodePushable.java:205)
        at org.apache.hyracks.api.rewriter.runtime.SuperActivityOperatorNodePushable$1.call(SuperActivityOperatorNodePushable.java:202)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        ... 3 more
Caused by: org.apache.hyracks.api.exceptions.HyracksDataException: org.apache.hyracks.api.exceptions.HyracksDataException:
Long Parser - a digit is expected. But, encountered this character: V in the incoming input:
VLSID 05 Proceedings of the 18th International Conference on VLSI Design held jointly with
4th International Conference on Embedded Systems Design
        at org.apache.asterix.external.dataflow.RecordDataFlowController.start(RecordDataFlowController.java:65)
        at org.apache.asterix.external.dataset.adapter.GenericAdapter.start(GenericAdapter.java:37)
        at org.apache.asterix.external.operators.ExternalDataScanOperatorDescriptor$1.initialize(ExternalDataScanOperatorDescriptor.java:62)
        ... 7 more
Caused by: org.apache.hyracks.api.exceptions.HyracksDataException: Long Parser - a digit is
expected. But, encountered this character: V in the incoming input: VLSID 05 Proceedings of
the 18th International Conference on VLSI Design held jointly with 4th International Conference
on Embedded Systems Design
        at org.apache.hyracks.dataflow.common.data.parsers.LongParserFactory$1.parse(LongParserFactory.java:74)
        at org.apache.asterix.external.parser.DelimitedDataParser.parseRecord(DelimitedDataParser.java:144)
        at org.apache.asterix.external.parser.DelimitedDataParser.parse(DelimitedDataParser.java:159)
        at org.apache.asterix.external.dataflow.RecordDataFlowController.start(RecordDataFlowController.java:57)
        ... 9 more
Apr 24, 2016 11:38:19 AM org.apache.hyracks.control.common.work.WorkQueue$WorkerThread run
INFO: Executing: NotifyTaskFailure
Apr 24, 2016 11:38:19 AM org.apache.hyracks.control.common.work.WorkQueue$WorkerThread run
INFO: Executing: AbortTasks
Apr 24, 2016 11:38:19 AM org.apache.hyracks.control.nc.work.AbortTasksWork run
INFO: Aborting Tasks: JID:19:[TAID:TID:ANID:ODID:1:0:12:0, TAID:TID:ANID:ODID:1:0:13:0, TAID:TID:ANID:ODID:1:0:14:0,
TAID:TID:ANID:ODID:1:0:15:0, TAID:TID:ANID:ODID:1:0:16:0, TAID:TID:ANID:ODID:1:0:17:0, TAID:TID:ANID:ODID:1:0:18:0,
TAID:TID:ANID:ODID:1:0:19:0, TAID:TID:ANID:ODID:1:0:20:0, TAID:TID:ANID:ODID:1:0:21:0, TAID:TID:ANID:ODID:1:0:22:0,
TAID:TID:ANID:ODID:1:0:23:0]
java.lang.InterruptedException
        at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1302)
        at java.util.concurrent.Semaphore.acquire(Semaphore.java:467)
        at org.apache.hyracks.control.nc.Task.run(Task.java:299)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)
java.lang.InterruptedException
        at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1302)
        at java.util.concurrent.Semaphore.acquire(Semaphore.java:467)
        at org.apache.hyracks.control.nc.Task.run(Task.java:299)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)
java.lang.InterruptedException
        at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1302)
        at java.util.concurrent.Semaphore.acquire(Semaphore.java:467)
        at org.apache.hyracks.control.nc.Task.run(Task.java:299)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)
java.lang.InterruptedException
        at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1302)
        at java.util.concurrent.Semaphore.acquire(Semaphore.java:467)
        at org.apache.hyracks.control.nc.Task.run(Task.java:299)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)
java.lang.InterruptedException
        at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1302)
        at java.util.concurrent.Semaphore.acquire(Semaphore.java:467)
        at org.apache.hyracks.control.nc.Task.run(Task.java:299)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)
java.lang.InterruptedException
        at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1302)
        at java.util.concurrent.Semaphore.acquire(Semaphore.java:467)
        at org.apache.hyracks.control.nc.Task.run(Task.java:299)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)
java.lang.InterruptedException
        at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1302)
        at java.util.concurrent.Semaphore.acquire(Semaphore.java:467)
        at org.apache.hyracks.control.nc.Task.run(Task.java:299)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)
java.lang.InterruptedException
        at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1302)
        at java.util.concurrent.Semaphore.acquire(Semaphore.java:467)
        at org.apache.hyracks.control.nc.Task.run(Task.java:299)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)
java.lang.InterruptedException
        at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1302)
        at java.util.concurrent.Semaphore.acquire(Semaphore.java:467)
        at org.apache.hyracks.control.nc.Task.run(Task.java:299)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)
java.lang.InterruptedException
        at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1302)
        at java.util.concurrent.Semaphore.acquire(Semaphore.java:467)
        at org.apache.hyracks.control.nc.Task.run(Task.java:299)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)
java.lang.InterruptedException
        at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1302)
        at java.util.concurrent.Semaphore.acquire(Semaphore.java:467)
        at org.apache.hyracks.control.nc.Task.run(Task.java:299)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
 java.lang.InterruptedException
        at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1302)
        at java.util.concurrent.Semaphore.acquire(Semaphore.java:467)
        at org.apache.hyracks.control.nc.Task.run(Task.java:299)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)
Apr 24, 2016 11:38:19 AM org.apache.hyracks.control.common.work.WorkQueue$WorkerThread run
INFO: Executing: CleanupJoblet

{noformat}


> Batch import errors
> -------------------
>
>                 Key: ASTERIXDB-1412
>                 URL: https://issues.apache.org/jira/browse/ASTERIXDB-1412
>             Project: Apache AsterixDB
>          Issue Type: Bug
>          Components: Data Formats, Hyracks
>         Environment: Linux ubuntu12.04, 24cores, 128GB memory;
> Configure: 2NCs X 12partitions, 10GB/per NCs
>            Reporter: Wenhai
>            Assignee: Abdullah Alamoudi
>
> When we importing about 2 million ACM datasets with the following schema, we got a data
fields mismatching error. While we splitting the records into 6 parts, the error disappears.
Always, we put the dataset onto the following site for verifying.
> Dataset
> {noformat}
> http://pan.baidu.com/s/1c26Y9LQ
> {noformat}
> Schema
> {noformat}
> drop dataverse fuzzytest if exists;
> create dataverse fuzzytest;
> use dataverse fuzzytest;
> create type PaperType as open {
>   tid:uuid,
>   title: string,
>   authors: string?,
>   year: int?,
>   conf: string?,
>   idx: string,
>   abstract: string?
> }
> {noformat}
> Splitting import successfully
> {noformat}
> use dataverse test;
> create dataset ACM(PaperType) primary key tid autogenerated;
> load dataset ACM
> using localfs
> (("path"="127.0.0.1:///home/hadoop/Downloads/doccorpus/acm_split.aa,127.0.0.1:///home/hadoop/Downloads/doccorpus/acm_split.ab,127.0.0.1:///home/hadoop/Downloads/doccorpus/acm_split.ac,127.0.0.1:///home/hadoop/Downloads/doccorpus/acm_split.ad,127.0.0.1:///home/hadoop/Downloads/doccorpus/acm_split.ae"),("format"="delimited-text"),("delimiter"="#"),("quote"="\u0000"));
> {noformat}
> Batch Import with failure
> {noformat}
> use dataverse test;
> drop dataset ACM if exists;
> create dataset ACM(PaperType) primary key tid autogenerated;
> load dataset ACM
> using localfs
> (("path"="127.0.0.1:///home/hadoop/Downloads/doccorpus/reproduce/acm_raw.txt"),("format"="delimited-text"),("delimiter"="#"),("quote"="\u0000"));
> {noformat}
> Error message
> {noformat}
> Long Parser - a digit is expected. But, encountered this character: V in the incoming
input: VLSID 05 Proceedings of the 18th International Conference on VLSI Design held jointly
with 4th International Conference on Embedded Systems Design [HyracksDataException]
> {noformat}
> Error trace
> {noformat}
> org.apache.hyracks.api.exceptions.HyracksDataException: java.util.concurrent.ExecutionException:
org.apache.hyracks.api.exceptions.HyracksDataException: org.apache.hyracks.api.exceptions.HyracksDataException:
org.apache.hyracks.api.exceptions.HyracksDataException: Long Parser - a digit is expected.
But, encountered this character: V in the incoming input: VLSID 05 Proceedings of the 18th
International Conference on VLSI Design held jointly with 4th International Conference on
Embedded Systems Design
>         at org.apache.hyracks.api.rewriter.runtime.SuperActivityOperatorNodePushable.runInParallel(SuperActivityOperatorNodePushable.java:218)
>         at org.apache.hyracks.api.rewriter.runtime.SuperActivityOperatorNodePushable.initialize(SuperActivityOperatorNodePushable.java:83)
>         at org.apache.hyracks.control.nc.Task.run(Task.java:263)
>         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>         at java.lang.Thread.run(Thread.java:745)
> Caused by: java.util.concurrent.ExecutionException: org.apache.hyracks.api.exceptions.HyracksDataException:
org.apache.hyracks.api.exceptions.HyracksDataException: org.apache.hyracks.api.exceptions.HyracksDataException:
Long Parser - a digit is expected. But, encountered this character: V in the incoming input:
VLSID 05 Proceedings of the 18th International Conference on VLSI Design held jointly with
4th International Conference on Embedded Systems Design
>         at java.util.concurrent.FutureTask.report(FutureTask.java:122)
>         at java.util.concurrent.FutureTask.get(FutureTask.java:192)
>         at org.apache.hyracks.api.rewriter.runtime.SuperActivityOperatorNodePushable.runInParallel(SuperActivityOperatorNodePushable.java:212)
>         ... 5 more
> Caused by: org.apache.hyracks.api.exceptions.HyracksDataException: org.apache.hyracks.api.exceptions.HyracksDataException:
org.apache.hyracks.api.exceptions.HyracksDataException: Long Parser - a digit is expected.
But, encountered this character: V in the incoming input: VLSID 05 Proceedings of the 18th
International Conference on VLSI Design held jointly with 4th International Conference on
Embedded Systems Design
>         at org.apache.asterix.external.operators.ExternalDataScanOperatorDescriptor$1.initialize(ExternalDataScanOperatorDescriptor.java:65)
>         at org.apache.hyracks.api.rewriter.runtime.SuperActivityOperatorNodePushable.lambda$initialize$0(SuperActivityOperatorNodePushable.java:83)
>         at org.apache.hyracks.api.rewriter.runtime.SuperActivityOperatorNodePushable$1.call(SuperActivityOperatorNodePushable.java:205)
>         at org.apache.hyracks.api.rewriter.runtime.SuperActivityOperatorNodePushable$1.call(SuperActivityOperatorNodePushable.java:202)
>         at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>         ... 3 more
> Caused by: org.apache.hyracks.api.exceptions.HyracksDataException: org.apache.hyracks.api.exceptions.HyracksDataException:
Long Parser - a digit is expected. But, encountered this character: V in the incoming input:
VLSID 05 Proceedings of the 18th International Conference on VLSI Design held jointly with
4th International Conference on Embedded Systems Design
>         at org.apache.asterix.external.dataflow.RecordDataFlowController.start(RecordDataFlowController.java:65)
>         at org.apache.asterix.external.dataset.adapter.GenericAdapter.start(GenericAdapter.java:37)
>         at org.apache.asterix.external.operators.ExternalDataScanOperatorDescriptor$1.initialize(ExternalDataScanOperatorDescriptor.java:62)
>         ... 7 more
> Caused by: org.apache.hyracks.api.exceptions.HyracksDataException: Long Parser - a digit
is expected. But, encountered this character: V in the incoming input: VLSID 05 Proceedings
of the 18th International Conference on VLSI Design held jointly with 4th International Conference
on Embedded Systems Design
>         at org.apache.hyracks.dataflow.common.data.parsers.LongParserFactory$1.parse(LongParserFactory.java:74)
>         at org.apache.asterix.external.parser.DelimitedDataParser.parseRecord(DelimitedDataParser.java:144)
>         at org.apache.asterix.external.parser.DelimitedDataParser.parse(DelimitedDataParser.java:159)
>         at org.apache.asterix.external.dataflow.RecordDataFlowController.start(RecordDataFlowController.java:57)
>         ... 9 more
> Apr 24, 2016 11:38:19 AM org.apache.hyracks.control.common.work.WorkQueue$WorkerThread
run
> INFO: Executing: NotifyTaskFailure
> Apr 24, 2016 11:38:19 AM org.apache.hyracks.control.common.work.WorkQueue$WorkerThread
run
> INFO: Executing: AbortTasks
> Apr 24, 2016 11:38:19 AM org.apache.hyracks.control.nc.work.AbortTasksWork run
> INFO: Aborting Tasks: JID:19:[TAID:TID:ANID:ODID:1:0:12:0, TAID:TID:ANID:ODID:1:0:13:0,
TAID:TID:ANID:ODID:1:0:14:0, TAID:TID:ANID:ODID:1:0:15:0, TAID:TID:ANID:ODID:1:0:16:0, TAID:TID:ANID:ODID:1:0:17:0,
TAID:TID:ANID:ODID:1:0:18:0, TAID:TID:ANID:ODID:1:0:19:0, TAID:TID:ANID:ODID:1:0:20:0, TAID:TID:ANID:ODID:1:0:21:0,
TAID:TID:ANID:ODID:1:0:22:0, TAID:TID:ANID:ODID:1:0:23:0]
> java.lang.InterruptedException
>         at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1302)
>         at java.util.concurrent.Semaphore.acquire(Semaphore.java:467)
>         at org.apache.hyracks.control.nc.Task.run(Task.java:299)
>         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>         at java.lang.Thread.run(Thread.java:745)
> java.lang.InterruptedException
>         at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1302)
>         at java.util.concurrent.Semaphore.acquire(Semaphore.java:467)
>         at org.apache.hyracks.control.nc.Task.run(Task.java:299)
>         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>         at java.lang.Thread.run(Thread.java:745)
> java.lang.InterruptedException
>         at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1302)
>         at java.util.concurrent.Semaphore.acquire(Semaphore.java:467)
>         at org.apache.hyracks.control.nc.Task.run(Task.java:299)
>         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>         at java.lang.Thread.run(Thread.java:745)
> java.lang.InterruptedException
>         at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1302)
>         at java.util.concurrent.Semaphore.acquire(Semaphore.java:467)
>         at org.apache.hyracks.control.nc.Task.run(Task.java:299)
>         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>         at java.lang.Thread.run(Thread.java:745)
> java.lang.InterruptedException
>         at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1302)
>         at java.util.concurrent.Semaphore.acquire(Semaphore.java:467)
>         at org.apache.hyracks.control.nc.Task.run(Task.java:299)
>         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>         at java.lang.Thread.run(Thread.java:745)
> java.lang.InterruptedException
>         at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1302)
>         at java.util.concurrent.Semaphore.acquire(Semaphore.java:467)
>         at org.apache.hyracks.control.nc.Task.run(Task.java:299)
>         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>         at java.lang.Thread.run(Thread.java:745)
> java.lang.InterruptedException
>         at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1302)
>         at java.util.concurrent.Semaphore.acquire(Semaphore.java:467)
>         at org.apache.hyracks.control.nc.Task.run(Task.java:299)
>         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>         at java.lang.Thread.run(Thread.java:745)
> java.lang.InterruptedException
>         at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1302)
>         at java.util.concurrent.Semaphore.acquire(Semaphore.java:467)
>         at org.apache.hyracks.control.nc.Task.run(Task.java:299)
>         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>         at java.lang.Thread.run(Thread.java:745)
> java.lang.InterruptedException
>         at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1302)
>         at java.util.concurrent.Semaphore.acquire(Semaphore.java:467)
>         at org.apache.hyracks.control.nc.Task.run(Task.java:299)
>         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>         at java.lang.Thread.run(Thread.java:745)
> java.lang.InterruptedException
>         at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1302)
>         at java.util.concurrent.Semaphore.acquire(Semaphore.java:467)
>         at org.apache.hyracks.control.nc.Task.run(Task.java:299)
>         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>         at java.lang.Thread.run(Thread.java:745)
> java.lang.InterruptedException
>         at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1302)
>         at java.util.concurrent.Semaphore.acquire(Semaphore.java:467)
>         at org.apache.hyracks.control.nc.Task.run(Task.java:299)
>         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>  java.lang.InterruptedException
>         at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1302)
>         at java.util.concurrent.Semaphore.acquire(Semaphore.java:467)
>         at org.apache.hyracks.control.nc.Task.run(Task.java:299)
>         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>         at java.lang.Thread.run(Thread.java:745)
> Apr 24, 2016 11:38:19 AM org.apache.hyracks.control.common.work.WorkQueue$WorkerThread
run
> INFO: Executing: CleanupJoblet
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


Mime
View raw message