Mailing-List: contact user-help@accumulo.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@accumulo.apache.org
Received-SPF: softfail (nike.apache.org: transitioning domain of
 wilhelm.von.cloud@accumulo.net does not designate 209.85.213.50 as permitted
 sender)
MIME-Version: 1.0
In-Reply-To: <1396964408022-8904.post@n5.nabble.com>
References: <1396964408022-8904.post@n5.nabble.com>
Date: Tue, 8 Apr 2014 10:30:12 -0400
Message-ID: 
 <CAMz+Dusctp+msG2a8jWgBjWbQPRy6Ap=7kdEesuYFMseiVKJ3Q@mail.gmail.com>
Subject: Re: bulk ingest without mapred
From: William Slacum <wilhelm.von.cloud@accumulo.net>
To: user@accumulo.apache.org
Content-Type: multipart/alternative; boundary=20cf303e9f3c679fdb04f688d281

--20cf303e9f3c679fdb04f688d281
Content-Type: text/plain; charset=ISO-8859-1

 java.io.FileNotFoundException: File does not exist:
bulk/entities_fails/failures

sticks out to me. it looks like a relative path. where does that directory
exist on your file system?


On Tue, Apr 8, 2014 at 9:40 AM, pdread <paul.read@siginttech.com> wrote:

> Hi
>
> I interface to an accumulo cloud (100s of nodes) which I don't maintain.
> I'll try and keep this short, the interface App is used to ingest millions
> of docs/week from various streams, some are required near real time. A
> problem came up where the tservers would not stay up and our ingest would
> halt. Now the admins are working on fixing this but I'm not optimistic.
> Others who have run into this tell me its the use of Mutations that is
> causing the problem and it will go away if I do bulk ingest. However
> mapreduce is way to slow to spin up and does not map to our arch.
>
> So here is what I have been trying to do. After much research I think I
> should be able to bulk ingest if I create the RFile and feed this to
> TableOperations.importDirectory(). I can create the RFile ok, at least I
> thinks so, I create the "failure" directory using hadoops' file system. I
> check that the failure directory is there and is a directory but when I
> feed
> it to the import I get an error over on the accumulo master log that the it
> can not find the failure directory. Now the interesting thing is I have
> traced the code thourgh the accumulo client it checks successfully for the
> load file and the failure directory. What am I doing wrong?
>
> First the client error:
>
> org.apache.accumulo.core.client.AccumuloException: Internal error
> processing
> waitForTableOperation
>         at
>
> org.apache.accumulo.core.client.admin.TableOperationsImpl.doTableOperation(TableOperationsImpl.java:290)
>         at
>
> org.apache.accumulo.core.client.admin.TableOperationsImpl.doTableOperation(TableOperationsImpl.java:258)
>         at
>
> org.apache.accumulo.core.client.admin.TableOperationsImpl.importDirectory(TableOperationsImpl.java:945)
>         at
>
> airs.medr.accumulo.server.table.EntityTable.writeEntities(EntityTable.java:130)
>
> Now the master log exception:
>
> 2014-04-08 08:33:50,609 [thrift.MasterClientService$Processor] ERROR:
> Internal error processing waitForTableOperation
> java.lang.RuntimeException: java.io.FileNotFoundException: File does not
> exist: bulk/entities_fails/failures
>         at
>
> org.apache.accumulo.server.master.Master$MasterClientServiceHandler.waitForTableOperation(Master.java:1053)
>         at sun.reflect.GeneratedMethodAccessor24.invoke(Unknown Source)
>         at
>
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>         at java.lang.reflect.Method.invoke(Method.java:597)
>         at
>
> org.apache.accumulo.cloudtrace.instrument.thrift.TraceWrap$1.invoke(TraceWrap.java:59)
>         at $Proxy6.waitForTableOperation(Unknown Source)
>         at
>
> org.apache.accumulo.core.master.thrift.MasterClientService$Processor$waitForTableOperation.process(MasterClientService.java:2004)
>         at
>
> org.apache.accumulo.core.master.thrift.MasterClientService$Processor.process(MasterClientService.java:1472)
>         at
>
> org.apache.accumulo.server.util.TServerUtils$TimedProcessor.process(TServerUtils.java:154)
>         at
>
> org.apache.thrift.server.TNonblockingServer$FrameBuffer.invoke(TNonblockingServer.java:631)
>         at
>
> org.apache.accumulo.server.util.TServerUtils$THsHaServer$Invocation.run(TServerUtils.java:202)
>         at
>
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>         at
>
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>         at
> org.apache.accumulo.core.util.LoggingRunnable.run(LoggingRunnable.java:34)
>         at java.lang.Thread.run(Thread.java:662)
> Caused by: java.io.FileNotFoundException: File does not exist:
> bulk/entities_fails/failures
>         at
>
> org.apache.hadoop.hdfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:528)
>         at
>
> org.apache.accumulo.server.trace.TraceFileSystem.getFileStatus(TraceFileSystem.java:797)
>         at
>
> org.apache.accumulo.server.master.tableOps.BulkImport.call(BulkImport.java:157)
>         at
>
> org.apache.accumulo.server.master.tableOps.BulkImport.call(BulkImport.java:110)
>         at
>
> org.apache.accumulo.server.master.tableOps.TraceRepo.call(TraceRepo.java:65)
>         at
> org.apache.accumulo.server.fate.Fate$TransactionRunner.run(Fate.java:65)
>
>
> Thoughts?
>
> Thanks
>
> Paul
>
>
>
>
> --
> View this message in context:
> http://apache-accumulo.1065345.n5.nabble.com/bulk-ingest-without-mapred-tp8904.html
> Sent from the Users mailing list archive at Nabble.com.
>

--20cf303e9f3c679fdb04f688d281
Content-Type: text/html; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr"><div> java.io.FileNotFoundException: File does not exist:<=
br>
bulk/entities_fails/failures <br><br></div>sticks out to me. it looks like =
a relative path. where does that directory exist on your file system?<br></=
div><div class=3D"gmail_extra"><br><br><div class=3D"gmail_quote">On Tue, A=
pr 8, 2014 at 9:40 AM, pdread <span dir=3D"ltr">&lt;<a href=3D"mailto:paul.=
read@siginttech.com" target=3D"_blank">paul.read@siginttech.com</a>&gt;</sp=
an> wrote:<br>
<blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1p=
x #ccc solid;padding-left:1ex">Hi<br>
<br>
I interface to an accumulo cloud (100s of nodes) which I don&#39;t maintain=
.<br>
I&#39;ll try and keep this short, the interface App is used to ingest milli=
ons<br>
of docs/week from various streams, some are required near real time. A<br>
problem came up where the tservers would not stay up and our ingest would<b=
r>
halt. Now the admins are working on fixing this but I&#39;m not optimistic.=
<br>
Others who have run into this tell me its the use of Mutations that is<br>
causing the problem and it will go away if I do bulk ingest. However<br>
mapreduce is way to slow to spin up and does not map to our arch.<br>
<br>
So here is what I have been trying to do. After much research I think I<br>
should be able to bulk ingest if I create the RFile and feed this to<br>
TableOperations.importDirectory(). I can create the RFile ok, at least I<br=
>
thinks so, I create the &quot;failure&quot; directory using hadoops&#39; fi=
le system. I<br>
check that the failure directory is there and is a directory but when I fee=
d<br>
it to the import I get an error over on the accumulo master log that the it=
<br>
can not find the failure directory. Now the interesting thing is I have<br>
traced the code thourgh the accumulo client it checks successfully for the<=
br>
load file and the failure directory. What am I doing wrong?<br>
<br>
First the client error:<br>
<br>
org.apache.accumulo.core.client.AccumuloException: Internal error processin=
g<br>
waitForTableOperation<br>
=A0 =A0 =A0 =A0 at<br>
org.apache.accumulo.core.client.admin.TableOperationsImpl.doTableOperation(=
TableOperationsImpl.java:290)<br>
=A0 =A0 =A0 =A0 at<br>
org.apache.accumulo.core.client.admin.TableOperationsImpl.doTableOperation(=
TableOperationsImpl.java:258)<br>
=A0 =A0 =A0 =A0 at<br>
org.apache.accumulo.core.client.admin.TableOperationsImpl.importDirectory(T=
ableOperationsImpl.java:945)<br>
=A0 =A0 =A0 =A0 at<br>
airs.medr.accumulo.server.table.EntityTable.writeEntities(EntityTable.java:=
130)<br>
<br>
Now the master log exception:<br>
<br>
2014-04-08 08:33:50,609 [thrift.MasterClientService$Processor] ERROR:<br>
Internal error processing waitForTableOperation<br>
java.lang.RuntimeException: java.io.FileNotFoundException: File does not<br=
>
exist: bulk/entities_fails/failures<br>
=A0 =A0 =A0 =A0 at<br>
org.apache.accumulo.server.master.Master$MasterClientServiceHandler.waitFor=
TableOperation(Master.java:1053)<br>
=A0 =A0 =A0 =A0 at sun.reflect.GeneratedMethodAccessor24.invoke(Unknown Sou=
rce)<br>
=A0 =A0 =A0 =A0 at<br>
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImp=
l.java:25)<br>
=A0 =A0 =A0 =A0 at java.lang.reflect.Method.invoke(Method.java:597)<br>
=A0 =A0 =A0 =A0 at<br>
org.apache.accumulo.cloudtrace.instrument.thrift.TraceWrap$1.invoke(TraceWr=
ap.java:59)<br>
=A0 =A0 =A0 =A0 at $Proxy6.waitForTableOperation(Unknown Source)<br>
=A0 =A0 =A0 =A0 at<br>
org.apache.accumulo.core.master.thrift.MasterClientService$Processor$waitFo=
rTableOperation.process(MasterClientService.java:2004)<br>
=A0 =A0 =A0 =A0 at<br>
org.apache.accumulo.core.master.thrift.MasterClientService$Processor.proces=
s(MasterClientService.java:1472)<br>
=A0 =A0 =A0 =A0 at<br>
org.apache.accumulo.server.util.TServerUtils$TimedProcessor.process(TServer=
Utils.java:154)<br>
=A0 =A0 =A0 =A0 at<br>
org.apache.thrift.server.TNonblockingServer$FrameBuffer.invoke(TNonblocking=
Server.java:631)<br>
=A0 =A0 =A0 =A0 at<br>
org.apache.accumulo.server.util.TServerUtils$THsHaServer$Invocation.run(TSe=
rverUtils.java:202)<br>
=A0 =A0 =A0 =A0 at<br>
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.j=
ava:886)<br>
=A0 =A0 =A0 =A0 at<br>
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:=
908)<br>
=A0 =A0 =A0 =A0 at<br>
org.apache.accumulo.core.util.LoggingRunnable.run(LoggingRunnable.java:34)<=
br>
=A0 =A0 =A0 =A0 at java.lang.Thread.run(Thread.java:662)<br>
Caused by: java.io.FileNotFoundException: File does not exist:<br>
bulk/entities_fails/failures<br>
=A0 =A0 =A0 =A0 at<br>
org.apache.hadoop.hdfs.DistributedFileSystem.getFileStatus(DistributedFileS=
ystem.java:528)<br>
=A0 =A0 =A0 =A0 at<br>
org.apache.accumulo.server.trace.TraceFileSystem.getFileStatus(TraceFileSys=
tem.java:797)<br>
=A0 =A0 =A0 =A0 at<br>
org.apache.accumulo.server.master.tableOps.BulkImport.call(BulkImport.java:=
157)<br>
=A0 =A0 =A0 =A0 at<br>
org.apache.accumulo.server.master.tableOps.BulkImport.call(BulkImport.java:=
110)<br>
=A0 =A0 =A0 =A0 at<br>
org.apache.accumulo.server.master.tableOps.TraceRepo.call(TraceRepo.java:65=
)<br>
=A0 =A0 =A0 =A0 at<br>
org.apache.accumulo.server.fate.Fate$TransactionRunner.run(Fate.java:65)<br=
>
<br>
<br>
Thoughts?<br>
<br>
Thanks<br>
<br>
Paul<br>
<br>
<br>
<br>
<br>
--<br>
View this message in context: <a href=3D"http://apache-accumulo.1065345.n5.=
nabble.com/bulk-ingest-without-mapred-tp8904.html" target=3D"_blank">http:/=
/apache-accumulo.1065345.n5.nabble.com/bulk-ingest-without-mapred-tp8904.ht=
ml</a><br>

Sent from the Users mailing list archive at Nabble.com.<br>
</blockquote></div><br></div>

--20cf303e9f3c679fdb04f688d281--