Return-Path: X-Original-To: apmail-accumulo-user-archive@www.apache.org Delivered-To: apmail-accumulo-user-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 7E6DA111B5 for ; Wed, 25 Jun 2014 18:38:58 +0000 (UTC) Received: (qmail 2419 invoked by uid 500); 25 Jun 2014 18:38:58 -0000 Delivered-To: apmail-accumulo-user-archive@accumulo.apache.org Received: (qmail 2368 invoked by uid 500); 25 Jun 2014 18:38:58 -0000 Mailing-List: contact user-help@accumulo.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@accumulo.apache.org Delivered-To: mailing list user@accumulo.apache.org Received: (qmail 2359 invoked by uid 99); 25 Jun 2014 18:38:58 -0000 Received: from minotaur.apache.org (HELO minotaur.apache.org) (140.211.11.9) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 25 Jun 2014 18:38:58 +0000 Received: from localhost (HELO mail-ve0-f171.google.com) (127.0.0.1) (smtp-auth username vines, mechanism plain) by minotaur.apache.org (qpsmtpd/0.29) with ESMTP; Wed, 25 Jun 2014 18:38:58 +0000 Received: by mail-ve0-f171.google.com with SMTP id jz11so2465940veb.2 for ; Wed, 25 Jun 2014 11:38:57 -0700 (PDT) X-Received: by 10.52.138.14 with SMTP id qm14mr2820219vdb.49.1403721537123; Wed, 25 Jun 2014 11:38:57 -0700 (PDT) MIME-Version: 1.0 Reply-To: vines@apache.org Received: by 10.220.114.148 with HTTP; Wed, 25 Jun 2014 11:38:17 -0700 (PDT) In-Reply-To: References: From: John Vines Date: Wed, 25 Jun 2014 14:38:17 -0400 Message-ID: Subject: Re: Mapreduce output format killing tablet servers To: "user@accumulo.apache.org" Content-Type: multipart/alternative; boundary=bcaec51b1daf9587d704fcad634e --bcaec51b1daf9587d704fcad634e Content-Type: text/plain; charset=UTF-8 And you're certain your using the standalone example and not the native-standalone? Those expect the native libraries to be extant and if not will eventually cause an OOM. On Wed, Jun 25, 2014 at 2:33 PM, Jacob Rust wrote: > Accumulo version 1.5.1.2.1.2.1-471 > Hadoop version 2.4.0.2.1.2.1-471 > tserver debug log http://pastebin.com/BHdTkxeK > > I what you mean about the memory. I am using the memory settings from the > example files > https://github.com/apache/accumulo/tree/master/conf/examples/512MB/standalone. > I also ran into this problem using the 1GB example memory settings. Each > node has 4GB RAM. > > Thanks > > > On Wed, Jun 25, 2014 at 2:10 PM, Sean Busbey wrote: > >> What version of Accumulo? >> >> What version of Hadoop? >> >> What does your server memory and per-role allocation look like? >> >> Can you paste the tserver debug log? >> >> >> >> On Wed, Jun 25, 2014 at 1:01 PM, Jacob Rust >> wrote: >> >>> I am trying to create an inverted text index for a table using accumulo >>> input/output format in a java mapreduce program. When the job reaches the >>> reduce phase and creates the table / tries to write to it the tablet >>> servers begin to die. >>> >>> Now when I do a start-all.sh the tablet servers start for about a minute >>> and then die again. Any idea as to why the mapreduce job is killing the >>> tablet servers and/or how to bring the tablet servers back up without >>> failing? >>> >>> This is on a 12 node cluster with low quality hardware. >>> >>> The java code I am running is here http://pastebin.com/ti7Qz19m >>> >>> The log files on each tablet server only display the startup >>> information, no errors. The log files on the master server show these >>> errors http://pastebin.com/LymiTfB7 >>> >>> >>> >>> >>> -- >>> Jacob Rust >>> Software Intern >>> >> >> >> >> -- >> Sean >> > > > > -- > Jacob Rust > Software Intern > --bcaec51b1daf9587d704fcad634e Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable
And you're certain your using the standalone example a= nd not the native-standalone? Those expect the native libraries to be extan= t and if not will eventually cause an OOM.
=

On Wed, Jun 25, 2014 at 2:33 PM, Jacob Rust = <jrust@clearedgeit.com> wrote:
Accumulo version =C2=A0 1.5.1.2.1.2.1-471
Hadoop versi= on =C2=A0 =C2=A0 =C2=A0 2.4.0.2.1.2.1-471
tserver debug log = =C2=A0 =C2=A0htt= p://pastebin.com/BHdTkxeK

I what you mean about the memory. I am using the memory settings from the e= xample files=C2=A0https://github.com/apach= e/accumulo/tree/master/conf/examples/512MB/standalone. I also ran into = this problem using the 1GB example memory settings. Each node has 4GB RAM.= =C2=A0

Thanks


On Wed, Jun 25, 2014 at 2:10 PM, S= ean Busbey <busbey@cloudera.com> wrote:
What version of Accumulo?

What version of Hadoop?

What does your server memory and per-role allocation look li= ke?

Can you paste the tserver debug log?



On Wed, Jun 25, 2014 at 1:01 PM, Jacob Rust <jrust@c= learedgeit.com> wrote:
I am trying to create an=C2= =A0inverted=C2=A0text index for a table using accumulo input/output format = in a java mapreduce=C2=A0program. =C2=A0When the job reaches the reduce pha= se and creates the table / tries to write to it the tablet servers begin to= die.

Now when I do a start-all.sh the tablet servers start for ab= out a minute and then die again.=C2=A0Any idea as to why the mapreduce job = is killing the tablet servers and/or how to bring the tablet servers back u= p without failing?

This is on a 12 node cluster with low quality hardware.=C2= =A0
=C2=A0
The java code I am running is here http://pastebin.com/ti7= Qz19m

The log files on each tablet server only display the startup informati= on, no errors. The log files on the master server show these errors=C2=A0http://pastebin.co= m/LymiTfB7




--
Jacob Rust
Software Intern



<= font color=3D"#888888">--
Sean



--
Jacob Rust
Software Intern

--bcaec51b1daf9587d704fcad634e--