Return-Path: X-Original-To: apmail-accumulo-user-archive@www.apache.org Delivered-To: apmail-accumulo-user-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 8E110E0BE for ; Thu, 31 Jan 2013 15:52:55 +0000 (UTC) Received: (qmail 19834 invoked by uid 500); 31 Jan 2013 15:52:55 -0000 Delivered-To: apmail-accumulo-user-archive@accumulo.apache.org Received: (qmail 19570 invoked by uid 500); 31 Jan 2013 15:52:55 -0000 Mailing-List: contact user-help@accumulo.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@accumulo.apache.org Delivered-To: mailing list user@accumulo.apache.org Received: (qmail 19550 invoked by uid 99); 31 Jan 2013 15:52:54 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 31 Jan 2013 15:52:54 +0000 X-ASF-Spam-Status: No, hits=3.6 required=5.0 tests=HTML_MESSAGE,HTML_OBFUSCATE_10_20,RCVD_IN_DNSWL_LOW,SPF_SOFTFAIL X-Spam-Check-By: apache.org Received-SPF: softfail (athena.apache.org: transitioning domain of wilhelm.von.cloud@accumulo.net does not designate 209.85.216.181 as permitted sender) Received: from [209.85.216.181] (HELO mail-qc0-f181.google.com) (209.85.216.181) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 31 Jan 2013 15:52:48 +0000 Received: by mail-qc0-f181.google.com with SMTP id x40so1330300qcp.12 for ; Thu, 31 Jan 2013 07:52:26 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=mime-version:x-received:x-originating-ip:in-reply-to:references :date:message-id:subject:from:to:content-type:x-gm-message-state; bh=/UzXDoo8aZ1AyjNQvPl5pSAQBWI1Jv2pTNBk3h7N7ZQ=; b=gSI/bFqN/RNfgOqMNpQRJll6+nkH8yGvHbrkEH85KcPNmeAxm6/pPIJLdQiz2BmmPK iE/YwUe7tomohUVG5LFZJ5R+M44+4n3zCY8Z3zJs4S6H8JDyfrkQpmG9DOI7EIOP2amO mbwUaDdox/zR5Avj3ddSyx35vxbsjJnD5SkmDZeKycl6yKEhRh/sTVFsgGoGLZuMtXLj 7p7miFYQ27Do849qspfp80HLzEGEBWBQjodCFkp4RAe8LZJV1cC7ey6WR247GWd5zhfn nhz7NpsWUYjZC63z1VjTkDEomI0cXz9ZyWLQKn5gdtC4j7zb3BZt+dCkyEZ4hKWrOEqd BJLA== MIME-Version: 1.0 X-Received: by 10.224.72.197 with SMTP id n5mr9531262qaj.38.1359647546701; Thu, 31 Jan 2013 07:52:26 -0800 (PST) Received: by 10.229.139.139 with HTTP; Thu, 31 Jan 2013 07:52:26 -0800 (PST) X-Originating-IP: [144.51.26.95] In-Reply-To: References: Date: Thu, 31 Jan 2013 10:52:26 -0500 Message-ID: Subject: Re: Accumulo Configuration Question From: William Slacum To: user@accumulo.apache.org Content-Type: multipart/alternative; boundary=20cf3071d1da0a77e404d4979d4e X-Gm-Message-State: ALoCoQnzfVrJc2VhWmMlprlzD1G3Qa7tqmwkVdM21nlcx17lbdo+H9uGLNb+smup4CVlgpA6fqwI X-Virus-Checked: Checked by ClamAV on apache.org --20cf3071d1da0a77e404d4979d4e Content-Type: text/plain; charset=ISO-8859-1 This doesn't have much to do with your cluster set up, but what does the monitor say as your jobs are nearing completion and things start failing? Are there hold times for the table(s) you are writing to? On Thu, Jan 31, 2013 at 10:19 AM, Parker, Matthew - IS < Matthew.Parker@exelisinc.com> wrote: > TWIMC: > > I'm new to Accumulo and I've been trying to come up with a good > architecture for a 20 node cluster. I have been running a map/reduce > program, and it encounters issues when it comes to running the Accumulo > section of the code. Once the job's completion rate exceeds 93, it startsdropping 10's ofta > sks during the process, because they eventually timeout. The completionrate drops back down, but it the > job eventually finishes. I have a suspicion it's due to the way I have > the system configured and I wanted to get some feedback as to what's thegenerally preferred > architecture when installing Accumulo? > > Since you have the choice of installing hdfs, map/reduce, and tablet > servers on any three, the general guideline is to install two per machine > (data node and table server, or data node and map/reduce) as per the > Hardware section in the Administration documentation. > > http://accumulo.apache.org/1.4/user_manual/Administration.html#Hardware > > Does that mean you have one large group of data nodes that's installed on > the majority of the cluster, or are they somehow split into two groups such > that map/reduce & hdfs runs on one set of nodes, and Accumulo tablet > servers and hdfs uses another? > > I was wondering whether people would comment on what a working > configuration might look like? > > TIA, > > Matt > > ------------------------------ > > This e-mail and any files transmitted with it may be proprietary and are > intended solely for the use of the individual or entity to whom they are > addressed. If you have received this e-mail in error please notify the > sender. Please note that any views or opinions presented in this e-mail are > solely those of the author and do not necessarily represent those of Exelis > Inc. The recipient should check this e-mail and any attachments for the > presence of viruses. Exelis Inc. accepts no liability for any damage caused > by any virus transmitted by this e-mail. > --20cf3071d1da0a77e404d4979d4e Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable This doesn't have much to do with your cluster set up, but what does th= e monitor say as your jobs are nearing completion and things start failing?= Are there hold times for the table(s) you are writing to?

On Thu, Jan 31, 2013 at 10:19 AM, Parker, Matthew - IS &l= t;Matthew= .Parker@exelisinc.com> wrote:
TWIMC:

I'm new to Accumulo and I've been trying to = come up with a good architecture for a 20 node cluster. I have been running a map/reduce program, and it encounters issues when it comes to r= unning the Accumulo section of the code. Once the job's completion rate exceeds 93, it starts dropping 10's of tasks during the process, because they eventually timeout. The completion rate drops bac= k down, but it=A0= the job eventually finishes. I have a suspicion it's due to the way I have the sys= tem configured and I wanted to get some feedback as to what's the generally preferred architecture when installing= Accumulo?

Since you have the choice of installing hdfs, map/reduce, and tablet = servers on any three, the general guideline is to install two per ma= chine (data node and table server, or data node and map/reduce)= as per the Hardware section in the Administration documentation.

http:/= /accumulo.apache.org/1.4/user_manual/Administration.html#Hardware

Does that mean you have one large group of data nodes that's installed = on the majority of the cluster, or are they somehow split into two groups s= uch that map/reduce & hdfs runs on one set of nodes, and Accumulo table= t servers and hdfs uses another?

I was wondering whether people would comment on what a working configuratio= n might look like?

TIA,

Matt


This e-mail and any files transmitted with it may be proprietary and are in= tended solely for the use of the individual or entity to whom they are addr= essed. If you have received this e-mail in error please notify the sender. = Please note that any views or opinions presented in this e-mail are solely those of the author and do not necessa= rily represent those of Exelis Inc. The recipient should check this e-mail = and any attachments for the presence of viruses. Exelis Inc. accepts no lia= bility for any damage caused by any virus transmitted by this e-mail.

--20cf3071d1da0a77e404d4979d4e--