From notifications-return-7521-apmail-accumulo-notifications-archive=accumulo.apache.org@accumulo.apache.org Fri May 24 13:52:22 2013 Return-Path: X-Original-To: apmail-accumulo-notifications-archive@minotaur.apache.org Delivered-To: apmail-accumulo-notifications-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 35862D780 for ; Fri, 24 May 2013 13:52:22 +0000 (UTC) Received: (qmail 59645 invoked by uid 500); 24 May 2013 13:52:22 -0000 Delivered-To: apmail-accumulo-notifications-archive@accumulo.apache.org Received: (qmail 59443 invoked by uid 500); 24 May 2013 13:52:21 -0000 Mailing-List: contact notifications-help@accumulo.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: jira@apache.org Delivered-To: mailing list notifications@accumulo.apache.org Received: (qmail 59400 invoked by uid 99); 24 May 2013 13:52:20 -0000 Received: from arcas.apache.org (HELO arcas.apache.org) (140.211.11.28) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 24 May 2013 13:52:20 +0000 Date: Fri, 24 May 2013 13:52:20 +0000 (UTC) From: "Dave Marion (JIRA)" To: notifications@accumulo.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Commented] (ACCUMULO-118) accumulo could work across HDFS instances, which would help it to scale past a single namenode MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 [ https://issues.apache.org/jira/browse/ACCUMULO-118?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13666314#comment-13666314 ] Dave Marion commented on ACCUMULO-118: -------------------------------------- Personally I am not a fan of the hash idea. I would rather see a mapping of namespace prefix to NN in the configuration (ns1 = hdfs://host:port, ns2 = hdfs://host:port). I'm thinking forward to table file load balancing across namespaces and backups (see my comment from 3/Apr/12). If for example you quiesced the database and performed a backup, then you could change the namespace mapping such that ns1 and ns2 point to the same hdfs://host:port if for some reason you lost the 2nd hdfs instance (it crashed, you wanted to remove it, etc). This could also allow for of Hadoop wile Accumulo is still running. Think about the scenario where ns1 is on racks 1&2 and ns2 is on racks 3&4 of a cluster and the files of table T are spread across ns1 and ns2. You could change the configuration of the table file load balancer (new feature) that puts new files on ns2. You recompact the table so now all new files are on ns2. When done for all tables (and walogs), then you can shutdown ns1 and upgrade to a new version of Hadoop. > accumulo could work across HDFS instances, which would help it to scale past a single namenode > ---------------------------------------------------------------------------------------------- > > Key: ACCUMULO-118 > URL: https://issues.apache.org/jira/browse/ACCUMULO-118 > Project: Accumulo > Issue Type: Improvement > Components: master, tserver > Reporter: Eric Newton > Assignee: Eric Newton > Priority: Blocker > Fix For: 1.6.0 > > Attachments: ACCUMULO-118-01.txt > > Original Estimate: 2,016h > Remaining Estimate: 2,016h > > Consider using full path names to files, which would allow the servers to access the files on any HDFS file system. > Work may exist elsewhere to run HDFS using a number of NameNode instances to break up the namespace. > We may need a pluggable strategy to determine namespace for new files. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira