Return-Path: 
 <issues-return-37966-apmail-hbase-issues-archive=hbase.apache.org@hbase.apache.org>
X-Original-To: apmail-hbase-issues-archive@www.apache.org
Delivered-To: apmail-hbase-issues-archive@www.apache.org
Received: from mail.apache.org (hermes.apache.org [140.211.11.3])
	by minotaur.apache.org (Postfix) with SMTP id 6ECBC9654
	for <apmail-hbase-issues-archive@www.apache.org>;
 Sun,  4 Mar 2012 05:46:32 +0000 (UTC)
Received: (qmail 29653 invoked by uid 500); 4 Mar 2012 05:46:32 -0000
Delivered-To: apmail-hbase-issues-archive@hbase.apache.org
Received: (qmail 29607 invoked by uid 500); 4 Mar 2012 05:46:31 -0000
Mailing-List: contact issues-help@hbase.apache.org; run by ezmlm
Precedence: bulk
List-Help: <mailto:issues-help@hbase.apache.org>
List-Unsubscribe: <mailto:issues-unsubscribe@hbase.apache.org>
List-Post: <mailto:issues@hbase.apache.org>
List-Id: <issues.hbase.apache.org>
Delivered-To: mailing list issues@hbase.apache.org
Received: (qmail 29573 invoked by uid 99); 4 Mar 2012 05:46:31 -0000
Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136)
    by apache.org (qpsmtpd/0.29) with ESMTP; Sun, 04 Mar 2012 05:46:31 +0000
X-ASF-Spam-Status: No, hits=-2000.0 required=5.0
	tests=ALL_TRUSTED,T_RP_MATCHES_RCVD
X-Spam-Check-By: apache.org
Received: from [140.211.11.116] (HELO hel.zones.apache.org) (140.211.11.116)
    by apache.org (qpsmtpd/0.29) with ESMTP; Sun, 04 Mar 2012 05:46:30 +0000
Received: from hel.zones.apache.org (hel.zones.apache.org [140.211.11.116])
	by hel.zones.apache.org (Postfix) with ESMTP id DD62A4B38
	for <issues@hbase.apache.org>; Sun,  4 Mar 2012 05:46:09 +0000 (UTC)
Date: Sun, 4 Mar 2012 05:46:09 +0000 (UTC)
From: "Lars Hofhansl (Commented) (JIRA)" <jira@apache.org>
To: issues@hbase.apache.org
Message-ID: 
 <362317313.19216.1330839969919.JavaMail.tomcat@hel.zones.apache.org>
In-Reply-To: 
 <1467548248.10772.1330668299893.JavaMail.tomcat@hel.zones.apache.org>
Subject: [jira] [Commented] (HBASE-5509) MR based copier for copying HFiles
 (trunk version)
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394
X-Virus-Checked: Checked by ClamAV on apache.org


    [ https://issues.apache.org/jira/browse/HBASE-5509?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13221796#comment-13221796 ] 

Lars Hofhansl commented on HBASE-5509:
--------------------------------------

bq. For getStoreFileList():{code}
+   * @param families
+   *          a comma separated list of column families for which we need to
{code} I think List<String> may be better data type for families parameter. This would make this method more general in that 

Families is taken from the command line; it might be (1) an indicator for all CFs, or  (2) a list of specific CFs. In the former case we cannot get the list of CFs before we know the RegionServer, which only happens later.

Again, this is not my code. I can refactor and have extra command line flags and extra code for this.
                
> MR based copier for copying HFiles (trunk version)
> --------------------------------------------------
>
>                 Key: HBASE-5509
>                 URL: https://issues.apache.org/jira/browse/HBASE-5509
>             Project: HBase
>          Issue Type: Sub-task
>          Components: documentation, regionserver
>            Reporter: Karthik Ranganathan
>            Assignee: Lars Hofhansl
>             Fix For: 0.94.0, 0.96.0
>
>         Attachments: 5509.txt
>
>
> This copier is a modification of the distcp tool in HDFS. It does the following:
> 1. List out all the regions in the HBase cluster for the required table
> 2. Write the above out to a file
> 3. Each mapper 
>    3.1 lists all the HFiles for a given region by querying the regionserver
>    3.2 copies all the HFiles
>    3.3 outputs success if the copy succeeded, failure otherwise. Failed regions are retried in another loop
> 4. Mappers are placed on nodes which have maximum locality for a given region to speed up copying

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira