Mailing-List: contact user-help@hbase.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@hbase.apache.org
MIME-Version: 1.0
In-Reply-To: 
 <CAFb6HiKGk5hZxYyg62BQezmG5WWzdk4whbr2C6NWWkeoso1KJg@mail.gmail.com>
References: 
 <CA+sHBGte9hQpOGdD3V6Df1tLnWt_Y-B1mMRyg2iw4B4-ikCFNw@mail.gmail.com>
	<CAFb6HiKGk5hZxYyg62BQezmG5WWzdk4whbr2C6NWWkeoso1KJg@mail.gmail.com>
Date: Thu, 4 Feb 2016 19:14:57 -0800
Message-ID: 
 <CALte62xv813JA7is=HmMBrFJW=BWcu+CUUJR_=JpqyLRo=m5Aw@mail.gmail.com>
Subject: Re: Regd: ExportSnapshot Tool (export Hfiles in parts)
From: Ted Yu <yuzhihong@gmail.com>
To: "user@hbase.apache.org" <user@hbase.apache.org>
Content-Type: multipart/alternative; boundary=001a11432108801858052afd4184

--001a11432108801858052afd4184
Content-Type: text/plain; charset=UTF-8

Vishnu:
Please take a look
at hadoop-common-project/hadoop-common/src/main/resources/core-default.xml
for multipart related config parameters (other than the one mentioned by
Matteo):

fs.s3n.multipart.uploads.block.size
fs.s3n.multipart.copy.block.size

Cheers

On Thu, Feb 4, 2016 at 7:00 PM, Matteo Bertozzi <theo.bertozzi@gmail.com>
wrote:

> the multipart upload is on the s3 connector.
> you can tune your connector, to use multipart
> fs.s3n.multipart.uploads.enabled = true
>
> Matteo
>
>
> On Thu, Feb 4, 2016 at 6:34 PM, Vishnu Amdiyala <vishnuamdiyala@gmail.com>
> wrote:
>
> > Hi,
> >
> > I am trying to back up snapshots of Hbase table to S3 bucket of which
> each
> > Hfile is sized>5GB which fails due to S3 bucket's 5gb limitation.  The
> > export snapshot's source says that the mappers are set to max of  total
> > number of files. Is there a way to use this tool to split files and
> upload
> > to S3 in parts?
> >
> >
> > Thanks!
> > Vishnu
> >
>

--001a11432108801858052afd4184--