Mailing-List: contact user-help@hbase.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@hbase.apache.org
Received-SPF: pass (nike.apache.org: domain of michael_segel@hotmail.com
 designates 65.55.111.79 as permitted sender)
Message-ID: <BLU0-SMTP5049336EAF11D36ADA3A7C8FBA0@phx.gbl>
Content-Type: text/plain; charset="us-ascii"
MIME-Version: 1.0 (Mac OS X Mail 6.3 \(1503\))
Subject: Re: Export / Import and table splits
From: Michael Segel <michael_segel@hotmail.com>
In-Reply-To: 
 <CAPQV63XRSPPa3C2Hyu4XzkasqW_J0+K-Ap=0nko_iwEJANQMbQ@mail.gmail.com>
Date: Tue, 7 May 2013 18:11:00 -0500
Content-Transfer-Encoding: quoted-printable
References: 
 <CAPQV63V7_Tk_quzq6OozVdxNnH1y8wJrU311e2cD4e4ZcVge+Q@mail.gmail.com>
 <CALte62ydYw83CeYi0bhQY52pwnO833fbKz5aZ0rtQdp20fUT7A@mail.gmail.com>
 <BLU0-SMTP59906301633128ADF912388FBA0@phx.gbl>
 <CAPQV63XRSPPa3C2Hyu4XzkasqW_J0+K-Ap=0nko_iwEJANQMbQ@mail.gmail.com>
To: user@hbase.apache.org

I don't see much value in duplicating the table's structure, but IMHO, =
the jury is still out.=20


On May 7, 2013, at 6:02 PM, Jean-Marc Spaggiari =
<jean-marc@spaggiari.org> wrote:

> @Mohammad: The end goal is really more regarding the splits more than
> the model. So I don't think Lars' options are good for this usecase.
> @Mike: I agree that things were not configured correctly. User should
> have had split the table before doing the import. I like the idea of
> looking at the files to get the regions boundaries. That way you don't
> need to have the source_table still there...
>=20
> So we have 2 different things here.
> 1) a command on the shell to duplicate a table structure
> 2) an option on the import command to split the table regions based on
> the files names.
>=20
> If we agree on that I will open one JIRA for each...
>=20
> JM
>=20
> 2013/5/7 Michael Segel <michael_segel@hotmail.com>:
>> Silly question...
>>=20
>> If you're doing a simple export, then you end up with all of your =
prior regions as separate files in a directory, right?
>>=20
>> So in theory, you could find the first row and the last complete row =
of each file and then do your pre-splits based on the start key and end =
key that you find.
>>=20
>> That would be your tool so to speak.
>>=20
>> But to the point that reading back in these files will cause you to =
crash your RS and HBase?
>> That doesn't sound like its well tuned or right.
>>=20
>> HTH
>> -Mike
>>=20
>> On May 7, 2013, at 5:29 PM, Ted Yu <yuzhihong@gmail.com> wrote:
>>=20
>>> I am not aware of a tool which can pre-split table using another =
table's
>>> region boundaries as template.
>>>=20
>>> Such a tool would be nice to have.
>>>=20
>>> Cheers
>>>=20
>>> On Tue, May 7, 2013 at 3:23 PM, Jean-Marc Spaggiari =
<jean-marc@spaggiari.org
>>>> wrote:
>>>=20
>>>> Hi,
>>>>=20
>>>> When we are doing an export, we are only exporting the data. Then =
when
>>>> we are importing that back, we need to make sure the table is
>>>> pre-splitted correctly else we might hotspot some servers.
>>>>=20
>>>> If you simply export then import without pre-splitting at all, you
>>>> will most probably brought some servers down because they will be
>>>> overwhelmed with splits and compactions.
>>>>=20
>>>> Do we have any tool to pre-split a table the same way another table =
is
>>>> already pre-splitted?
>>>>=20
>>>> Something like
>>>>> duplicate 'source_table', 'target_table'
>>>>=20
>>>> Which will create a new table called 'target_table' with exactly =
the
>>>> same parameters as 'source_table' and the same regions boundaries?
>>>>=20
>>>> If we don't have, will it be useful to have one?
>>>>=20
>>>> Or event something like:
>>>>> create 'target_table', 'f1', {SPLITS_MODEL =3D> 'source_table'}
>>>>=20
>>>>=20
>>>> JM
>>>>=20
>>=20
>=20