hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Stack <st...@duboce.net>
Subject Re: Creation of Hfiles for multiple tables using Single Bulk Load Job?
Date Mon, 26 Sep 2011 16:26:55 GMT
Can you play around w/ the class to make it work for your case?
St.Ack

On Mon, Sep 26, 2011 at 9:24 AM, Arsalan Bilal <charsalanbilal@gmail.com> wrote:
> @Andrew
>
> Can u help me? U suggest that Try this: https://gist.github.com/1237770 , i
> saw it, How we can we use it actually?
> i try to use it as job.setOutputFormatClass(MultiHFileOutputFormat.class);
> but it generate different column Families(not correct) inside one
> folder(recursively).  I need to generate different column families in
> different folders.
>  And there is also lack of configureIncrementalLoad Function in your pointed
> file.
>
> Plz help me Andrew, Stack, i need your help.
>
>
> On Mon, Sep 26, 2011 at 6:48 PM, Shuja Rehman <shujamughal@gmail.com> wrote:
>
>> @arslan,
>> I did not get chance to look at it.
>>
>>
>> On Mon, Sep 26, 2011 at 12:11 PM, Arsalan Bilal <charsalanbilal@gmail.com>wrote:
>>
>>> I am also facing same kind of problem, I am using one Map only and need to
>>> generate multiple Hfiles for multiple table for insertion in Hbase?
>>> Have you found any solution ? Can u help me to find solution of this one?
>>>
>>>
>>> On Fri, Sep 23, 2011 at 8:11 PM, Andrew Purtell <apurtell@apache.org>wrote:
>>>
>>>> Try this: https://gist.github.com/1237770
>>>>
>>>> See line 135.
>>>>
>>>> Best regards,
>>>>
>>>>
>>>>    - Andy
>>>>
>>>>
>>>> Problems worthy of attack prove their worth by hitting back. - Piet Hein
>>>> (via Tom White)
>>>>
>>>>
>>>> >________________________________
>>>> >From: Shuja Rehman <shujamughal@gmail.com>
>>>> >To: user@hbase.apache.org; Andrew Purtell <apurtell@apache.org>
>>>> >Sent: Friday, September 23, 2011 8:18 AM
>>>> >Subject: Re: Creation of Hfiles for multiple tables using Single Bulk
>>>> Load Job?
>>>> >
>>>> >
>>>> >andrew,
>>>> >
>>>> >can you guide me where to change and what to change? it will help.
>>>> >thnx
>>>> >
>>>> >
>>>> >On Thu, Sep 22, 2011 at 9:35 PM, Shuja Rehman <shujamughal@gmail.com>
>>>> wrote:
>>>> >
>>>> >Can you update it so I can use it? :)
>>>> >>
>>>> >>Thanks
>>>> >>
>>>> >>
>>>> >>
>>>> >>On Thu, Sep 22, 2011 at 8:40 PM, Andrew Purtell <apurtell@apache.org>
>>>> wrote:
>>>> >>
>>>> >>> From: Shuja Rehman <shujamughal@gmail.com>
>>>> >>>
>>>> >>>
>>>> >>>> I am using bulk load to insert data into hbase. Its runs
fine if I
>>>> need to
>>>> >>>> insert in one table. But Now, I have the requirement in
which I need
>>>> to
>>>> >>>> insert data into more than  one table.
>>>> >>>
>>>> >>>
>>>> >>>We started some support for this here:
>>>> https://issues.apache.org/jira/browse/HBASE-3727
>>>> >>>
>>>> >>>It needs to be updated for HFile v2, I think, a couple of single
line
>>>> changes.
>>>> >>>
>>>> >>>Best regards,
>>>> >>>
>>>> >>>
>>>> >>>   - Andy
>>>> >>>
>>>> >>>Problems worthy of attack prove their worth by hitting back.
- Piet
>>>> Hein (via Tom White)
>>>> >>>
>>>> >>>
>>>> >>>
>>>> >>>----- Original Message -----
>>>> >>>> From: Shuja Rehman <shujamughal@gmail.com>
>>>> >>>> To: user@hbase.apache.org
>>>> >>>> Cc:
>>>> >>>> Sent: Thursday, September 22, 2011 8:01 AM
>>>> >>>> Subject: Creation of Hfiles for multiple tables using Single
Bulk
>>>> Load Job?
>>>> >>>>
>>>> >>>> Hi
>>>> >>>>
>>>> >>>> I am using bulk load to insert data into hbase. Its runs
fine if I
>>>> need to
>>>> >>>> insert in one table. But Now, I have the requirement in
which I need
>>>> to
>>>> >>>> insert data into more than  one table. But it seems that
using bulk
>>>> load, we
>>>> >>>> can generate hfiles for 1 table only.  So the question
is that is
>>>> there any
>>>> >>>> solution which enable us to generate hfiles in multiple
output paths
>>>> and
>>>> >>>> then we can load these different files in different hbase
tables.
>>>> >>>>
>>>> >>>> Thanks in advance
>>>> >>>>
>>>> >>>>
>>>> >>>> --
>>>> >>>> Regards
>>>> >>>> Shuja-ur-Rehman Baig
>>>> >>>
>>>> >>>> <http://pk.linkedin.com/in/shujamughal>
>>>> >>>>
>>>> >>>
>>>> >>
>>>> >>
>>>> >>--
>>>> >>Regards
>>>> >>Shuja-ur-Rehman Baig
>>>> >>
>>>> >>
>>>> >>
>>>> >
>>>> >
>>>> >--
>>>> >Regards
>>>> >Shuja-ur-Rehman Baig
>>>> >
>>>> >
>>>> >
>>>> >
>>>> >
>>>>
>>>
>>>
>>>
>>> --
>>> Best Regards,
>>> Arsalan Bilal
>>>
>>>
>>>
>>
>>
>> --
>> Regards
>> Shuja-ur-Rehman Baig
>> <http://pk.linkedin.com/in/shujamughal>
>>
>>
>
>
> --
> Best Regards,
> Arsalan Bilal
>

Mime
View raw message