hive-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Saurabh B <saurabh.wri...@gmail.com>
Subject Re: Converting from textfile to sequencefile using Hive
Date Mon, 30 Sep 2013 18:54:56 GMT
Hi Nitin,

No offense taken. Thank you for your response. Part of this is also trying
to find the right tool for the job.

I am doing queries to determine the cuts of tweets that I want, then doing
some modest normalization (through a python script) and then I want to
create sequenceFiles from that.

So far Hive seems to be the most convenient way to do this. But I can take
a look at PIG too. It looked like the "STORED AS SEQUENCEFILE" gets me 99%
way there. So I was wondering if there was a way to get those ids in there
as well. The last piece is always the stumbler :)

Thanks again,

S




On Mon, Sep 30, 2013 at 2:41 PM, Nitin Pawar <nitinpawar432@gmail.com>wrote:

> are you using hive to just convert your text files to sequence files?
> If thats the case then you may want to look at the purpose why hive was
> developed.
>
> If you want to modify data or process data which does not involve any kind
> of analytics functions on a routine basis.
>
> If you want to do a data manipulation or enrichment and do not want to
> code a lot of map reduce job, you can take a look at pig scripts.
> basically what you want to do is generate an  UUID for each of your tweet
> and then feed it to mahout algorithms.
>
> Sorry if I understood it wrong or it sounds rude.
>

Mime
View raw message