hive-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Bejoy Ks <bejoy...@yahoo.com>
Subject Re: Twitter Data analyse with HIVE
Date Tue, 05 Jun 2012 16:07:33 GMT
Hi Babak

In very simple terms, you have a set of machines on linux OS. You may have your files in that
linux file system . From this linux file system you need to copy to hdfs. In hdfs create a
directory like you do in normal systems which ought to be your destination directory. copy
the file/dir from this linux file system to hdfs.


to list the contents in hdfs
hadoop fs -ls /

make a dir in hdfs
hadoop fs -mkdir /userdata/user/twitterData

copy from lfs to hdfs
hadoop fs -copyFromLocal /home/user/tweetData /userdata/user/twitterData 

It'll be much better if you can start off your hive experiments after reading 'Hadoop - The
Definitive Guide' by Tom White, It is a amazing  resource for hadoop starters.


Regards
Bejoy KS



________________________________
 From: Babak Bastan <babakbsn@gmail.com>
To: user@hive.apache.org; Bejoy Ks <bejoy_ks@yahoo.com> 
Sent: Tuesday, June 5, 2012 9:27 PM
Subject: Re: Twitter Data analyse with HIVE
 

Thank you for your answer
location of file in lfs

That means the location of my *txt file on my computer ? and I have no destination address
in hdfs,where can I get this location?
could you please write an example?


On Tue, Jun 5, 2012 at 4:29 PM, Bejoy Ks <bejoy_ks@yahoo.com> wrote:

Hi Babak
>
>
>There isn't anything called hdfs files. Hdfs is just a file system that can store any
type of file. You just need to transfer your file from lfs to hdfs and the following command
helps you out for that
>
>
>hadoop fs -copyFromLocal <location of file in lfs> <destination location in hdfs>
>
>
>Regards
>Bejoy KS
>
>
>
>________________________________
> From: Babak Bastan <babakbsn@gmail.com>
>To: user@hive.apache.org 
>Sent: Tuesday, June 5, 2012 7:54 PM
>Subject: Re: Twitter Data analyse with HIVE
> 
>
>ok, no difference for me records in a line or not
> 2009-06-08 21:49:37 - http://twitter.com/evionblablabla- I think data mining is awesome!
>2009-06-08 21:49:37 - http://twitter.com/hyungjinbliblibli -  I don’t think so. I
don’t like data mining
>....
>
>
>How can I do that.I think that I should change my text file to hdfs file,correct? how
can I do this one?
>Sorry I'm very new in this field :(
>
>
>On Tue, Jun 5, 2012 at 4:07 PM, Edward Capriolo <edlinuxguru@gmail.com> wrote:
>
>If you get output onto a single line it will be much easier for hive to process.
>>
>>
>>On Tue, Jun 5, 2012 at 5:20 AM, Babak Bastan <babakbsn@gmail.com> wrote:
>>> Hi experts
>>>
>>> I'm very new in Hive and Hadoop and I want to create a very simple demo to
>>> analyse sample twitts like this:
>>>
>>> T 2009-06-08 21:49:37
>>> U http://twitter.com/evion
>>> W I think data mining is awesome!
>>>
>>> T 2009-06-08 21:49:37
>>> U http://twitter.com/hyungjin
>>> W I don’t think so. I don’t like data mining
>>> ....
>>> Generally is it possible to do that?
>>> but I don't know exactly from which point should I strat.Do you know any
>>> simple and clear reference to do this job? or  would you please inform me
>>> (not in detail) what should I do?
>>>
>>> Thank you very much for your helps
>>> Babak
>>
>
>
>
Mime
View raw message