hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "S. Venkatesh" <venkat...@innerzeal.com>
Subject Re: Importing log files from various machines
Date Tue, 29 Jun 2010 04:34:21 GMT
You could write a simple map-only job with each map pulling a bunch of
files from each of the servers. You could use a NLineInputFormat and
tweak N based on the # of maps, # of files, etc.

Venkatesh

On Tue, Jun 29, 2010 at 5:40 AM, Blargy <zmanods@hotmail.com> wrote:
>
> I am currently looking into importing all of our application log files (~100+
> host machines) into HDFS. Can someone point me in the right direction or
> walk me through the process of how I can accomplish this? Any good reading
> material on this subject? Videos?
>
> I hope I don't need to physically copy all of the log files to one target
> machine before importing.
>
> Thanks
> --
> View this message in context: http://lucene.472066.n3.nabble.com/Importing-log-files-from-various-machines-tp929423p929423.html
> Sent from the Hadoop lucene-users mailing list archive at Nabble.com.
>



-- 
Regards,
Venkatesh

“Perfection (in design) is achieved not when there is nothing more to
add, but rather when there is nothing more to take away.”
- Antoine de Saint-Exupéry

Mime
View raw message