hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Xiaobin She <xiaobin...@gmail.com>
Subject Re: Can I write to an compressed file which is located in hdfs?
Date Mon, 06 Feb 2012 08:41:50 GMT
sorry, this sentence is wrong,

I can't compress these logs every hour and them put them into hdfs.

it should be

I can  compress these logs every hour and them put them into hdfs.




2012/2/6 Xiaobin She <xiaobinshe@gmail.com>

>
> hi all,
>
> I'm testing hadoop and hive, and I want to use them in log analysis.
>
> Here I have a question, can I write/append log to  an compressed file
> which is located in hdfs?
>
> Our system generate lots of log files every day, I can't compress these
> logs every hour and them put them into hdfs.
>
> But what if I want to write logs into files that was already in the hdfs
> and was compressed?
>
> Is these files were not compressed, then this job seems easy, but how to
> write or append logs into an compressed log?
>
> Can I do that?
>
> Can anyone give me some advices or give me some examples?
>
> Thank you very much!
>
> xiaobin
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message