hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Xiaobin She <xiaobin...@gmail.com>
Subject Can I write to an compressed file which is located in hdfs?
Date Mon, 06 Feb 2012 08:40:23 GMT
hi all,

I'm testing hadoop and hive, and I want to use them in log analysis.

Here I have a question, can I write/append log to  an compressed file which
is located in hdfs?

Our system generate lots of log files every day, I can't compress these
logs every hour and them put them into hdfs.

But what if I want to write logs into files that was already in the hdfs
and was compressed?

Is these files were not compressed, then this job seems easy, but how to
write or append logs into an compressed log?

Can I do that?

Can anyone give me some advices or give me some examples?

Thank you very much!


  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message