hadoop-general mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Arun C Murthy <...@yahoo-inc.com>
Subject Re: Question - Remote Logging to Hadoop
Date Mon, 18 May 2009 07:39:29 GMT

On May 17, 2009, at 1:35 AM, Scott Mackenzie wrote:

> All,
> Does anyone have a working method to capture system and application  
> logging for remote servers directly into HDFS in real-time?
> We have tested a few methods but all seem reliant on a batch process  
> run every 5 to 10 minutes.
> Any ideas would be appreciated?

Not exactly 'real-time', but you might want to take a look at Chukwa (http://svn.apache.org/viewvc/hadoop/chukwa/


Unfortunately I can't seem to find documentation for it, I'll try and  
push the chukwa-devs.


> Warm Regards,
> Scott
> ________________________________
> Disclaimer: This email and any attachments thereto are confidential  
> and it may contain information protected by intellectual Property  
> Laws or otherwise legally privileged. It is intended solely for the  
> use of the addressee(s). If you are not the intended recipient,  
> please notify the sender immediately and delete this email and any  
> attachments from your system and destroy any printout thereof. Any  
> unauthorized use or disclosure of this email and its attachments is  
> strictly prohibited and may be unlawful. MEEZA QSTP-LLC (MEEZA) does  
> not certify that this email or any attachments are free of viruses  
> or defects and recommends that you undertake your own virus checking  
> procedures before opening this e-mail or any attachments. MEEZA does  
> not accept any liability for any loss or damage resulting from the  
> use of this e-mail or any attachments. The contents of this e-mail  
> and any attachments do not necessarily represent the views or  
> polices of MEEZA. As the integrity of e-mails across the internet  
> cannot be guaranteed, MEEZA does not accept any liability for any e- 
> mails and attachments which may violate the relevant applicable laws.

View raw message