hadoop-common-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Apache Wiki <wikidi...@apache.org>
Subject [Hadoop Wiki] Update of "Chukwa Adaptors List" by AriRabkin
Date Thu, 18 Jun 2009 21:49:38 GMT
Dear Wiki user,

You have subscribed to a wiki page or wiki category on "Hadoop Wiki" for change notification.

The following page has been changed by AriRabkin:
http://wiki.apache.org/hadoop/Chukwa_Adaptors_List

------------------------------------------------------------------------------
  
  '''!FileAdaptor''':  Push a whole file, as one Chunk, then exit.  Requires file name as
parameter.
  
- Example:   {{{add FileTailer }}}
+ Example:   {{{add FileTailer FooData /tmp/foo 0}}}
+  This pushes file `/tmp/foo`, labelling the data as `FooData`.
  
- '''!filetailer.FileTailingAdaptor''': The 
+ '''filetailer.!FileTailingAdaptor''': 
+   Takes a file name as mandatory parameter.   
    Pushes the file as flat bytes, ignoring the content. Useful for streaming binary data.
+ 
+ Example:  `add filetailer.FileTailingAdaptor BarData /foo/bar 0`
+ 
- '''!filetailer.CharFileTailingAdaptorUTF8'''
+ '''filetailer.CharFileTailingAdaptorUTF8'''
   The same, but breaking chunks only at carriage returns.  Useful for most ASCII log file
formats.
- '''!filetailer.CharFileTailingAdaptorUT!F8NewLineEscaped'''  
+ '''filetailer.CharFileTailingAdaptorUT!F8NewLineEscaped'''  
   The same, but breaking chunks only at non-escaped carriage returns.  Useful for pushing
Chukwa-formatted log files, where exception stack traces stay in a single chunk.
+ '''DirTailingAdaptor'''  
+ Takes a directory path and a second adaptor name as mandatory parameters; repeatedly scans
that directory and all subdirectories, and starts the indicated adaptor running on each file.
 
  
+ Example: `add DirTailingAdaptor logs /var/log/ filetailer.CharFileTailingAdaptorUTF8 0`
+ 
+ '''ExecAdaptor'''
+ Takes a frequency (in miliseconds) as optional parameter, and then program name as mandatory
parameter.  Runs that program repeatedly at a rate specified by frequency.
+ 
+ Example:
+ `add ExecAdaptor Df 60000 /bin/df -x nfs -x none 0`
+ Runs `df` every minute, labelling output as `Df`.
+ 

Mime
View raw message