flume-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From lancexxx <fervent.li...@gmail.com>
Subject Re: hdfs sink output mojibake
Date Sat, 03 Nov 2012 12:29:41 GMT
the logs of flume with no error,flume-ng looks like completely normal except the web logs what
collecting and output by flume-ng on hdfs is messy code 。
please look at the e-mail attachment.  

--  
lancexxx


On 2012年11月3日Saturday at 下午3:06, Alexander Lorenz wrote:

> Please attach the logs or console debug messages. What's exactly the issue?
>  
> - alex  
>  
> On Nov 2, 2012, at 4:16 AM, lancexxx <fervent.linux@gmail.com (mailto:fervent.linux@gmail.com)>
wrote:
>  
> > I got a error seems mojibake.
> >  
> >  
> >  
> > this is my flume.conf,is it any wrong?
> > ---------
> > agent.sources = source1
> > agent.channels = channel1
> > agent.sinks = sink1
> >  
> > agent.channels.channel1.type = memory
> > agent.channels.channel1.capacity = 100000
> > agent.channels.channel1.transactionCapacity = 8000
> > agent.channels.channel1.keep-alive = 3
> >  
> > agent.sources.source1.type = avro
> > agent.sources.source1.bind = 0.0.0.0
> > agent.sources.source1.port = 4141
> > agent.sources.source1.channels = channel1
> >  
> >  
> >  
> > agent.sinks.sink1.type = hdfs
> > agent.sinks.sink1.hdfs.path = hdfs://flume:9000/g
> > agent.sinks.sink1.hdfs.fileType = DataStream
> > agent.sinks.sink1.hdfs.rollInterval = 300
> > agent.sinks.sink1.hdfs.rollSize = 0
> > agent.sinks.sink1.hdfs.rollCount = 0
> > agent.sinks.sink1.hdfs.batchSize = 1000
> > agent.sinks.sink1.serializer = AVRO_EVENT
> > agent.sinks.sink1.serializer.compressionCodec = snappy
> > agent.sinks.sink1.channel = channel1
> > -------------
> >  
> >  
> > --
> > lancexxx
> >  
> > <_g_FlumeData.1351825007314.tmp>
>  
> --
> Alexander Alten-Lorenz
> http://mapredit.blogspot.com
> German Hadoop LinkedIn Group: http://goo.gl/N8pCF
>  
>  



Mime
View raw message