flume-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From alo alt <wget.n...@googlemail.com>
Subject Re: RollingFile Sink problems
Date Tue, 22 May 2012 08:28:32 GMT
The .sink was in the main config available, but the identifier not. 

@ Guillaume
agent1.sinks.file-sink-1.sink.channel = ch1
agent1.sinks.file-sink-1.sink.directory = /foo/bar
agent1.sinks.file-sink-1.sink.type = file_roll

works.

Not the "file-sink-1" identifier after .sinks.

I was reopened the Jira, but it is a doc gap, so I close again (sorry Mike). I'll file a doc
jira about.

Thanks
- Alex


--
Alexander Alten-Lorenz
http://mapredit.blogspot.com
German Hadoop LinkedIn Group: http://goo.gl/N8pCF

On May 22, 2012, at 10:11 AM, Mike Percy wrote:

> Some of the flume NG config options are a little strange. This is one of them.
> 
> You have to specify agent.sinks.file-sink.sink.directory = /some/dir
> 
> It doesn't make a lot of sense, but it's historical.
> 
> HTH,
> Mike
> 
> On Monday, May 21, 2012 at 11:57 PM, Guillaume Polaert wrote:
> 
>> Thank you for helping me out.
>> 
>> Guillaume Polaert | Cyrès Conseil
>> 
>> 
>> -----Message d'origine-----
>> De : alo alt [mailto:wget.null@googlemail.com]
>> Envoyé : mardi 22 mai 2012 08:34
>> À : flume-user@incubator.apache.org
>> Objet : Re: RollingFile Sink problems
>> 
>> created https://issues.apache.org/jira/browse/FLUME-1222
>> 
>> --
>> Alexander Alten-Lorenz
>> http://mapredit.blogspot.com
>> German Hadoop LinkedIn Group: http://goo.gl/N8pCF
>> 
>> On May 21, 2012, at 6:27 PM, alo alt wrote:
>> 
>>> Looks like a config.parser issue in RollingFileSink.java or Configuration.java.
Please open a Jira for.
>>> The error messages comes from the Preconditions check (Google Guava library)
in RollingFileSink.java, indeed the variable for the directory is empty.
>>> 
>>> - Alex
>>> 
>>> --
>>> Alexander Alten-Lorenz
>>> http://mapredit.blogspot.com
>>> German Hadoop LinkedIn Group: http://goo.gl/N8pCF
>>> 
>>> On May 21, 2012, at 3:50 PM, Guillaume Polaert wrote:
>>> 
>>>> Thank you for the link.
>>>> I've created a empty directory /tmp/log (mkdir+chmod).
>>>> 
>>>> Do I have to create other things ?
>>>> 
>>>> -----Message d'origine-----
>>>> De : alo alt [mailto:wget.null@googlemail.com] Envoyé : lundi 21 mai
>>>> 2012 15:31 À : flume-user@incubator.apache.org Objet : Re:
>>>> RollingFile Sink problems
>>>> 
>>>> Hi,
>>>> 
>>>> http://archive.cloudera.com/cdh4/cdh/4/flume-ng-1.1.0-cdh4.0.0b2/Flum
>>>> eUserGuide.html#h.frlqql6vu68
>>>> 
>>>> Means, /tmp/log is empty or non existent, isn't it?
>>>> 
>>>> - Alex
>>>> 
>>>> --
>>>> Alexander Alten-Lorenz
>>>> http://mapredit.blogspot.com
>>>> German Hadoop LinkedIn Group: http://goo.gl/N8pCF
>>>> 
>>>> On May 21, 2012, at 3:03 PM, Guillaume Polaert wrote:
>>>> 
>>>>> Hi everybody,
>>>>> 
>>>>> I've problems with RollingFile sink.
>>>>> First, I'm seeking about the sink's properties. I've found two properties
in RollingFileSink.java : rollInterval and directory. Are there anymore ?
>>>>> 
>>>>> Then, I've this error when I'm running the node :
>>>>> 
>>>>> .
>>>>> SINKS: {sink=ComponentConfiguration[sink]
>>>>> CONFIG: {directory=/tmp/log, type=file_roll}
>>>>> RUNNER: ComponentConfiguration[runner]
>>>>> CONFIG: {}
>>>>> 
>>>>> 
>>>>> , log-sink1=ComponentConfiguration[log-sink1]
>>>>> CONFIG: {type=logger, channel=ch1}
>>>>> RUNNER: ComponentConfiguration[runner]
>>>>> CONFIG: {}
>>>>> 
>>>>> }
>>>>> .
>>>>> 2012-05-21 08:29:15,246 (conf-file-poller-0) [INFO -
>>>>> org.apache.flume.sink.DefaultSinkFactory.create(DefaultSinkFactory.j
>>>>> av a:69)] Creating instance of sink sink typefile_roll
>>>>> 2012-05-21 08:29:15,253 (conf-file-poller-0) [ERROR - org.apache.flume.conf.file.AbstractFileConfigurationProvider$FileWatcherRunnable.run(AbstractFileConfigurationProvider.java:205)]
Failed to load configuration data. Exception follows.
>>>>> java.lang.IllegalArgumentException: Directory may not be null
>>>>> at com.google.common.base.Preconditions.checkArgument(Preconditions.java:88)
>>>>> at org.apache.flume.sink.RollingFileSink.configure(RollingFileSink.java:74)
>>>>> at
>>>>> org.apache.flume.conf.Configurables.configure(Configurables.java:41)
>>>>> .
>>>>> 
>>>>> Flume.conf
>>>>> 
>>>>> # Define a memory channel called ch1 on agent1
>>>>> agent1.channels.ch1.type = memory
>>>>> 
>>>>> # Define an Avro source called avro-source1 on agent1 and tell it #
>>>>> to bind to 0.0.0.0:41414. Connect it to channel ch1.
>>>>> agent1.sources.avro-source1.channels = ch1
>>>>> agent1.sources.avro-source1.type = avro
>>>>> agent1.sources.avro-source1.bind = 0.0.0.0
>>>>> agent1.sources.avro-source1.port = 41414
>>>>> 
>>>>> # Define a logger sink that simply logs all events it receives # and
>>>>> connect it to the other end of the same channel.
>>>>> agent1.sinks.log-sink1.channel = ch1 agent1.sinks.log-sink1.type =
>>>>> logger agent1.sinks.sink.channel = ch1 agent1.sinks.sink.type =
>>>>> file_roll agent1.sinks.sink.directory=/tmp/log
>>>>> 
>>>>> # Finally, now that we've defined all of our components, tell #
>>>>> agent1 which ones we want to activate.
>>>>> agent1.channels = ch1
>>>>> agent1.sources = avro-source1
>>>>> agent1.sinks = sink
>>>>> 
>>>>> What is wrong ? I don't understand the problem.
>>>>> 
>>>>> Guillaume
> 


Mime
View raw message