flume-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Tzur Turkenitz" <tz...@vision.bi>
Subject RE: SpoolDir sub-directories
Date Mon, 04 Feb 2013 16:41:03 GMT
Will do. Thanks!

 

From: Brock Noland [mailto:brock@cloudera.com] 
Sent: Monday, February 4, 2013 11:14 AM
To: user@flume.apache.org
Subject: Re: SpoolDir sub-directories

 

Hi,


I don't believe sub directories are supported at this time. Perhaps you
should file a JIRA for this improvement?

 

Brock

 

On Mon, Feb 4, 2013 at 10:01 AM, Tzur Turkenitz <tzurt@vision.bi
<mailto:tzurt@vision.bi> > wrote:

Hi,

I couldn't find any information about this in the Wiki / User guide.

Are sub directories for the Flume SpoolDir supported in 1.3x?

Thanks!

 

On Thu, Jan 31, 2013 at 11:56 AM, Tzur Turkenitz <tzurt@vision.bi
<mailto:tzurt@vision.bi> > wrote:

Hi All,

 

I am experimenting with Flumes` 1.3 new SpoolDir adapter.

I was wondering is there an option to tell the adapter to walk down
directories and search for files.

 

Currently, if I have a directory within the source location I get java
exceptions.

13/01/30 17:41:30 WARN avro.SpoolingFileLineReader: Could not find file:
/home/fl/source/dir

java.io.FileNotFoundException: /home/fl/source/dir (Is a directory)

at java.io.FileInputStream.open(Native Method)

at java.io.FileInputStream.<init>(FileInputStream.java:120)

at java.io.FileReader.<init>(FileReader.java:55)

at
org.apache.flume.client.avro.SpoolingFileLineReader.getNextFile(SpoolingFile
LineReader.java:322)

at
org.apache.flume.client.avro.SpoolingFileLineReader.readLines(SpoolingFileLi
neReader.java:172)

at
org.apache.flume.source.SpoolDirectorySource$SpoolDirectoryRunnable.run(Spoo
lDirectorySource.java:135)

at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)

at
java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:317)

at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:150)

at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$
101(ScheduledThreadPoolExecutor.java:98)

at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.runPeri
odic(ScheduledThreadPoolExecutor.java:180)

at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Sch
eduledThreadPoolExecutor.java:204)

at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.ja
va:886)

at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:9
08)

at java.lang.Thread.run(Thread.java:662)

Thanks!

 

 

 

-- 

Regards,

Tzur Turkenitz

Vision.BI 

 <http://www.vision.bi/> http://www.vision.bi/

 

"Facts are stubborn things, but statistics are more pliable"

-Mark Twain





 

-- 

Regards,

Tzur Turkenitz

Vision.BI 

 <http://www.vision.bi/> http://www.vision.bi/

 

"Facts are stubborn things, but statistics are more pliable"

-Mark Twain





 

-- 
Apache MRUnit - Unit testing MapReduce - http://incubator.apache.org/mrunit/



Mime
View raw message