spark-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Apache Spark (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (SPARK-1795) Add recursive directory file search to fileInputStream
Date Sat, 19 Jul 2014 23:18:38 GMT

    [ https://issues.apache.org/jira/browse/SPARK-1795?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14067722#comment-14067722
] 

Apache Spark commented on SPARK-1795:
-------------------------------------

User 'patrickotoole' has created a pull request for this issue:
[https://github.com/apache/spark/pull/537|https://github.com/apache/spark/pull/537]

> Add recursive directory file search to fileInputStream
> ------------------------------------------------------
>
>                 Key: SPARK-1795
>                 URL: https://issues.apache.org/jira/browse/SPARK-1795
>             Project: Spark
>          Issue Type: Improvement
>          Components: Streaming
>            Reporter: Rick OToole
>
> When writing logs, they are often partitioned into a hierarchical directory structure.
This change will allow spark streaming to monitor all sub-directories of a parent directory
to find new files as they are added. 
> See https://github.com/apache/spark/pull/537



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Mime
View raw message