flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From kl0u <...@git.apache.org>
Subject [GitHub] flink issue #2109: [FLINK-3677] FileInputFormat: Allow to specify include/ex...
Date Fri, 08 Jul 2016 14:29:46 GMT
Github user kl0u commented on the issue:

    https://github.com/apache/flink/pull/2109
  
    Hi @mushketyk ,
    
    Sorry for the late reply. 
    I have some general comments on the PR.
    
    1) Given the FilePathFilter, I think that the GlobFilePathFilter is redundant, right?
It is a specific implementation that uses pattern matching. This can be provided by the programmer.
Given this, in the FileInputFormat, the filesFilter (which we could change the name to filePathFilter
or sth more expressive of its function) becomes a FilePathFilter.
    
    2) Given that the filter is now in the FileInputFormat, then the ContinuousFileMonitoringFunction
should change, as now it is the format that does the filtering. So the filter should be removed
from its constructor and the shouldIgnore() method becomes redundant.
    
    3) The affected methods in the StreamExecutionEnvironment should change too.
    
    It could help if you shared a design draft before integrating the changes, so that we
can discuss on them and figure out all the parts in the code that change and need testing.
    
    What do you think?



---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

Mime
View raw message