taverna-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Stian Soiland-Reyes <st...@apache.org>
Subject CWL Browser: File vs Path when parsing *.cwl
Date Thu, 23 Jun 2016 10:43:56 GMT

Thanks for your latest pull request.

Now I wonder if we could change from using the classic java.util.File
to the more modern java.nio.file.Path here:



(In theory this would also allow you to read a whole Research Object
bundle ZIP file containing many CWL descriptions)

If you combine this with Java 8 lambdas this should also simplify how
you do directory browsing, file name filtering and file opening, e.g.
something like:

     try (DirectoryStream<Path> stream = Files.newDirectoryStream(dir,
          p -> p.endsWith(".cwl");
      )) {
          for (Path p : stream) {

Also I see you do callBack.partialResults() with a growing list - it's
not a big problem as duplicates are skipped based on their identifier,
but - you can do result.clear() in-between to just provide the new
partial results.  Perhaps we should try this on a large directory (say
150 *.cwl files?) to see if it's fast enough - you could in theory
parse multiple CWL files concurrently (but not too many!) and deliver
them independently.

If you want to play with Java 8 streams and paralellism, perhaps something like:

Stream<Path> s = StreamSupport.stream(stream.spliterator(), true);
s.forEach(this::parseCwlFile) ;

with a new method

void parserCwlFile(Path p);

that does the parsing and adding of its partial results.

Stian Soiland-Reyes
Apache Taverna (incubating), Apache Commons

View raw message