avro-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ryan Blue (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (AVRO-1715) Compiling a schema leaves the schema file open
Date Fri, 02 Oct 2015 22:49:27 GMT

    [ https://issues.apache.org/jira/browse/AVRO-1715?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14941914#comment-14941914

Ryan Blue commented on AVRO-1715:

Thanks for looking at this, Pavel. My main concern with your patch is that it calls close
on a parser that is passed into the internal parse method. That parser could be instantiated
for an InputStream, Reader, or File. I think that it should only be closed for the File case
because the underlying stream must be closed. I'd prefer to put the try/finally in just the
{{public Schema parse(File)}} method.

For the other two input sources, InputStream and Reader, the situation is a little more difficult
because close wasn't being called before. So streams that were passed in weren't being closed
by the parser. In that case, we should continue to not close those streams, but the parser
by default closes the underlying stream because Jackson's Feature.AUTO_CLOSE_SOURCE defaults
to enabled.

We should probably disable that auto close feature and call close on the reader when done,
so I'm fine with either a solution that does that or one that just wraps the file variant
in a try/finally. Do you agree? If so, could you update your patch with one of those strategies?

> Compiling a schema leaves the schema file open
> ----------------------------------------------
>                 Key: AVRO-1715
>                 URL: https://issues.apache.org/jira/browse/AVRO-1715
>             Project: Avro
>          Issue Type: Bug
>          Components: java
>    Affects Versions: 1.7.7, 1.8.0
>            Reporter: Pavel Safrata
>             Fix For: 1.8.0
>         Attachments: Schema_close_parser.patch
> The schema compiler leaves the schema file open. This is demonstrated on the following
> File schemaFile = new File("schema.avsc");
> SpecificCompiler.compileSchema(new File[] { schemaFile }, new File("output"));
> Files.delete(schemaFile.toPath());
> On Windows I get "java.nio.file.FileSystemException: schema.avsc: The process cannot
access the file because it is being used by another process."
> The problem is in Schema.java, method parse(File). This method creates a JsonParser for
the schema file but never closes it, so the parser stays open together with the FileInputStream
created by it. Closing the parser fixes the problem. I believe the same should be done for
the other types of inputs as the json parsers do use more resources that should be freed.
So I propose to close the parser in the parse(JsonParser) method. I will attach the patch.
> This is almost a blocker for our project and I would really appreciate if we can have
this resolved for 1.8. Please let me know if that's possible and whether I should go ahead
with merging the change myself (it's my first contribution here).

This message was sent by Atlassian JIRA

View raw message