Return-Path: X-Original-To: apmail-avro-commits-archive@www.apache.org Delivered-To: apmail-avro-commits-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id B649C1727E for ; Wed, 7 Oct 2015 21:06:32 +0000 (UTC) Received: (qmail 61119 invoked by uid 500); 7 Oct 2015 21:06:32 -0000 Delivered-To: apmail-avro-commits-archive@avro.apache.org Received: (qmail 61083 invoked by uid 500); 7 Oct 2015 21:06:32 -0000 Mailing-List: contact commits-help@avro.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@avro.apache.org Delivered-To: mailing list commits@avro.apache.org Received: (qmail 61074 invoked by uid 99); 7 Oct 2015 21:06:32 -0000 Received: from Unknown (HELO spamd4-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 07 Oct 2015 21:06:32 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd4-us-west.apache.org (ASF Mail Server at spamd4-us-west.apache.org) with ESMTP id 29E8DC028F for ; Wed, 7 Oct 2015 21:06:32 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd4-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: 1.79 X-Spam-Level: * X-Spam-Status: No, score=1.79 tagged_above=-999 required=6.31 tests=[KAM_ASCII_DIVIDERS=0.8, KAM_LAZY_DOMAIN_SECURITY=1, T_RP_MATCHES_RCVD=-0.01] autolearn=disabled Received: from mx1-eu-west.apache.org ([10.40.0.8]) by localhost (spamd4-us-west.apache.org [10.40.0.11]) (amavisd-new, port 10024) with ESMTP id DUtnXH01QhhX for ; Wed, 7 Oct 2015 21:06:30 +0000 (UTC) Received: from mailrelay1-us-west.apache.org (mailrelay1-us-west.apache.org [209.188.14.139]) by mx1-eu-west.apache.org (ASF Mail Server at mx1-eu-west.apache.org) with ESMTP id C9D6227DB4 for ; Wed, 7 Oct 2015 21:06:29 +0000 (UTC) Received: from svn01-us-west.apache.org (svn.apache.org [10.41.0.6]) by mailrelay1-us-west.apache.org (ASF Mail Server at mailrelay1-us-west.apache.org) with ESMTP id EB1D1E0256 for ; Wed, 7 Oct 2015 21:06:28 +0000 (UTC) Received: from svn01-us-west.apache.org (localhost [127.0.0.1]) by svn01-us-west.apache.org (ASF Mail Server at svn01-us-west.apache.org) with ESMTP id E778C3A01E3 for ; Wed, 7 Oct 2015 21:06:28 +0000 (UTC) Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Subject: svn commit: r1707389 - in /avro/trunk: ./ lang/java/avro/src/main/java/org/apache/avro/ lang/java/tools/src/main/java/org/apache/avro/tool/ Date: Wed, 07 Oct 2015 21:06:28 -0000 To: commits@avro.apache.org From: blue@apache.org X-Mailer: svnmailer-1.0.9 Message-Id: <20151007210628.E778C3A01E3@svn01-us-west.apache.org> Author: blue Date: Wed Oct 7 21:06:28 2015 New Revision: 1707389 URL: http://svn.apache.org/viewvc?rev=1707389&view=rev Log: AVRO-1715. Java: Close files opened by the Schema parser. Contributed by Pavel Safrata. This also clarifies the InputStream parse method, which does not own the stream that is passed in and does not close it. The tools have been updated to correctly close streams passed to the parser. Modified: avro/trunk/CHANGES.txt avro/trunk/lang/java/avro/src/main/java/org/apache/avro/Schema.java avro/trunk/lang/java/tools/src/main/java/org/apache/avro/tool/BinaryFragmentToJsonTool.java avro/trunk/lang/java/tools/src/main/java/org/apache/avro/tool/CreateRandomFileTool.java avro/trunk/lang/java/tools/src/main/java/org/apache/avro/tool/DataFileWriteTool.java avro/trunk/lang/java/tools/src/main/java/org/apache/avro/tool/JsonToBinaryFragmentTool.java avro/trunk/lang/java/tools/src/main/java/org/apache/avro/tool/Util.java Modified: avro/trunk/CHANGES.txt URL: http://svn.apache.org/viewvc/avro/trunk/CHANGES.txt?rev=1707389&r1=1707388&r2=1707389&view=diff ============================================================================== --- avro/trunk/CHANGES.txt (original) +++ avro/trunk/CHANGES.txt Wed Oct 7 21:06:28 2015 @@ -195,6 +195,9 @@ Avro 1.8.0 (10 August 2014) AVRO-1700. C++: Fix avro_BufferStreambuf_hh__ header guard. (Liu Yanbo via blue) + AVRO-1715. Java: Close files opened by the Schema parser. + (Pavel Safrata via blue) + Avro 1.7.7 (23 July 2014) NEW FEATURES Modified: avro/trunk/lang/java/avro/src/main/java/org/apache/avro/Schema.java URL: http://svn.apache.org/viewvc/avro/trunk/lang/java/avro/src/main/java/org/apache/avro/Schema.java?rev=1707389&r1=1707388&r2=1707389&view=diff ============================================================================== --- avro/trunk/lang/java/avro/src/main/java/org/apache/avro/Schema.java (original) +++ avro/trunk/lang/java/avro/src/main/java/org/apache/avro/Schema.java Wed Oct 7 21:06:28 2015 @@ -997,9 +997,11 @@ public abstract class Schema extends Jso } /** Parse a schema from the provided stream. - * If named, the schema is added to the names known to this parser. */ + * If named, the schema is added to the names known to this parser. + * The input stream stays open after the parsing. */ public Schema parse(InputStream in) throws IOException { - return parse(FACTORY.createJsonParser(in)); + return parse(FACTORY.createJsonParser(in).disable( + JsonParser.Feature.AUTO_CLOSE_SOURCE)); } /** Read a schema from one or more json strings */ @@ -1030,6 +1032,7 @@ public abstract class Schema extends Jso } catch (JsonParseException e) { throw new SchemaParseException(e); } finally { + parser.close(); validateNames.set(saved); VALIDATE_DEFAULTS.set(savedValidateDefaults); } Modified: avro/trunk/lang/java/tools/src/main/java/org/apache/avro/tool/BinaryFragmentToJsonTool.java URL: http://svn.apache.org/viewvc/avro/trunk/lang/java/tools/src/main/java/org/apache/avro/tool/BinaryFragmentToJsonTool.java?rev=1707389&r1=1707388&r2=1707389&view=diff ============================================================================== --- avro/trunk/lang/java/tools/src/main/java/org/apache/avro/tool/BinaryFragmentToJsonTool.java (original) +++ avro/trunk/lang/java/tools/src/main/java/org/apache/avro/tool/BinaryFragmentToJsonTool.java Wed Oct 7 21:06:28 2015 @@ -66,7 +66,7 @@ public class BinaryFragmentToJsonTool im schema = new Schema.Parser().parse(nargs.get(0)); inputFile = nargs.get(1); } else { - schema = new Schema.Parser().parse(Util.openFromFS(schemaFile)); + schema = Util.parseSchemaFromFS(schemaFile); inputFile = nargs.get(0); } InputStream input = Util.fileOrStdin(inputFile, stdin); Modified: avro/trunk/lang/java/tools/src/main/java/org/apache/avro/tool/CreateRandomFileTool.java URL: http://svn.apache.org/viewvc/avro/trunk/lang/java/tools/src/main/java/org/apache/avro/tool/CreateRandomFileTool.java?rev=1707389&r1=1707388&r2=1707389&view=diff ============================================================================== --- avro/trunk/lang/java/tools/src/main/java/org/apache/avro/tool/CreateRandomFileTool.java (original) +++ avro/trunk/lang/java/tools/src/main/java/org/apache/avro/tool/CreateRandomFileTool.java Wed Oct 7 21:06:28 2015 @@ -78,7 +78,7 @@ public class CreateRandomFileTool implem return 1; } Schema schema = (schemafile != null) - ? new Schema.Parser().parse(Util.openFromFS(schemafile)) + ? Util.parseSchemaFromFS(schemafile) : new Schema.Parser().parse(schemastr); DataFileWriter writer = Modified: avro/trunk/lang/java/tools/src/main/java/org/apache/avro/tool/DataFileWriteTool.java URL: http://svn.apache.org/viewvc/avro/trunk/lang/java/tools/src/main/java/org/apache/avro/tool/DataFileWriteTool.java?rev=1707389&r1=1707388&r2=1707389&view=diff ============================================================================== --- avro/trunk/lang/java/tools/src/main/java/org/apache/avro/tool/DataFileWriteTool.java (original) +++ avro/trunk/lang/java/tools/src/main/java/org/apache/avro/tool/DataFileWriteTool.java Wed Oct 7 21:06:28 2015 @@ -80,7 +80,7 @@ public class DataFileWriteTool implement return 1; } Schema schema = (schemafile != null) - ? new Schema.Parser().parse(Util.openFromFS(schemafile)) + ? Util.parseSchemaFromFS(schemafile) : new Schema.Parser().parse(schemastr); DatumReader reader = new GenericDatumReader(schema); Modified: avro/trunk/lang/java/tools/src/main/java/org/apache/avro/tool/JsonToBinaryFragmentTool.java URL: http://svn.apache.org/viewvc/avro/trunk/lang/java/tools/src/main/java/org/apache/avro/tool/JsonToBinaryFragmentTool.java?rev=1707389&r1=1707388&r2=1707389&view=diff ============================================================================== --- avro/trunk/lang/java/tools/src/main/java/org/apache/avro/tool/JsonToBinaryFragmentTool.java (original) +++ avro/trunk/lang/java/tools/src/main/java/org/apache/avro/tool/JsonToBinaryFragmentTool.java Wed Oct 7 21:06:28 2015 @@ -62,7 +62,7 @@ public class JsonToBinaryFragmentTool im schema = new Schema.Parser().parse(nargs.get(0)); inputFile = nargs.get(1); } else { - schema = new Schema.Parser().parse(Util.openFromFS(schemaFile)); + schema = Util.parseSchemaFromFS(schemaFile); inputFile = nargs.get(0); } InputStream input = Util.fileOrStdin(inputFile, stdin); Modified: avro/trunk/lang/java/tools/src/main/java/org/apache/avro/tool/Util.java URL: http://svn.apache.org/viewvc/avro/trunk/lang/java/tools/src/main/java/org/apache/avro/tool/Util.java?rev=1707389&r1=1707388&r2=1707389&view=diff ============================================================================== --- avro/trunk/lang/java/tools/src/main/java/org/apache/avro/tool/Util.java (original) +++ avro/trunk/lang/java/tools/src/main/java/org/apache/avro/tool/Util.java Wed Oct 7 21:06:28 2015 @@ -153,6 +153,21 @@ class Util { } } + /** + * Parses a schema from the specified file. + * @param filename The file name to parse + * @return The parsed schema + * @throws IOException + */ + static Schema parseSchemaFromFS(String filename) throws IOException { + InputStream stream = openFromFS(filename); + try { + return new Schema.Parser().parse(stream); + } finally { + close(stream); + } + } + /**If pathname is a file, this method returns a list with a single absolute Path to that file, * if pathname is a directory, this method returns a list of Pathes to all the files within * this directory.