parquet-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From b...@apache.org
Subject [25/50] [abbrv] parquet-mr git commit: PARQUET-560: Synchronize writes to the finishCalled variable
Date Thu, 19 Jan 2017 01:27:36 GMT
PARQUET-560: Synchronize writes to the finishCalled variable

Reads of the `finishCalled` variable are properly synchronized, but writes are not -- so there's
some sort of inconsistent synch. going on here. This PR fixes that.

/cc @rdblue can you please take a look?

Author: Nezih Yigitbasi <nyigitbasi@netflix.com>

Closes #334 from nezihyigitbasi/sc-synch-fix and squashes the following commits:

a85cf0c [Nezih Yigitbasi] Synchronize writes to the finishCalled variable


Project: http://git-wip-us.apache.org/repos/asf/parquet-mr/repo
Commit: http://git-wip-us.apache.org/repos/asf/parquet-mr/commit/45e673fc
Tree: http://git-wip-us.apache.org/repos/asf/parquet-mr/tree/45e673fc
Diff: http://git-wip-us.apache.org/repos/asf/parquet-mr/diff/45e673fc

Branch: refs/heads/parquet-1.8.x
Commit: 45e673fc955177fad0aa46fd57daad45c092a3ae
Parents: fd22747
Author: Nezih Yigitbasi <nyigitbasi@netflix.com>
Authored: Mon Apr 25 15:05:11 2016 -0700
Committer: Ryan Blue <blue@apache.org>
Committed: Mon Jan 9 16:54:54 2017 -0800

----------------------------------------------------------------------
 .../java/org/apache/parquet/hadoop/codec/SnappyCompressor.java     | 2 +-
 pom.xml                                                            | 1 +
 2 files changed, 2 insertions(+), 1 deletion(-)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/parquet-mr/blob/45e673fc/parquet-hadoop/src/main/java/org/apache/parquet/hadoop/codec/SnappyCompressor.java
----------------------------------------------------------------------
diff --git a/parquet-hadoop/src/main/java/org/apache/parquet/hadoop/codec/SnappyCompressor.java
b/parquet-hadoop/src/main/java/org/apache/parquet/hadoop/codec/SnappyCompressor.java
index f099896..d0270ca 100644
--- a/parquet-hadoop/src/main/java/org/apache/parquet/hadoop/codec/SnappyCompressor.java
+++ b/parquet-hadoop/src/main/java/org/apache/parquet/hadoop/codec/SnappyCompressor.java
@@ -113,7 +113,7 @@ public class SnappyCompressor implements Compressor {
   }
 
   @Override
-  public void finish() {
+  public synchronized void finish() {
     finishCalled = true;
   }
 

http://git-wip-us.apache.org/repos/asf/parquet-mr/blob/45e673fc/pom.xml
----------------------------------------------------------------------
diff --git a/pom.xml b/pom.xml
index 08e1730..ba7e282 100644
--- a/pom.xml
+++ b/pom.xml
@@ -234,6 +234,7 @@
                      <exclude>org/apache/parquet/avro/SpecificDataSupplier</exclude>
<!-- made public -->
                      <exclude>org/apache/parquet/io/ColumnIOFactory$ColumnIOCreatorVisitor</exclude>
<!-- removed non-API class -->
                      <exclude>org/apache/parquet/io/ColumnIOFactory/**</exclude>
<!-- removed non-API class and methods-->
+		     <exclude>org/apache/parquet/hadoop/codec/SnappyCompressor</exclude> <!--
added synchronized modifier -->
                    </excludes>
                  </requireBackwardCompatibility>
                </rules>


Mime
View raw message