avro-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Mulugeta Mammo (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (AVRO-1760) Thread scalability problem with the use of SynchronizedMap
Date Mon, 23 Nov 2015 18:25:11 GMT

     [ https://issues.apache.org/jira/browse/AVRO-1760?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Mulugeta Mammo updated AVRO-1760:
---------------------------------
    Status: Patch Available  (was: Open)

--- https://github.com/apache/avro/blob/trunk/lang/java/avro/src/main/java/org/apache/avro/generic/GenericData.java
       2015-11-19 13:53:10.238342446 -0800
+++ avro/lang/java/avro/src/main/java/org/apache/avro/generic/GenericData.java  2015-11-18
15:10:01.217569133 -0800
@@ -23,7 +23,6 @@
 import java.util.AbstractList;
 import java.util.Arrays;
 import java.util.Collection;
-import java.util.Collections;
 import java.util.HashMap;
 import java.util.IdentityHashMap;
 import java.util.LinkedHashMap;
@@ -31,6 +30,7 @@
 import java.util.Iterator;
 import java.util.List;
 import java.util.Map;
+import java.util.concurrent.ConcurrentHashMap;

 import org.apache.avro.AvroRuntimeException;
 import org.apache.avro.AvroTypeException;
@@ -969,7 +969,7 @@
   }

   private final Map<Field, Object> defaultValueCache
-    = Collections.synchronizedMap(new WeakHashMap<Field, Object>());
+    = new ConcurrentHashMap<Field, Object>(new WeakHashMap<Field, Object>());

   /**
    * Gets the default value of the given field, if any.


> Thread scalability problem with the use of SynchronizedMap
> ----------------------------------------------------------
>
>                 Key: AVRO-1760
>                 URL: https://issues.apache.org/jira/browse/AVRO-1760
>             Project: Avro
>          Issue Type: Improvement
>          Components: java
>    Affects Versions: 1.7.7, 1.7.6, 1.7.5, 1.7.4, 1.8.0, 1.8.1, 1.9.0
>            Reporter: Mulugeta Mammo
>              Labels: patch, performance
>             Fix For: 1.7.8, 1.8.0, 1.8.1, 1.9.0, 1.7.7, 1.7.6, 1.7.5, 1.7.4
>
>
> While running Adam Genomics (which uses Avro) on Apache Spark, we discovered that threads
(tasks in Spark Context) block in Avro while executing the getDefaultValue(Field field) method
in https://github.com/apache/avro/blob/trunk/lang/java/avro/src/main/java/org/apache/avro/generic/GenericData.java



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message