spark-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Josh Rosen (JIRA)" <j...@apache.org>
Subject [jira] [Created] (SPARK-8106) Set derby.system.durability=test in order to speed up Hive compatibility tests
Date Thu, 04 Jun 2015 20:05:38 GMT
Josh Rosen created SPARK-8106:
---------------------------------

             Summary: Set derby.system.durability=test in order to speed up Hive compatibility
tests
                 Key: SPARK-8106
                 URL: https://issues.apache.org/jira/browse/SPARK-8106
             Project: Spark
          Issue Type: Improvement
          Components: SQL
            Reporter: Josh Rosen


Derby has a configuration property named {{derby.system.durability}} that disables I/O synchronization
calls for many writes.  This sacrifices durability but can result in large performance gains,
which is appropriate for tests.

We should enable this in our test system properties in order to speed up the Hive compatibility
tests.  I saw 2-3x speedups locally with this change.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org


Mime
View raw message