pig-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Nandor Kollar (JIRA)" <j...@apache.org>
Subject [jira] [Created] (PIG-5318) Unit test failures on Pig on Spark with Spark 2.2
Date Tue, 28 Nov 2017 14:43:00 GMT
Nandor Kollar created PIG-5318:
----------------------------------

             Summary: Unit test failures on Pig on Spark with Spark 2.2
                 Key: PIG-5318
                 URL: https://issues.apache.org/jira/browse/PIG-5318
             Project: Pig
          Issue Type: Bug
          Components: spark
            Reporter: Nandor Kollar
            Assignee: Nandor Kollar


There are sever failing cases when executing the unit tests with Spark 2.2:
{code}
 org.apache.pig.test.TestAssert#testNegativeWithoutFetch
 org.apache.pig.test.TestAssert#testNegative
 org.apache.pig.test.TestEvalPipeline2#testNonStandardDataWithoutFetch
 org.apache.pig.test.TestScalarAliases#testScalarErrMultipleRowsInInput
 org.apache.pig.test.TestStore#testCleanupOnFailureMultiStore
 org.apache.pig.test.TestStoreInstances#testBackendStoreCommunication
 org.apache.pig.test.TestStoreLocal#testCleanupOnFailureMultiStore
{code}

All of these are related to fixes/changes in Spark.

TestAssert, TestScalarAliases and TestEvalPipeline2 failures could be fixed by asserting on
the message of the exception's root cause, looks like on Spark 2.2 the exception is wrapped
into an additional layer.
TestStore and TestStoreLocal failure are also a test related problems: looks like SPARK-7953
is fixed in Spark 2.2
The root cause of TestStoreInstances is yet to be found out.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Mime
View raw message