sqoop-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Veena Basavaraj (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (SQOOP-1837) Sqoop2: Verify that executing job has been successful in ConnectorTestCase
Date Wed, 03 Dec 2014 20:27:13 GMT

    [ https://issues.apache.org/jira/browse/SQOOP-1837?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14233481#comment-14233481
] 

Veena Basavaraj edited comment on SQOOP-1837 at 12/3/14 8:26 PM:
-----------------------------------------------------------------

I tried this patch for my SQOOP-1815 use case.

No luck. the exception Info is null.  There is some place else that it is swallowing the real
issue underneath the 1815 use case

{code}
java.lang.AssertionError: Submission has failed with null expected:<SUCCEEDED> but was:<FAILED>
	at org.junit.Assert.fail(Assert.java:88)
	at org.junit.Assert.failNotEquals(Assert.java:743)
	at org.junit.Assert.assertEquals(Assert.java:118)
	at org.apache.sqoop.test.testcases.ConnectorTestCase.executeJob(ConnectorTestCase.java:253)
	at org.apache.sqoop.test.testcases.ConnectorTestCase.executeJob(ConnectorTestCase.java:264)
	at org.apache.sqoop.integration.connector.jdbc.generic.PartitionerTest.testSplitter(PartitionerTest.java:98)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
	at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
	at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
	at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
	at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
	at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
	at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:55)
	at org.junit.rules.RunRules.evaluate(RunRules.java:20)
	at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:271)
	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:70)
	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:50)
	at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238)
	at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:63)
	at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:236)
	at org.junit.runners.ParentRunner.access$000(ParentRunner.java:53)
	at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:229)
	at org.junit.runners.ParentRunner.run(ParentRunner.java:309)
	at org.junit.runners.Suite.runChild(Suite.java:127)
	at org.junit.runners.Suite.runChild(Suite.java:26)
	at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238)
	at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:63)
	at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:236)
	at org.junit.runners.ParentRunner.access$000(ParentRunner.java:53)
	at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:229)
	at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
	at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
	at org.junit.runners.ParentRunner.run(ParentRunner.java:309)
	at org.eclipse.jdt.internal.junit4.runner.JUnit4TestReference.run(JUnit4TestReference.java:50)
	at org.eclipse.jdt.internal.junit.runner.TestExecution.run(TestExecution.java:38)
	at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:459)
	at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:675)
	at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.run(RemoteTestRunner.java:382)
	at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.main(RemoteTestRunner.java:192)

{code}


The only plave we set exception info on the submission object is is in the  MR submission
engine

{code}

  ....
      String jobId = job.getJobID().toString();
      request.getSummary().setExternalId(jobId);
      request.getSummary().setExternalLink(job.getTrackingURL());

      LOG.debug("Executed new map-reduce job with id " + jobId);
    } catch (Exception e) {
      request.getSummary().setException(e);
      LOG.error("Error in submitting job", e);
      return false;
    }
    return true;
  }
{code}


was (Author: vybs):
I tried this patch for my SQOOP-1815 use case.

No luck. the exception Info is null.  There is some place else that it is swallowing the real
issue underneath the 1815 use case

{code}

  ....
      String jobId = job.getJobID().toString();
      request.getSummary().setExternalId(jobId);
      request.getSummary().setExternalLink(job.getTrackingURL());

      LOG.debug("Executed new map-reduce job with id " + jobId);
    } catch (Exception e) {
      request.getSummary().setException(e);
      LOG.error("Error in submitting job", e);
      return false;
    }
    return true;
  }
{code}

> Sqoop2: Verify that executing job has been successful in ConnectorTestCase
> --------------------------------------------------------------------------
>
>                 Key: SQOOP-1837
>                 URL: https://issues.apache.org/jira/browse/SQOOP-1837
>             Project: Sqoop
>          Issue Type: Bug
>    Affects Versions: 1.99.4
>            Reporter: Jarek Jarcec Cecho
>            Assignee: Jarek Jarcec Cecho
>             Fix For: 1.99.5
>
>         Attachments: SQOOP-1837.patch
>
>
> While investigating SQOOP-1815 we've spent a lot of time with [~vybs] trying to see why
we are not getting data that we are expecting. We finally realized that the submission itself
is failing and that is why we don't see any data. It seems that our [util classes|https://github.com/apache/sqoop/blob/sqoop2/test/src/main/java/org/apache/sqoop/test/testcases/ConnectorTestCase.java#L244]
for executing job are running the job without checking whether it was successful or not. Hence
the test didn't noticed that the job failed and move to the stage for checking job outputs
that haven't been there.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message