hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sergey Shelukhin (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HIVE-12072) scriptfile1 test on MiniTez fails (due to session reuse?)
Date Thu, 08 Oct 2015 20:17:26 GMT

     [ https://issues.apache.org/jira/browse/HIVE-12072?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Sergey Shelukhin updated HIVE-12072:
------------------------------------
    Description: 
I can repro by running the following combination of tests with this feature enabled (but it
doesn't repro for me if I run the entire MiniTez test, or just one test)
{noformat}
mvn test -Phadoop-2 -Dtest=TestMiniTezCliDriver -Dqfile=alter_merge_orc.q,auto_join30.q,auto_sortmerge_join_11.q,filter_join_breaktask.q,insert_into1.q,mergejoin.q,metadata_only_queries_with_filters.q,scriptfile1.q
{noformat}
It fails as follows:
{noformat}
Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error while processing
row {"key":"238","value":"val_238"}
        at org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:520)
        at org.apache.hadoop.hive.ql.exec.tez.MapRecordSource.processRow(MapRecordSource.java:86)
        ... 17 more
Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: [Error 20000]: Unable to initialize
custom script.
        at org.apache.hadoop.hive.ql.exec.ScriptOperator.process(ScriptOperator.java:416)
        at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:852)
        at org.apache.hadoop.hive.ql.exec.SelectOperator.process(SelectOperator.java:87)
        at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:852)
        at org.apache.hadoop.hive.ql.exec.TableScanOperator.process(TableScanOperator.java:97)
        at org.apache.hadoop.hive.ql.exec.MapOperator$MapOpCtx.forward(MapOperator.java:164)
        at org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:510)
        ... 18 more
Caused by: java.io.IOException: Cannot run program "/grid/5/sershe/tez-autobuild/hive/itests/qtest/target/hive/hive-localDir-nm-1_2/usercache/sershe/appcache/application_1444328939776_0001/container_1444328939776_0001_01_000004/./testgrep":
error=13, Permission denied
        at java.lang.ProcessBuilder.start(ProcessBuilder.java:1047)
        at org.apache.hadoop.hive.ql.exec.ScriptOperator.process(ScriptOperator.java:374)
        ... 24 more
Caused by: java.io.IOException: error=13, Permission denied
        at java.lang.UNIXProcess.forkAndExec(Native Method)
        at java.lang.UNIXProcess.<init>(UNIXProcess.java:186)
        at java.lang.ProcessImpl.start(ProcessImpl.java:130)
        at java.lang.ProcessBuilder.start(ProcessBuilder.java:1028)
        ... 25 more
{noformat}

I didn't dig deeper (i.e. minimizing what tests cause it), but it doesn't look like any of
these tests use this file.

It passes with the feature disabled, although tests do take much longer :(

  was:
I can repro by running the following combination of tests with this feature enabled (but it
doesn't repro for me if I run the entire MiniTez test, or just one test)
{noformat}
mvn test -Phadoop-2 -Dtest=TestMiniTezCliDriver -Dqfile=alter_merge_orc.q,auto_join30.q,auto_sortmerge_join_11.q,filter_join_breaktask.q,insert_into1.q,mergejoin.q,metadata_only_queries_with_filters.q,scriptfile1.q
{noformat}
It fails as follows:
{noformat}
Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error while processing
row {"key":"238","value":"val_238"}
        at org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:520)
        at org.apache.hadoop.hive.ql.exec.tez.MapRecordSource.processRow(MapRecordSource.java:86)
        ... 17 more
Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: [Error 20000]: Unable to initialize
custom script.
        at org.apache.hadoop.hive.ql.exec.ScriptOperator.process(ScriptOperator.java:416)
        at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:852)
        at org.apache.hadoop.hive.ql.exec.SelectOperator.process(SelectOperator.java:87)
        at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:852)
        at org.apache.hadoop.hive.ql.exec.TableScanOperator.process(TableScanOperator.java:97)
        at org.apache.hadoop.hive.ql.exec.MapOperator$MapOpCtx.forward(MapOperator.java:164)
        at org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:510)
        ... 18 more
Caused by: java.io.IOException: Cannot run program "/grid/5/sershe/tez-autobuild/hive/itests/qtest/target/hive/hive-localDir-nm-1_2/usercache/sershe/appcache/application_1444328939776_0001/container_1444328939776_0001_01_000004/./testgrep":
error=13, Permission denied
        at java.lang.ProcessBuilder.start(ProcessBuilder.java:1047)
        at org.apache.hadoop.hive.ql.exec.ScriptOperator.process(ScriptOperator.java:374)
        ... 24 more
Caused by: java.io.IOException: error=13, Permission denied
        at java.lang.UNIXProcess.forkAndExec(Native Method)
        at java.lang.UNIXProcess.<init>(UNIXProcess.java:186)
        at java.lang.ProcessImpl.start(ProcessImpl.java:130)
        at java.lang.ProcessBuilder.start(ProcessBuilder.java:1028)
        ... 25 more
{noformat}

I didn't dig deeper (i.e. minimizing what tests cause it), but it doesn't look like any of
these tests use this file.

It passes (or at least doesn't fail the same way) with the feature disabled.


> scriptfile1 test on MiniTez fails (due to session reuse?)
> ---------------------------------------------------------
>
>                 Key: HIVE-12072
>                 URL: https://issues.apache.org/jira/browse/HIVE-12072
>             Project: Hive
>          Issue Type: Bug
>            Reporter: Sergey Shelukhin
>
> I can repro by running the following combination of tests with this feature enabled (but
it doesn't repro for me if I run the entire MiniTez test, or just one test)
> {noformat}
> mvn test -Phadoop-2 -Dtest=TestMiniTezCliDriver -Dqfile=alter_merge_orc.q,auto_join30.q,auto_sortmerge_join_11.q,filter_join_breaktask.q,insert_into1.q,mergejoin.q,metadata_only_queries_with_filters.q,scriptfile1.q
> {noformat}
> It fails as follows:
> {noformat}
> Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error while
processing row {"key":"238","value":"val_238"}
>         at org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:520)
>         at org.apache.hadoop.hive.ql.exec.tez.MapRecordSource.processRow(MapRecordSource.java:86)
>         ... 17 more
> Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: [Error 20000]: Unable to
initialize custom script.
>         at org.apache.hadoop.hive.ql.exec.ScriptOperator.process(ScriptOperator.java:416)
>         at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:852)
>         at org.apache.hadoop.hive.ql.exec.SelectOperator.process(SelectOperator.java:87)
>         at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:852)
>         at org.apache.hadoop.hive.ql.exec.TableScanOperator.process(TableScanOperator.java:97)
>         at org.apache.hadoop.hive.ql.exec.MapOperator$MapOpCtx.forward(MapOperator.java:164)
>         at org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:510)
>         ... 18 more
> Caused by: java.io.IOException: Cannot run program "/grid/5/sershe/tez-autobuild/hive/itests/qtest/target/hive/hive-localDir-nm-1_2/usercache/sershe/appcache/application_1444328939776_0001/container_1444328939776_0001_01_000004/./testgrep":
error=13, Permission denied
>         at java.lang.ProcessBuilder.start(ProcessBuilder.java:1047)
>         at org.apache.hadoop.hive.ql.exec.ScriptOperator.process(ScriptOperator.java:374)
>         ... 24 more
> Caused by: java.io.IOException: error=13, Permission denied
>         at java.lang.UNIXProcess.forkAndExec(Native Method)
>         at java.lang.UNIXProcess.<init>(UNIXProcess.java:186)
>         at java.lang.ProcessImpl.start(ProcessImpl.java:130)
>         at java.lang.ProcessBuilder.start(ProcessBuilder.java:1028)
>         ... 25 more
> {noformat}
> I didn't dig deeper (i.e. minimizing what tests cause it), but it doesn't look like any
of these tests use this file.
> It passes with the feature disabled, although tests do take much longer :(



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message