falcon-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Murali Ramasami (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (FALCON-2071) Falcon Spark SQL failing with Yarn Client Mode
Date Mon, 11 Jul 2016 15:22:11 GMT

     [ https://issues.apache.org/jira/browse/FALCON-2071?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Murali Ramasami updated FALCON-2071:
------------------------------------
    Description: 
Falcon Spark SQL failing with Yarn Client Mode with error message "Only local python files
are supported". Please see the error message below from application log:

{noformat}
Error: Only local python files are supported: Parsed arguments:
  master                  yarn-client
  deployMode              client
  executorMemory          null
  executorCores           null
  totalExecutorCores      null
  propertiesFile          null
  driverMemory            null
  driverCores             null
  driverExtraClassPath    $PWD/*
  driverExtraLibraryPath  /usr/hdp/current/hadoop-client/lib/native
  driverExtraJavaOptions  -Dlog4j.configuration=spark-log4j.properties
  supervise               false
  queue                   null
  numExecutors            null
{noformat}

Process
{noformat}
<?xml version="1.0" encoding="UTF-8"?>
<process name="spark-hivesql-client" xmlns="uri:falcon:process:0.1">
    <clusters>
        <cluster name="primaryCluster">
            <validity start="2015-01-02T01:00Z" end="2015-01-02T01:04Z"/>
        </cluster>
    </clusters>
    <parallel>1</parallel>
    <order>FIFO</order>
    <frequency>minutes(5)</frequency>
    <timezone>UTC</timezone>
    <workflow engine="spark" path="/user/hrt_qa/falcon-regression/spark_hivesql"/>
    <spark-attributes>
        <master>yarn-client</master>
        <name>Spark-Hive</name>
        <class>Spark Hive</class>
        <jar>/user/hrt_qa/falcon-regression/spark_hivesql/spark-hivesql.py</jar>
    </spark-attributes>
    <retry policy="periodic" delay="minutes(3)" attempts="3"/>
    <ACL owner="hrt_qa" group="users" permission="0x755"/>
</process>

{noformat}


  was:
Falcon Spark SQL failing with Yarn Client Mode Only local python files are supported. Please
see the error message below from application log:

{noformat}
Error: Only local python files are supported: Parsed arguments:
  master                  yarn-client
  deployMode              client
  executorMemory          null
  executorCores           null
  totalExecutorCores      null
  propertiesFile          null
  driverMemory            null
  driverCores             null
  driverExtraClassPath    $PWD/*
  driverExtraLibraryPath  /usr/hdp/current/hadoop-client/lib/native
  driverExtraJavaOptions  -Dlog4j.configuration=spark-log4j.properties
  supervise               false
  queue                   null
  numExecutors            null
{noformat}

Process
{noformat}
<?xml version="1.0" encoding="UTF-8"?>
<process name="spark-hivesql-client" xmlns="uri:falcon:process:0.1">
    <clusters>
        <cluster name="primaryCluster">
            <validity start="2015-01-02T01:00Z" end="2015-01-02T01:04Z"/>
        </cluster>
    </clusters>
    <parallel>1</parallel>
    <order>FIFO</order>
    <frequency>minutes(5)</frequency>
    <timezone>UTC</timezone>
    <workflow engine="spark" path="/user/hrt_qa/falcon-regression/spark_hivesql"/>
    <spark-attributes>
        <master>yarn-client</master>
        <name>Spark-Hive</name>
        <class>Spark Hive</class>
        <jar>/user/hrt_qa/falcon-regression/spark_hivesql/spark-hivesql.py</jar>
    </spark-attributes>
    <retry policy="periodic" delay="minutes(3)" attempts="3"/>
    <ACL owner="hrt_qa" group="users" permission="0x755"/>
</process>

{noformat}



> Falcon Spark SQL failing with Yarn Client Mode 
> -----------------------------------------------
>
>                 Key: FALCON-2071
>                 URL: https://issues.apache.org/jira/browse/FALCON-2071
>             Project: Falcon
>          Issue Type: Bug
>          Components: process
>    Affects Versions: trunk
>            Reporter: Murali Ramasami
>            Priority: Critical
>             Fix For: trunk
>
>
> Falcon Spark SQL failing with Yarn Client Mode with error message "Only local python
files are supported". Please see the error message below from application log:
> {noformat}
> Error: Only local python files are supported: Parsed arguments:
>   master                  yarn-client
>   deployMode              client
>   executorMemory          null
>   executorCores           null
>   totalExecutorCores      null
>   propertiesFile          null
>   driverMemory            null
>   driverCores             null
>   driverExtraClassPath    $PWD/*
>   driverExtraLibraryPath  /usr/hdp/current/hadoop-client/lib/native
>   driverExtraJavaOptions  -Dlog4j.configuration=spark-log4j.properties
>   supervise               false
>   queue                   null
>   numExecutors            null
> {noformat}
> Process
> {noformat}
> <?xml version="1.0" encoding="UTF-8"?>
> <process name="spark-hivesql-client" xmlns="uri:falcon:process:0.1">
>     <clusters>
>         <cluster name="primaryCluster">
>             <validity start="2015-01-02T01:00Z" end="2015-01-02T01:04Z"/>
>         </cluster>
>     </clusters>
>     <parallel>1</parallel>
>     <order>FIFO</order>
>     <frequency>minutes(5)</frequency>
>     <timezone>UTC</timezone>
>     <workflow engine="spark" path="/user/hrt_qa/falcon-regression/spark_hivesql"/>
>     <spark-attributes>
>         <master>yarn-client</master>
>         <name>Spark-Hive</name>
>         <class>Spark Hive</class>
>         <jar>/user/hrt_qa/falcon-regression/spark_hivesql/spark-hivesql.py</jar>
>     </spark-attributes>
>     <retry policy="periodic" delay="minutes(3)" attempts="3"/>
>     <ACL owner="hrt_qa" group="users" permission="0x755"/>
> </process>
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message