spark-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Quincy HSIEH (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (SPARK-9776) Another instance of Derby may have already booted the database
Date Wed, 26 Jul 2017 12:06:03 GMT

    [ https://issues.apache.org/jira/browse/SPARK-9776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16101597#comment-16101597
] 

Quincy HSIEH edited comment on SPARK-9776 at 7/26/17 12:05 PM:
---------------------------------------------------------------

Hi,

In case that someone who has the same issue and comes to this page : here is the workaround
I use to resolve the problem :

1. Configure Spark with Yarn client mode.
2. Use FairScheduler in yarn-site.xml to enable multi-queques so that you can spark-submit
job to different queue name with --queue <QUEUE_NAME_YOU_PREFER>
    <property>
        <name>yarn.resourcemanager.scheduler.class</name>       <value>org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FairScheduler</value>
    </property>
3. Switch derby to memory mode by modify the connection string in 
   <property>
     <name>javax.jdo.option.ConnectionURL</name>
     <value>jdbc:derby*:memory*:<YOUR_STORAGE_DIR>/metastore_db;create=true</value>
     <description>JDBC connect string for a JDBC metastore</description>
   </property>


was (Author: quincy.tw):
Hi,

In case that someone who has the same issue and comes to this page : here is the workaround
I use to resolve the problem :

1. Configure Spark with Yarn client mode.
2. Use FairScheduler in yarn-site.xml to enable multi-queques so that you can spark-submit
job to different queue name with --queue <QUEUE_NAME_YOU_PREFER>
    <property>
        <name>yarn.resourcemanager.scheduler.class</name>
        <!--value>org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler</value-->
        <value>org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FairScheduler</value>
    </property>
3. Switch derby to memory mode by modify the connection string in 
   <property>
     <name>javax.jdo.option.ConnectionURL</name>
     <value>jdbc:derby*:memory*:<YOUR_STORAGE_DIR>/metastore_db;create=true</value>
     <description>JDBC connect string for a JDBC metastore</description>
   </property>

> Another instance of Derby may have already booted the database 
> ---------------------------------------------------------------
>
>                 Key: SPARK-9776
>                 URL: https://issues.apache.org/jira/browse/SPARK-9776
>             Project: Spark
>          Issue Type: Bug
>          Components: SQL
>    Affects Versions: 1.5.0
>         Environment: Mac Yosemite, spark-1.5.0
>            Reporter: Sudhakar Thota
>         Attachments: SPARK-9776-FL1.rtf
>
>
> val hiveContext = new org.apache.spark.sql.hive.HiveContext(sc) results in error. Though
the same works for spark-1.4.1.
> Caused by: ERROR XSDB6: Another instance of Derby may have already booted the database




--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org


Mime
View raw message