spark-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Vincent Warmerdam (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (SPARK-8596) Install and configure RStudio server on Spark EC2
Date Mon, 06 Jul 2015 19:50:04 GMT

    [ https://issues.apache.org/jira/browse/SPARK-8596?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14615523#comment-14615523
] 

Vincent Warmerdam edited comment on SPARK-8596 at 7/6/15 7:49 PM:
------------------------------------------------------------------

1. true. made commit: https://github.com/koaning/spark-ec2/blob/rstudio-install/rstudio/init.sh#L35-37.
do we wan this code here, or somewhere else? seems like this is something system wide and
not rstudio specific... what about /spark/init.sh instead of /rstudio/init.sh? 

2. it gives you a breaking error. 

```
> sc <- sparkR.init(spark_link) 
Launching java with spark-submit command /root/spark/bin/spark-submit  sparkr-shell /tmp/RtmpSaSV2q/backend_port53744f8e9f59

/root/spark/conf/spark-env.sh: line 30: ulimit: open files: cannot modify limit: Operation
not permitted
```
ulimit is a command that can only be run by root, not the rstudio user.

3. can i remove this line: https://github.com/koaning/spark-ec2/blob/branch-1.4/templates/root/spark/conf/spark-env.sh#L30


not all users will want to use rstudio so doing this might break things (im assuming pyspark
might use this script as well?). we need to change the file though if we want rstudio to work.
perhaps we can move the new `/etc/security/limits.conf` paremeters current set in /rstudio/init.sh
to /spark/init.sh? 




was (Author: cantdutchthis):
1. true. made commit: https://github.com/koaning/spark-ec2/blob/rstudio-install/rstudio/init.sh#L35-37.
do we wan this code here, or somewhere else? seems like this is something system wide and
not rstudio specific... what about /spark/init.sh instead of /rstudio/init.sh? 

2. it gives you a breaking error. 

```
> sc <- sparkR.init(spark_link) 
Launching java with spark-submit command /root/spark/bin/spark-submit  sparkr-shell /tmp/RtmpSaSV2q/backend_port53744f8e9f59

/root/spark/conf/spark-env.sh: line 30: ulimit: open files: cannot modify limit: Operation
not permitted
```
ulimit is a command that can only be run by root, not the rstudio user.

3. can i remove this line: https://github.com/koaning/spark-ec2/blob/branch-1.4/templates/root/spark/conf/spark-env.sh#L30


not all users will want to use rstudio so doing this might break things (im assuming pyspark
might use this script as well?). we need to change the file though if we want rstudio to work.
perhaps we can move the new `/etc/security/limits.conf` paremeters in here? 



> Install and configure RStudio server on Spark EC2
> -------------------------------------------------
>
>                 Key: SPARK-8596
>                 URL: https://issues.apache.org/jira/browse/SPARK-8596
>             Project: Spark
>          Issue Type: Improvement
>          Components: EC2, SparkR
>            Reporter: Shivaram Venkataraman
>
> This will make it convenient for R users to use SparkR from their browsers 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org


Mime
View raw message