spark-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Apache Spark (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (SPARK-11463) SparkContext fail to create in non-main thread in Python
Date Mon, 09 Nov 2015 19:43:11 GMT

    [ https://issues.apache.org/jira/browse/SPARK-11463?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14997231#comment-14997231
] 

Apache Spark commented on SPARK-11463:
--------------------------------------

User 'davies' has created a pull request for this issue:
https://github.com/apache/spark/pull/9574

> SparkContext fail to create in non-main thread in Python
> --------------------------------------------------------
>
>                 Key: SPARK-11463
>                 URL: https://issues.apache.org/jira/browse/SPARK-11463
>             Project: Spark
>          Issue Type: Bug
>          Components: PySpark
>            Reporter: Davies Liu
>            Assignee: Davies Liu
>            Priority: Blocker
>
> https://github.com/apache/spark/commit/2e572c4135c3f5ad3061c1f58cdb8a70bed0a9d3#commitcomment-14137386
> {code}
> Exception in thread Thread-3:
> Traceback (most recent call last):
>   File "/home/asoni/.conda/envs/py35_spark_django/lib/python3.5/threading.py", line 923,
in _bootstrap_inner
>     self.run()
>   File "/home/asoni/.conda/envs/py35_spark_django/lib/python3.5/threading.py", line 871,
in run
>     self._target(*self._args, **self._kwargs)
>   File "/home/asoni/spark-server/spark_server/spark_jobs/views.py", line 67, in process_job
>     from spark_jobs.job_runner import JobRunner
>   File "/home/asoni/spark-server/spark_server/spark_jobs/job_runner.py", line 2, in <module>
>     from spark_jobs.extraction_plan_group_runner import *
>   File "/home/asoni/spark-server/spark_server/spark_jobs/extraction_plan_group_runner.py",
line 1, in <module>
>     from spark_jobs.spark_connection import SparkConnection
>   File "/home/asoni/spark-server/spark_server/spark_jobs/spark_connection.py", line 5,
in <module>
>     class SparkConnection:
>   File "/home/asoni/spark-server/spark_server/spark_jobs/spark_connection.py", line 7,
in SparkConnection
>     sc = SparkContext(conf=conf)
>   File "/home/asoni/spark-1.6.0-SNAPSHOT-bin-custom-spark/python/lib/pyspark.zip/pyspark/context.py",
line 114, in __init__
>     conf, jsc, profiler_cls)
>   File "/home/asoni/spark-1.6.0-SNAPSHOT-bin-custom-spark/python/lib/pyspark.zip/pyspark/context.py",
line 225, in _do_init
>     signal.signal(signal.SIGINT, signal_handler)
>   File "/home/asoni/.conda/envs/py35_spark_django/lib/python3.5/signal.py", line 47,
in signal
>     handler = _signal.signal(_enum_to_int(signalnum), _enum_to_int(handler))
> ValueError: signal only works in main thread
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org


Mime
View raw message