spark-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Apache Spark (JIRA)" <j...@apache.org>
Subject [jira] [Assigned] (SPARK-9942) Broken pandas could crash PySpark SQL
Date Thu, 13 Aug 2015 20:43:46 GMT

     [ https://issues.apache.org/jira/browse/SPARK-9942?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Apache Spark reassigned SPARK-9942:
-----------------------------------

    Assignee: Apache Spark  (was: Davies Liu)

> Broken pandas could crash PySpark SQL
> -------------------------------------
>
>                 Key: SPARK-9942
>                 URL: https://issues.apache.org/jira/browse/SPARK-9942
>             Project: Spark
>          Issue Type: Bug
>          Components: PySpark, SQL
>    Affects Versions: 1.3.1, 1.4.1, 1.5.0
>            Reporter: Davies Liu
>            Assignee: Apache Spark
>            Priority: Blocker
>
> {code}
> Error from python worker:
>   Traceback (most recent call last):
>     File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/runpy.py",
line 151, in _run_module_as_main
>       mod_name, loader, code, fname = _get_module_details(mod_name)
>     File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/runpy.py",
line 101, in _get_module_details
>       loader = get_loader(mod_name)
>     File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pkgutil.py",
line 464, in get_loader
>       return find_loader(fullname)
>     File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pkgutil.py",
line 474, in find_loader
>       for importer in iter_importers(fullname):
>     File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pkgutil.py",
line 430, in iter_importers
>       __import__(pkg)
>     File "/Users/davies/work/spark/python/pyspark/__init__.py", line 52, in <module>
>       from pyspark.sql import SQLContext, HiveContext, SchemaRDD, Row
>     File "/Users/davies/work/spark/python/pyspark/sql/__init__.py", line 63, in <module>
>       from pyspark.sql.context import SQLContext, HiveContext
>     File "/Users/davies/work/spark/python/pyspark/sql/context.py", line 40, in <module>
>       import pandas
>     File "/Library/Python/2.7/site-packages/pandas/__init__.py", line 7, in <module>
>       from pandas import hashtable, tslib, lib
>     File "pandas/src/numpy.pxd", line 157, in init pandas.hashtable (pandas/hashtable.c:23654)
>   ValueError: numpy.dtype has the wrong size, try recompiling
> PYTHONPATH was:
>   /Users/davies/work/spark/python/lib/pyspark.zip:/Users/davies/work/spark/python/lib/py4j-0.8.2.1-src.zip:/Users/davies/work/spark/sbin/../python/lib/py4j-0.8.2.1-src.zip:/Users/davies/work/spark/sbin/../python:/Users/davies/work/spark/sbin/../python/lib/py4j-0.8.2.1-src.zip:/Users/davies/work/spark/sbin/../python:
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org


Mime
View raw message