spark-reviews mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From BryanCutler <>
Subject [GitHub] spark issue #18378: [SPARK-21163][SQL] DataFrame.toPandas should respect the...
Date Mon, 16 Apr 2018 18:02:53 GMT
Github user BryanCutler commented on the issue:
    @edlee123 a Spark `DoubleType` will produce a `float64` dtype in Pandas and `FloatType`
will be `float32`.  `DateType` will be Python objects.  Also keep in mind that
if you have integer data with null values, then Pandas will treat it as floats and represent
the null values as `NaN`s.  In this case, Spark will not change the dtype.


To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message