spark-reviews mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From GitBox <...@apache.org>
Subject [GitHub] [spark] viirya commented on a change in pull request #29569: [SPARK-32722][PYTHON][DOCS] Update document type conversion for Pandas UDFs (pyarrow 1.0.1, pandas 1.1.1, Python 3.7)
Date Fri, 28 Aug 2020 06:23:03 GMT

viirya commented on a change in pull request #29569:
URL: https://github.com/apache/spark/pull/29569#discussion_r478854820



##########
File path: python/pyspark/sql/pandas/functions.py
##########
@@ -294,30 +294,30 @@ def calculate(iterator: Iterator[pd.Series]) -> Iterator[pd.Series]:
     # future. The table might have to be eventually documented externally.
     # Please see SPARK-28132's PR to see the codes in order to generate the table below.
     #
-    # +-----------------------------+----------------------+------------------+------------------+------------------+--------------------+--------------------+------------------+------------------+------------------+------------------+--------------+--------------+--------------+-----------------------------------+-----------------------------------------------------+-----------------+--------------------+-----------------------------+--------------+-----------------+------------------+-----------+--------------------------------+
 # noqa
-    # |SQL Type \ Pandas Value(Type)|None(object(NoneType))|        True(bool)|         
 1(int8)|          1(int16)|            1(int32)|            1(int64)|          1(uint8)|
        1(uint16)|         1(uint32)|         1(uint64)|  1.0(float16)|  1.0(float32)|  1.0(float64)|1970-01-01
00:00:00(datetime64[ns])|1970-01-01 00:00:00-05:00(datetime64[ns, US/Eastern])|a(object(string))|
 1(object(Decimal))|[1 2 3](object(array[int32]))| 1.0(float128)|(1+0j)(complex64)|(1+0j)(complex128)|A(category)|1
days 00:00:00(timedelta64[ns])|  # noqa
-    # +-----------------------------+----------------------+------------------+------------------+------------------+--------------------+--------------------+------------------+------------------+------------------+------------------+--------------+--------------+--------------+-----------------------------------+-----------------------------------------------------+-----------------+--------------------+-----------------------------+--------------+-----------------+------------------+-----------+--------------------------------+
 # noqa
-    # |                      boolean|                  None|              True|         
    True|              True|                True|                True|              True|
             True|              True|              True|          True|          True|   
      True|                                  X|                                          
         X|                X|                   X|                            X|         
   X|                X|                 X|          X|                               X|  #
noqa
-    # |                      tinyint|                  None|                 1|         
       1|                 1|                   1|                   1|                 1|
                1|                 1|                 1|             1|             1|   
         1|                                  X|                                          
         X|                X|                   1|                            X|         
   X|                X|                 X|          0|                               X|  #
noqa
-    # |                     smallint|                  None|                 1|         
       1|                 1|                   1|                   1|                 1|
                1|                 1|                 1|             1|             1|   
         1|                                  X|                                          
         X|                X|                   1|                            X|         
   X|                X|                 X|          X|                               X|  #
noqa
-    # |                          int|                  None|                 1|         
       1|                 1|                   1|                   1|                 1|
                1|                 1|                 1|             1|             1|   
         1|                                  X|                                          
         X|                X|                   1|                            X|         
   X|                X|                 X|          X|                               X|  #
noqa
-    # |                       bigint|                  None|                 1|         
       1|                 1|                   1|                   1|                 1|
                1|                 1|                 1|             1|             1|   
         1|                                  0|                                       18000000000000|
               X|                   1|                            X|             X|      
         X|                 X|          X|                               X|  # noqa

Review comment:
       bigint -> 1 days 00:00:00(timedelta64[ns]) is changed.




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


Mime
View raw message