spark-reviews mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From HyukjinKwon <...@git.apache.org>
Subject [GitHub] spark pull request #21130: [SPARK-24054][R] Add array_position function / el...
Date Mon, 23 Apr 2018 14:59:40 GMT
GitHub user HyukjinKwon opened a pull request:

    https://github.com/apache/spark/pull/21130

    [SPARK-24054][R] Add array_position function / element_at functions

    ## What changes were proposed in this pull request?
    
    This PR proposes to add array_position and element_at in R side too.
    
    array_position:
    
    ```r
    df <- createDataFrame(cbind(model = rownames(mtcars), mtcars))
    mutated <- mutate(df, v1 = create_array(df$gear, df$am, df$carb))
    head(select(mutated, array_position(mutated$v1, 1)))
    ```
    
    ```
      array_position(v1, 1.0)
    1                       2
    2                       2
    3                       2
    4                       3
    5                       0
    6                       3
    ```
    
    element_at:
    
    ```r
    df <- createDataFrame(cbind(model = rownames(mtcars), mtcars))
    mutated <- mutate(df, v1 = create_array(df$mpg, df$cyl, df$hp))
    head(select(mutated, element_at(mutated$v1, 1)))
    ```
    
    ```
      element_at(v1, 1.0)
    1                21.0
    2                21.0
    3                22.8
    4                21.4
    5                18.7
    6                18.1
    ```
    
    ```r
    df <- createDataFrame(cbind(model = rownames(mtcars), mtcars))
    mutated <- mutate(df, v3 = create_map(df$model, df$cyl))
    head(select(mutated, element_at(mutated$v3, "Valiant")))
    ```
    
    ```
      element_at(v3, Valiant)
    1                      NA
    2                      NA
    3                      NA
    4                      NA
    5                      NA
    6                       6
    ```
    
    ## How was this patch tested?
    
    Unit tests were added in `R/pkg/tests/fulltests/test_sparkSQL.R` and manually tested.
Documentation was manually built and verified.

You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/HyukjinKwon/spark sparkr_array_position_element_at

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/spark/pull/21130.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #21130
    
----
commit 443034ac3180ea3f18f692040e7be4e6ca11553b
Author: hyukjinkwon <gurwls223@...>
Date:   2018-04-23T14:11:59Z

    Add array_position function / element_at functions

----


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


Mime
View raw message