spark-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Felix Cheung (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (SPARK-11886) R function name conflicts with base or stats package ones
Date Tue, 01 Dec 2015 00:37:10 GMT

    [ https://issues.apache.org/jira/browse/SPARK-11886?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15032810#comment-15032810
] 

Felix Cheung commented on SPARK-11886:
--------------------------------------

I see this if I load dplyr after SparkR

{code}
> library(dplyr)

Attaching package: ‘dplyr’

The following objects are masked from ‘package:SparkR’:

    arrange, between, collect, count, cume_dist, dense_rank, desc,
    distinct, explain, filter, first, group_by, intersect, lag, last,
    lead, mutate, n, n_distinct, ntile, percent_rank, rename,
    row_number, sample_frac, select, sql, summarize

The following objects are masked from ‘package:stats’:

    filter, lag

The following objects are masked from ‘package:base’:

    intersect, setdiff, setequal, union

> select(a, row_number(a$Sepal_Length))
Error in UseMethod("select_") :
  no applicable method for 'select_' applied to an object of class "DataFrame"
> select(a, a$Sepal_Length)
Error in UseMethod("select_") :
  no applicable method for 'select_' applied to an object of class "DataFrame"
> select(a, "*")
Error in UseMethod("select_") :
  no applicable method for 'select_' applied to an object of class "DataFrame"
> r <- row_number(a$Sepal_Length)
Error in x[!nas] : object of type 'S4' is not subsettable
In addition: Warning message:
In is.na(x) : is.na() applied to non-(list or vector) of type 'S4'
> r <- dense_rank(a$Sepal_Length)
Error in x[!nas] : object of type 'S4' is not subsettable
In addition: Warning message:
In is.na(x) : is.na() applied to non-(list or vector) of type 'S4'
> collect(df)
Error in UseMethod("collect") :
  no applicable method for 'collect' applied to an object of class "DataFrame"
{code}

Seems like a bunch of things are broken

> R function name conflicts with base or stats package ones
> ---------------------------------------------------------
>
>                 Key: SPARK-11886
>                 URL: https://issues.apache.org/jira/browse/SPARK-11886
>             Project: Spark
>          Issue Type: Bug
>          Components: SparkR
>    Affects Versions: 1.5.2
>            Reporter: Felix Cheung
>            Priority: Minor
>
> See https://github.com/apache/spark/pull/9785
> Currently these are masked:
> stats::cov
> stats::filter
> base::sample
> base::table
> [~shivaram] suggested:
> "
> If we have same name but the param types completely don't match (and no room for ...)
then we override those functions but (This is true for sample, table, cov right now I guess)
we should try to limit the number of functions where we do this. Also we should revisit some
of these to see if we can avoid it (for example table can be renamed ?)
> "



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org


Mime
View raw message