Return-Path: X-Original-To: archive-asf-public-internal@cust-asf2.ponee.io Delivered-To: archive-asf-public-internal@cust-asf2.ponee.io Received: from cust-asf.ponee.io (cust-asf.ponee.io [163.172.22.183]) by cust-asf2.ponee.io (Postfix) with ESMTP id 6F197200B8B for ; Tue, 4 Oct 2016 19:32:52 +0200 (CEST) Received: by cust-asf.ponee.io (Postfix) id 6D852160AF4; Tue, 4 Oct 2016 17:32:52 +0000 (UTC) Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by cust-asf.ponee.io (Postfix) with SMTP id 04DDA160B08 for ; Tue, 4 Oct 2016 19:32:47 +0200 (CEST) Received: (qmail 76745 invoked by uid 500); 4 Oct 2016 17:32:46 -0000 Mailing-List: contact commits-help@spark.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Delivered-To: mailing list commits@spark.apache.org Received: (qmail 75717 invoked by uid 99); 4 Oct 2016 17:32:46 -0000 Received: from git1-us-west.apache.org (HELO git1-us-west.apache.org) (140.211.11.23) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 04 Oct 2016 17:32:46 +0000 Received: by git1-us-west.apache.org (ASF Mail Server at git1-us-west.apache.org, from userid 33) id 0F855E08AD; Tue, 4 Oct 2016 17:32:46 +0000 (UTC) Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit From: rxin@apache.org To: commits@spark.apache.org Date: Tue, 04 Oct 2016 17:33:11 -0000 Message-Id: <8e97d2db1a7b4f40af7d5fa84d0f94df@git.apache.org> In-Reply-To: <5d47eae5bac6491eb122ccc7d7f8edff@git.apache.org> References: <5d47eae5bac6491eb122ccc7d7f8edff@git.apache.org> X-Mailer: ASF-Git Admin Mailer Subject: [27/51] [partial] spark-website git commit: Add doc for 2.0.1 archived-at: Tue, 04 Oct 2016 17:32:52 -0000 http://git-wip-us.apache.org/repos/asf/spark-website/blob/7c7b45c8/site/docs/2.0.1/api/R/structField.html ---------------------------------------------------------------------- diff --git a/site/docs/2.0.1/api/R/structField.html b/site/docs/2.0.1/api/R/structField.html new file mode 100644 index 0000000..2f6aa38 --- /dev/null +++ b/site/docs/2.0.1/api/R/structField.html @@ -0,0 +1,84 @@ + +R: structField + + + + + + + + + +
structField {SparkR}R Documentation
+ +

structField

+ +

Description

+ +

Create a structField object that contains the metadata for a single field in a schema. +

+ + +

Usage

+ +
+structField(x, ...)
+
+## S3 method for class 'jobj'
+structField(x, ...)
+
+## S3 method for class 'character'
+structField(x, type, nullable = TRUE, ...)
+
+ + +

Arguments

+ + + + + + + + + + +
x +

the name of the field.

+
... +

additional argument(s) passed to the method.

+
type +

The data type of the field

+
nullable +

A logical vector indicating whether or not the field is nullable

+
+ + +

Value

+ +

A structField object. +

+ + +

Note

+ +

structField since 1.4.0 +

+ + +

Examples

+ +
## Not run: 
+##D field1 <- structField("a", "integer")
+##D field2 <- structField("c", "string")
+##D field3 <- structField("avg", "double")
+##D schema <-  structType(field1, field2, field3)
+##D df1 <- gapply(df, list("a", "c"),
+##D               function(key, x) { y <- data.frame(key, mean(x$b), stringsAsFactors = FALSE) },
+##D               schema)
+## End(Not run)
+
+ + +
[Package SparkR version 2.0.1 Index]
+ http://git-wip-us.apache.org/repos/asf/spark-website/blob/7c7b45c8/site/docs/2.0.1/api/R/structType.html ---------------------------------------------------------------------- diff --git a/site/docs/2.0.1/api/R/structType.html b/site/docs/2.0.1/api/R/structType.html new file mode 100644 index 0000000..d94e7e8 --- /dev/null +++ b/site/docs/2.0.1/api/R/structType.html @@ -0,0 +1,75 @@ + +R: structType + + + + + + + + + +
structType {SparkR}R Documentation
+ +

structType

+ +

Description

+ +

Create a structType object that contains the metadata for a SparkDataFrame. Intended for +use with createDataFrame and toDF. +

+ + +

Usage

+ +
+structType(x, ...)
+
+## S3 method for class 'jobj'
+structType(x, ...)
+
+## S3 method for class 'structField'
+structType(x, ...)
+
+ + +

Arguments

+ + + + + + +
x +

a structField object (created with the field() function)

+
... +

additional structField objects

+
+ + +

Value

+ +

a structType object +

+ + +

Note

+ +

structType since 1.4.0 +

+ + +

Examples

+ +
## Not run: 
+##D schema <-  structType(structField("a", "integer"), structField("c", "string"),
+##D                       structField("avg", "double"))
+##D df1 <- gapply(df, list("a", "c"),
+##D               function(key, x) { y <- data.frame(key, mean(x$b), stringsAsFactors = FALSE) },
+##D               schema)
+## End(Not run)
+
+ + +
[Package SparkR version 2.0.1 Index]
+ http://git-wip-us.apache.org/repos/asf/spark-website/blob/7c7b45c8/site/docs/2.0.1/api/R/subset.html ---------------------------------------------------------------------- diff --git a/site/docs/2.0.1/api/R/subset.html b/site/docs/2.0.1/api/R/subset.html new file mode 100644 index 0000000..d32e8a9 --- /dev/null +++ b/site/docs/2.0.1/api/R/subset.html @@ -0,0 +1,309 @@ + +R: Subset + + + + + + + + + +
[[ {SparkR}R Documentation
+ +

Subset

+ +

Description

+ +

Return subsets of SparkDataFrame according to given conditions +

+ + +

Usage

+ +
+## S4 method for signature 'SparkDataFrame,numericOrcharacter'
+x[[i]]
+
+## S4 method for signature 'SparkDataFrame'
+x[i, j, ..., drop = F]
+
+## S4 method for signature 'SparkDataFrame'
+subset(x, subset, select, drop = F, ...)
+
+subset(x, ...)
+
+ + +

Arguments

+ + + + + + + + + + + + +
x +

a SparkDataFrame.

+
i,subset +

(Optional) a logical expression to filter on rows.

+
j,select +

expression for the single Column or a list of columns to select from the SparkDataFrame.

+
... +

currently not used.

+
drop +

if TRUE, a Column will be returned if the resulting dataset has only one column. +Otherwise, a SparkDataFrame will always be returned.

+
+ + +

Value

+ +

A new SparkDataFrame containing only the rows that meet the condition with selected columns. +

+ + +

Note

+ +

[[ since 1.4.0 +

+

[ since 1.4.0 +

+

subset since 1.5.0 +

+ + +

See Also

+ +

Other SparkDataFrame functions: $, +$,SparkDataFrame-method, $<-, +$<-,SparkDataFrame-method, +select, select, +select,SparkDataFrame,Column-method, +select,SparkDataFrame,character-method, +select,SparkDataFrame,list-method; +SparkDataFrame-class; agg, +agg, agg, +agg,GroupedData-method, +agg,SparkDataFrame-method, +summarize, summarize, +summarize, +summarize,GroupedData-method, +summarize,SparkDataFrame-method; +arrange, arrange, +arrange, +arrange,SparkDataFrame,Column-method, +arrange,SparkDataFrame,character-method, +orderBy,SparkDataFrame,characterOrColumn-method; +as.data.frame, +as.data.frame,SparkDataFrame-method; +attach, +attach,SparkDataFrame-method; +cache, cache, +cache,SparkDataFrame-method; +collect, collect, +collect,SparkDataFrame-method; +colnames, colnames, +colnames,SparkDataFrame-method, +colnames<-, colnames<-, +colnames<-,SparkDataFrame-method, +columns, columns, +columns,SparkDataFrame-method, +names, +names,SparkDataFrame-method, +names<-, +names<-,SparkDataFrame-method; +coltypes, coltypes, +coltypes,SparkDataFrame-method, +coltypes<-, coltypes<-, +coltypes<-,SparkDataFrame,character-method; +count,SparkDataFrame-method, +nrow, nrow, +nrow,SparkDataFrame-method; +createOrReplaceTempView, +createOrReplaceTempView, +createOrReplaceTempView,SparkDataFrame,character-method; +dapplyCollect, dapplyCollect, +dapplyCollect,SparkDataFrame,function-method; +dapply, dapply, +dapply,SparkDataFrame,function,structType-method; +describe, describe, +describe, +describe,SparkDataFrame,ANY-method, +describe,SparkDataFrame,character-method, +describe,SparkDataFrame-method, +summary, summary, +summary,SparkDataFrame-method; +dim, +dim,SparkDataFrame-method; +distinct, distinct, +distinct,SparkDataFrame-method, +unique, +unique,SparkDataFrame-method; +dropDuplicates, +dropDuplicates, +dropDuplicates,SparkDataFrame-method; +dropna, dropna, +dropna,SparkDataFrame-method, +fillna, fillna, +fillna,SparkDataFrame-method, +na.omit, na.omit, +na.omit,SparkDataFrame-method; +drop, drop, +drop, drop,ANY-method, +drop,SparkDataFrame-method; +dtypes, dtypes, +dtypes,SparkDataFrame-method; +except, except, +except,SparkDataFrame,SparkDataFrame-method; +explain, explain, +explain,SparkDataFrame-method; +filter, filter, +filter,SparkDataFrame,characterOrColumn-method, +where, where, +where,SparkDataFrame,characterOrColumn-method; +first, first, +first, +first,SparkDataFrame-method, +first,characterOrColumn-method; +gapplyCollect, gapplyCollect, +gapplyCollect, +gapplyCollect,GroupedData-method, +gapplyCollect,SparkDataFrame-method; +gapply, gapply, +gapply, +gapply,GroupedData-method, +gapply,SparkDataFrame-method; +groupBy, groupBy, +groupBy,SparkDataFrame-method, +group_by, group_by, +group_by,SparkDataFrame-method; +head, +head,SparkDataFrame-method; +histogram, +histogram,SparkDataFrame,characterOrColumn-method; +insertInto, insertInto, +insertInto,SparkDataFrame,character-method; +intersect, intersect, +intersect,SparkDataFrame,SparkDataFrame-method; +isLocal, isLocal, +isLocal,SparkDataFrame-method; +join, +join,SparkDataFrame,SparkDataFrame-method; +limit, limit, +limit,SparkDataFrame,numeric-method; +merge, merge, +merge,SparkDataFrame,SparkDataFrame-method; +mutate, mutate, +mutate,SparkDataFrame-method, +transform, transform, +transform,SparkDataFrame-method; +ncol, +ncol,SparkDataFrame-method; +persist, persist, +persist,SparkDataFrame,character-method; +printSchema, printSchema, +printSchema,SparkDataFrame-method; +randomSplit, randomSplit, +randomSplit,SparkDataFrame,numeric-method; +rbind, rbind, +rbind,SparkDataFrame-method; +registerTempTable, +registerTempTable, +registerTempTable,SparkDataFrame,character-method; +rename, rename, +rename,SparkDataFrame-method, +withColumnRenamed, +withColumnRenamed, +withColumnRenamed,SparkDataFrame,character,character-method; +repartition, repartition, +repartition,SparkDataFrame-method; +sample, sample, +sample,SparkDataFrame,logical,numeric-method, +sample_frac, sample_frac, +sample_frac,SparkDataFrame,logical,numeric-method; +saveAsParquetFile, +saveAsParquetFile, +saveAsParquetFile,SparkDataFrame,character-method, +write.parquet, write.parquet, +write.parquet,SparkDataFrame,character-method; +saveAsTable, saveAsTable, +saveAsTable,SparkDataFrame,character-method; +saveDF, saveDF, +saveDF,SparkDataFrame,character-method, +write.df, write.df, +write.df, +write.df,SparkDataFrame,character-method; +schema, schema, +schema,SparkDataFrame-method; +selectExpr, selectExpr, +selectExpr,SparkDataFrame,character-method; +showDF, showDF, +showDF,SparkDataFrame-method; +show, show, +show,Column-method, +show,GroupedData-method, +show,SparkDataFrame-method, +show,WindowSpec-method; str, +str,SparkDataFrame-method; +take, take, +take,SparkDataFrame,numeric-method; +union, union, +union,SparkDataFrame,SparkDataFrame-method, +unionAll, unionAll, +unionAll,SparkDataFrame,SparkDataFrame-method; +unpersist, unpersist, +unpersist,SparkDataFrame-method; +withColumn, withColumn, +withColumn,SparkDataFrame,character,Column-method; +with, +with,SparkDataFrame-method; +write.jdbc, write.jdbc, +write.jdbc,SparkDataFrame,character,character-method; +write.json, write.json, +write.json,SparkDataFrame,character-method; +write.orc, write.orc, +write.orc,SparkDataFrame,character-method; +write.text, write.text, +write.text,SparkDataFrame,character-method +

+

Other subsetting functions: $, +$,SparkDataFrame-method, $<-, +$<-,SparkDataFrame-method, +select, select, +select,SparkDataFrame,Column-method, +select,SparkDataFrame,character-method, +select,SparkDataFrame,list-method; +filter, filter, +filter,SparkDataFrame,characterOrColumn-method, +where, where, +where,SparkDataFrame,characterOrColumn-method +

+ + +

Examples

+ +
## Not run: 
+##D   # Columns can be selected using [[ and [
+##D   df[[2]] == df[["age"]]
+##D   df[,2] == df[,"age"]
+##D   df[,c("name", "age")]
+##D   # Or to filter rows
+##D   df[df$age > 20,]
+##D   # SparkDataFrame can be subset on both rows and Columns
+##D   df[df$name == "Smith", c(1,2)]
+##D   df[df$age %in% c(19, 30), 1:2]
+##D   subset(df, df$age %in% c(19, 30), 1:2)
+##D   subset(df, df$age %in% c(19), select = c(1,2))
+##D   subset(df, select = c(1,2))
+## End(Not run)
+
+ + +
[Package SparkR version 2.0.1 Index]
+ http://git-wip-us.apache.org/repos/asf/spark-website/blob/7c7b45c8/site/docs/2.0.1/api/R/substr.html ---------------------------------------------------------------------- diff --git a/site/docs/2.0.1/api/R/substr.html b/site/docs/2.0.1/api/R/substr.html new file mode 100644 index 0000000..6adb81f --- /dev/null +++ b/site/docs/2.0.1/api/R/substr.html @@ -0,0 +1,67 @@ + +R: substr + + + + +
substr {SparkR}R Documentation
+ +

substr

+ +

Description

+ +

An expression that returns a substring. +

+ + +

Usage

+ +
+## S4 method for signature 'Column'
+substr(x, start, stop)
+
+ + +

Arguments

+ + + + + + + + +
x +

a Column.

+
start +

starting position.

+
stop +

ending position.

+
+ + +

Note

+ +

substr since 1.4.0 +

+ + +

See Also

+ +

Other colum_func: alias, +alias,Column-method; between, +between, +between,Column-method; cast, +cast, cast,Column-method; +endsWith, endsWith, +endsWith,Column-method; +otherwise, otherwise, +otherwise,Column-method; +over, over, +over,Column,WindowSpec-method; +startsWith, startsWith, +startsWith,Column-method +

+ +
[Package SparkR version 2.0.1 Index]
+ http://git-wip-us.apache.org/repos/asf/spark-website/blob/7c7b45c8/site/docs/2.0.1/api/R/substring_index.html ---------------------------------------------------------------------- diff --git a/site/docs/2.0.1/api/R/substring_index.html b/site/docs/2.0.1/api/R/substring_index.html new file mode 100644 index 0000000..6e88805 --- /dev/null +++ b/site/docs/2.0.1/api/R/substring_index.html @@ -0,0 +1,129 @@ + +R: substring_index + + + + + + + + + +
substring_index {SparkR}R Documentation
+ +

substring_index

+ +

Description

+ +

Returns the substring from string str before count occurrences of the delimiter delim. +If count is positive, everything the left of the final delimiter (counting from left) is +returned. If count is negative, every to the right of the final delimiter (counting from the +right) is returned. substring_index performs a case-sensitive match when searching for delim. +

+ + +

Usage

+ +
+## S4 method for signature 'Column,character,numeric'
+substring_index(x, delim, count)
+
+substring_index(x, delim, count)
+
+ + +

Arguments

+ + + + + + + + +
x +

a Column.

+
delim +

a delimiter string.

+
count +

number of occurrences of delim before the substring is returned. +A positive number means counting from the left, while negative means +counting from the right.

+
+ + +

Note

+ +

substring_index since 1.5.0 +

+ + +

See Also

+ +

Other string_funcs: ascii, +ascii, ascii,Column-method; +base64, base64, +base64,Column-method; +concat_ws, concat_ws, +concat_ws,character,Column-method; +concat, concat, +concat,Column-method; decode, +decode, +decode,Column,character-method; +encode, encode, +encode,Column,character-method; +format_number, format_number, +format_number,Column,numeric-method; +format_string, format_string, +format_string,character,Column-method; +initcap, initcap, +initcap,Column-method; instr, +instr, +instr,Column,character-method; +length, length,Column-method; +levenshtein, levenshtein, +levenshtein,Column-method; +locate, locate, +locate,character,Column-method; +lower, lower, +lower,Column-method; lpad, +lpad, +lpad,Column,numeric,character-method; +ltrim, ltrim, +ltrim,Column-method; +regexp_extract, +regexp_extract, +regexp_extract,Column,character,numeric-method; +regexp_replace, +regexp_replace, +regexp_replace,Column,character,character-method; +reverse, reverse, +reverse,Column-method; rpad, +rpad, +rpad,Column,numeric,character-method; +rtrim, rtrim, +rtrim,Column-method; soundex, +soundex, +soundex,Column-method; +translate, translate, +translate,Column,character,character-method; +trim, trim, +trim,Column-method; unbase64, +unbase64, +unbase64,Column-method; +upper, upper, +upper,Column-method +

+ + +

Examples

+ +
## Not run: 
+##D substring_index(df$c, '.', 2)
+##D substring_index(df$c, '.', -1)
+## End(Not run)
+
+ + +
[Package SparkR version 2.0.1 Index]
+ http://git-wip-us.apache.org/repos/asf/spark-website/blob/7c7b45c8/site/docs/2.0.1/api/R/sum.html ---------------------------------------------------------------------- diff --git a/site/docs/2.0.1/api/R/sum.html b/site/docs/2.0.1/api/R/sum.html new file mode 100644 index 0000000..a6c82b7 --- /dev/null +++ b/site/docs/2.0.1/api/R/sum.html @@ -0,0 +1,104 @@ + +R: sum + + + + + + + + + +
sum {SparkR}R Documentation
+ +

sum

+ +

Description

+ +

Aggregate function: returns the sum of all values in the expression. +

+ + +

Usage

+ +
+## S4 method for signature 'Column'
+sum(x)
+
+ + +

Arguments

+ + + + +
x +

Column to compute on.

+
+ + +

Note

+ +

sum since 1.5.0 +

+ + +

See Also

+ +

Other agg_funcs: agg, agg, +agg, agg,GroupedData-method, +agg,SparkDataFrame-method, +summarize, summarize, +summarize, +summarize,GroupedData-method, +summarize,SparkDataFrame-method; +avg, avg, +avg,Column-method; +countDistinct, countDistinct, +countDistinct,Column-method, +n_distinct, n_distinct, +n_distinct,Column-method; +count, count, +count,Column-method, +count,GroupedData-method, n, +n, n,Column-method; +first, first, +first, +first,SparkDataFrame-method, +first,characterOrColumn-method; +kurtosis, kurtosis, +kurtosis,Column-method; last, +last, +last,characterOrColumn-method; +max, max,Column-method; +mean, mean,Column-method; +min, min,Column-method; +sd, sd, +sd,Column-method, stddev, +stddev, stddev,Column-method; +skewness, skewness, +skewness,Column-method; +stddev_pop, stddev_pop, +stddev_pop,Column-method; +stddev_samp, stddev_samp, +stddev_samp,Column-method; +sumDistinct, sumDistinct, +sumDistinct,Column-method; +var_pop, var_pop, +var_pop,Column-method; +var_samp, var_samp, +var_samp,Column-method; var, +var, var,Column-method, +variance, variance, +variance,Column-method +

+ + +

Examples

+ +
## Not run: sum(df$c)
+
+ + +
[Package SparkR version 2.0.1 Index]
+ http://git-wip-us.apache.org/repos/asf/spark-website/blob/7c7b45c8/site/docs/2.0.1/api/R/sumDistinct.html ---------------------------------------------------------------------- diff --git a/site/docs/2.0.1/api/R/sumDistinct.html b/site/docs/2.0.1/api/R/sumDistinct.html new file mode 100644 index 0000000..e2fcc37 --- /dev/null +++ b/site/docs/2.0.1/api/R/sumDistinct.html @@ -0,0 +1,105 @@ + +R: sumDistinct + + + + + + + + + +
sumDistinct {SparkR}R Documentation
+ +

sumDistinct

+ +

Description

+ +

Aggregate function: returns the sum of distinct values in the expression. +

+ + +

Usage

+ +
+## S4 method for signature 'Column'
+sumDistinct(x)
+
+sumDistinct(x)
+
+ + +

Arguments

+ + + + +
x +

Column to compute on.

+
+ + +

Note

+ +

sumDistinct since 1.4.0 +

+ + +

See Also

+ +

Other agg_funcs: agg, agg, +agg, agg,GroupedData-method, +agg,SparkDataFrame-method, +summarize, summarize, +summarize, +summarize,GroupedData-method, +summarize,SparkDataFrame-method; +avg, avg, +avg,Column-method; +countDistinct, countDistinct, +countDistinct,Column-method, +n_distinct, n_distinct, +n_distinct,Column-method; +count, count, +count,Column-method, +count,GroupedData-method, n, +n, n,Column-method; +first, first, +first, +first,SparkDataFrame-method, +first,characterOrColumn-method; +kurtosis, kurtosis, +kurtosis,Column-method; last, +last, +last,characterOrColumn-method; +max, max,Column-method; +mean, mean,Column-method; +min, min,Column-method; +sd, sd, +sd,Column-method, stddev, +stddev, stddev,Column-method; +skewness, skewness, +skewness,Column-method; +stddev_pop, stddev_pop, +stddev_pop,Column-method; +stddev_samp, stddev_samp, +stddev_samp,Column-method; +sum, sum,Column-method; +var_pop, var_pop, +var_pop,Column-method; +var_samp, var_samp, +var_samp,Column-method; var, +var, var,Column-method, +variance, variance, +variance,Column-method +

+ + +

Examples

+ +
## Not run: sumDistinct(df$c)
+
+ + +
[Package SparkR version 2.0.1 Index]
+ http://git-wip-us.apache.org/repos/asf/spark-website/blob/7c7b45c8/site/docs/2.0.1/api/R/summarize.html ---------------------------------------------------------------------- diff --git a/site/docs/2.0.1/api/R/summarize.html b/site/docs/2.0.1/api/R/summarize.html new file mode 100644 index 0000000..a48644b --- /dev/null +++ b/site/docs/2.0.1/api/R/summarize.html @@ -0,0 +1,330 @@ + +R: Summarize data across columns + + + + + + + + + +
agg {SparkR}R Documentation
+ +

Summarize data across columns

+ +

Description

+ +

Compute aggregates by specifying a list of columns +

+

Aggregates on the entire SparkDataFrame without groups. +The resulting SparkDataFrame will also contain the grouping columns. +

+ + +

Usage

+ +
+## S4 method for signature 'SparkDataFrame'
+agg(x, ...)
+
+## S4 method for signature 'SparkDataFrame'
+summarize(x, ...)
+
+agg(x, ...)
+
+summarize(x, ...)
+
+## S4 method for signature 'GroupedData'
+agg(x, ...)
+
+## S4 method for signature 'GroupedData'
+summarize(x, ...)
+
+ + +

Arguments

+ + + + + + +
x +

a SparkDataFrame or GroupedData.

+
... +

further arguments to be passed to or from other methods.

+
+ + +

Details

+ +

df2 <- agg(df, <column> = <aggFunction>) +df2 <- agg(df, newColName = aggFunction(column)) +

+ + +

Value

+ +

A SparkDataFrame. +

+ + +

Note

+ +

agg since 1.4.0 +

+

summarize since 1.4.0 +

+

agg since 1.4.0 +

+

summarize since 1.4.0 +

+ + +

See Also

+ +

Other SparkDataFrame functions: $, +$,SparkDataFrame-method, $<-, +$<-,SparkDataFrame-method, +select, select, +select,SparkDataFrame,Column-method, +select,SparkDataFrame,character-method, +select,SparkDataFrame,list-method; +SparkDataFrame-class; [, +[,SparkDataFrame-method, [[, +[[,SparkDataFrame,numericOrcharacter-method, +subset, subset, +subset,SparkDataFrame-method; +arrange, arrange, +arrange, +arrange,SparkDataFrame,Column-method, +arrange,SparkDataFrame,character-method, +orderBy,SparkDataFrame,characterOrColumn-method; +as.data.frame, +as.data.frame,SparkDataFrame-method; +attach, +attach,SparkDataFrame-method; +cache, cache, +cache,SparkDataFrame-method; +collect, collect, +collect,SparkDataFrame-method; +colnames, colnames, +colnames,SparkDataFrame-method, +colnames<-, colnames<-, +colnames<-,SparkDataFrame-method, +columns, columns, +columns,SparkDataFrame-method, +names, +names,SparkDataFrame-method, +names<-, +names<-,SparkDataFrame-method; +coltypes, coltypes, +coltypes,SparkDataFrame-method, +coltypes<-, coltypes<-, +coltypes<-,SparkDataFrame,character-method; +count,SparkDataFrame-method, +nrow, nrow, +nrow,SparkDataFrame-method; +createOrReplaceTempView, +createOrReplaceTempView, +createOrReplaceTempView,SparkDataFrame,character-method; +dapplyCollect, dapplyCollect, +dapplyCollect,SparkDataFrame,function-method; +dapply, dapply, +dapply,SparkDataFrame,function,structType-method; +describe, describe, +describe, +describe,SparkDataFrame,ANY-method, +describe,SparkDataFrame,character-method, +describe,SparkDataFrame-method, +summary, summary, +summary,SparkDataFrame-method; +dim, +dim,SparkDataFrame-method; +distinct, distinct, +distinct,SparkDataFrame-method, +unique, +unique,SparkDataFrame-method; +dropDuplicates, +dropDuplicates, +dropDuplicates,SparkDataFrame-method; +dropna, dropna, +dropna,SparkDataFrame-method, +fillna, fillna, +fillna,SparkDataFrame-method, +na.omit, na.omit, +na.omit,SparkDataFrame-method; +drop, drop, +drop, drop,ANY-method, +drop,SparkDataFrame-method; +dtypes, dtypes, +dtypes,SparkDataFrame-method; +except, except, +except,SparkDataFrame,SparkDataFrame-method; +explain, explain, +explain,SparkDataFrame-method; +filter, filter, +filter,SparkDataFrame,characterOrColumn-method, +where, where, +where,SparkDataFrame,characterOrColumn-method; +first, first, +first, +first,SparkDataFrame-method, +first,characterOrColumn-method; +gapplyCollect, gapplyCollect, +gapplyCollect, +gapplyCollect,GroupedData-method, +gapplyCollect,SparkDataFrame-method; +gapply, gapply, +gapply, +gapply,GroupedData-method, +gapply,SparkDataFrame-method; +groupBy, groupBy, +groupBy,SparkDataFrame-method, +group_by, group_by, +group_by,SparkDataFrame-method; +head, +head,SparkDataFrame-method; +histogram, +histogram,SparkDataFrame,characterOrColumn-method; +insertInto, insertInto, +insertInto,SparkDataFrame,character-method; +intersect, intersect, +intersect,SparkDataFrame,SparkDataFrame-method; +isLocal, isLocal, +isLocal,SparkDataFrame-method; +join, +join,SparkDataFrame,SparkDataFrame-method; +limit, limit, +limit,SparkDataFrame,numeric-method; +merge, merge, +merge,SparkDataFrame,SparkDataFrame-method; +mutate, mutate, +mutate,SparkDataFrame-method, +transform, transform, +transform,SparkDataFrame-method; +ncol, +ncol,SparkDataFrame-method; +persist, persist, +persist,SparkDataFrame,character-method; +printSchema, printSchema, +printSchema,SparkDataFrame-method; +randomSplit, randomSplit, +randomSplit,SparkDataFrame,numeric-method; +rbind, rbind, +rbind,SparkDataFrame-method; +registerTempTable, +registerTempTable, +registerTempTable,SparkDataFrame,character-method; +rename, rename, +rename,SparkDataFrame-method, +withColumnRenamed, +withColumnRenamed, +withColumnRenamed,SparkDataFrame,character,character-method; +repartition, repartition, +repartition,SparkDataFrame-method; +sample, sample, +sample,SparkDataFrame,logical,numeric-method, +sample_frac, sample_frac, +sample_frac,SparkDataFrame,logical,numeric-method; +saveAsParquetFile, +saveAsParquetFile, +saveAsParquetFile,SparkDataFrame,character-method, +write.parquet, write.parquet, +write.parquet,SparkDataFrame,character-method; +saveAsTable, saveAsTable, +saveAsTable,SparkDataFrame,character-method; +saveDF, saveDF, +saveDF,SparkDataFrame,character-method, +write.df, write.df, +write.df, +write.df,SparkDataFrame,character-method; +schema, schema, +schema,SparkDataFrame-method; +selectExpr, selectExpr, +selectExpr,SparkDataFrame,character-method; +showDF, showDF, +showDF,SparkDataFrame-method; +show, show, +show,Column-method, +show,GroupedData-method, +show,SparkDataFrame-method, +show,WindowSpec-method; str, +str,SparkDataFrame-method; +take, take, +take,SparkDataFrame,numeric-method; +union, union, +union,SparkDataFrame,SparkDataFrame-method, +unionAll, unionAll, +unionAll,SparkDataFrame,SparkDataFrame-method; +unpersist, unpersist, +unpersist,SparkDataFrame-method; +withColumn, withColumn, +withColumn,SparkDataFrame,character,Column-method; +with, +with,SparkDataFrame-method; +write.jdbc, write.jdbc, +write.jdbc,SparkDataFrame,character,character-method; +write.json, write.json, +write.json,SparkDataFrame,character-method; +write.orc, write.orc, +write.orc,SparkDataFrame,character-method; +write.text, write.text, +write.text,SparkDataFrame,character-method +

+

Other agg_funcs: avg, avg, +avg,Column-method; +countDistinct, countDistinct, +countDistinct,Column-method, +n_distinct, n_distinct, +n_distinct,Column-method; +count, count, +count,Column-method, +count,GroupedData-method, n, +n, n,Column-method; +first, first, +first, +first,SparkDataFrame-method, +first,characterOrColumn-method; +kurtosis, kurtosis, +kurtosis,Column-method; last, +last, +last,characterOrColumn-method; +max, max,Column-method; +mean, mean,Column-method; +min, min,Column-method; +sd, sd, +sd,Column-method, stddev, +stddev, stddev,Column-method; +skewness, skewness, +skewness,Column-method; +stddev_pop, stddev_pop, +stddev_pop,Column-method; +stddev_samp, stddev_samp, +stddev_samp,Column-method; +sumDistinct, sumDistinct, +sumDistinct,Column-method; +sum, sum,Column-method; +var_pop, var_pop, +var_pop,Column-method; +var_samp, var_samp, +var_samp,Column-method; var, +var, var,Column-method, +variance, variance, +variance,Column-method +

+ + +

Examples

+ +
## Not run: 
+##D  df2 <- agg(df, age = "sum")  # new column name will be created as 'SUM(age#0)'
+##D  df3 <- agg(df, ageSum = sum(df$age)) # Creates a new column named ageSum
+##D  df4 <- summarize(df, ageSum = max(df$age))
+## End(Not run)
+
+ + +
[Package SparkR version 2.0.1 Index]
+ http://git-wip-us.apache.org/repos/asf/spark-website/blob/7c7b45c8/site/docs/2.0.1/api/R/summary.html ---------------------------------------------------------------------- diff --git a/site/docs/2.0.1/api/R/summary.html b/site/docs/2.0.1/api/R/summary.html new file mode 100644 index 0000000..39d8bdf --- /dev/null +++ b/site/docs/2.0.1/api/R/summary.html @@ -0,0 +1,286 @@ + +R: summary + + + + + + + + + +
describe {SparkR}R Documentation
+ +

summary

+ +

Description

+ +

Computes statistics for numeric columns. +If no columns are given, this function computes statistics for all numerical columns. +

+ + +

Usage

+ +
+## S4 method for signature 'SparkDataFrame,character'
+describe(x, col, ...)
+
+## S4 method for signature 'SparkDataFrame,ANY'
+describe(x)
+
+## S4 method for signature 'SparkDataFrame'
+summary(object, ...)
+
+describe(x, col, ...)
+
+summary(object, ...)
+
+ + +

Arguments

+ + + + + + + + + + +
x +

a SparkDataFrame to be computed.

+
col +

a string of name.

+
... +

additional expressions.

+
object +

a SparkDataFrame to be summarized.

+
+ + +

Value

+ +

A SparkDataFrame. +

+ + +

Note

+ +

describe(SparkDataFrame, character) since 1.4.0 +

+

describe(SparkDataFrame) since 1.4.0 +

+

summary(SparkDataFrame) since 1.5.0 +

+ + +

See Also

+ +

Other SparkDataFrame functions: $, +$,SparkDataFrame-method, $<-, +$<-,SparkDataFrame-method, +select, select, +select,SparkDataFrame,Column-method, +select,SparkDataFrame,character-method, +select,SparkDataFrame,list-method; +SparkDataFrame-class; [, +[,SparkDataFrame-method, [[, +[[,SparkDataFrame,numericOrcharacter-method, +subset, subset, +subset,SparkDataFrame-method; +agg, agg, agg, +agg,GroupedData-method, +agg,SparkDataFrame-method, +summarize, summarize, +summarize, +summarize,GroupedData-method, +summarize,SparkDataFrame-method; +arrange, arrange, +arrange, +arrange,SparkDataFrame,Column-method, +arrange,SparkDataFrame,character-method, +orderBy,SparkDataFrame,characterOrColumn-method; +as.data.frame, +as.data.frame,SparkDataFrame-method; +attach, +attach,SparkDataFrame-method; +cache, cache, +cache,SparkDataFrame-method; +collect, collect, +collect,SparkDataFrame-method; +colnames, colnames, +colnames,SparkDataFrame-method, +colnames<-, colnames<-, +colnames<-,SparkDataFrame-method, +columns, columns, +columns,SparkDataFrame-method, +names, +names,SparkDataFrame-method, +names<-, +names<-,SparkDataFrame-method; +coltypes, coltypes, +coltypes,SparkDataFrame-method, +coltypes<-, coltypes<-, +coltypes<-,SparkDataFrame,character-method; +count,SparkDataFrame-method, +nrow, nrow, +nrow,SparkDataFrame-method; +createOrReplaceTempView, +createOrReplaceTempView, +createOrReplaceTempView,SparkDataFrame,character-method; +dapplyCollect, dapplyCollect, +dapplyCollect,SparkDataFrame,function-method; +dapply, dapply, +dapply,SparkDataFrame,function,structType-method; +dim, +dim,SparkDataFrame-method; +distinct, distinct, +distinct,SparkDataFrame-method, +unique, +unique,SparkDataFrame-method; +dropDuplicates, +dropDuplicates, +dropDuplicates,SparkDataFrame-method; +dropna, dropna, +dropna,SparkDataFrame-method, +fillna, fillna, +fillna,SparkDataFrame-method, +na.omit, na.omit, +na.omit,SparkDataFrame-method; +drop, drop, +drop, drop,ANY-method, +drop,SparkDataFrame-method; +dtypes, dtypes, +dtypes,SparkDataFrame-method; +except, except, +except,SparkDataFrame,SparkDataFrame-method; +explain, explain, +explain,SparkDataFrame-method; +filter, filter, +filter,SparkDataFrame,characterOrColumn-method, +where, where, +where,SparkDataFrame,characterOrColumn-method; +first, first, +first, +first,SparkDataFrame-method, +first,characterOrColumn-method; +gapplyCollect, gapplyCollect, +gapplyCollect, +gapplyCollect,GroupedData-method, +gapplyCollect,SparkDataFrame-method; +gapply, gapply, +gapply, +gapply,GroupedData-method, +gapply,SparkDataFrame-method; +groupBy, groupBy, +groupBy,SparkDataFrame-method, +group_by, group_by, +group_by,SparkDataFrame-method; +head, +head,SparkDataFrame-method; +histogram, +histogram,SparkDataFrame,characterOrColumn-method; +insertInto, insertInto, +insertInto,SparkDataFrame,character-method; +intersect, intersect, +intersect,SparkDataFrame,SparkDataFrame-method; +isLocal, isLocal, +isLocal,SparkDataFrame-method; +join, +join,SparkDataFrame,SparkDataFrame-method; +limit, limit, +limit,SparkDataFrame,numeric-method; +merge, merge, +merge,SparkDataFrame,SparkDataFrame-method; +mutate, mutate, +mutate,SparkDataFrame-method, +transform, transform, +transform,SparkDataFrame-method; +ncol, +ncol,SparkDataFrame-method; +persist, persist, +persist,SparkDataFrame,character-method; +printSchema, printSchema, +printSchema,SparkDataFrame-method; +randomSplit, randomSplit, +randomSplit,SparkDataFrame,numeric-method; +rbind, rbind, +rbind,SparkDataFrame-method; +registerTempTable, +registerTempTable, +registerTempTable,SparkDataFrame,character-method; +rename, rename, +rename,SparkDataFrame-method, +withColumnRenamed, +withColumnRenamed, +withColumnRenamed,SparkDataFrame,character,character-method; +repartition, repartition, +repartition,SparkDataFrame-method; +sample, sample, +sample,SparkDataFrame,logical,numeric-method, +sample_frac, sample_frac, +sample_frac,SparkDataFrame,logical,numeric-method; +saveAsParquetFile, +saveAsParquetFile, +saveAsParquetFile,SparkDataFrame,character-method, +write.parquet, write.parquet, +write.parquet,SparkDataFrame,character-method; +saveAsTable, saveAsTable, +saveAsTable,SparkDataFrame,character-method; +saveDF, saveDF, +saveDF,SparkDataFrame,character-method, +write.df, write.df, +write.df, +write.df,SparkDataFrame,character-method; +schema, schema, +schema,SparkDataFrame-method; +selectExpr, selectExpr, +selectExpr,SparkDataFrame,character-method; +showDF, showDF, +showDF,SparkDataFrame-method; +show, show, +show,Column-method, +show,GroupedData-method, +show,SparkDataFrame-method, +show,WindowSpec-method; str, +str,SparkDataFrame-method; +take, take, +take,SparkDataFrame,numeric-method; +union, union, +union,SparkDataFrame,SparkDataFrame-method, +unionAll, unionAll, +unionAll,SparkDataFrame,SparkDataFrame-method; +unpersist, unpersist, +unpersist,SparkDataFrame-method; +withColumn, withColumn, +withColumn,SparkDataFrame,character,Column-method; +with, +with,SparkDataFrame-method; +write.jdbc, write.jdbc, +write.jdbc,SparkDataFrame,character,character-method; +write.json, write.json, +write.json,SparkDataFrame,character-method; +write.orc, write.orc, +write.orc,SparkDataFrame,character-method; +write.text, write.text, +write.text,SparkDataFrame,character-method +

+ + +

Examples

+ +
## Not run: 
+##D sparkR.session()
+##D path <- "path/to/file.json"
+##D df <- read.json(path)
+##D describe(df)
+##D describe(df, "col1")
+##D describe(df, "col1", "col2")
+## End(Not run)
+
+ + +
[Package SparkR version 2.0.1 Index]
+ http://git-wip-us.apache.org/repos/asf/spark-website/blob/7c7b45c8/site/docs/2.0.1/api/R/tableNames.html ---------------------------------------------------------------------- diff --git a/site/docs/2.0.1/api/R/tableNames.html b/site/docs/2.0.1/api/R/tableNames.html new file mode 100644 index 0000000..845c255 --- /dev/null +++ b/site/docs/2.0.1/api/R/tableNames.html @@ -0,0 +1,62 @@ + +R: Table Names + + + + + + + + + +
tableNames {SparkR}R Documentation
+ +

Table Names

+ +

Description

+ +

Returns the names of tables in the given database as an array. +

+ + +

Usage

+ +
+## Default S3 method:
+tableNames(databaseName = NULL)
+
+ + +

Arguments

+ + + + +
databaseName +

name of the database

+
+ + +

Value

+ +

a list of table names +

+ + +

Note

+ +

tableNames since 1.4.0 +

+ + +

Examples

+ +
## Not run: 
+##D sparkR.session()
+##D tableNames("hive")
+## End(Not run)
+
+ + +
[Package SparkR version 2.0.1 Index]
+ --------------------------------------------------------------------- To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org For additional commands, e-mail: commits-help@spark.apache.org