spark-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Milad Bourhani (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (SPARK-3947) Support Scala/Java UDAF
Date Mon, 23 Nov 2015 12:54:11 GMT

    [ https://issues.apache.org/jira/browse/SPARK-3947?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15022051#comment-15022051
] 

Milad Bourhani commented on SPARK-3947:
---------------------------------------

Just for completeness, I'm attaching the [^logs.zip]. For the record, it looks as if the first
time you run the clustered computation (right after you started {{./sbin/start-all.sh}}),
the computation is OK, even though the race condition error shows up in the log. After that,
it fails. So the attached logs contain exactly two executions: the first gives a correct answer,
the second doesn't.
To reproduce, run these commands on the unzipped project [^spark-udaf-adapted-1.5.2.zip]:
{noformat}
mvn clean install
java -jar `ls target/uber*.jar` `ls target/uber*.jar` spark://master_host:7077
java -jar `ls target/uber*.jar` `ls target/uber*.jar` spark://master_host:7077
{noformat}
where {{spark://master_host:7077}} is your master URL.

> Support Scala/Java UDAF
> -----------------------
>
>                 Key: SPARK-3947
>                 URL: https://issues.apache.org/jira/browse/SPARK-3947
>             Project: Spark
>          Issue Type: Sub-task
>          Components: SQL
>            Reporter: Pei-Lun Lee
>            Assignee: Yin Huai
>             Fix For: 1.5.0
>
>         Attachments: logs.zip, spark-udaf-adapted-1.5.2.zip, spark-udaf.zip
>
>
> Right now only Hive UDAFs are supported. It would be nice to have UDAF similar to UDF
through SQLContext.registerFunction.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org


Mime
View raw message