crunch-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Barry,Nathan" <NBA...@CERNER.COM>
Subject Re: convert a scrunch.PCollection to a crunch.PCollection ?
Date Wed, 12 Mar 2014 15:57:46 GMT
_doh_

Missed that one

From: Josh Wills <jwills@cloudera.com<mailto:jwills@cloudera.com>>
Reply-To: "user@crunch.apache.org<mailto:user@crunch.apache.org>" <user@crunch.apache.org<mailto:user@crunch.apache.org>>
Date: Wednesday, March 12, 2014 at 10:44 AM
To: "user@crunch.apache.org<mailto:user@crunch.apache.org>" <user@crunch.apache.org<mailto:user@crunch.apache.org>>
Subject: Re: convert a scrunch.PCollection to a crunch.PCollection ?

Scrunch PCollections have a member called "native" that returns the underlying Java PCollection.
I think we could allow the conversion to happen automatically by updating o.a.c.s.Conversions
to include implicit methods for going _back_ from a Scrunch PCollection to a Java PCollection
(right now, only the Java -> Scala methods are provided.)


On Wed, Mar 12, 2014 at 8:27 AM, Barry,Nathan <NBARRY@cerner.com<mailto:NBARRY@cerner.com>>
wrote:
Hi

I'm migrating some code from crunch to scrunch and I'm running into an issue leveraging a
crunch based API that takes a crunch PCollection as a parameter but I have a scrunch PCollection
which results in compilation errors when trying to pass the scrunch Pcollection into the API

What am I missing?  I _think_ I have the proper conversion imports in place.

[ERROR]  found   : org.apache.crunch.scrunch.PCollection[org.apache.hadoop.hbase.client.Put]
[ERROR]  required: org.apache.crunch.PCollection[org.apache.hadoop.hbase.client.Put]

Thanks,

Nathan

CONFIDENTIALITY NOTICE This message and any included attachments are from Cerner Corporation
and are intended only for the addressee. The information contained in this message is confidential
and may constitute inside or non-public information under international, federal, or state
securities laws. Unauthorized forwarding, printing, copying, distribution, or use of such
information is strictly prohibited and may be unlawful. If you are not the addressee, please
promptly delete this message and notify the sender of the delivery error by e-mail or you
may call Cerner's corporate offices in Kansas City, Missouri, U.S.A at (+1) (816)221-1024<tel:%28%2B1%29%20%28816%29221-1024>.



--
Director of Data Science
Cloudera<https://urldefense.proofpoint.com/v1/url?u=http://www.cloudera.com&k=PmKqfXspAHNo6iYJ48Q45A%3D%3D%0A&r=bSlK0OoXv%2FhAR3K6OwkDzQ7GANpYeh6g8gdpXiWlUWw%3D%0A&m=rS1hwZwuXljlQh3r9aRV1rqOhPpYA7Og49uiDg5M9nw%3D%0A&s=25da0547e6d98dfef47f42328782c82f4453fcd421f86080e034da44032f672e>
Twitter: @josh_wills<https://urldefense.proofpoint.com/v1/url?u=http://twitter.com/josh_wills&k=PmKqfXspAHNo6iYJ48Q45A%3D%3D%0A&r=bSlK0OoXv%2FhAR3K6OwkDzQ7GANpYeh6g8gdpXiWlUWw%3D%0A&m=rS1hwZwuXljlQh3r9aRV1rqOhPpYA7Og49uiDg5M9nw%3D%0A&s=38d71c8b9d15c622c35a99bb2211d2329cbe88933309c5b4d78a0e6481c0b0a3>

Mime
View raw message