From user-return-2904-archive-asf-public=cust-asf.ponee.io@beam.apache.org Thu Feb 8 19:17:17 2018 Return-Path: X-Original-To: archive-asf-public@eu.ponee.io Delivered-To: archive-asf-public@eu.ponee.io Received: from cust-asf.ponee.io (cust-asf.ponee.io [163.172.22.183]) by mx-eu-01.ponee.io (Postfix) with ESMTP id A953D18064F for ; Thu, 8 Feb 2018 19:17:17 +0100 (CET) Received: by cust-asf.ponee.io (Postfix) id 98A01160C4A; Thu, 8 Feb 2018 18:17:17 +0000 (UTC) Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by cust-asf.ponee.io (Postfix) with SMTP id B85FB160C34 for ; Thu, 8 Feb 2018 19:17:16 +0100 (CET) Received: (qmail 82177 invoked by uid 500); 8 Feb 2018 18:17:15 -0000 Mailing-List: contact user-help@beam.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@beam.apache.org Delivered-To: mailing list user@beam.apache.org Received: (qmail 82167 invoked by uid 99); 8 Feb 2018 18:17:15 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd4-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 08 Feb 2018 18:17:15 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd4-us-west.apache.org (ASF Mail Server at spamd4-us-west.apache.org) with ESMTP id 2A2CDC0042 for ; Thu, 8 Feb 2018 18:17:15 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd4-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: 1.888 X-Spam-Level: * X-Spam-Status: No, score=1.888 tagged_above=-999 required=6.31 tests=[DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, HTML_MESSAGE=2, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_PASS=-0.001, T_RP_MATCHES_RCVD=-0.01] autolearn=disabled Authentication-Results: spamd4-us-west.apache.org (amavisd-new); dkim=pass (2048-bit key) header.d=google.com Received: from mx1-lw-us.apache.org ([10.40.0.8]) by localhost (spamd4-us-west.apache.org [10.40.0.11]) (amavisd-new, port 10024) with ESMTP id TIx1Z8KMJCJH for ; Thu, 8 Feb 2018 18:17:12 +0000 (UTC) Received: from mail-io0-f177.google.com (mail-io0-f177.google.com [209.85.223.177]) by mx1-lw-us.apache.org (ASF Mail Server at mx1-lw-us.apache.org) with ESMTPS id C162B5F23E for ; Thu, 8 Feb 2018 18:17:11 +0000 (UTC) Received: by mail-io0-f177.google.com with SMTP id d13so6772939iog.5 for ; Thu, 08 Feb 2018 10:17:11 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to; bh=0k2zDzj92VVcO3q7+qs1VwVRfyPIa6GUvC2HZzmNfoA=; b=XNl91rs/PHp7a7f7TfFHLo1t9D5ir1btW62zEGb0J90PWXLJ+9ot4N8TP9jpQtjbjT NbdduQeIqK7u7vj8dQoxCkKNbgph2Bc2UFmJvNclPU5opKtZP0i2bOcjizkn4TqrpHAB 0xV1WnsVpovGCta+qLJWqk7JdGchur5GsihWpmaEAjE/bRRBYDjKvICR0efVLrXict7B hcjQiENvV/9DvYtMECJaq41W0ylCd0lAT+f3eNsFrL+Yg0NwMNE4qiZHD4yDbnUwfi3N XW3fTwmr0l9oAqFVJBD0Od9Z6u65E/EiKHfQdWV6C0KFKlesKIvk53IuoztRK23quoPi Buzg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to; bh=0k2zDzj92VVcO3q7+qs1VwVRfyPIa6GUvC2HZzmNfoA=; b=B5YPSUM+U/oij1bW3+bvEEGMvqqxL0Ay2f1thWeAprJkOPCipubVcA7ijeYKCQNZrn E5NcQx9bv74u9giain+PavuC6v3riapRGVY6tAUE5rIz4//U5ovSqF/avEsrFGYNB5n+ p+s9MaKh+Fb8k5zYJnwNVYfzftKOy+fcCI+5voptSgZatevVcWSfeQ1X7kCV35ajEFut abM84b56/qEehJ2xnOPfi6rQtoGIBrle8nl3qCJScrNLV0JWL1y0VTr/gPSTTdTmW6VZ j7L7WxWU7A2KEX5KN+1IzM2N3HP1uk07LlYbO3/IpHKcGwjWJqGNHYr2NIyoeh3Qg4hd SOdQ== X-Gm-Message-State: APf1xPC+yecKbRl9SEQcPTzSWQP4+ilIoybz1tQSrxkkfaQcBVi48XCA LQ8Ay21hilGw4II3nlYmI/acAoeuRhoyVdoO9cexos7q X-Google-Smtp-Source: AH8x227n//277CKcw8tuRfjPwzDw20xFzdTHJPFVBapEFmLcbJBjQnw4LmgrfyJ1WfvPqF71aPNuTZWB21SKsKFQjLk= X-Received: by 10.107.2.150 with SMTP id 144mr67285ioc.283.1518113830718; Thu, 08 Feb 2018 10:17:10 -0800 (PST) MIME-Version: 1.0 References: <5A7C2C15.3010302@steiner-buchholz.de> In-Reply-To: <5A7C2C15.3010302@steiner-buchholz.de> From: Eugene Kirpichov Date: Thu, 08 Feb 2018 18:17:00 +0000 Message-ID: Subject: Re: CoGroupByKey question To: user@beam.apache.org Content-Type: multipart/alternative; boundary="001a1139efc29cae250564b76bd0" --001a1139efc29cae250564b76bd0 Content-Type: text/plain; charset="UTF-8" You typically get this compile error when you have a raw type (e.g. PCollection instead of PCollection) sneaking somewhere. Perhaps tempPCollection or humPCollection is declared as a PCollection without a type? On Thu, Feb 8, 2018 at 2:53 AM Steiner Patrick wrote: > Hi all, > > I'm trying to combine two PCollections> via CoGroupByKey > > When adapting the sample code > > final TupleTag t1 = new TupleTag<>(); > final TupleTag t2 = new TupleTag<>(); > > PCollection> coGbkResultCollection = > KeyedPCollectionTuple.of(t1, pt1) > .and(t2, pt2) > .apply(CoGroupByKey.create()); > > I do get > > final TupleTag tempTuple = new TupleTag<>(); > final TupleTag humTuple = new TupleTag<>(); > > PCollection> coGbkResultCollection = > KeyedPCollectionTuple.of(tempTuple, tempPCollection) > .and(humTuple, humPCollection) > .apply(CoGroupByKey.create()); > > which results in > > Type mismatch: cannot convert from POutput to > PCollection> > > I guess it's a simple, stupid thing I'm either missing or not > understanding, when combining my two PCollections to one. > > Any advice is appreciated > > > Patrick > --001a1139efc29cae250564b76bd0 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
You typically get this compile error when you have a raw t= ype (e.g. PCollection instead of PCollection<Something>) sneaking som= ewhere. Perhaps tempPCollection or humPCollection is declared as a PCollect= ion without a type?

On= Thu, Feb 8, 2018 at 2:53 AM Steiner Patrick <patrick@steiner-buchholz.de> wrote:
Hi all,

I'm trying to combine two PCollections<KV<String, Double>>= =20 via CoGroupByKey

When adapting the sample code

final TupleTag<V1> t1 =3D new=20 TupleTag<>();
final TupleTag<V2> t2 =3D new TupleTag<>();

PCollection<KV<K, CoGbkResult>> coGbkResultCollection =3D
=C2=A0=C2=A0=C2=A0 =C2=A0=C2=A0=C2=A0 KeyedPCollectionTuple.of(t1, pt1)
=C2=A0=C2=A0=C2=A0 =C2=A0=C2=A0=C2=A0 .and(t2, pt2)
=C2=A0=C2=A0=C2=A0 =C2=A0=C2=A0=C2=A0 .apply(CoGroupByKey.<K>create()= );

I do get

final TupleTag<Double> tempTuple =3D new TupleTag<>();
final TupleTag<Double> humTuple =3D new TupleTag<>();
=C2=A0=C2=A0=C2=A0 =C2=A0=C2=A0=C2=A0
PCollection<KV<String,=20 CoGbkResult>> coGbkResultCollection =3D
=C2=A0=C2=A0=C2=A0 =C2=A0=C2=A0=C2=A0 KeyedPCollectionTuple.of(tempTuple, t= empPCollection)
=C2=A0=C2=A0=C2=A0 =C2=A0=C2=A0=C2=A0 .and(humTuple, humPCollection)
=C2=A0=C2=A0=C2=A0 =C2=A0=C2=A0=C2=A0 .apply(CoGroupByKey.<String>cre= ate());

which results in

Type mismatch: cannot convert from=20 POutput to PCollection<KV<String,CoGbkResult>>

I guess it's a simple, stupid thing I'm either missing or not=20 understanding, when combining my two PCollections to one.

Any advice is appreciated

Patrick
--001a1139efc29cae250564b76bd0--