Return-Path: X-Original-To: apmail-cassandra-user-archive@www.apache.org Delivered-To: apmail-cassandra-user-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id A5CE118B95 for ; Thu, 12 Nov 2015 13:41:47 +0000 (UTC) Received: (qmail 93726 invoked by uid 500); 12 Nov 2015 13:41:45 -0000 Delivered-To: apmail-cassandra-user-archive@cassandra.apache.org Received: (qmail 93661 invoked by uid 500); 12 Nov 2015 13:41:45 -0000 Mailing-List: contact user-help@cassandra.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@cassandra.apache.org Delivered-To: mailing list user@cassandra.apache.org Received: (qmail 93651 invoked by uid 99); 12 Nov 2015 13:41:45 -0000 Received: from Unknown (HELO spamd1-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 12 Nov 2015 13:41:45 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd1-us-west.apache.org (ASF Mail Server at spamd1-us-west.apache.org) with ESMTP id D5D31C8D85 for ; Thu, 12 Nov 2015 13:41:44 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd1-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: 2.898 X-Spam-Level: ** X-Spam-Status: No, score=2.898 tagged_above=-999 required=6.31 tests=[DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, HTML_MESSAGE=3, RCVD_IN_MSPIKE_H2=-0.001, SPF_PASS=-0.001] autolearn=disabled Authentication-Results: spamd1-us-west.apache.org (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com Received: from mx1-eu-west.apache.org ([10.40.0.8]) by localhost (spamd1-us-west.apache.org [10.40.0.7]) (amavisd-new, port 10024) with ESMTP id qGXQiKXBoU0L for ; Thu, 12 Nov 2015 13:41:44 +0000 (UTC) Received: from mail-yk0-f172.google.com (mail-yk0-f172.google.com [209.85.160.172]) by mx1-eu-west.apache.org (ASF Mail Server at mx1-eu-west.apache.org) with ESMTPS id AD2C020512 for ; Thu, 12 Nov 2015 13:41:43 +0000 (UTC) Received: by ykdv3 with SMTP id v3so94714031ykd.0 for ; Thu, 12 Nov 2015 05:41:42 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type; bh=5lcAgvoxdmQ9w6HjRQvIvo3fBnGwqwrHX68n1hPU/7I=; b=TL688ADx7yzJ0XuMKHZGA036iVfVLYOe1dLV45j3yKJc76nAWIA0Xjyf8Z2gkKuDPI 8GRiEV5BNd1pmvRcR1E8yq5LrwYUC3zv6dySF88zyKIbqd3rpZvpWFg2kYRjb6DXEtej VBf19B6j9wJz/Qi+tL9bk+T40c9J13Wy1qK7UPb+QaLtAQdku/ibvGKOnPBfTDkNOU7E erdsRLlZXNgSyNpqgbrKST8QL8z51myOk1+DgjV+y2f0AM26fhZwO06ODozpX2PO2JmP qYsC5qQlcNdHYUiZlEgrs6mVJuFg3tLFDo3P8BXxXtuEdjCYyrNPWC30t6DrOZdWeQQX bFfg== MIME-Version: 1.0 X-Received: by 10.129.145.11 with SMTP id i11mr15209366ywg.118.1447335702676; Thu, 12 Nov 2015 05:41:42 -0800 (PST) Received: by 10.13.221.15 with HTTP; Thu, 12 Nov 2015 05:41:42 -0800 (PST) In-Reply-To: References: Date: Thu, 12 Nov 2015 14:41:42 +0100 Message-ID: Subject: Re: Spark on cassandra From: DuyHai Doan To: user@cassandra.apache.org Content-Type: multipart/alternative; boundary=94eb2c096e806dcf100524581acd --94eb2c096e806dcf100524581acd Content-Type: text/plain; charset=UTF-8 Hello Prem I believe it's better to ask your question on the ML of the Spark Cassandra connector: http://groups.google.com/a/lists.datastax.com/forum/#!forum/spark-connector-user Second "we need to join multiple table from multiple keyspaces. How can we do that?", the response is given in your example, you pass the keyspace name as parameter for each operation with Cassandra. On Thu, Nov 12, 2015 at 2:12 PM, Prem Yadav wrote: > Hi, > Is it better to use Spark APIs to do join on cassandra tables or should we > use SPARK-SQL. > We have been struggling with SPARK-SQL as we need to do multiple large > table joins and there is always failure. > > I tried to do joins using the API like this: > val join1 = > sc.cassandraTable("Keyspace1","table1").joinWithCassandraTable("keyspace1","table2").on(SomeColumns("column1")) > > however, we need to join multiple table from multiple keyspaces. How can > we do that? > > > Thanks, > Prem > --94eb2c096e806dcf100524581acd Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable
Hello Prem

I believe it's better to= ask your question on the ML of the Spark Cassandra connector:=C2=A0http://groups.google.com/a/lists.datastax.com/forum/#!forum/spar= k-connector-user

Second "we need to join multiple table from multiple keyspaces. Ho= w can we do that?", the response is given in your example, you pass th= e keyspace name as parameter for each operation with Cassandra.

On Thu, No= v 12, 2015 at 2:12 PM, Prem Yadav <ipremyadav@gmail.com> = wrote:
Hi,
Is it bet= ter to use Spark APIs to do join on cassandra tables or should we use SPARK= -SQL.
We have been struggling with SPARK-SQL as we need to do mul= tiple large table joins and there is always failure.

I tried to do joins using the API like this:
val join1 =3D sc.= cassandraTable("Keyspace1","table1").joinWithCassandraT= able("keyspace1","table2").on(SomeColumns("column1= "))

however, we need to join multiple tab= le from multiple keyspaces. How can we do that?

Thanks,
Prem

--94eb2c096e806dcf100524581acd--