Mailing-List: contact dev-help@spark.apache.org; run by ezmlm
Precedence: bulk
From: Deepu Raj <deepuraj_tech@outlook.com>
To: "dev@spark.apache.org" <dev@spark.apache.org>
Subject: Spark Streaming Cassandra Transformation
Thread-Topic: Spark Streaming Cassandra Transformation
Thread-Index: AQHSzUP7MP5I4Olv20y+znw3ysASEQ==
Date: Mon, 15 May 2017 06:25:02 +0000
Message-ID: <ME1PR01MB08354B850A2604F40615548281E10@ME1PR01MB0835.ausprd01.prod.outlook.com>
Accept-Language: en-AU, en-US
Content-Language: en-US
x-ms-exchange-messagesentrepresentingtype: 1
x-ms-publictraffictype: Email
x-microsoft-exchange-diagnostics: 1;HK2APC01HT180;5:5vkqHbuptYmg5HWFL7jbO2Hpf/qlrpykF9DCcvFDYidNbSl8AQfGHYmtM0TIbRc1XIoOzOW7jkH9YrdFtXdCeZC2OAI55UBzFnOODKwA5xx4Fyng3tRwOTULLyHNy+bPFLWGN8s9uMcwvozBQDid5Q==;24:+fHQOMnaDRnMoZfpNTyECZZx+Dyd0k/4/otaUjbscVffP/wtOi0UW72lHQNHKqLbSFy7MvOMnmyTZ4u9ueofULdvwCGcfkw6gKXxfiBsjc8=;7:GcFL0MS3p3DVn86SKyLaevX5N/U1icmbIWPGQ2eg0vw8JI36JROjlaspRFvfF6j0kBixUbZl0ffkrhA8+VHS9bPAe20DVdhQdMMvGtrVgwLyGBISqhlMTPQZNybgftGQSCFoNhssyMAUz9OOhahIVT3oCVn1xEicuDt9iyXKxX4CeFDTE9Ig1n3mUZpaYEMt4E1FUWCO4ZY1xRu2MoDcl4I+kzUsw+U0DsCAM7hvevaLVb8sP5Vqm40aOeVInopJO5w8RnZOkXo0gdpKkV2UbGv3Of2EzN9Fzy5+UE4UZQvpTjeYkQF7SLoadU7jqWww
x-incomingheadercount: 40
x-eopattributedmessage: 0
x-forefront-antispam-report: EFV:NLI;SFV:NSPM;SFS:(7070007)(98901004);DIR:OUT;SFP:1901;SCL:1;SRVR:HK2APC01HT180;H:ME1PR01MB0835.ausprd01.prod.outlook.com;FPR:;SPF:None;LANG:en;
x-ms-office365-filtering-correlation-id: 03531347-2c59-4d5d-937b-08d49b5b1d0e
x-microsoft-antispam: UriScan:;BCL:0;PCL:0;RULEID:(22001)(201702061074)(5061506573)(5061507331)(1603103135)(2017031320274)(2017031324274)(2017031323274)(2017031322274)(1601125374)(1603101448)(1701031045);SRVR:HK2APC01HT180;
x-exchange-antispam-report-cfa-test: BCL:0;PCL:0;RULEID:(444000031);SRVR:HK2APC01HT180;BCL:0;PCL:0;RULEID:;SRVR:HK2APC01HT180;
x-forefront-prvs: 0308EE423E
spamdiagnosticoutput: 1:99
spamdiagnosticmetadata: NSPM
Content-Type: text/plain; charset="iso-8859-15"
Content-ID: <78DE63416708324D830FE1EA88EAE7EA@ausprd01.prod.outlook.com>
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-OriginatorOrg: outlook.com
X-MS-Exchange-CrossTenant-originalarrivaltime: 15 May 2017 06:25:02.1055
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Internet
X-MS-Exchange-CrossTenant-id: 84df9e7f-e9f6-40af-b435-aaaaaaaaaaaa
X-MS-Exchange-Transport-CrossTenantHeadersStamped: HK2APC01HT180
archived-at: Mon, 15 May 2017 06:25:17 -0000

Hi Team,


1. Can someone please guide on  how to change the following transformation =
=20
change for the below data in Spark 2.0?

{"name":"Michael", "age":30, "state":AZ, "Job": SoftwareDev}
{"name":"Andy", "age":30,  "state":WA, "Job": Mana}
{"name":"Justin", "age":19, , "state":VIC, "Job": DBAdmin}

KafkaUtils.createStream[String, String, StringDecoder, StringDecoder](
ssc, kafkaParams, topicMap, StorageLevel.MEMORY_ONLY)
.map{ case (_,json) =3D> =20
JsonParser.parse(json).extract[MonthlyCommits]}.saveToCassandra("github_sta=
ts","commits_aggr")


2. What is the best way to do this push JSON into Cassandra without using =
=20
DF?

Thanks,
Deepu Raj
--=20
Using Opera's mail client: http://www.opera.com/mail/

---------------------------------------------------------------------
To unsubscribe e-mail: dev-unsubscribe@spark.apache.org