spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Divya Gehlot <divya.htco...@gmail.com>
Subject RE: Spark JDBC connection - data writing success or failure cases
Date Sun, 21 Feb 2016 00:08:59 GMT
Thanks for the input everyone .
What I am trying to understand is if I use oracle to store my data after
Spark job processing.
And if any spark job fails half the way.
What happens then..
Does rollback happens or we have to programatically  handle this kind of
situation in spark job itself?
How transaction are being handled n spark to oracle storage ?
My apologies for such a naive question .
Thanks,
Divya

agreed



Dr Mich Talebzadeh



LinkedIn * https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw
<https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw>*



http://talebzadehmich.wordpress.com



NOTE: The information in this email is proprietary and confidential. This
message is for the designated recipient only, if you are not the intended
recipient, you should destroy it immediately. Any information in this
message shall not be understood as given or endorsed by Peridale Technology
Ltd, its subsidiaries or their employees, unless expressly so stated. It is
the responsibility of the recipient to ensure that this email is virus
free, therefore neither Peridale Technology Ltd, its subsidiaries nor their
employees accept any responsibility.





*From:* Russell Jurney [mailto:russell.jurney@gmail.com]
*Sent:* 19 February 2016 16:49
*To:* Jörn Franke <jornfranke@gmail.com>
*Cc:* Divya Gehlot <divya.htconex@gmail.com>; user @spark <
user@spark.apache.org>
*Subject:* Re: Spark JDBC connection - data writing success or failure cases



Oracle is a perfectly reasonable endpoint for publishing data processed in
Spark. I've got to assume he's using it that way and not as a stand in for
HDFS?

On Friday, February 19, 2016, Jörn Franke <jornfranke@gmail.com> wrote:

Generally oracle db should not be used as a storage layer for spark due to
performance reasons. You should consider HDFS. This will help you also with
fault - tolerance.

> On 19 Feb 2016, at 03:35, Divya Gehlot <divya.htconex@gmail.com> wrote:
>
> Hi,
> I am a Spark job which connects to RDBMS (in mycase its Oracle).
> How can we check that complete data writing is successful?
> Can I use commit in case of success or rollback in case of failure ?
>
>
>
> Thanks,
> Divya

---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
For additional commands, e-mail: user-help@spark.apache.org



-- 

Russell Jurney twitter.com/rjurney russell.jurney@gmail.com relato.io

Mime
View raw message