spark-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From 牛兆捷 <nzjem...@gmail.com>
Subject Re: Questions about Fault tolerance of Spark
Date Sat, 11 Jul 2015 14:34:02 GMT
I am in the beginning, would you like to share something on this area?

2015-07-10 22:05 GMT+08:00 MIKE HYNES <91mbbh@gmail.com>:

> Gentle bump on this topic; how to test the fault tolerance and previous
> benchmark results are both things we are interested in as well.
> Mike
>
>
> -------- Original message --------
> From: 牛兆捷
> Date:07-09-2015 04:19 (GMT-05:00)
> To: dev@spark.apache.org, user@spark.apache.org
> Subject: Questions about Fault tolerance of Spark
>
> Hi All:
>
> We already know that Spark utilizes the lineage to recompute the RDDs when
> failure occurs.
> I want to study the performance of this fault-tolerant approach and have
> some questions about it.
>
> 1) Is there any benchmark (or standard failure model) to test the fault
> tolerance of these kinds of in-memory data processing systems?
>
> 2) How do you emulate the failures in testing spark?  (e.g., kill a
> computation task? or kill the computation nodes?)
>
> Thanks!!!
>
> --
> *Regards,*
> *Zhaojie*
>
>


-- 
*Regards,*
*Zhaojie*

Mime
View raw message