Mailing-List: contact dev-help@spark.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@spark.apache.org
Received-SPF: pass (athena.apache.org: local policy includes SPF record at
 spf.trusted-forwarder.org)
MIME-Version: 1.0
In-Reply-To: 
 <CAAzo=r-+uO0GZF4iZw7QfojQ5bn7WC9sPB6Ze_2dkdwPcSHbLw@mail.gmail.com>
References: 
 <CAAzo=r-+uO0GZF4iZw7QfojQ5bn7WC9sPB6Ze_2dkdwPcSHbLw@mail.gmail.com>
Date: Tue, 4 Mar 2014 16:55:11 -0800
Message-ID: 
 <CAPh_B=bofbTnKV+PXt3kKYXKStF-XcuvqUPXXQDf-RdKPkhMhA@mail.gmail.com>
Subject: Re: a question about fault recovery
From: Reynold Xin <rxin@databricks.com>
To: dev@spark.apache.org
Content-Type: multipart/alternative; boundary=001a11c3a98407604f04f3d1791b

--001a11c3a98407604f04f3d1791b
Content-Type: text/plain; charset=ISO-8859-1

BlockManager is only responsible for in-memory/on-disk storage. It has
nothing to do with re-computation.

All the recomputation / retry code are done in the DAGScheduler. Note that
when a node crashes, due to lazy evaluation, there is no task that needs to
be re-run. Those tasks are re-run only when their outputs are needed for
another task/job.


On Tue, Mar 4, 2014 at 4:51 PM, dachuan <hdc1112@gmail.com> wrote:

> Hello, developers,
>
> I am just curious about the following two things which seems to be
> contradictory to each other, please help me find out my understanding
> mistakes:
>
> 1) Excerpted from sosp 2013 paper, "Then, when a node fails, the system
> detects all missing RDD partitions and launches tasks to recompute them
> from the last checkpoint. Many tasks can be launched at the same time to
> compute different RDD partitions, allowing the whole cluster to partake in
> recovery."
>
> 2) Excerpted from code, this function is called when there's one dead
> BlockManager, this function didn't launch tasks to recover lost partitions,
> instead, it updated many meta-data.
>
>   private def removeBlockManager(blockManagerId: BlockManagerId) {
>     val info = blockManagerInfo(blockManagerId)
>
>     // Remove the block manager from blockManagerIdByExecutor.
>     blockManagerIdByExecutor -= blockManagerId.executorId
>
>     // Remove it from blockManagerInfo and remove all the blocks.
>     blockManagerInfo.remove(blockManagerId)
>     val iterator = info.blocks.keySet.iterator
>     while (iterator.hasNext) {
>       val blockId = iterator.next
>       val locations = blockLocations.get(blockId)
>       locations -= blockManagerId
>       if (locations.size == 0) {
>         blockLocations.remove(locations)
>       }
>     }
>   }
>
> thanks,
> dachuan.
>
> --
> Dachuan Huang
> Cellphone: 614-390-7234
> 2015 Neil Avenue
> Ohio State University
> Columbus, Ohio
> U.S.A.
> 43210
>

--001a11c3a98407604f04f3d1791b--