mesos-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Vetoshkin Nikita <nikita.vetosh...@gmail.com>
Subject Re: HDFS on Mesos
Date Thu, 26 Jun 2014 15:23:15 GMT
What about long term solution? Any ideas? Twitter's Manhattan database
claims to use Mesos for scaling up and down. Can you shed some light how do
they deal with the situation like this?
On Jun 26, 2014 5:01 AM, "Vinod Kone" <vinodkone@gmail.com> wrote:

> Thanks for listing this out Adam.
>
> Data Residency:
> > - Should we destroy the sandbox/hdfs-data when shutting down a DN?
> > - If starting DN on node that was previously running a DN, can/should we
> > try to revive the existing data?
> >
>
> I think this is one of the key challenges for a production quality HDFS on
> Mesos. Currently, since sandbox is deleted after a task exits, if all the
> data nodes that hold a block (and its replicas) get lost/killed for
> whatever reason there would be data loss. A short terms solution would be
> to write outside sandbox and use slave attributes to track where to
> re-launch data node tasks.
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message