mesos-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Wong, Steven" <>
Subject RE: Re: external storage with dvdi module
Date Wed, 18 Nov 2015 22:30:24 GMT
Regarding inquiry by Marica Antonacci on Wed, 18 Nov 2015 04:43:47 -0800

Failover scenarios are anticipated in the DVDI module and are being tracked as this issue
on github

The resolution of this issue requires that the underlying volume driver supports a "pre-emptive
mount" whereby any prior mounts are terminated automatically by a mount from another node.

Many but not all storage providers are capable of supporting this behavior.

The intention is that DVDI will handle scenarios including:

1. Task crashes on an agent node and is rescheduled on a different node. Note in this case,
if the slave process remains running, it is very likely that the dvdi module will remove the
mount automatically now. However this might not occur BEFORE the task is rescheduled on another
cluster node, so the upcoming pre-emptive mount feature could be useful in this scenario.

2. Entire Agent node crashes while task is running and does not restart or return. The agent
machine is a smoking piece of molten metal. This will REQUIRE the new dvdi feature and support
from the underlying storage driver.

3. Agent crashes and restarts. Similar to scenario #1 but perhaps the unmounts takes longer.
This scenario should result in removal of the original mount upon restart, and is implemented
now. If the Agent restart takes a long time, the roadmap pre-emptive mount feature will be
of value.

As a related note, some storage providers and some filesystems actually allow use of concurrent
mounts from multiple cluster nodes, although this is uncommon. The architecture of the dvdi
module is intended to support this, although no volume driver available today enable the functionality.

In summary pre-emptive mount is a roadmap feature for the dvdi module. I expect it to be delivered
shortly, after the feature is enabled in the rexray driver. My best guess is that this will
not occur within the next two weeks, but it may occur within the next two months. 

Steve Wong
Contributor to the Mesos dvdi module project Developer Advocate EMC{code} - the open source
advocacy group within EMC



View raw message