mesos-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jie Yu (JIRA)" <>
Subject [jira] [Issue Comment Deleted] (MESOS-6360) The handling of whiteout files in provisioner is not correct
Date Fri, 14 Oct 2016 19:57:20 GMT


Jie Yu updated MESOS-6360:
    Comment: was deleted

(was: commit 3f6503861e92896a619e3b26a65f9b679a2dd3a9
Author: Qian Zhang <>
Date:   Fri Oct 14 11:41:02 2016 -0700

    Added backend suffix to image layer rootfs path.
    Previously image layer rootfs path is in the format below regardless
    of which backend is used.
    This introduced an issue: when agent is restarted with a different
    backend, we will wrongly handle the whiteout files since different
    backends(e.g., aufs and overlay) have different whiteout standard.
    In this commit, we added backend suffix to image layer rootfs path
    for overlay backend like below.
    For non-overlay backends, it is still in the previous format, this
    is because they share the same whiteout standard (aufs standard),
    and also used to handle backward compatibility.
    So the expected result of this commit is:
    1. If user switches backend from overlay to a non-overlay (or vice
    versa) when restarting agent, all the image layers of the previous
    backend in the store will just be ignored.
    2. In the upgrade case, if user starts a new version of Mesos agent
    with overlay backend, then all the image layers in the store pulled
    by the old agent will just be ignored, but if user starts the new
    agent with a non-overlay backend, then all the image layers in the
    store can still be used.

> The handling of whiteout files in provisioner is not correct
> ------------------------------------------------------------
>                 Key: MESOS-6360
>                 URL:
>             Project: Mesos
>          Issue Type: Bug
>            Reporter: Qian Zhang
>            Assignee: Qian Zhang
>            Priority: Blocker
> Currently when user launches a container from a Docker image via universal containerizer,
we always handle the whiteout files in {{ProvisionerProcess::__provision()}} regardless of
which backend is used.
> However this is actually not correct, because the way to handle whiteout files is backend
dependent, that means for different backends, we need to handle whiteout files in different
ways, e.g.:
> * AUFS backend: It seems the AUFS whiteout ({{.wh.<filename/dirname>}} and {{.wh..wh..opq}})
is the whiteout standard in Docker (see [this comment |]
for details), so that means after the Docker image is pulled, its whiteout files in the store
are already in aufs format, then we do not need to do anything about whiteout file handling
because the aufs mount done in {{AufsBackendProcess::provision()}} will handle it automatically.
> * Overlay backend: Overlayfs has its own whiteout files (see [this doc |]
for details), so we need to convert the aufs whiteout files to overlayfs whiteout files before
we do the overlay mount in {{OverlayBackendProcess::provision}} which will automatically handle
the overlayfs whiteout files.
> * Copy backend: We need to manually handle the aufs whiteout files when we copy each
layer in {{CopyBackendProcess::_provision()}}.

This message was sent by Atlassian JIRA

View raw message