Mailing-List: contact hdfs-issues-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: hdfs-issues@hadoop.apache.org
Date: Tue, 21 Jan 2014 21:14:21 +0000 (UTC)
From: "Chris Nauroth (JIRA)" <jira@apache.org>
To: hdfs-issues@hadoop.apache.org
Message-ID: <JIRA.12610859.1349722326641.41913.1390338861097@arcas>
In-Reply-To: <JIRA.12610859.1349722326641@arcas>
References: <JIRA.12610859.1349722326641@arcas>
Subject: [jira] [Commented] (HDFS-4019) FSShell should support creating
 symlinks
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable


    [ https://issues.apache.org/jira/browse/HDFS-4019?page=3Dcom.atlassian.=
jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=3D13877=
861#comment-13877861 ]=20

Chris Nauroth commented on HDFS-4019:
-------------------------------------

Hi, Colin.  This patch mostly looks good to me.  I took it for a test run, =
and it worked as advertised.  Could you please update FileSystemShell.apt.v=
m for the new command?

I did notice a discrepancy in the behavior compared to local file system ln=
 implementations I've used.  If the link_path is already a symlink that exi=
sts pointing to a directory, then the default behavior I've seen is to foll=
ow the symlink and put the new symlink underneath that target directory.  S=
ee below for an example run on OS X to demonstrate it.

When I run through the same sequence on HDFS, the behavior is different.  I=
nstead, the second ln fails due to file already exists.

The symlink following behavior is typically disabled with the -n or -h flag=
 (resulting in file already exists error), and then additionally you can ad=
d -f to force overwrite of link_path (resulting in successful replacement o=
f the link with the new target).  It seems the current patch is doing the e=
quivalent of ln -sn (but not the -f flag).

I can think of a few different ways we can handle this:
# Add more code in the shell to traverse {{linkPath}} if it exists and is a=
 symlink.
# Don't code for this case, but force callers to specify -n, just like the =
patch currently forces them to specify -s.
# Document the difference in behavior.

Additionally, I'd like to see support for a -f flag eventually.  I think we=
're going to need different API support from the NameNode for this though, =
so that's probably going to be a different patch.  ln -sfn can be a useful =
building block for atomically publishing changes to a file.  For example, y=
ou could have readers access a file named "currentView", which is really a =
symlink to the last published version of the file.  Then, a writer can stag=
e an update in a separate file and use ln -sfn to swap in the changes atomi=
cally.

Let me know your thoughts on this.  Thanks.

{code}
[chris@Chriss-MacBook-Pro:ttys006] lntest                                  =
                                        =20
> mkdir dir1 dir2
[chris@Chriss-MacBook-Pro:ttys006] lntest                                  =
                                        =20
> ln -s dir1 link1
[chris@Chriss-MacBook-Pro:ttys006] lntest                                  =
                                        =20
> ln -s dir2 link1
[chris@Chriss-MacBook-Pro:ttys006] lntest                                  =
                                        =20
> tree
.
=E2=94=9C=E2=94=80=E2=94=80 dir1
=E2=94=82=C2=A0=C2=A0 =E2=94=94=E2=94=80=E2=94=80 dir2 -> dir2
=E2=94=9C=E2=94=80=E2=94=80 dir2
=E2=94=94=E2=94=80=E2=94=80 link1 -> dir1

3 directories, 1 file
{code}


> FSShell should support creating symlinks
> ----------------------------------------
>
>                 Key: HDFS-4019
>                 URL: https://issues.apache.org/jira/browse/HDFS-4019
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>          Components: tools
>    Affects Versions: 2.0.3-alpha
>            Reporter: Colin Patrick McCabe
>            Assignee: Colin Patrick McCabe
>            Priority: Minor
>         Attachments: HDFS-4019.001.patch, HDFS-4019.002.patch, HDFS-4019.=
003.patch
>
>
> FSShell should support creating symlinks.  This would allow users to crea=
te symlinks from the shell without having to write a Java program.
> One thing that makes this complicated is that FSShell currently uses File=
System internally, and symlinks are currently only supported by the FileCon=
text API.  So either FSShell would have to be ported to FileContext, or sym=
links would have to be added to FileSystem.  Or perhaps we could open a Fil=
eContext only when symlinks were necessary, but that seems messy.


--
This message was sent by Atlassian JIRA
(v6.1.5#6160)