Mailing-List: contact user-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@hadoop.apache.org
Received-SPF: neutral (athena.apache.org: local policy)
MIME-Version: 1.0
In-Reply-To: <loom.20141030T184842-507@post.gmane.org>
References: <loom.20141030T184842-507@post.gmane.org>
Date: Fri, 31 Oct 2014 09:45:28 +0100
Message-ID: 
 <CAH70K77fdCyFKDWW8wfFQ2iY9o3eyQKnB4rqz=9vNs4onb6i_Q@mail.gmail.com>
Subject: Re: About building dummy Custom Distributed File system for hadoop.
From: Thomas Demoor <thomas.demoor@amplidata.com>
To: user@hadoop.apache.org
Content-Type: multipart/alternative; boundary=001a11c1215add55e70506b404a1

--001a11c1215add55e70506b404a1
Content-Type: text/plain; charset=UTF-8

Hi Vikrant,

YARN uses AbstractFileSystem. Filesystem is the legacy interface (Hadoop
v1).

However, there is an abstract class DelegateToFileSystem that (trivially)
enables using a legacy FileSystem in YARN  (see f.i. the Wasb class in
hadoop-azure)

So what you should implement depends on if you want to run YARN, legacy
Hadoop or both. Distributed filesystems are hard, nice to see you are
ambitious.


Good luck,
Thomas


On Thu, Oct 30, 2014 at 6:52 PM, Vikrant Dhimate <vikrantdhimate@gmail.com>
wrote:

> We are working on B.Tech project and we want to build dummy Custom
> Distributed File system for hadoop.
>
> In 2.5.1 version there are 2 classes : 1) FileSystem and 2)
> AbstractFileSystem.
> To make our own custom DFS which one of these two classes should we
> implement or extend to our custom filesystem class. Because in earlier
> version there was only FileSystem class.
>
> Can you please tell us what is difference between FileSystem and
> AbstractFileSystem classes and when to use each one sand which one should
> we
> implement in our case?
>
> Thanks,
> Vikrant
>
>

--001a11c1215add55e70506b404a1
Content-Type: text/html; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr"><div>Hi Vikrant,</div><div><br></div>YARN uses=C2=A0<span =
style=3D"font-size:13px;font-family:arial,sans-serif">AbstractFileSystem. F=
ilesystem is the legacy interface (Hadoop v1).=C2=A0</span><div><div><span =
style=3D"font-family:arial,sans-serif;font-size:13px"><br></span></div><div=
><span style=3D"font-family:arial,sans-serif;font-size:13px">However, there=
 is an</span><font face=3D"arial, sans-serif">=C2=A0abstract class Delegate=
ToFileSystem that (trivially) enables using a legacy FileSystem in YARN =C2=
=A0(see f.i. the Wasb class in hadoop-azure)</font></div><div><font face=3D=
"arial, sans-serif"><br></font></div><div>So what you should implement depe=
nds on if you want to run YARN, legacy Hadoop or both. Distributed filesyst=
ems are hard, nice to see you are ambitious.</div><div><br></div><div><br><=
/div><div>Good luck,</div><div>Thomas</div><div><br></div></div><div class=
=3D"gmail_extra"><div><div class=3D"gmail_signature"><div dir=3D"ltr"><div>=
<br></div></div></div></div>
<br><div class=3D"gmail_quote">On Thu, Oct 30, 2014 at 6:52 PM, Vikrant Dhi=
mate <span dir=3D"ltr">&lt;<a href=3D"mailto:vikrantdhimate@gmail.com" targ=
et=3D"_blank">vikrantdhimate@gmail.com</a>&gt;</span> wrote:<br><blockquote=
 class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1px #ccc soli=
d;padding-left:1ex">We are working on B.Tech project and we want to build d=
ummy Custom<br>
Distributed File system for hadoop.<br>
<br>
In 2.5.1 version there are 2 classes : 1) FileSystem and 2) AbstractFileSys=
tem.<br>
To make our own custom DFS which one of these two classes should we<br>
implement or extend to our custom filesystem class. Because in earlier<br>
version there was only FileSystem class.<br>
<br>
Can you please tell us what is difference between FileSystem and<br>
AbstractFileSystem classes and when to use each one sand which one should w=
e<br>
implement in our case?<br>
<br>
Thanks,<br>
Vikrant<br>
<br>
</blockquote></div><br></div></div>

--001a11c1215add55e70506b404a1--