hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Peter Marron <Peter.Mar...@trilliumsoftware.com>
Subject RE: libhdfs on windows
Date Thu, 25 Oct 2012 23:53:03 GMT
Hi Colin,

OK, I didn't know there was a hdfs-dev.
I'm happy to ask there.
(However there's a lot of mail on dev@hadoop
and user@hadoop as well as user@hive
and it's a bit of a commitment to track them all.)
As for webhdfs, I did think about that, and in
some ways it's a beautiful solution as it
gives me a platform- and language-neutral
access mechanism. I was just a little worried
about the HTTP overhead if I am reading a single
record at a time. Also I will need some way to launch
my Map/Reduce jobs as well. So I'll probably
end up using the C++/JNI/Java route to do
that anyway. Unless there's a better way?
Is there a web Map/Reduce interface?

Many thanks,

Z

> -----Original Message-----
> From: rarecactus@gmail.com
> [mailto:rarecactus@gmail.com] On
> Behalf Of Colin McCabe
> Sent: 25 October 2012 18:24
> To: common-dev@hadoop.apache.org
> Subject: Re: libhdfs on windows
> 
> Hi Peter,
> 
> This might be a good question for hdfs-
> dev?
> 
> As Harsh pointed out below, HDFS-573
> was never committed.  I don't even see a
> patch attached, although there is some
> discussion.
> 
> In the mean time, might I suggest using
> the webhdfs interface on Windows?
> webhdfs was intended as a stable REST
> interface that can be accessed from any
> platform.
> 
> cheers,
> Colin
> 
> 
> On Thu, Oct 25, 2012 at 7:19 AM, Peter
> Marron
> <Peter.Marron@trilliumsoftware.com>
> wrote:
> > Hi,
> >
> > I've been looking at using libhdfs and I
> would like to use it on windows.
> > I have found HDFS-573 and the
> information on this page:
> >
> http://issues.apache.org/jira/browse/HD
> FS-573?page=com.atlassian.jira.
> > plugin.system.issuetabpanels:all-
> tabpanel
> > which suggests that quite a lot of work
> was done on this way back in 2009.
> > So is there some source from this
> effort retained somewhere? If so,
> where?
> > Or do I have to start from scratch?
> Apologies if this has already been asked
> recently.
> >
> > Any help appreciated.
> >
> > Peter Marron



Mime
View raw message