Mailing-List: contact hbase-user-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: hbase-user@hadoop.apache.org
Received-SPF: pass (nike.apache.org: local policy)
Content-class: urn:content-classes:message
MIME-Version: 1.0
Content-Type: text/plain;
	charset="iso-8859-1"
Content-Transfer-Encoding: quoted-printable
Subject: RE: Basic question about using C# with Hadoop filesystems
Date: Sun, 10 Jan 2010 21:58:26 +0100
Message-ID: 
 <219D8244D980254ABF28AB469AD4E98F03469FCE@VF-MBX13.internal.vodafone.com>
Thread-Topic: Basic question about using C# with Hadoop filesystems
Thread-Index: AcqSLZUaOFbrmX5KTKClAoIF/PF/XwABUWWyAAEzNxo=
References: <27096203.post@talk.nabble.com>
  <78568af11001100254ob01e571r57b070912e663428@mail.gmail.com>
  <27098395.post@talk.nabble.com>
  <93baab321001100706i6280784x4db697a18bc2da6a@mail.gmail.com>
  <93baab321001100710h63545d4eoc517f6005579df44@mail.gmail.com>
  <27099222.post@talk.nabble.com>
  <93baab321001100736r3edfc41awf0915671d79a5474@mail.gmail.com>
 <31a243e71001100857t5373ab68ib3e2a98e6fe81a7@mail.gmail.com>
 <7743.24671.qm@web65507.mail.ac4.yahoo.com>
 <995786.96680.qm@web65504.mail.ac4.yahoo.com>
 <219D8244D980254ABF28AB469AD4E98F03469FCD@VF-MBX13.internal.vodafone.com>
From: "Gibbon, Robert, VF-Group" <Robert.Gibbon@vodafone.com>
To: "Gibbon, Robert, VF-Group" <Robert.Gibbon@vodafone.com>,
	<hbase-user@hadoop.apache.org>


http://code.google.com/p/jar2ikvmc/

This tool referenced from IKVM, referenced from the Mono project claims =
to resolve dependencies and convert java bytecode class files collected =
in JARs to Mono MSil DLLs. You might be able to use this to make a =
native MSil .NET port of the hbase client.

HTH
R

-----Original Message-----
From: Gibbon, Robert, VF-Group
Sent: Sun 1/10/2010 9:30 PM
To: hbase-user@hadoop.apache.org
Subject: RE: Basic question about using C# with Hadoop filesystems
=20
Options for you:

http://www.markhneedham.com/blog/2008/08/29/c-thrift-examples/

Thrift is one of the main ways into HBase and HDFS. Above is an IDL for =
C# blog post.

http://www.mono-project.com/using/relnotes/1.0-beta1.html

Mono has a bytecode to MSil translator built in. Not likely to give high =
performance though, and I doubt it will work TBH

http://caffeine.berlios.de

Provides mono-java interop via JNI: no dynamic bytecode->MSil =
translation. I have never used it and the project looks quite dead but =
might still do what you want.


-----Original Message-----
From: Andrew Purtell [mailto:apurtell@apache.org]
Sent: Sun 1/10/2010 8:45 PM
To: hbase-user@hadoop.apache.org
Subject: Re: Basic question about using C# with Hadoop filesystems
=20
Just to clarify:

> On Windows especially context switching during I/O like that has a=20
> high penalty.

should read

> Context switching during I/O like that has a penalty.

I know we are talking about Mono on Linux here. After all the subject
is FUSE. I forgot to fix that statement before hitting 'send'. :-)


----- Original Message ----
> From: Andrew Purtell <apurtell@apache.org>
> To: hbase-user@hadoop.apache.org
> Sent: Sun, January 10, 2010 11:30:42 AM
> Subject: Re: Basic question about using C# with Hadoop filesystems
>=20
> Bear in mind that hdfs-fuse has something like a 30% performance =
impact
> when compared with direct access via the Java API. The data path is
> something like:
>=20
>     your app -> kernel -> libfuse -> JVM -> kernel -> HDFS
>=20
>     HDFS -> kernel-> JVM -> libfuse -> kernel -> your app
>=20
> On Windows especially context switching during I/O like that has a=20
> high penalty. Maybe it would be better to bind the C libhdfs API
> directly via a C# wrapper (see http://wiki.apache.org/hadoop/LibHDFS).
> But, at that point, you have pulled the Java Virtual Machine into the
> address space of your process and are bridging between Java land and
> C# land over the JNI and the C# equivalent. So, at this point, why not
> just use Java instead of C#? Or, just use C and limit the damage to
> only one native-to-managed interface instead of two?
>=20
> The situation will change somewhat when/if all HDFS RPC is moved to
> some RPC and serialization scheme which is truly language independent,
> i.e. Avro. I have no idea when or if that will happen. Even if that
> happens, as Ryan said before, the HDFS client is fat. Just talking
> the RPC gets you maybe 25% of the way toward a functional HDFS
> client.=20
>=20
> The bottom line is the Hadoop software ecosystem has a strong Java
> affinity.=20
>=20
>    - Andy
>=20
>=20
>=20
> ----- Original Message ----
> > From: Jean-Daniel Cryans=20
> > To: hbase-user@hadoop.apache.org
> > Sent: Sun, January 10, 2010 8:57:32 AM
> > Subject: Re: Basic question about using C# with Hadoop filesystems
> >=20
> > http://code.google.com/p/hdfs-fuse/
> >=20
> > On Sun, Jan 10, 2010 at 7:36 AM, Aram Mkhitaryan
> > wrote:
> > > ah, sorry, forgot to mention, it's in hdfs-user mailing list
> > > hdfs-user@hadoop.apache.org


     =20