hadoop-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Bart Verwilst <li...@verwilst.be>
Subject Re: Pydoop 0.7.0-rc1 released
Date Fri, 16 Nov 2012 21:02:21 GMT
Hi Simone,

I was wondering, is it possible to write AVRO files to hadoop straight 
from your lib ( mixed with avro libs ofcourse )? I'm currently trying to 
come up with a way to read from mysql ( but more complicated than sqoop 
can handle ) and write it out to avro files on HDFS. Is something like 
this feasible with this? How do you see it?

Thanks!

Bart

Simone Leo schreef op 13.11.2012 14:11:
> Hello everyone,
>
> we're happy to announce that we have just released Pydoop 0.7.0-rc1
> (http://pydoop.sourceforge.net).
>
> The main changes with respect to the previous version are:
>
>  * support for CDH4 (MapReduce v1 only)
>  * tested with the latest releases of other supported Hadoop versions
>  * simpler build process
>  * Pydoop scripts can now accept user-defined configuration 
> parameters
>  * new wrapper object makes it easier to interact with the JobConf
>  * new hdfs.path functions: isdir, isfile, kind
>  * HDFS: support for string description of permission modes in chmod
>  * several bug fixes
>
> This is a release candidate.  We're working on binary packages for
> the final release.  As usual, we're happy to receive your feedback on
> the forum:
>
> http://sourceforge.net/projects/pydoop/forums/forum/990018
>
> Pydoop is a Python MapReduce and HDFS API for Hadoop, built upon the 
> C++
> Pipes and the C libhdfs APIs, that allows to write full-fledged
> MapReduce applications with HDFS access. Pydoop has been maturing
> nicely and is currently in production use at CRS4 as we have a few
> scientific projects that are based on it, including Seal
> (http://biodoop-seal.sourceforge.net), Biodoop-BLAST
> (http://biodoop.sourceforge.net/blast), and more yet to be released.
>
> Links:
>
>  * full release notes: http://pydoop.sourceforge.net/docs/news.html
>  * download page on sf: http://sourceforge.net/projects/pydoop/files
>  * download page on PyPI: 
> http://pypi.python.org/pypi/pydoop/0.7.0-rc1
>  * git repo:
> 
> http://pydoop.git.sourceforge.net/git/gitweb.cgi?p=pydoop/pydoop;a=summary
>
> Happy pydooping!
>
>
> The Pydoop Team

Mime
View raw message