accumulo-notifications mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Billie Rinaldi (JIRA)" <>
Subject [jira] [Commented] (ACCUMULO-804) Hadoop 2.0 Support
Date Sat, 13 Oct 2012 16:46:03 GMT


Billie Rinaldi commented on ACCUMULO-804:

The test issues have already been addressed by creating a ContextFactory for ACCUMULO-564.
 I would be in favor of fixing the MapReduce tests anyway and getting rid of the ContextFactory.

The remaining issues appear to be:

1) Pulling JobTracker information for org.apache.accumulo.server.monitor.servlets.DefaultServlet.
 We could potentially pull ResourceManager information instead, and do something with reflection
to determine when to do that (like with the ContextFactory) or we could remove that information
from the Accumulo monitor.

2) Instantiating SocketInputStream in org.apache.accumulo.core.util.TTimeoutTransport.  This
class should be instantiated with NetUtils.getInputStream, which also exists in Hadoop 1.
 While we're at it, we might as well replace SocketOutputStream with NetUtils.getOutputStream,
although it isn't broken in Hadoop 2.
> Hadoop 2.0 Support
> ------------------
>                 Key: ACCUMULO-804
>                 URL:
>             Project: Accumulo
>          Issue Type: Improvement
>          Components: client
>    Affects Versions: 1.5.0, 1.4.3
>            Reporter: Ed Kohlwey
>            Assignee: Billie Rinaldi
> We should start thinking about Hadoop 2 support now that it is Cloudera's recommended
distribution and many new Hadoop users will probably be adopting it.
> When I investigated this first a few months ago it seemed like the biggest barrier to
this was that all the Map/Reduce related tests are implemented using pseudo-private constructors
from Hadoop 1.0 that are no-longer present in Hadoop 2.0.
> The main strategy to fix this should probably be to adopt the Map/Reduce cluster test
object for testing the various Accumulo input formats instead of instrumenting them directly.
I have used this convenience object successfully on tests utilizing MockInstance, so I think
it should work fine.
> There may also be some filesystem API issues but I don't think they will be too severe.
> The other main issue is that we will need to actually deploy on Hadoop 1 and 2 and run
the integration tests once we start supporting both, so that will be a headache for release
testing that we should think through.

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see:

View raw message