accumulo-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Steve Loughran <ste...@hortonworks.com>
Subject Re: [DISCUSS] Hadoop 2 and Accumulo 1.6.0
Date Thu, 24 Oct 2013 19:40:10 GMT
at a lower level, Hadoop 2.2 has some other opportunities too



- Move up the dependencies (inc things like SLF4J, protobuf) to be
consistent.


-co-exist with YARN. That shouldn't take any effort, merely that
a YARN node manager can run on every server node, so that YARN-scheduled
work can be executed on it -code that may want to work with Accumulo.

-deploy under YARN. This is what we are doing with Hoya

-Move to the Hadoop 2 APIs, which for Accumulo means changes in the
filesystem
APIs (FileContext alongside the classic FileSystem). While FileContext
promises
a more consistent set of semantics, I don't see any single tangible feature
in
the API that isn't in FileSystem (I speak as someone who has just
implemented
a new FileSystem implementation for Hadoop 1 & 2)/

- Move to the new "undeprecated" configuration options. This is very much
a committing move, except in the special case that you are referencing
strings in the Hadoop source, and they change in those JARs. (remember,
strings
don't get copied at compile time, only the types where sizeof() < =8 ).
As the deprecated options get migrated, not moving merely adds more
warning messages into the logs. Which you can turn off by cranking back on
the level
of org.apache.hadoop.conf.Configuration.deprecation (Hadoop-9487 by one
Stevel)

The least traumatic options would be
 -stay with the APIs and constants that work with 1.x for now
 -run from the 2.0.5 APIs, switch Hadoop 2 to 2.2.0
 -test and run on it: find bugreps, file them now
 -help us with Hoya -we are drafting it for incubation- so that anyone with
 a Hadoop 2 cluster can bring up an Accumulo cluster on top of it -without
 even needing to talk to ops about it.



On 24 October 2013 05:51, William Slacum <wilhelm.von.cloud@accumulo.net>wrote:

> There wasn't any discussions in those tickets as to what Hadoop 2 provides
> Accumulo. If we're going to still support 1, then any new features only
> possible with 2 have to become optional until we ditch support for 1. Is
> there anything people have in mind, feature wise, that Hadoop 2 would help
> with?
>
>
> On Wed, Oct 23, 2013 at 7:05 PM, Josh Elser <josh.elser@gmail.com> wrote:
>
> > To ensure that we get broader community interaction than only on a Jira
> > issue [1], I want to get community feedback about the version of Hadoop
> > which the default, deployed Accumulo artifacts will be compiled against.
> >
> > Currently, Accumulo builds against a Hadoop-1 series release
> > (1.5.1-SNAPSHOT and 1.6.0-SNAPSHOT build against 1.2.1, and 1.5.0 builds
> > against 1.0.4). Last week, the Apache Hadoop community voted to release
> > 2.2.0 as GA (general availability) -- in other words, the Apache Hadoop
> > community is calling Hadoop-2.2.0 "stable".
> >
> > As has been discussed across various issues on Jira, this means a few
> > different things for Accumulo. Most importantly, this serves as a
> > recommendation by us that users should be trying to use Hadoop-2.2.0 with
> > Accumulo 1.6.0. This does *not* mean that we do not support Hadoop1 ([2]
> > 1.2.1 specifically). Hadoop-1 support would still be "guaranteed" by us
> for
> > 1.6.0.
> >
> > - Josh
> >
> > [1] https://issues.apache.org/**jira/browse/ACCUMULO-1419<
> https://issues.apache.org/jira/browse/ACCUMULO-1419>
> > [2] https://issues.apache.org/**jira/browse/ACCUMULO-1643<
> https://issues.apache.org/jira/browse/ACCUMULO-1643>
> >
>

-- 
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to 
which it is addressed and may contain information that is confidential, 
privileged and exempt from disclosure under applicable law. If the reader 
of this message is not the intended recipient, you are hereby notified that 
any printing, copying, dissemination, distribution, disclosure or 
forwarding of this communication is strictly prohibited. If you have 
received this communication in error, please contact the sender immediately 
and delete it from your system. Thank You.

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message