accumulo-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Josh Elser <josh.el...@gmail.com>
Subject Re: [DISCUSS] Hadoop 2 and Accumulo 1.6.0
Date Thu, 24 Oct 2013 17:11:42 GMT
On the contrary, it actually can. There's an example in the following 
repo which I've used to actually run Accumulo on YARN. It's still a 
little rough around the edges in some cases, but it does work.

https://github.com/hortonworks/hoya/

Point being though, there's nothing special that Accumulo needs to do 
here. The only real concern is the compatibility with Hadoop1 and 
Hadoop2 APIs (which is already done).

There are no architectural tradeoffs that I am aware of between hadoop 1 
and 2.

On 10/24/13 10:01 AM, John Vines wrote:
> Accumulo isn't going to run in YARN, it's just a matter of replacing MR
> compatibility with YARN compatibility.
>
>
> On Thu, Oct 24, 2013 at 7:04 AM, David Medinets <david.medinets@gmail.com>wrote:
>
>> Can an application work with parts in Hadoop 1 and parts in YARN? Could we
>> convert one component at a time to work inside YARN? For example, first
>> move monitor then tracer, etc. Where there any architectural tradeoffs made
>> for Hadoop 1 that no longer apply in Hadoop 2?
>>
>>
>> On Thu, Oct 24, 2013 at 12:58 AM, Josh Elser <josh.elser@gmail.com> wrote:
>>
>>> Highlights from hadoop.apache.org
>>>
>>>      YARN - A general purpose resource management system for Hadoop to
>>> allow MapReduce and other other data processing frameworks and services
>>>      High Availability for HDFS
>>>      HDFS Federation
>>>      HDFS Snapshots
>>>      NFSv3 access to data in HDFS
>>>      Support for running Hadoop on Microsoft Windows
>>>      Binary Compatibility for MapReduce applications built on hadoop-1.x
>>>      Substantial amount of integration testing with rest of projects in
>> the
>>> ecosystem
>>>
>>> Additionally, I believe there are substantial improvements in HDFS which
>>> should improve general performance for all filesystem ops.
>>>
>>>
>>> On 10/23/13 9:51 PM, William Slacum wrote:
>>>
>>>> There wasn't any discussions in those tickets as to what Hadoop 2
>> provides
>>>> Accumulo. If we're going to still support 1, then any new features only
>>>> possible with 2 have to become optional until we ditch support for 1. Is
>>>> there anything people have in mind, feature wise, that Hadoop 2 would
>> help
>>>> with?
>>>>
>>>>
>>>> On Wed, Oct 23, 2013 at 7:05 PM, Josh Elser <josh.elser@gmail.com>
>> wrote:
>>>>
>>>>   To ensure that we get broader community interaction than only on a Jira
>>>>> issue [1], I want to get community feedback about the version of Hadoop
>>>>> which the default, deployed Accumulo artifacts will be compiled
>> against.
>>>>>
>>>>> Currently, Accumulo builds against a Hadoop-1 series release
>>>>> (1.5.1-SNAPSHOT and 1.6.0-SNAPSHOT build against 1.2.1, and 1.5.0
>> builds
>>>>> against 1.0.4). Last week, the Apache Hadoop community voted to release
>>>>> 2.2.0 as GA (general availability) -- in other words, the Apache Hadoop
>>>>> community is calling Hadoop-2.2.0 "stable".
>>>>>
>>>>> As has been discussed across various issues on Jira, this means a few
>>>>> different things for Accumulo. Most importantly, this serves as a
>>>>> recommendation by us that users should be trying to use Hadoop-2.2.0
>> with
>>>>> Accumulo 1.6.0. This does *not* mean that we do not support Hadoop1
>> ([2]
>>>>> 1.2.1 specifically). Hadoop-1 support would still be "guaranteed" by
us
>>>>> for
>>>>> 1.6.0.
>>>>>
>>>>> - Josh
>>>>>
>>>>> [1] https://issues.apache.org/****jira/browse/ACCUMULO-1419<
>> https://issues.apache.org/**jira/browse/ACCUMULO-1419>
>>>>> <http**s://issues.apache.org/jira/**browse/ACCUMULO-1419<
>> https://issues.apache.org/jira/browse/ACCUMULO-1419>
>>>>>>
>>>>> [2] https://issues.apache.org/****jira/browse/ACCUMULO-1643<
>> https://issues.apache.org/**jira/browse/ACCUMULO-1643>
>>>>> <http**s://issues.apache.org/jira/**browse/ACCUMULO-1643<
>> https://issues.apache.org/jira/browse/ACCUMULO-1643>
>>>>>>
>>>>>
>>>>>
>>>>
>>
>

Mime
View raw message