Mailing-List: contact dev-help@ignite.incubator.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@ignite.incubator.apache.org
Received-SPF: pass (athena.apache.org: domain of dsetrakyan@gridgain.com
 designates 209.85.212.174 as permitted sender)
MIME-Version: 1.0
From: Dmitriy Setrakyan <dsetrakyan@gridgain.com>
Date: Tue, 30 Dec 2014 10:56:20 -0800
Message-ID: 
 <CA+0=VoVNdbqMFQxH6ybJzUNcbpTcG8S2hBF=4-_-iYsTyRT8HA@mail.gmail.com>
Subject: [DISCUSS] Hadoop Accelerator as part of Apache Ignite
To: dev@ignite.incubator.apache.org
Content-Type: multipart/alternative; boundary=089e013d202857cc1f050b738e10

--089e013d202857cc1f050b738e10
Content-Type: text/plain; charset=UTF-8

Hi,

I want to open up a discussion about adding GridGain Hadoop Accelerator as
a feature of Apache Ignite.

As some of you may know, Hadoop Accelerator is now offered as a part of
GridGain open source edition. It is built on top of Ignite In-Memory Data
Fabric technology and provides plug-n-play acceleration for Hadoop. It also
recently has been integrated with Apache BigTop.

The acceleration is achieved by providing the following Hadoop components
in memory:

   - IgniteFS, in-memory Hadoop-compliant file system, which natively plugs
   into Hadoop, and is built on top of Ignite data grid.
   - Ignite MapReduce, very fast Hadoop MapReduce implementation, which is
   built on top of Ignite computation framework.

I anticipate that some questions will arise around how Ignite Hadoop
Accelerator is different from Apache Spark. The reality is that they are
very different.

One of the main differences is that Ignite Hadoop Accelerator will offer
acceleration of the existing Hadoop MapReduce computations that run
natively on Hadoop, while Spark essentially takes you off of Hadoop
MapReduce into its own DSL. Additionally, because of the fast MapReduce
implementation, Ignite Hadoop Accelerator will also accelerate native Hive
queries, while Spark provides its own SQL engine.

More information about Hadoop Accelerator can be found here:
http://hadoop.gridgain.org/

Please let me know your thoughts.

--089e013d202857cc1f050b738e10--