Return-Path: X-Original-To: apmail-datafu-dev-archive@minotaur.apache.org Delivered-To: apmail-datafu-dev-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id C3AF517BA5 for ; Mon, 20 Apr 2015 17:17:20 +0000 (UTC) Received: (qmail 67177 invoked by uid 500); 20 Apr 2015 17:17:20 -0000 Delivered-To: apmail-datafu-dev-archive@datafu.apache.org Received: (qmail 67126 invoked by uid 500); 20 Apr 2015 17:17:20 -0000 Mailing-List: contact dev-help@datafu.incubator.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@datafu.incubator.apache.org Delivered-To: mailing list dev@datafu.incubator.apache.org Received: (qmail 67111 invoked by uid 99); 20 Apr 2015 17:17:16 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 20 Apr 2015 17:17:16 +0000 X-ASF-Spam-Status: No, hits=3.2 required=5.0 tests=HTML_MESSAGE,SPF_SOFTFAIL X-Spam-Check-By: apache.org Received-SPF: softfail (athena.apache.org: transitioning domain of matthew.terence.hayes@gmail.com does not designate 54.191.145.13 as permitted sender) Received: from [54.191.145.13] (HELO mx1-us-west.apache.org) (54.191.145.13) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 20 Apr 2015 17:17:10 +0000 Received: from mail-la0-f52.google.com (mail-la0-f52.google.com [209.85.215.52]) by mx1-us-west.apache.org (ASF Mail Server at mx1-us-west.apache.org) with ESMTPS id 60B6120C83 for ; Mon, 20 Apr 2015 17:16:49 +0000 (UTC) Received: by lagv1 with SMTP id v1so132139914lag.3 for ; Mon, 20 Apr 2015 10:16:47 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type; bh=SkZAc+SqFSR4hVnCu1SIAzyL2LsBngkzYF5FRRrbpEk=; b=kWQ+0V04EZBhcHj+FDw0v2DMSmydjouitjAYWE+ztH3YKgP/JD0EZcI/SBzjpENZWp B1vfLGJs2+P7i2U1ZfE2mpsdoBWkgnqCKwKPFiZvnU8A9iZeiNMuOYWvNYkpnuOz3Hge 3HznfSH1iR1bfr6X6JKmuhk7pM4S64Wc5vceBGdOvG9sUUwEfHwQEUmh4wnp+EbqUG5h 5mfXM7TNLAYI4xmPskAxHQpoQOCKBiXoqKomX2CWABJKrDlYUkyf+oZKo18eShLSAMsh Lim4PXa3m4m7uU1PNNMdcre8wbq5rQsedelh1u6jeo4qZyZz+2oQvuzQpSFEXqHwQdRc 9Jzg== MIME-Version: 1.0 X-Received: by 10.112.130.129 with SMTP id oe1mr16222782lbb.37.1429550207880; Mon, 20 Apr 2015 10:16:47 -0700 (PDT) Received: by 10.114.1.212 with HTTP; Mon, 20 Apr 2015 10:16:47 -0700 (PDT) In-Reply-To: References: Date: Mon, 20 Apr 2015 10:16:47 -0700 Message-ID: Subject: Re: HourGlass From: Matthew Hayes To: DataFu , hodgesz@gmail.com Content-Type: multipart/alternative; boundary=e89a8f64732f54543f05142b1830 X-Virus-Checked: Checked by ClamAV on apache.org --e89a8f64732f54543f05142b1830 Content-Type: text/plain; charset=UTF-8 Hey Jonathan, sorry for the late response. I'm glad to hear you are finding the framework useful :) So far I haven't thought much about moving this to other processing engines like Spark and we don't have any immediate plans to do so. If you have some ideas though about this I would love to hear them. -Matt On Wed, Apr 1, 2015 at 8:25 AM, Jonathan Hodges wrote: > Hi Guys, > > First off I wanted to praise you guys for a great framework for incremental > processing over immutable data. We are using it with nice success in our > organization. We are now starting to move more towards streaming and > memory processing via Spark. So my quick question is are there any plans > to move this from MR to other processing engines like Spark? If so, we > would like to help. > > Thanks again for a great open source project! > Jonathan > --e89a8f64732f54543f05142b1830--