Return-Path: X-Original-To: apmail-hadoop-user-archive@minotaur.apache.org Delivered-To: apmail-hadoop-user-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 2B58718126 for ; Fri, 4 Dec 2015 13:25:36 +0000 (UTC) Received: (qmail 1832 invoked by uid 500); 4 Dec 2015 13:25:30 -0000 Delivered-To: apmail-hadoop-user-archive@hadoop.apache.org Received: (qmail 1716 invoked by uid 500); 4 Dec 2015 13:25:30 -0000 Mailing-List: contact user-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Delivered-To: mailing list user@hadoop.apache.org Received: (qmail 1704 invoked by uid 99); 4 Dec 2015 13:25:30 -0000 Received: from Unknown (HELO spamd1-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 04 Dec 2015 13:25:30 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd1-us-west.apache.org (ASF Mail Server at spamd1-us-west.apache.org) with ESMTP id 93D81C13FE for ; Fri, 4 Dec 2015 13:25:29 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd1-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: 4.001 X-Spam-Level: **** X-Spam-Status: No, score=4.001 tagged_above=-999 required=6.31 tests=[FSL_HELO_BARE_IP_2=1, HTML_MESSAGE=3, URIBL_BLOCKED=0.001] autolearn=disabled Received: from mx1-us-west.apache.org ([10.40.0.8]) by localhost (spamd1-us-west.apache.org [10.40.0.7]) (amavisd-new, port 10024) with ESMTP id nlp3VRb4eKue for ; Fri, 4 Dec 2015 13:25:20 +0000 (UTC) Received: from relayvx11b.securemail.intermedia.net (relayvx11b.securemail.intermedia.net [64.78.52.184]) by mx1-us-west.apache.org (ASF Mail Server at mx1-us-west.apache.org) with ESMTPS id 2EBEB201FB for ; Fri, 4 Dec 2015 13:25:20 +0000 (UTC) Received: from securemail.intermedia.net (localhost [127.0.0.1]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by emg-ca-1-1.localdomain (Postfix) with ESMTPS id C4AC753F2A; Fri, 4 Dec 2015 05:25:19 -0800 (PST) Subject: RE: Falcon usecases MIME-Version: 1.0 x-echoworx-msg-id: fde073fb-a31f-43e1-8594-8685f8f3adfb x-echoworx-emg-received: Fri, 4 Dec 2015 05:25:19.737 -0800 x-echoworx-action: delivered Received: from 10.254.155.14 ([10.254.155.14]) by emg-ca-1-1 (JAMES SMTP Server 2.3.2) with SMTP ID 722; Fri, 4 Dec 2015 05:25:19 -0800 (PST) Received: from MBX080-W4-CO-2.exch080.serverpod.net (unknown [10.224.117.102]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by emg-ca-1-1.localdomain (Postfix) with ESMTPS id 7DA2653F2A; Fri, 4 Dec 2015 05:25:19 -0800 (PST) Received: from MBX080-W4-CO-2.exch080.serverpod.net (10.224.117.102) by MBX080-W4-CO-2.exch080.serverpod.net (10.224.117.102) with Microsoft SMTP Server (TLS) id 15.0.1130.7; Fri, 4 Dec 2015 05:25:18 -0800 Received: from MBX080-W4-CO-2.exch080.serverpod.net ([10.224.117.102]) by mbx080-w4-co-2.exch080.serverpod.net ([10.224.117.102]) with mapi id 15.00.1130.005; Fri, 4 Dec 2015 05:25:18 -0800 From: Biren Saini To: praveenesh kumar , Chris Nauroth CC: "user@hadoop.apache.org" Thread-Topic: Falcon usecases Thread-Index: AQHRLojuU0JYlLPU2EmcDjLKpDH8T5660V3I Date: Fri, 4 Dec 2015 13:25:17 +0000 Message-ID: <2brh12rvv9cyhtewj7qvltwf.1449235138153@email.android.com> References: , In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-ms-exchange-transport-fromentityheader: Hosted x-source-routing-agent: Processed Content-Type: multipart/alternative; boundary="_000_2brh12rvv9cyhtewj7qvltwf1449235138153emailandroidcom_" --_000_2brh12rvv9cyhtewj7qvltwf1449235138153emailandroidcom_ Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Praveenesh, I am the Governance SME lead at Hortonworks which includes Falcon. Like Chr= is said falcon mailing list is a better group for this question but here is= the crux of what you are looking for - We have many clients (including F100) who have been using Falcon in product= ion very successfully. There are a ton on features in the roadmap that I am= looking forward to. Falcon plays a critical role in the overall data gover= nance story for Hadoop. Check out - hortonworks.com/hadoop/falcon for overview of the tool and more= details. Here is a tutorial that will get you started - http://hortonworks.com/hadoop-tutorial/defining-processing-data-end-end-dat= a-pipeline-apache-falcon/ Sample data pipeline built using Falcon in my github repo - https://github= .com/sainib/hadoop-data-pipeline For any more follow up questions - please try the falcon distribution. Biren. -------- Original message -------- From: praveenesh kumar Date:12/04/2015 6:43 AM (GMT-05:00) To: Chris Nauroth Cc: user@hadoop.apache.org Subject: Re: Falcon usecases Thanks Chris for pointing me to the mailing list and HDP support forums. Ho= wever my question is more general and generic that is why I thought of putt= ing it here. All I am trying to understand from anyone in the hadoop commun= ity who has encountered Falcon before to understand how the community is re= sponding towards it. Does anyone using it or trying to use it. I can unders= tand that falcon mailing list currently doesn't support user mailing list t= hat is why I thought of putting this question here rather than subscribing = to one more mailing list. @Chris - What is the reason HDP is backing it and delivering it in the HDP = distribution? Do you see any future/current client use cases which kinds of= highlighting its necessities. FYI - I am trying to be working on falcon for past 2 weeks and trying to un= derstand it much better from the industry point of view, hence asking this = to understand whether I am on a right path or its still a long way to go be= fore falcon can be used as a production tool. On Wed, Dec 2, 2015 at 5:26 PM, Chris Nauroth > wrote: Hello Prav, You might have better luck getting a response to this question by directly = asking the Falcon community. I don't see a user@ mailing list for Falcon, = but I do see a dev@ list. More details are here: http://falcon.apache.org/mail-lists.html For questions related specifically to HDP Sandbox, you'll likely get more h= elp from Hortonworks support forums. (This is generally true for any vendo= r product that differentiates from the Apache distro.) I hope this helps. --Chris Nauroth From: praveenesh kumar > Date: Wednesday, December 2, 2015 at 10:01 AM To: "user@hadoop.apache.org" > Subject: Falcon usecases Hello hadoopers Just curious to understand what is the current state of falcon.. How much i= t is currently being adopted in the industry.. Anyone even using it other t= han the creators? There is not much information on the internet about falcon examples and use= cases but then it is coming along in HDP distribution. Hence this question= on understanding what are the best engineering/deployment principles aroun= d it ? I personally tried the GUI and it doesn't seems to be working properly on H= DP sandbox 2.3.2, but that is another question to dig later. Before that I = wanted to understand the current adoption of Falcon around big data industr= y. Anyone with any insights, please share..!! Regards Prav --_000_2brh12rvv9cyhtewj7qvltwf1449235138153emailandroidcom_ Content-Type: text/html; charset="us-ascii" Content-Transfer-Encoding: quoted-printable
Praveenesh, 
I am the Governance SME lead at Hortonworks which includes Falcon. Lik= e Chris said falcon mailing list is a better group for this question but he= re is the crux of what you are looking for - 

We have many clients (including F100) who have been using Falcon in pr= oduction very successfully. There are a ton on features in the roadmap that= I am looking forward to. Falcon plays a critical role in the overall data = governance story for Hadoop. 

Check out - hortonworks.com/hadoop/falcon for overview of the tool and= more details.

Here is a tutorial that will get you started - 

http://hortonworks.com/hadoop-tutorial/defining-processing-data-end-en= d-data-pipeline-apache-falcon/

Sample data pipeline built using Falcon in my github repo  -=  https://github.com/sainib/hadoop-data-pipeline

For any more follow up questions - please try the falcon distribution.=  

Biren.


-------- Original message --------
From: praveenesh kumar <praveenesh@gmail.com>
Date:12/04/2015 6:43 AM (GMT-05:00)
To: Chris Nauroth <cnauroth@hortonworks.com>
Cc: user@hadoop.apache.org
Subject: Re: Falcon usecases

Thanks Chris for pointing me to the mailing list and HDP s= upport forums. However my question is more general and generic that is why = I thought of putting it here. All I am trying to understand from anyone in = the hadoop community who has encountered Falcon before to understand how the community is responding towards it. Do= es anyone using it or trying to use it. I can understand that falcon mailin= g list currently doesn't support user mailing list that is why I thought of= putting this question here rather than subscribing to one more mailing list. 

@Chris - What is the reason HDP is backing it and delivering it in the= HDP distribution? Do you see any future/current client use cases which kin= ds of highlighting its necessities. 

FYI - I am trying to be working on falcon for past 2 weeks and trying = to understand it much better from the industry point of view, hence asking = this to understand whether I am on a right path or its still a long way to = go before falcon can be used as a production tool.


On Wed, Dec 2, 2015 at 5:26 PM, Chris Nauroth <cnauroth@= hortonworks.com> wrote:
Hello Prav,

You might have better luck getting a response to this question by dire= ctly asking the Falcon community.  I don't see a user@ mailing list fo= r Falcon, but I do see a dev@ list.  More details are here:


For questions related specifically to HDP Sandbox, you'll likely get m= ore help from Hortonworks support forums.  (This is generally true for= any vendor product that differentiates from the Apache distro.)

I hope this helps.

--Chris Nauroth<= /font>

From: praveenesh kumar <praveenesh@gmail.com= >
Date: Wednesday, December 2, 2015 a= t 10:01 AM
To: "user@hadoop.apache.org" <user@hadoop.apache= .org>
Subject: Falcon usecases

Hello hadoopers

Just curious to understand what is the current state of falcon.. How m= uch it is currently being adopted in the industry.. Anyone even using it ot= her than the creators?

There is not much information on the internet about falcon examples an= d use cases but then it is coming along in HDP distribution. Hence this que= stion on understanding what are the best engineering/deployment principles = around it ?

I personally tried the GUI and it doesn't seems to be working properly= on HDP sandbox 2.3.2, but that is another question to dig later. Before th= at I wanted to understand the current adoption of Falcon around big data in= dustry. 

Anyone with any insights, please share..!!

Regards
Prav

--_000_2brh12rvv9cyhtewj7qvltwf1449235138153emailandroidcom_--