hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Chris Nauroth <cnaur...@hortonworks.com>
Subject Re: Falcon usecases
Date Fri, 04 Dec 2015 17:51:56 GMT
I'll just add a bit to Biren's response by saying that I personally find Falcon compelling
as a user.  I was a user of Hadoop before I became a developer on Hadoop.  As a user, I found
that most of my engineering effort went into figuring out how to get data into Hadoop and
then figuring out how to get job results back out to other external systems.  I wrote a lot
of custom one-off code to do this for different projects.  Eventually, I helped build a somewhat
Falcon-like system to simplify the process of onboarding new data sets into my Hadoop cluster.
 Falcon has a richer feature set though compared to that system I helped build.  If I was
still in my prior role, I'd be giving Falcon a serious evaluation as a replacement.

--Chris Nauroth

From: Biren Saini <bsaini@hortonworks.com<mailto:bsaini@hortonworks.com>>
Date: Friday, December 4, 2015 at 6:25 AM
To: praveenesh kumar <praveenesh@gmail.com<mailto:praveenesh@gmail.com>>, Chris
Nauroth <cnauroth@hortonworks.com<mailto:cnauroth@hortonworks.com>>
Cc: "user@hadoop.apache.org<mailto:user@hadoop.apache.org>" <user@hadoop.apache.org<mailto:user@hadoop.apache.org>>
Subject: RE: Falcon usecases

I am the Governance SME lead at Hortonworks which includes Falcon. Like Chris said falcon
mailing list is a better group for this question but here is the crux of what you are looking
for -

We have many clients (including F100) who have been using Falcon in production very successfully.
There are a ton on features in the roadmap that I am looking forward to. Falcon plays a critical
role in the overall data governance story for Hadoop.

Check out - hortonworks.com/hadoop/falcon for overview of the tool and more details.

Here is a tutorial that will get you started -


Sample data pipeline built using Falcon in my github repo  - https://github.com/sainib/hadoop-data-pipeline

For any more follow up questions - please try the falcon distribution.


-------- Original message --------
From: praveenesh kumar <praveenesh@gmail.com<mailto:praveenesh@gmail.com>>
Date:12/04/2015 6:43 AM (GMT-05:00)
To: Chris Nauroth <cnauroth@hortonworks.com<mailto:cnauroth@hortonworks.com>>
Cc: user@hadoop.apache.org<mailto:user@hadoop.apache.org>
Subject: Re: Falcon usecases

Thanks Chris for pointing me to the mailing list and HDP support forums. However my question
is more general and generic that is why I thought of putting it here. All I am trying to understand
from anyone in the hadoop community who has encountered Falcon before to understand how the
community is responding towards it. Does anyone using it or trying to use it. I can understand
that falcon mailing list currently doesn't support user mailing list that is why I thought
of putting this question here rather than subscribing to one more mailing list.

@Chris - What is the reason HDP is backing it and delivering it in the HDP distribution? Do
you see any future/current client use cases which kinds of highlighting its necessities.

FYI - I am trying to be working on falcon for past 2 weeks and trying to understand it much
better from the industry point of view, hence asking this to understand whether I am on a
right path or its still a long way to go before falcon can be used as a production tool.

On Wed, Dec 2, 2015 at 5:26 PM, Chris Nauroth <cnauroth@hortonworks.com<mailto:cnauroth@hortonworks.com>>
Hello Prav,

You might have better luck getting a response to this question by directly asking the Falcon
community.  I don't see a user@ mailing list for Falcon, but I do see a dev@ list.  More details
are here:


For questions related specifically to HDP Sandbox, you'll likely get more help from Hortonworks
support forums.  (This is generally true for any vendor product that differentiates from the
Apache distro.)

I hope this helps.

--Chris Nauroth

From: praveenesh kumar <praveenesh@gmail.com<mailto:praveenesh@gmail.com>>
Date: Wednesday, December 2, 2015 at 10:01 AM
To: "user@hadoop.apache.org<mailto:user@hadoop.apache.org>" <user@hadoop.apache.org<mailto:user@hadoop.apache.org>>
Subject: Falcon usecases

Hello hadoopers

Just curious to understand what is the current state of falcon.. How much it is currently
being adopted in the industry.. Anyone even using it other than the creators?

There is not much information on the internet about falcon examples and use cases but then
it is coming along in HDP distribution. Hence this question on understanding what are the
best engineering/deployment principles around it ?

I personally tried the GUI and it doesn't seems to be working properly on HDP sandbox 2.3.2,
but that is another question to dig later. Before that I wanted to understand the current
adoption of Falcon around big data industry.

Anyone with any insights, please share..!!


View raw message