hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Alejandro Abdelnur <t...@cloudera.com>
Subject Re: Bringing Hadoop to Fedora
Date Tue, 09 Jul 2013 00:53:47 GMT
suresh, 

thanks for jumping in from webhdfs side. 

apparently, the issues that were affecting us with jetty 6 have been address with newer jetty
versions. 

my take would be we move forward with the version change and we make sure webhdfs is not affected,
do you have a test that exercises webhdfs under load? that would definitely help

thx 
  
Alejandro
(phone typing)

On Jul 8, 2013, at 17:43, Suresh Srinivas <suresh@hortonworks.com> wrote:

> Isn't Jetty used by WebHDFS? Given that, Jetty performance is still
> important.
> 
> 
> On Mon, Jul 8, 2013 at 2:06 PM, Alejandro Abdelnur <tucu@cloudera.com>wrote:
> 
>> [moving bigtop to bcc]
>> 
>> Tim,
>> 
>> Except from HADOOP-9680 which has significant code changes and some false
>> changes (which I did not go thru), all other changes seem OK.
>> 
>> * Have you had a change to run ALL Hadoop testcases with them applied to
>> make sure there are not regression?
>> 
>> * Have you look at the output of running 'mvn dependency:tree' without/with
>> the patches to verify unwanted dependencies are not sneaking in?
>> 
>> On HADOOP-9650 (not included in the above list):
>> 
>> Moving from Jetty6 to Jetty9 should be OK if there are not regressions. I
>> was chatting with ToddL about the issues we had before in and according to
>> Jetty guys they've been fixed in newer Jetty versions. Also, we are not
>> using Jetty for shuffle anymore (we use Netty), so Jetty is not 'stress'
>> that much anymore as it is used for the web UIs and for NN-SNN
>> checkpointing.
>> 
>> From the patch some code changes are required, once that is take care we
>> should repeat the bullet item above for this patch.
>> 
>> In addition, we have to see how this will play with other projects like
>> HBase that are using HttpServer from hadoop-common. I'll forward this part
>> of the email to their dev@ so they can watch/jump-in if necessary in the
>> JIRA.
>> 
>> Thanks.
>> 
>> 
>> On Mon, Jul 8, 2013 at 11:28 AM, Tim St Clair <tstclair@redhat.com> wrote:
>> 
>>> Arun,
>>> 
>>> I realize you're probably pretty busy (or on vacation), but I figured I
>>> would re-ping this thread to inquire about the status of the patch set
>>> listed below.?.?
>>> 
>>> https://issues.apache.org/jira/browse/HADOOP-9594
>>> https://issues.apache.org/jira/browse/HADOOP-9610
>>> https://issues.apache.org/jira/browse/HADOOP-9611
>>> https://issues.apache.org/jira/browse/HADOOP-9613
>>> https://issues.apache.org/jira/browse/HADOOP-9680 / 9623
>>> 
>>> Cheers,
>>> Tim
>>> 
>>> ----- Original Message -----
>>>> From: "Arun C Murthy" <acm@hortonworks.com>
>>>> To: yarn-dev@hadoop.apache.org
>>>> Cc: common-dev@hadoop.apache.org, mapreduce-dev@hadoop.apache.org,
>>> hdfs-dev@hadoop.apache.org, dev@bigtop.apache.org
>>>> Sent: Tuesday, June 11, 2013 12:54:11 PM
>>>> Subject: Re: Bringing Hadoop to Fedora
>>>> 
>>>> Tim,
>>>> 
>>>> This is great! I'll take a look at some of these patches, welcome!
>>>> 
>>>> Arun
>>>> 
>>>> On Jun 11, 2013, at 8:54 AM, Tim St Clair wrote:
>>>> 
>>>>> Greetings Hadoop Development Community,
>>>>> 
>>>>> As some of you are aware, we've been looking to bring the upstream
>> 2.X
>>>>> series into the Fedora channels.  We believe that there are several
>>>>> benefits that this can bring to the community:
>>>>> 
>>>>> - Updated & streamlined rpm packaging
>>>>> - Deeper level of system integration
>>>>> - System managed dependencies (security + defect tracking)
>>>>> - Proving ground for OpenJDK7
>>>>> - Greater exposure for the community (both adoption and defect
>>> tracking)
>>>>> ...
>>>>> 
>>>>> During our evaluation (
>> https://fedoraproject.org/wiki/Features/Hadoop
>>> ),
>>>>> we've come across some build issues as they relate to Fedora
>> packaging
>>>>> guidelines (http://fedoraproject.org/wiki/Packaging:Java).  Most of
>>> these
>>>>> issues (listed below) are very minor dependency changes, and we were
>>>>> wondering if there are folks who would be willing to review.
>>>>> 
>>>>> fedora-patch-math: https://issues.apache.org/jira/browse/HADOOP-9594
>>>>> fedora-patch-collections:
>>> https://issues.apache.org/jira/browse/HADOOP-9610
>>>>> fedora-patch-cglib:
>> https://issues.apache.org/jira/browse/HADOOP-9611
>>>>> fedora-patch-jersey:
>> https://issues.apache.org/jira/browse/HADOOP-9613
>>>>> fedora-patch-jets3t:
>> https://issues.apache.org/jira/browse/HADOOP-9623
>>>>> (uber deprecated)
>>>>> 
>>>>> There are other patches we have in the queue but they still need to
>> be
>>>>> flushed out, and in an effort to follow the KISS principles we figure
>>> this
>>>>> would be a good start.
>>>>> 
>>>>> Cheers,
>>>>> Tim
>>>> 
>>>> --
>>>> Arun C. Murthy
>>>> Hortonworks Inc.
>>>> http://hortonworks.com/
>> 
>> 
>> 
>> --
>> Alejandro
> 
> 
> 
> -- 
> http://hortonworks.com/download/

Mime
View raw message