hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From R Nair <ravishankar.n...@gmail.com>
Subject Re: Scheduling in YARN according to available resources
Date Sat, 21 Feb 2015 23:07:33 GMT
Hi Tariq,

Glad to see that your issue is resolved, thank you. This re-affirms the
compatibility issue with openJDK. Thanks

Regards,
Ravi

On Sat, Feb 21, 2015 at 1:40 PM, tesmai4@gmail.com <tesmai4@gmail.com>
wrote:

> Dear Nair,
>
> Your tip in your first email saved my day. Tahnks once again. I am happy
> with Oracle JDK.
>
> Regards,
> Tariq
>
> On Sat, Feb 21, 2015 at 4:05 PM, R Nair <ravishankar.nair@gmail.com>
> wrote:
>
>> one of it is in the forum, if you search in google you will get more. I
>> am not saying it may not work, but you will have to select and apply some
>> patches. One of my friends also had the same problem and with too much
>> difficulty, he got this into work. So better avoid :)
>>
>> https://github.com/elasticsearch/elasticsearch-hadoop/issues/197
>>
>> Thanks and regards,
>> Nair
>>
>> On Sat, Feb 21, 2015 at 8:20 AM, tesmai4@gmail.com <tesmai4@gmail.com>
>> wrote:
>>
>>> Thanks Nair.
>>>
>>> Managed installing Oracle JDK and it is working great. Thanks for the
>>> tip.
>>>
>>> Any idea why OpenJDK is crashing and Oracle JDK works?
>>>
>>> Regards,
>>> Tariq
>>>
>>>
>>>
>>>
>>> On Sat, Feb 21, 2015 at 7:14 AM, tesmai4@gmail.com <tesmai4@gmail.com>
>>> wrote:
>>>
>>>> Thanks for your answer Nair,
>>>> Is installing Oracle JDK on Ubuntu is that complicated as described in
>>>> this link
>>>>
>>>> http://askubuntu.com/questions/56104/how-can-i-install-sun-oracles-proprietary-java-jdk-6-7-8-or-jre
>>>>
>>>> Is there an alternate?
>>>>
>>>> Regards
>>>>
>>>>
>>>> On Sat, Feb 21, 2015 at 6:50 AM, R Nair <ravishankar.nair@gmail.com>
>>>> wrote:
>>>>
>>>>> I had an issue very similar, I changed and used Oracle JDK. There is
>>>>> nothing I see wrong with your configuration in my first look, thanks
>>>>>
>>>>> Regards,
>>>>> Nair
>>>>>
>>>>> On Sat, Feb 21, 2015 at 1:42 AM, tesmai4@gmail.com <tesmai4@gmail.com>
>>>>> wrote:
>>>>>
>>>>>> I have 7 nodes in my Hadoop cluster [8GB RAM and 4VCPUs to each
>>>>>> nodes], 1 Namenode + 6 datanodes.
>>>>>>
>>>>>> I followed the link from Hortonwroks [
>>>>>> http://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.0.6.0/bk_installing_manually_book/content/rpm-chap1-11.html
>>>>>> ] and made calculation according to the hardware configruation on
my
>>>>>> nodes. Added the update mapred-site and yarn-site.xml files in my
question.
>>>>>> Still my application is crashing with the same exection
>>>>>>
>>>>>> My mapreduce application has 34 input splits with a block size of
>>>>>> 128MB.
>>>>>>
>>>>>> **mapred-site.xml** has the  following properties:
>>>>>>
>>>>>>     mapreduce.framework.name  = yarn
>>>>>>     mapred.child.java.opts    = -Xmx2048m
>>>>>>     mapreduce.map.memory.mb   = 4096
>>>>>>     mapreduce.map.java.opts   = -Xmx2048m
>>>>>>
>>>>>> **yarn-site.xml** has the  following properties:
>>>>>>
>>>>>>     yarn.resourcemanager.hostname        = hadoop-master
>>>>>>     yarn.nodemanager.aux-services        = mapreduce_shuffle
>>>>>>     yarn.nodemanager.resource.memory-mb  = 6144
>>>>>>     yarn.scheduler.minimum-allocation-mb = 2048
>>>>>>     yarn.scheduler.maximum-allocation-mb = 6144
>>>>>>
>>>>>>
>>>>>>  Exception from container-launch: ExitCodeException exitCode=134:
>>>>>> /bin/bash: line 1:  3876 Aborted  (core dumped)
>>>>>> /usr/lib/jvm/java-7-openjdk-amd64/bin/java
>>>>>> -Djava.net.preferIPv4Stack=true
>>>>>> -Dhadoop.metrics.log.level=WARN -Xmx8192m
>>>>>> -Djava.io.tmpdir=/tmp/hadoop-ubuntu/nm-local-dir/usercache/ubuntu/appcache/application_1424264025191_0002/container_1424264025191_0002_01_000011/tmp
>>>>>> -Dlog4j.configuration=container-log4j.properties
>>>>>> -Dyarn.app.container.log.dir=/home/ubuntu/hadoop/logs/userlogs/application_1424264025191_0002/container_1424264025191_0002_01_000011
>>>>>> -Dyarn.app.container.log.filesize=0
>>>>>> -Dhadoop.root.logger=INFO,CLA org.apache.hadoop.mapred.YarnChild
>>>>>> 192.168.0.12 50842 attempt_1424264025191_0002_m_000005_0 11 >
>>>>>>
>>>>>> /home/ubuntu/hadoop/logs/userlogs/application_1424264025191_0002/container_1424264025191_0002_01_000011/stdout
>>>>>> 2>
>>>>>>
>>>>>> /home/ubuntu/hadoop/logs/userlogs/application_1424264025191_0002/container_1424264025191_0002_01_000011/stderr
>>>>>>
>>>>>>
>>>>>> How can avoid this?any help is appreciated
>>>>>>
>>>>>> It looks to me that YAN is trying to launch all the container
>>>>>> simultaneously and anot according to the available resources. Is
>>>>>> there an option to restrict number of containers on hadoop ndoes?
>>>>>>
>>>>>> Regards,
>>>>>> Tariq
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Warmest Regards,
>>>>>
>>>>> Ravi Shankar
>>>>>
>>>>
>>>>
>>>
>>
>>
>> --
>> Warmest Regards,
>>
>> Ravi Shankar
>>
>
>


-- 
Warmest Regards,

Ravi Shankar

Mime
View raw message