hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Brahma Reddy Battula (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-15180) branch-2 : daemon processes' sysout overwrites 'ulimit -a' in daemon's out file
Date Wed, 18 Apr 2018 17:20:00 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-15180?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16442890#comment-16442890
] 

Brahma Reddy Battula commented on HADOOP-15180:
-----------------------------------------------

+1 on latest patch.will commit soon

> branch-2 : daemon processes' sysout overwrites 'ulimit -a' in daemon's out file
> -------------------------------------------------------------------------------
>
>                 Key: HADOOP-15180
>                 URL: https://issues.apache.org/jira/browse/HADOOP-15180
>             Project: Hadoop Common
>          Issue Type: Bug
>          Components: scripts
>    Affects Versions: 2.7.2
>            Reporter: Ranith Sardar
>            Assignee: Ranith Sardar
>            Priority: Minor
>         Attachments: HADOOP-15180-branch-2-002.patch, HADOOP-15180_branch-2.diff
>
>
> Whenever the balancer starts, it will redirect the sys out to the out log file.  And
balancer writes the system output to the log file, at the same time  script also will try
to append ulimit output. 
> {noformat}
>  # capture the ulimit output
>     if [ "true" = "$starting_secure_dn" ]; then
>       echo "ulimit -a for secure datanode user $HADOOP_SECURE_DN_USER" >> $log
>       # capture the ulimit info for the appropriate user
>       su --shell=/bin/bash $HADOOP_SECURE_DN_USER -c 'ulimit -a' >> $log 2>&1
>     elif [ "true" = "$starting_privileged_nfs" ]; then
>         echo "ulimit -a for privileged nfs user $HADOOP_PRIVILEGED_NFS_USER" >>
$log
>         su --shell=/bin/bash $HADOOP_PRIVILEGED_NFS_USER -c 'ulimit -a' >> $log
2>&1
>     else
>       echo "ulimit -a for user $USER" >> $log
>       ulimit -a >> $log 2>&1
>     fi
>     sleep 3;
>     if ! ps -p $! > /dev/null ; then
>       exit 1
>     fi
> {noformat}
> But the problem is first few lines of ulimit is overridding by the log of balancer.
> {noformat}
> vm1:/opt/install/hadoop/namenode/sbin # cat /opt/HA/AIH283/install/hadoop/namenode/logs/hadoop-root-balancer-vm1.out
> Time Stamp               Iteration#  Bytes Already Moved  Bytes Left To Move  Bytes Being
Moved
> The cluster is balanced. Exiting...
> Jan 9, 2018 6:26:26 PM            0                  0 B                 0 B        
       0 B
> Jan 9, 2018 6:26:26 PM   Balancing took 3.446 seconds
> x memory size         (kbytes, -m) 13428300
> open files                      (-n) 1024
> pipe size            (512 bytes, -p) 8
> POSIX message queues     (bytes, -q) 819200
> real-time priority              (-r) 0
> stack size              (kbytes, -s) 8192
> cpu time               (seconds, -t) unlimited
> max user processes              (-u) 127350
> virtual memory          (kbytes, -v) 15992160
> file locks                      (-x) unlimited
> {noformat}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org


Mime
View raw message