hadoop-hdfs-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ranith Sardar (JIRA)" <j...@apache.org>
Subject [jira] [Created] (HDFS-13002) Ulimit log is overridden by the balancer sysout log in balancer out file
Date Tue, 09 Jan 2018 10:49:00 GMT
Ranith Sardar created HDFS-13002:

             Summary: Ulimit log is overridden by the balancer sysout log in balancer out
                 Key: HDFS-13002
                 URL: https://issues.apache.org/jira/browse/HDFS-13002
             Project: Hadoop HDFS
          Issue Type: Bug
          Components: balancer & mover
    Affects Versions: 2.7.2
            Reporter: Ranith Sardar
            Priority: Minor

Whenever the balancer starts, it will redirect the sys out to the out log file.  And balancer
writes the system output to the log file, at the same time  script also will try to append
ulimit output. 
 # capture the ulimit output
    if [ "true" = "$starting_secure_dn" ]; then
      echo "ulimit -a for secure datanode user $HADOOP_SECURE_DN_USER" >> $log
      # capture the ulimit info for the appropriate user
      su --shell=/bin/bash $HADOOP_SECURE_DN_USER -c 'ulimit -a' >> $log 2>&1
    elif [ "true" = "$starting_privileged_nfs" ]; then
        echo "ulimit -a for privileged nfs user $HADOOP_PRIVILEGED_NFS_USER" >> $log
        su --shell=/bin/bash $HADOOP_PRIVILEGED_NFS_USER -c 'ulimit -a' >> $log 2>&1
      echo "ulimit -a for user $USER" >> $log
      ulimit -a >> $log 2>&1
    sleep 3;
    if ! ps -p $! > /dev/null ; then
      exit 1

But the problem is first few lines of ulimit is overridding by the log of balancer.

vm1:/opt/install/hadoop/namenode/sbin # cat /opt/HA/AIH283/install/hadoop/namenode/logs/hadoop-root-balancer-vm1.out
Time Stamp               Iteration#  Bytes Already Moved  Bytes Left To Move  Bytes Being
The cluster is balanced. Exiting...
Jan 9, 2018 6:26:26 PM            0                  0 B                 0 B             
  0 B
Jan 9, 2018 6:26:26 PM   Balancing took 3.446 seconds
x memory size         (kbytes, -m) 13428300
open files                      (-n) 1024
pipe size            (512 bytes, -p) 8
POSIX message queues     (bytes, -q) 819200
real-time priority              (-r) 0
stack size              (kbytes, -s) 8192
cpu time               (seconds, -t) unlimited
max user processes              (-u) 127350
virtual memory          (kbytes, -v) 15992160
file locks                      (-x) unlimited

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org

View raw message