drill-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (DRILL-143) Support CGROUPs resource management
Date Wed, 04 Apr 2018 05:22:01 GMT

    [ https://issues.apache.org/jira/browse/DRILL-143?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16425030#comment-16425030
] 

ASF GitHub Bot commented on DRILL-143:
--------------------------------------

Github user paul-rogers commented on a diff in the pull request:

    https://github.com/apache/drill/pull/1200#discussion_r179027761
  
    --- Diff: distribution/src/resources/drillbit.sh ---
    @@ -127,6 +127,44 @@ check_before_start()
       fi
     }
     
    +check_after_start(){
    +    #check if the process is running
    +    if [ -f $pid ]; then
    +      dbitProc=$(ps -ef | grep `cat $pid` | grep Drillbit)
    +      if [ -n "$dbitProc" ]; then
    +        # Check and enforce for CGroup
    +        if [ -n "$DRILLBIT_CGROUP" ]; then 
    +          check_and_enforce_cgroup `cat $pid`
    +        fi
    +      fi
    +    fi
    +}
    +
    +check_and_enforce_cgroup(){
    +    dbitPid=$1;
    +    #if [ $(`ps -o cgroup` | grep -c $DRILLBIT_CGROUP ) -eq 1 ]; then 
    +    if [ -f /cgroup/cpu/${DRILLBIT_CGROUP}/cgroup.procs ]; then 
    +      echo $dbitPid > /cgroup/cpu/${DRILLBIT_CGROUP}/cgroup.procs
    +      # Verify Enforcement
    +      cgroupStatus=`grep -w $pid /cgroup/cpu/${DRILLBIT_CGROUP}/cgroup.procs`
    +      if [ -z "$cgroupStatus" ]; then
    +        #Ref: https://www.kernel.org/doc/Documentation/scheduler/sched-bwc.txt
    +        let cpu_quota=`cat /cgroup/cpu/${DRILLBIT_CGROUP}/cpu.cfs_quota_us`
    +        let cpu_period=`cat /cgroup/cpu/${DRILLBIT_CGROUP}/cpu.cfs_period_us`
    +        if [ $cpu_period -gt 0 ] && [ $cpu_quota -gt 0 ]; then
    +          coresAllowed="(up to "`echo $(( 100 * $cpu_quota / $cpu_period )) | sed 's/..$/.&/'`"
cores allowed)"
    +        fi
    +        echo "WARN: Drillbit's CPU resource usage will be managed under the CGroup :
$DRILLBIT_CGROUP "$coresAllowed
    --- End diff --
    
    Why is this a WARN? Isn't this exactly what I requested?
    
    Maybe emit this as a message (no WARN) if a -v (verbose) flag is set.
    
    Also, put the variable inside the quotes; bash is handy that way...
    
    Message will be "...CGroup : drillcpu 5". Maybe, "CGroup drillcpu will limit Drill to
5 cpu(s)".


> Support CGROUPs resource management
> -----------------------------------
>
>                 Key: DRILL-143
>                 URL: https://issues.apache.org/jira/browse/DRILL-143
>             Project: Apache Drill
>          Issue Type: New Feature
>            Reporter: Jacques Nadeau
>            Assignee: Kunal Khatua
>            Priority: Major
>             Fix For: 1.14.0
>
>         Attachments: 253ce178-ddeb-e482-cd64-44ab7284ad1c.sys.drill
>
>
> For the purpose of playing nice on clusters that don't have YARN, we should write up
configuration and scripts to allows users to run Drill next to existing workloads without
sharing resources.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message