mesos-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "James Peach (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (MESOS-7521) Major performance regression in drf sorter
Date Thu, 18 May 2017 17:23:04 GMT

    [ https://issues.apache.org/jira/browse/MESOS-7521?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16016112#comment-16016112
] 

James Peach commented on MESOS-7521:
------------------------------------

Avoiding the double hash lookup in {{calculateShare}} makes a measurable difference. The benchmarks
below also include changes equivalent to [~neilc]'s patch.

{code}
@@ -535,6 +555,7 @@ double DRFSorter::calculateShare(const Node* node) const
       continue;
     }

+#if 0
     if (scalar.value() > 0.0 &&
         node->allocation.totals.contains(resourceName)) {
       const double allocation =
@@ -542,6 +563,18 @@ double DRFSorter::calculateShare(const Node* node) const

       share = std::max(share, allocation / scalar.value());
     }
+
+#else
+
+    if (scalar.value() > 0.0) {
+      Option<Value::Scalar> allocation =
+        node->allocation.totals.get(resourceName);
+      if (allocation.isSome()) {
+        share = std::max(share, allocation->value() / scalar.value());
+      }
+    }
+#endif
+
   }
{code}

**Before:**
{noformat}
[ RUN      ] SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.SuppressOffers/7
Using 1000 agents and 6000 frameworks
Added 6000 frameworks in 1.57426096secs
Added 1000 agents in 16.706671624secs
allocate() took 12.456627188secs to make 1000 offers with 1200 out of 6000 frameworks suppressing
offers
allocate() took 12.328605156secs to make 1000 offers with 2400 out of 6000 frameworks suppressing
offers
allocate() took 12.029362287secs to make 1000 offers with 3600 out of 6000 frameworks suppressing
offers
allocate() took 8.499285498secs to make 1000 offers with 4800 out of 6000 frameworks suppressing
offers
allocate() took 406.640714ms to make 0 offers with 6000 out of 6000 frameworks suppressing
offers
[       OK ] SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.SuppressOffers/7
(69693 ms)
[----------] 1 test from SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test (69693
ms total)
{noformat}

**After:**
{noformat}
[ RUN      ] SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.SuppressOffers/7
Using 1000 agents and 6000 frameworks
Added 6000 frameworks in 1.083730869secs
Added 1000 agents in 12.969247272secs
allocate() took 12.047631503secs to make 1000 offers with 1200 out of 6000 frameworks suppressing
offers
allocate() took 10.914298377secs to make 1000 offers with 2400 out of 6000 frameworks suppressing
offers
allocate() took 9.041474536secs to make 1000 offers with 3600 out of 6000 frameworks suppressing
offers
allocate() took 8.520769241secs to make 1000 offers with 4800 out of 6000 frameworks suppressing
offers
allocate() took 399.175742ms to make 0 offers with 6000 out of 6000 frameworks suppressing
offers
[       OK ] SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.SuppressOffers/7
(60591 ms)
[----------] 1 test from SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test (60591
ms total)
{noformat}


> Major performance regression in drf sorter
> ------------------------------------------
>
>                 Key: MESOS-7521
>                 URL: https://issues.apache.org/jira/browse/MESOS-7521
>             Project: Mesos
>          Issue Type: Bug
>          Components: allocation
>    Affects Versions: 1.3.0
>            Reporter: Dario Rexin
>            Assignee: Neil Conway
>            Priority: Blocker
>              Labels: perfomance
>
> The addition of hierarchical roles to the framework sorter (https://github.com/apache/mesos/commit/e5ef1992b2b8e84b5d1487f1578f18f2291cd082)
has introduced a major performance regression to 1.2. Suppressing offers for frameworks does
not seem to reduce allocation time anymore, like it used to in 1.2. Here are some relevant
benchmark results:
> Mesos 1.2:
> {noformat}
> [ RUN      ] SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.SuppressOffers/7
> Using 1000 agents and 6000 frameworks
> Added 6000 frameworks in 105957us
> Added 1000 agents in 34.937438secs
> allocate() took 27.408828secs to make 1000 offers with 1200 out of 6000 frameworks suppressing
offers
> allocate() took 20.121897secs to make 1000 offers with 2400 out of 6000 frameworks suppressing
offers
> allocate() took 12.964302secs to make 1000 offers with 3600 out of 6000 frameworks suppressing
offers
> allocate() took 6.534221secs to make 1000 offers with 4800 out of 6000 frameworks suppressing
offers
> allocate() took 8953us to make 0 offers with 6000 out of 6000 frameworks suppressing
offers
> [       OK ] SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.SuppressOffers/7
(106198 ms)
> {noformat}
> Mesos 1.3:
> {noformat}
> [ RUN      ] SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.SuppressOffers/7
> Using 1000 agents and 6000 frameworks
> Added 6000 frameworks in 1.036217secs
> Added 1000 agents in 10.093938secs
> allocate() took 10.629448secs to make 1000 offers with 1200 out of 6000 frameworks suppressing
offers
> allocate() took 11.607185secs to make 1000 offers with 2400 out of 6000 frameworks suppressing
offers
> allocate() took 12.896578secs to make 1000 offers with 3600 out of 6000 frameworks suppressing
offers
> allocate() took 14.162431secs to make 1000 offers with 4800 out of 6000 frameworks suppressing
offers
> allocate() took 257060us to make 0 offers with 6000 out of 6000 frameworks suppressing
offers
> [       OK ] SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.SuppressOffers/7
(64011 ms)
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

Mime
View raw message