aurora-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Zameer Manji (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (AURORA-1825) Enable async logging by default
Date Thu, 24 Nov 2016 00:22:58 GMT

    [ https://issues.apache.org/jira/browse/AURORA-1825?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15691758#comment-15691758
] 

Zameer Manji commented on AURORA-1825:
--------------------------------------

Locally I removed the expensive parts of our logback config with:
{noformat}
diff --git c/src/main/resources/logback.xml w/src/main/resources/logback.xml
index 84c175c..6206806 100644
--- c/src/main/resources/logback.xml
+++ w/src/main/resources/logback.xml
@@ -23,7 +23,7 @@ limitations under the License.
     <target>System.err</target>
     <encoder>
       <pattern>
-        %.-1level%date{MMdd HH:mm:ss.SSS} [%thread, %class{0}:%line] %message %xThrowable%n
+        %.-1level%date{MMdd HH:mm:ss.SSS} [%thread] %message %xThrowable%n
       </pattern>
     </encoder>
   </appender>

{noformat}

Before:
{noformat}
Benchmark                                               (numPendingTasks)  (numTasksToDelete)
  Mode  Cnt  Score   Error  Units
StateManagerBenchmarks.DeleteTasksBenchmark.run                       N/A                1000
 thrpt   10  2.510 ± 0.557  ops/s
StateManagerBenchmarks.DeleteTasksBenchmark.run                       N/A               10000
 thrpt   10  0.272 ± 0.030  ops/s
StateManagerBenchmarks.DeleteTasksBenchmark.run                       N/A               50000
 thrpt   10  0.053 ± 0.011  ops/s
StateManagerBenchmarks.InsertPendingTasksBenchmark.run               1000                
N/A  thrpt   10  2.446 ± 0.698  ops/s
StateManagerBenchmarks.InsertPendingTasksBenchmark.run              10000                
N/A  thrpt   10  0.246 ± 0.018  ops/s
StateManagerBenchmarks.InsertPendingTasksBenchmark.run              50000                
N/A  thrpt   10  0.041 ± 0.006  ops/s
{noformat}

After:

{noformat}
Benchmark                                               (numPendingTasks)  (numTasksToDelete)
  Mode  Cnt  Score   Error  Units
StateManagerBenchmarks.DeleteTasksBenchmark.run                       N/A                1000
 thrpt   10  8.640 ± 1.431  ops/s
StateManagerBenchmarks.DeleteTasksBenchmark.run                       N/A               10000
 thrpt   10  0.892 ± 0.066  ops/s
StateManagerBenchmarks.DeleteTasksBenchmark.run                       N/A               50000
 thrpt   10  0.172 ± 0.010  ops/s
StateManagerBenchmarks.InsertPendingTasksBenchmark.run               1000                
N/A  thrpt   10  4.837 ± 1.511  ops/s
StateManagerBenchmarks.InsertPendingTasksBenchmark.run              10000                
N/A  thrpt   10  0.510 ± 0.315  ops/s
StateManagerBenchmarks.InsertPendingTasksBenchmark.run              50000                
N/A  thrpt   10  0.079 ± 0.052  ops/s
{noformat}

I picked this benchmark because it logs a lot in the critical path.

We could probably fix this problem by removing line number and removing class name with the
logger name. The net result would be no line numbers but way faster logging.

> Enable async logging by default
> -------------------------------
>
>                 Key: AURORA-1825
>                 URL: https://issues.apache.org/jira/browse/AURORA-1825
>             Project: Aurora
>          Issue Type: Task
>            Reporter: Zameer Manji
>            Assignee: Jing Chen
>            Priority: Minor
>
> Based on my experience while working on AURORA-1823 and [~StephanErb]'s work on logging
recently, I think it would be best if we enabled async logging.
> For example if one attempts to parallelize the work inside {{StateManagerImpl}} there
isn't much benefit because all of the state transitions are logged and all of the threads
would contend for the lock.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message