Return-Path: <dev-return-1627-archive-asf-public=cust-asf.ponee.io@tephra.incubator.apache.org>
X-Original-To: archive-asf-public-internal@cust-asf2.ponee.io
Delivered-To: archive-asf-public-internal@cust-asf2.ponee.io
Received: from cust-asf.ponee.io (cust-asf.ponee.io [163.172.22.183])
	by cust-asf2.ponee.io (Postfix) with ESMTP id F3BB8200CF3
	for <archive-asf-public-internal@cust-asf2.ponee.io>; Wed, 13 Sep 2017 19:38:16 +0200 (CEST)
Received: by cust-asf.ponee.io (Postfix)
	id F28881609CA; Wed, 13 Sep 2017 17:38:16 +0000 (UTC)
Delivered-To: archive-asf-public@cust-asf.ponee.io
Received: from mail.apache.org (hermes.apache.org [140.211.11.3])
	by cust-asf.ponee.io (Postfix) with SMTP id 447841609C3
	for <archive-asf-public@cust-asf.ponee.io>; Wed, 13 Sep 2017 19:38:16 +0200 (CEST)
Received: (qmail 33948 invoked by uid 500); 13 Sep 2017 17:38:15 -0000
Mailing-List: contact dev-help@tephra.incubator.apache.org; run by ezmlm
Precedence: bulk
List-Help: <mailto:dev-help@tephra.incubator.apache.org>
List-Unsubscribe: <mailto:dev-unsubscribe@tephra.incubator.apache.org>
List-Post: <mailto:dev@tephra.incubator.apache.org>
List-Id: <dev.tephra.incubator.apache.org>
Reply-To: dev@tephra.incubator.apache.org
Delivered-To: mailing list dev@tephra.incubator.apache.org
Received: (qmail 33932 invoked by uid 99); 13 Sep 2017 17:38:15 -0000
Received: from pnap-us-west-generic-nat.apache.org (HELO spamd2-us-west.apache.org) (209.188.14.142)
    by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 13 Sep 2017 17:38:15 +0000
Received: from localhost (localhost [127.0.0.1])
	by spamd2-us-west.apache.org (ASF Mail Server at spamd2-us-west.apache.org) with ESMTP id A1B951A5398
	for <dev@tephra.apache.org>; Wed, 13 Sep 2017 17:38:14 +0000 (UTC)
X-Virus-Scanned: Debian amavisd-new at spamd2-us-west.apache.org
X-Spam-Flag: NO
X-Spam-Score: -4.02
X-Spam-Level: 
X-Spam-Status: No, score=-4.02 tagged_above=-999 required=6.31
	tests=[KAM_LAZY_DOMAIN_SECURITY=1, RCVD_IN_DNSWL_HI=-5,
	RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01,
	RP_MATCHES_RCVD=-0.001, URIBL_BLOCKED=0.001] autolearn=disabled
Received: from mx1-lw-eu.apache.org ([10.40.0.8])
	by localhost (spamd2-us-west.apache.org [10.40.0.9]) (amavisd-new, port 10024)
	with ESMTP id yuFYUyCPTVhE for <dev@tephra.apache.org>;
	Wed, 13 Sep 2017 17:38:12 +0000 (UTC)
Received: from mail.apache.org (hermes.apache.org [140.211.11.3])
	by mx1-lw-eu.apache.org (ASF Mail Server at mx1-lw-eu.apache.org) with SMTP id 83D6E60D28
	for <dev@tephra.incubator.apache.org>; Wed, 13 Sep 2017 17:38:11 +0000 (UTC)
Received: (qmail 32631 invoked by uid 99); 13 Sep 2017 17:38:10 -0000
Received: from git1-us-west.apache.org (HELO git1-us-west.apache.org) (140.211.11.23)
    by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 13 Sep 2017 17:38:10 +0000
Received: by git1-us-west.apache.org (ASF Mail Server at git1-us-west.apache.org, from userid 33)
	id BA905F5713; Wed, 13 Sep 2017 17:38:10 +0000 (UTC)
From: anew <git@git.apache.org>
To: dev@tephra.incubator.apache.org
Reply-To: dev@tephra.incubator.apache.org
References: <git-pr-53-incubator-tephra@git.apache.org>
In-Reply-To: <git-pr-53-incubator-tephra@git.apache.org>
Subject: [GitHub] incubator-tephra pull request #53: (TEPHRA-243) Improve logging for slow log...
Content-Type: text/plain
Message-Id: <20170913173810.BA905F5713@git1-us-west.apache.org>
Date: Wed, 13 Sep 2017 17:38:10 +0000 (UTC)
archived-at: Wed, 13 Sep 2017 17:38:17 -0000

Github user anew commented on a diff in the pull request:

    https://github.com/apache/incubator-tephra/pull/53#discussion_r138688512
  
    --- Diff: tephra-core/src/main/java/org/apache/tephra/persist/AbstractTransactionLog.java ---
    @@ -165,26 +203,36 @@ private void sync() throws IOException {
           // prevent writer being dereferenced
           tmpWriter = writer;
     
    -      List<Entry> currentPending = getPendingWrites();
    -      if (!currentPending.isEmpty()) {
    -        tmpWriter.commitMarker(currentPending.size());
    -      }
    -
    -      // write out all accumulated entries to log.
    -      for (Entry e : currentPending) {
    -        tmpWriter.append(e);
    -        entryCount++;
    -        latestSeq = Math.max(latestSeq, e.getKey().get());
    +      Entry[] currentPending = getPendingWrites();
    +      if (currentPending != null) {
    +        entryCount = currentPending.length;
    +        startTimerIfNeeded(tmpWriter, entryCount);
    +        tmpWriter.commitMarker(entryCount);
    +        for (Entry e : currentPending) {
    +          tmpWriter.append(e);
    +          latestSeq = Math.max(latestSeq, e.getKey().get());
    +        }
    +        writtenUpTo.set(latestSeq);
    --- End diff --
    
    I was thinking about that. However, it would add latency. In the following case: Suppose:
    - thread 1 writes up to seq id 5
    - thread 2 writes up to seq id 10
    - thread 2 syncs and sets syncedUpto to 10
    - thread 3 writes up to seq id 15
    - now thread 1 checks whether it needs to sync. Because its latestSeq is 5, it can return. But if it compares with writtenUpto, then it would try to sync. 
    
    That would add latency to thread 1 while thread 3 must wait anyway. That's why I introduced the writtenUpto
    
    By the way, the existing code had syncedUpto wrong. It was always set to latestSeq of the thread that syncs. But it might have synced more than its own entries. For example, in the scenario above, if thread 1 syncs instead of thread 2, then it has synced everything up to 10, but it would set syncedUpto to 5. With the consequence that thread would sync again, needlessly.


---