Return-Path: X-Original-To: archive-asf-public-internal@cust-asf2.ponee.io Delivered-To: archive-asf-public-internal@cust-asf2.ponee.io Received: from cust-asf.ponee.io (cust-asf.ponee.io [163.172.22.183]) by cust-asf2.ponee.io (Postfix) with ESMTP id 21A3B200C6E for ; Mon, 8 May 2017 20:24:53 +0200 (CEST) Received: by cust-asf.ponee.io (Postfix) id 2034B160BA5; Mon, 8 May 2017 18:24:53 +0000 (UTC) Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by cust-asf.ponee.io (Postfix) with SMTP id 67918160BA2 for ; Mon, 8 May 2017 20:24:52 +0200 (CEST) Received: (qmail 14505 invoked by uid 500); 8 May 2017 18:24:51 -0000 Mailing-List: contact dev-help@activemq.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@activemq.apache.org Delivered-To: mailing list dev@activemq.apache.org Received: (qmail 14494 invoked by uid 99); 8 May 2017 18:24:51 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd2-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 08 May 2017 18:24:51 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd2-us-west.apache.org (ASF Mail Server at spamd2-us-west.apache.org) with ESMTP id DFD671AF919 for ; Mon, 8 May 2017 18:24:50 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd2-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: 3.487 X-Spam-Level: *** X-Spam-Status: No, score=3.487 tagged_above=-999 required=6.31 tests=[DKIM_ADSP_CUSTOM_MED=0.001, NML_ADSP_CUSTOM_MED=1.2, RCVD_IN_DNSWL_NONE=-0.0001, SPF_SOFTFAIL=0.972, URI_HEX=1.313, URI_TRY_3LD=0.001] autolearn=disabled Received: from mx1-lw-eu.apache.org ([10.40.0.8]) by localhost (spamd2-us-west.apache.org [10.40.0.9]) (amavisd-new, port 10024) with ESMTP id 7phwC1dhICjq for ; Mon, 8 May 2017 18:24:47 +0000 (UTC) Received: from mwork.nabble.com (mwork.nabble.com [162.253.133.43]) by mx1-lw-eu.apache.org (ASF Mail Server at mx1-lw-eu.apache.org) with ESMTP id A1B105F1B3 for ; Mon, 8 May 2017 18:24:46 +0000 (UTC) Received: from mjoe.nabble.com (unknown [162.253.133.57]) by mwork.nabble.com (Postfix) with ESMTP id B58E6400AFBA1 for ; Mon, 8 May 2017 11:24:45 -0700 (MST) Date: Mon, 8 May 2017 11:12:25 -0700 (PDT) From: nigro_franz To: dev@activemq.apache.org Message-ID: <1494267145875-4725780.post@n4.nabble.com> In-Reply-To: <1494230008352-4725735.post@n4.nabble.com> References: <1494179564937-4725727.post@n4.nabble.com> <1494230008352-4725735.post@n4.nabble.com> Subject: Re: Adapting TimedBuffer and NIO Buffer Pooling MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit archived-at: Mon, 08 May 2017 18:24:53 -0000 In the meantime, I'm searching good articles to add other disk I/O limiter strategies, maybe based on IOPS credits consumption. A possible idea about an IOPS limiter: - a single writer/appender/sync thread to perform the real writes & flush - a bounded queue of bytes (I've already designed a couple pretty fast) to submit any requests - a proper batch strategy on the appender/sync thread based on the current IOPS and unflushed writes The bounded queue/ringbuffer will be useful to propagate backpressure "naturally" to the request producer, bounding the max latency and with bounded memory footprint. What do you think? Right now I've found only this article: https://engineering.linkedin.com/blog/2016/05/designing-ssd-friendly-applications-for-better-application-perfo But it doesn't take in account sync operations and is specific only for SSDs!! If any of you have something good on the argument, please share :) -- View this message in context: http://activemq.2283324.n4.nabble.com/Adapting-TimedBuffer-and-NIO-Buffer-Pooling-tp4725727p4725780.html Sent from the ActiveMQ - Dev mailing list archive at Nabble.com.