Return-Path: X-Original-To: archive-asf-public-internal@cust-asf2.ponee.io Delivered-To: archive-asf-public-internal@cust-asf2.ponee.io Received: from cust-asf.ponee.io (cust-asf.ponee.io [163.172.22.183]) by cust-asf2.ponee.io (Postfix) with ESMTP id B4104200BFC for ; Tue, 6 Dec 2016 22:19:00 +0100 (CET) Received: by cust-asf.ponee.io (Postfix) id B2821160B17; Tue, 6 Dec 2016 21:19:00 +0000 (UTC) Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by cust-asf.ponee.io (Postfix) with SMTP id E0BFB160B1B for ; Tue, 6 Dec 2016 22:18:59 +0100 (CET) Received: (qmail 55774 invoked by uid 500); 6 Dec 2016 21:18:58 -0000 Mailing-List: contact commits-help@cassandra.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@cassandra.apache.org Delivered-To: mailing list commits@cassandra.apache.org Received: (qmail 55663 invoked by uid 99); 6 Dec 2016 21:18:58 -0000 Received: from arcas.apache.org (HELO arcas) (140.211.11.28) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 06 Dec 2016 21:18:58 +0000 Received: from arcas.apache.org (localhost [127.0.0.1]) by arcas (Postfix) with ESMTP id 919112C0086 for ; Tue, 6 Dec 2016 21:18:58 +0000 (UTC) Date: Tue, 6 Dec 2016 21:18:58 +0000 (UTC) From: "Jay Zhuang (JIRA)" To: commits@cassandra.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Updated] (CASSANDRA-13008) Add vm.max_map_count startup StartupCheck MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 archived-at: Tue, 06 Dec 2016 21:19:00 -0000 [ https://issues.apache.org/jira/browse/CASSANDRA-13008?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jay Zhuang updated CASSANDRA-13008: ----------------------------------- Fix Version/s: 3.0.x Reproduced In: 3.0.10 Status: Patch Available (was: Open) > Add vm.max_map_count startup StartupCheck > ----------------------------------------- > > Key: CASSANDRA-13008 > URL: https://issues.apache.org/jira/browse/CASSANDRA-13008 > Project: Cassandra > Issue Type: Improvement > Components: Configuration > Reporter: Jay Zhuang > Priority: Minor > Fix For: 3.0.x > > > It's recommended to set {{vm.max_map_count}} to 1048575 (CASSANDRA-3563) > When the max_map_count is low, it throws OOM exception, which is hard to link to the real issue of vm.max_map_count. > The problem happened when we tried to remove one node, all the other nodes in cluster crashed. As each node was trying to load more local SSTable files for streaming. > I would suggest to add a StartupCheck for max_map_count, at least it could give a warning message to help the debug. > {code} > ERROR [STREAM-IN-] JVMStabilityInspector.java:140 - JVM state determined to be unstable. Exiting forcefully due to: > java.lang.OutOfMemoryError: Map failed > at sun.nio.ch.FileChannelImpl.map0(Native Method) ~[na:1.8.0_112] > at sun.nio.ch.FileChannelImpl.map(FileChannelImpl.java:937) ~[na:1.8.0_112] > at org.apache.cassandra.io.util.ChannelProxy.map(ChannelProxy.java:152) ~[apache-cassandra-3.0.10.jar:3.0.10] > at org.apache.cassandra.io.util.MmappedRegions$State.add(MmappedRegions.java:280) ~[apache-cassandra-3.0.10.jar:3.0.10] > at org.apache.cassandra.io.util.MmappedRegions$State.access$400(MmappedRegions.java:216) ~[apache-cassandra-3.0.10.jar:3.0.10] > at org.apache.cassandra.io.util.MmappedRegions.updateState(MmappedRegions.java:173) ~[apache-cassandra-3.0.10.jar:3.0.10] > at org.apache.cassandra.io.util.MmappedRegions.(MmappedRegions.java:70) ~[apache-cassandra-3.0.10.jar:3.0.10] > at org.apache.cassandra.io.util.MmappedRegions.(MmappedRegions.java:58) ~[apache-cassandra-3.0.10.jar:3.0.10] > at org.apache.cassandra.io.util.MmappedRegions.map(MmappedRegions.java:96) ~[apache-cassandra-3.0.10.jar:3.0.10] > at org.apache.cassandra.io.util.CompressedSegmentedFile.(CompressedSegmentedFile.java:47) ~[apache-cassandra-3.0.10.jar:3.0.10] > at org.apache.cassandra.io.util.CompressedSegmentedFile$Builder.complete(CompressedSegmentedFile.java:132) ~[apache-cassandra-3.0.10.jar:3.0.10] > at org.apache.cassandra.io.util.SegmentedFile$Builder.complete(SegmentedFile.java:177) ~[apache-cassandra-3.0.10.jar:3.0.10] > at org.apache.cassandra.io.util.SegmentedFile$Builder.buildData(SegmentedFile.java:193) ~[apache-cassandra-3.0.10.jar:3.0.10] > at org.apache.cassandra.io.sstable.format.big.BigTableWriter.openFinal(BigTableWriter.java:276) ~[apache-cassandra-3.0.10.jar:3.0.10] > at org.apache.cassandra.io.sstable.format.big.BigTableWriter.access$600(BigTableWriter.java:50) ~[apache-cassandra-3.0.10.jar:3.0.10] > at org.apache.cassandra.io.sstable.format.big.BigTableWriter$TransactionalProxy.doPrepare(BigTableWriter.java:313) ~[apache-cassandra-3.0.10.jar:3.0.10] > at org.apache.cassandra.utils.concurrent.Transactional$AbstractTransactional.prepareToCommit(Transactional.java:173) ~[apache-cassandra-3.0.10.jar:3.0.10] > at org.apache.cassandra.utils.concurrent.Transactional$AbstractTransactional.finish(Transactional.java:184) ~[apache-cassandra-3.0.10.jar:3.0.10] > at org.apache.cassandra.io.sstable.format.SSTableWriter.finish(SSTableWriter.java:213) ~[apache-cassandra-3.0.10.jar:3.0.10] > at org.apache.cassandra.io.sstable.SimpleSSTableMultiWriter.finish(SimpleSSTableMultiWriter.java:56) ~[apache-cassandra-3.0.10.jar:3.0.10] > at org.apache.cassandra.streaming.StreamReceiveTask.received(StreamReceiveTask.java:109) ~[apache-cassandra-3.0.10.jar:3.0.10] > at org.apache.cassandra.streaming.StreamSession.receive(StreamSession.java:599) ~[apache-cassandra-3.0.10.jar:3.0.10] > at org.apache.cassandra.streaming.StreamSession.messageReceived(StreamSession.java:482) ~[apache-cassandra-3.0.10.jar:3.0.10] > at org.apache.cassandra.streaming.ConnectionHandler$IncomingMessageHandler.run(ConnectionHandler.java:296) ~[apache-cassandra-3.0.10.jar:3.0.10] > at java.lang.Thread.run(Thread.java:745) [na:1.8.0_112] > {code} -- This message was sent by Atlassian JIRA (v6.3.4#6332)