Return-Path: X-Original-To: apmail-ambari-dev-archive@www.apache.org Delivered-To: apmail-ambari-dev-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 0A15D18985 for ; Tue, 28 Jul 2015 19:54:11 +0000 (UTC) Received: (qmail 91996 invoked by uid 500); 28 Jul 2015 19:54:07 -0000 Delivered-To: apmail-ambari-dev-archive@ambari.apache.org Received: (qmail 91968 invoked by uid 500); 28 Jul 2015 19:54:07 -0000 Mailing-List: contact dev-help@ambari.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@ambari.apache.org Delivered-To: mailing list dev@ambari.apache.org Received: (qmail 91947 invoked by uid 99); 28 Jul 2015 19:54:07 -0000 Received: from reviews-vm.apache.org (HELO reviews.apache.org) (140.211.11.40) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 28 Jul 2015 19:54:07 +0000 Received: from reviews.apache.org (localhost [127.0.0.1]) by reviews.apache.org (Postfix) with ESMTP id 5735FD67F2; Tue, 28 Jul 2015 19:54:06 +0000 (UTC) Content-Type: multipart/alternative; boundary="===============6662726298501480474==" MIME-Version: 1.0 Subject: Review Request 36895: Cluster creates stuck at 9x% (deadlock sql exception) From: "Jonathan Hurley" To: "Alejandro Fernandez" , "Nate Cole" , "Sumit Mohanty" Cc: "Jonathan Hurley" , "Ambari" Date: Tue, 28 Jul 2015 19:54:06 -0000 Message-ID: <20150728195406.1540.63755@reviews.apache.org> X-ReviewBoard-URL: https://reviews.apache.org/ Auto-Submitted: auto-generated Sender: "Jonathan Hurley" X-ReviewGroup: Ambari X-Auto-Response-Suppress: DR, RN, OOF, AutoReply X-ReviewRequest-URL: https://reviews.apache.org/r/36895/ X-Sender: "Jonathan Hurley" Reply-To: "Jonathan Hurley" X-ReviewRequest-Repository: ambari --===============6662726298501480474== MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: 7bit ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/36895/ ----------------------------------------------------------- Review request for Ambari, Alejandro Fernandez, Nate Cole, and Sumit Mohanty. Bugs: AMBARI-12570 https://issues.apache.org/jira/browse/AMBARI-12570 Repository: ambari Description ------- Similar to AMBARI-12526, Ambari installation via a blueprint on SQL Azure gets stuck somewhere between 90% and 100% because of a SQL Database deadlock. - We have dual X-locks on hostcomponentstate asking for U-locks when updating the CLUSTERED INDEX. - Both dual X-locks, from different transactions and different processes, are on the same row (technically impossible) - based on the XML execution plan, we can see that the concurrent UPDATE statements are executing on different rows due to their CLUSTERED INDEX predicate. - In Java, Ambari has locks which prevent concurrent U- or X-locks on the same row - Only happens on SQL Server My best suspicion right now is that we have a key hash collision happening on this table. That's why two processes appear to have the same lock even though they are on different rows. Restricting row-level locking on this table will prevent locking on hash keys which could collide. Diffs ----- ambari-server/src/main/resources/Ambari-DDL-SQLServer-CREATE.sql 0ff1aff Diff: https://reviews.apache.org/r/36895/diff/ Testing ------- Deployed a clean cluster on SQL Server and then ran 10+ deployments on SQL Azure without seeing a deadlock. Thanks, Jonathan Hurley --===============6662726298501480474==--