Return-Path: X-Original-To: apmail-accumulo-dev-archive@www.apache.org Delivered-To: apmail-accumulo-dev-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id E2A051041B for ; Tue, 13 Aug 2013 22:10:39 +0000 (UTC) Received: (qmail 37617 invoked by uid 500); 13 Aug 2013 22:10:39 -0000 Delivered-To: apmail-accumulo-dev-archive@accumulo.apache.org Received: (qmail 37547 invoked by uid 500); 13 Aug 2013 22:10:39 -0000 Mailing-List: contact dev-help@accumulo.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@accumulo.apache.org Delivered-To: mailing list dev@accumulo.apache.org Received: (qmail 37539 invoked by uid 99); 13 Aug 2013 22:10:39 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 13 Aug 2013 22:10:39 +0000 X-ASF-Spam-Status: No, hits=1.5 required=5.0 tests=HTML_MESSAGE,NORMAL_HTTP_TO_IP,RCVD_IN_DNSWL_LOW,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: domain of todd@cloudera.com designates 209.85.217.172 as permitted sender) Received: from [209.85.217.172] (HELO mail-lb0-f172.google.com) (209.85.217.172) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 13 Aug 2013 22:10:34 +0000 Received: by mail-lb0-f172.google.com with SMTP id o7so6315408lbv.3 for ; Tue, 13 Aug 2013 15:10:13 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:content-type; bh=eY4iFOWYOoKMblARggnMMVleLvfL1DjvSGPMj8HQyZE=; b=A4zHToHXtXnQUAKq93BMVul9bCuFdpRFdGJz3E9AkSuVCBrFeNvVrKvORVGYfD4Gjk EVcqbC9m7tmNuypcVxq8NUhY3iIaR4n6+AQ4RkAV/DMGZNmPs/nuC+C4A4/M7uNL2fCD C13Uix8isFWhF8BPb0R8TtoGuMjkf08/aPmt+DR+OjQ7776wihZWNoSqq+zeRDk8hoqN h3/JtZpfCTH2HhgHPxF+3k24c5qBQkub6PTMbuRctzcqWx3XA8lwbBxi0yzajHDWAQtV 36/JSQj6a9x4FzXmoxevkHIhy7o3VmhpDWoop3NQPeuJy8C/obHP7/cB3+5TumuRBRIW N65Q== X-Gm-Message-State: ALoCoQl3LNuc0r5xjVVt+wv+xxhros2r1wbrUunTS/1cZh2B/7SII1xoADbguXTa2chUheqf74Hj X-Received: by 10.152.22.65 with SMTP id b1mr23771laf.46.1376431812833; Tue, 13 Aug 2013 15:10:12 -0700 (PDT) MIME-Version: 1.0 Received: by 10.112.78.138 with HTTP; Tue, 13 Aug 2013 15:09:52 -0700 (PDT) In-Reply-To: References: From: Todd Lipcon Date: Tue, 13 Aug 2013 15:09:52 -0700 Message-ID: Subject: Re: WAL issues in 1.5.0 To: dev@accumulo.apache.org, vines@apache.org Content-Type: multipart/alternative; boundary=089e0158b92a42fd1a04e3db81f9 X-Virus-Checked: Checked by ClamAV on apache.org --089e0158b92a42fd1a04e3db81f9 Content-Type: text/plain; charset=ISO-8859-1 Out of disk space? HDFS won't write to a volume if you don't have 5x the block size available. -Todd On Tue, Aug 13, 2013 at 3:06 PM, John Vines wrote: > I had a few instances of it before, but I was never able to concretely > create it in a non-virtual environment. Except today, I had a next clean > checkout of first 1.5.1-SNAPSHOT and then 1.5.0 from git, with a fresh hdfs > directory and I got the never ending stream of > > " java.lang.RuntimeException: org.apache.hadoop.ipc.RemoteException: > java.io.IOException: File > /accumulo/wal/127.0.0.1+9997/86c798b2-2de3-4860-ba84-645cc9d38cc7 > could only be replicated to 0 nodes, instead of 1" > > > Normally when this happens, restarting the namenode is all I need to do to > fix it, but not this time. I'm willing to bet when I restart my computer it > will be fine. But, while this is happening, I'm seeing the number of files > in hdfs under the wal directory ever growing. I'm wondering if we have an > overly time sensitive contstraint or if there is a check we need to do > before giving up? I am seeing that error echoed in the namenode, so I'm not > quite sure. This is on hadoop 1.0.4. > -- Todd Lipcon Software Engineer, Cloudera --089e0158b92a42fd1a04e3db81f9--