Mailing-List: contact hadoop-user-help@lucene.apache.org; run by ezmlm
Precedence: bulk
Reply-To: hadoop-user@lucene.apache.org
Received-SPF: pass (herse.apache.org: domain of jagadeesh.vn@gmail.com
 designates 209.85.132.245 as permitted sender)
DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws;
        s=beta; d=gmail.com;
        h=received:to:subject:date:mime-version:content-type:content-transfer-encoding:x-mailer:x-mimeole:thread-index:in-reply-to:from:message-id;
        b=LDYEKhd/DON1N+w5O+hZkRnLUpreyoB59r0ywkMmyuWNxu4eF6q0a0dvWU84Bo0Q6tucGjyKhIuUD2lq+q45FdO3suhSljAsE7HDk620Pms0MJa5LykQxM6aVZbwB+1enIEWcYKAjbtJ3KwtXlB02ft74i93Wg2OACuP83bhT7g=
To: <hadoop-user@lucene.apache.org>
Subject: Urgent: Production Issues
Date: Mon, 18 Dec 2006 11:30:11 +0530
MIME-Version: 1.0
Content-Type: text/plain;
	charset="us-ascii"
Content-Transfer-Encoding: 7bit
Thread-Index: Accgf7TSDM1G8uZDQDij2yGafr9tsgB6APIA
In-Reply-To: <4582F704.2080007@yahoo-inc.com>
From: Jagadeesh <jagadeesh.vn@gmail.com>
Message-ID: <45862e17.2890e523.03cb.42e2@mx.google.com>

Hi All,

I am running Hadoop 0.7.2 in a production environment and it has stored
~170GB of data. Please read below the deployment architecture I am using.

I am using 4 nodes with 1.3TB storage each and the master node is not being
used for storage. So I have 5 servers in total out of which 4 servers are
running Hadoop nodes. This setup was working fine for the last 20-25 days
and there were no issues. As mentioned earlier, now the total storage has
gone upto ~170GB. 

Couple of days back, I noticed an error where Hadoop was not accepting new
files, I mean the upload always failed, but download was still working
great. I was getting the exception, writing <filename>.crc failed. When I
tried restarting the service, I was getting the message, jobtracker not
available and tasktracker not available. Then I had to kill all the
processes in the master node as well as in the client nodes to restart the
service.

After that everything worked fine for a day more and now I keep on getting
the message 

failure closing block of file /user/root/.LICENSE.txt2233331.crc to node
node1:50010

Even if I restart the service, I get this message after 10 minutes.

I read in the mailing list that this issues is resolved in 0.9.0, but I am a
bit skeptical about moving to 0.9.0 as I don't know whether I will end up
loosing the files that are already stored. Kindly confirm this and I wil
move to 0.9.0 and also please tell me the steps or pre-cautions I should
take before moving to 0.9.0.

Thanks and Regards
Jugs