Return-Path: 
 <cassandra-commits-return-8780-apmail-incubator-cassandra-commits-archive=incubator.apache.org@incubator.apache.org>
Delivered-To: apmail-incubator-cassandra-commits-archive@minotaur.apache.org
Received: (qmail 13458 invoked from network); 21 Jan 2010 15:24:15 -0000
Received: from hermes.apache.org (HELO mail.apache.org) (140.211.11.3)
  by minotaur.apache.org with SMTP; 21 Jan 2010 15:24:15 -0000
Received: (qmail 60612 invoked by uid 500); 21 Jan 2010 15:24:15 -0000
Delivered-To: apmail-incubator-cassandra-commits-archive@incubator.apache.org
Received: (qmail 60500 invoked by uid 500); 21 Jan 2010 15:24:15 -0000
Mailing-List: contact cassandra-commits-help@incubator.apache.org;
 run by ezmlm
Precedence: bulk
List-Help: <mailto:cassandra-commits-help@incubator.apache.org>
List-Unsubscribe: <mailto:cassandra-commits-unsubscribe@incubator.apache.org>
List-Post: <mailto:cassandra-commits@incubator.apache.org>
List-Id: <cassandra-commits.incubator.apache.org>
Reply-To: cassandra-dev@incubator.apache.org
Delivered-To: mailing list cassandra-commits@incubator.apache.org
Received: (qmail 60413 invoked by uid 99); 21 Jan 2010 15:24:15 -0000
Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136)
    by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 21 Jan 2010 15:24:15 +0000
X-ASF-Spam-Status: No, hits=-2000.0 required=10.0
	tests=ALL_TRUSTED
X-Spam-Check-By: apache.org
Received: from [140.211.11.140] (HELO brutus.apache.org) (140.211.11.140)
    by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 21 Jan 2010 15:24:14 +0000
Received: from brutus.apache.org (localhost [127.0.0.1])
	by brutus.apache.org (Postfix) with ESMTP id 874BA234C1F0
	for <cassandra-commits@incubator.apache.org>;
 Thu, 21 Jan 2010 07:23:54 -0800 (PST)
Message-ID: <1704932275.396221264087434552.JavaMail.jira@brutus.apache.org>
Date: Thu, 21 Jan 2010 15:23:54 +0000 (UTC)
From: "Gary Dusbabek (JIRA)" <jira@apache.org>
To: cassandra-commits@incubator.apache.org
Subject: [jira] Assigned: (CASSANDRA-716) bootstrapping does not work
 properly using multiple DataFileDirectory
In-Reply-To: <852149580.329781263884214844.JavaMail.jira@brutus.apache.org>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394


     [ https://issues.apache.org/jira/browse/CASSANDRA-716?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Gary Dusbabek reassigned CASSANDRA-716:
---------------------------------------

    Assignee: Gary Dusbabek

> bootstrapping does not work properly using multiple DataFileDirectory
> ---------------------------------------------------------------------
>
>                 Key: CASSANDRA-716
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-716
>             Project: Cassandra
>          Issue Type: Bug
>          Components: Core
>    Affects Versions: 0.5
>         Environment: storage-conf.xml:
>   <CommitLogDirectory>/home/store1/commitlog</CommitLogDirectory>
>   <DataFileDirectories>
>       <DataFileDirectory>/home/store0/data</DataFileDirectory>
>       <DataFileDirectory>/home/store1/data</DataFileDirectory>
>   </DataFileDirectories>
>   <CalloutLocation>/home/store1/cassandra/callouts</CalloutLocation>
>   <StagingFileDirectory>/home/store1/cassandra/staging</StagingFileDirectory>
>            Reporter: david.pan
>            Assignee: Gary Dusbabek
>   Original Estimate: 24h
>  Remaining Estimate: 24h
>
> I was adding a new machine A which has 2 DataFileDirectories into the ring. The A will throw exception while bootstrapping.
> DEBUG [MESSAGING-SERVICE-POOL:4] 2010-01-19 11:43:32,837 ContentStreamState.java (line 88) Removing stream context /home/store0/data/pic/raw_data-tmp-1-Data.db:209833142
>  WARN [MESSAGING-SERVICE-POOL:4] 2010-01-19 11:43:32,837 TcpConnection.java (line 484) Problem reading from socket connected to : java.nio.channels.SocketChannel[connected local
> =/10.81.37.65:7000 remote=/10.81.42.26:10418]
>  WARN [MESSAGING-SERVICE-POOL:4] 2010-01-19 11:43:32,837 TcpConnection.java (line 485) Exception was generated at : 01/19/2010 11:43:32 on thread MESSAGING-SERVICE-POOL:4
> java.io.IOException: rename failed of /home/store0/data/pic/raw_data-1-Filter.db
> java.io.IOError: java.io.IOException: rename failed of /home/store0/data/pic/raw_data-1-Filter.db
>         at org.apache.cassandra.io.SSTableWriter.rename(SSTableWriter.java:154)
>         at org.apache.cassandra.io.SSTableWriter.renameAndOpen(SSTableWriter.java:162)
>         at org.apache.cassandra.io.Streaming$StreamCompletionHandler.onStreamCompletion(Streaming.java:284)
>         at org.apache.cassandra.net.io.ContentStreamState.handleStreamCompletion(ContentStreamState.java:108)
>         at org.apache.cassandra.net.io.ContentStreamState.read(ContentStreamState.java:90)
>         at org.apache.cassandra.net.io.TcpReader.read(TcpReader.java:96)
>         at org.apache.cassandra.net.TcpConnection$ReadWorkItem.run(TcpConnection.java:445)
>         at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:885)
>         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)
>         at java.lang.Thread.run(Thread.java:619)
> Caused by: java.io.IOException: rename failed of /home/store0/data/pic/raw_data-1-Filter.db
>         at org.apache.cassandra.utils.FBUtilities.renameWithConfirm(FBUtilities.java:306)
>         at org.apache.cassandra.io.SSTableWriter.rename(SSTableWriter.java:150)
>         ... 9 more
> I traced the exception and maybe found the reason.
> StreamInitiateVerbHandler::doVerb() will create 3 temporary files(index, filter, data) for each ssTable. The name for each file is generated by getNewFileNameFromOldContextAndNames(). This method will generate a file name and a path for each ssTable, but the path is generated with DatabaseDescriptor.getDataFileLocationForTable() which will return different path for ech call when we configure multi-DataFileDirectory. 
> eg: the ssTable raw_data-1 may have 3 temporary files : 
> /home/store0/data/pic/raw_data-tmp-1-Index.db
> /home/store1/data/pic/raw_data-tmp-1-Filter.db
> /home/store0/data/pic/raw_data-tmp-1-Data.db
> After receiving all data, StreamCompletionHandler::onStreamCompletion() will rename all temporary files and this method think all ssTable files will have the same path as data.db file. 
>             if (streamContext.getTargetFile().contains("-Data.db"))
>             {
>                ......
>                 try
>                 {
>                     SSTableReader sstable = SSTableWriter.renameAndOpen(streamContext.getTargetFile());
>                     ......
>                 }
>                 ......
>             }
> Then the renameAndOpen() will throw that exception.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.