Return-Path: X-Original-To: apmail-cassandra-user-archive@www.apache.org Delivered-To: apmail-cassandra-user-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id ACF9DD6A8 for ; Thu, 18 Oct 2012 20:34:58 +0000 (UTC) Received: (qmail 45588 invoked by uid 500); 18 Oct 2012 20:34:56 -0000 Delivered-To: apmail-cassandra-user-archive@cassandra.apache.org Received: (qmail 45559 invoked by uid 500); 18 Oct 2012 20:34:56 -0000 Mailing-List: contact user-help@cassandra.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@cassandra.apache.org Delivered-To: mailing list user@cassandra.apache.org Received: (qmail 45546 invoked by uid 99); 18 Oct 2012 20:34:56 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 18 Oct 2012 20:34:56 +0000 X-ASF-Spam-Status: No, hits=-2.3 required=5.0 tests=RCVD_IN_DNSWL_MED,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: domain of btv1==63880444583==mkjellman@barracuda.com designates 64.235.145.82 as permitted sender) Received: from [64.235.145.82] (HELO bsf02.barracuda.com) (64.235.145.82) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 18 Oct 2012 20:34:51 +0000 X-ASG-Debug-ID: 1350592470-03e1a5400531cd60001-f7dORa Received: from bn-scl-fe05.Cudanet.local (bn-scl-fe05.cudanet.local [10.8.1.46]) by bsf02.barracuda.com with ESMTP id Nd1EbjY8BU4PJo9d (version=TLSv1 cipher=AES128-SHA bits=128 verify=NO) for ; Thu, 18 Oct 2012 13:34:30 -0700 (PDT) X-Barracuda-Envelope-From: mkjellman@barracuda.com Received: from bn-scl-be03.Cudanet.local (10.8.1.54) by bn-scl-fe05.Cudanet.local (10.8.1.46) with Microsoft SMTP Server (TLS) id 8.3.279.1; Thu, 18 Oct 2012 13:34:30 -0700 Received: from bn-scl-be03.Cudanet.local ([::1]) by bn-scl-be03.Cudanet.local ([::1]) with mapi; Thu, 18 Oct 2012 13:34:29 -0700 From: Michael Kjellman X-Barracuda-Apparent-Source-IP: ::1 To: "user@cassandra.apache.org" Date: Thu, 18 Oct 2012 13:34:24 -0700 Subject: Re: hadoop consistency level Thread-Topic: hadoop consistency level X-ASG-Orig-Subj: Re: hadoop consistency level Thread-Index: Ac2tb/k0zJ1eCEhpRvCz5VyMnoS1+Q== Message-ID: In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: user-agent: Microsoft-MacOutlook/14.2.3.120616 acceptlanguage: en-US Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-Barracuda-Connect: bn-scl-fe05.cudanet.local[10.8.1.46] X-Barracuda-Start-Time: 1350592470 X-Barracuda-Encrypted: AES128-SHA X-Barracuda-URL: http://bsf02.barracuda.com:8000/cgi-mod/mark.cgi Received-SPF: softfail (barracuda.com: domain of transitioning mkjellman@barracuda.com does not designate ::1 as permitted sender) X-Barracuda-BRTS-Status: 1 X-Virus-Scanned: by bsmtpd at barracuda.com X-Barracuda-Spam-Score: 0.14 X-Barracuda-Spam-Status: No, SCORE=0.14 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=9.0 tests=BSF_SPF_SOFTFAIL, CN_BODY_332, THREAD_INDEX, THREAD_TOPIC X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.2.111720 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- 0.01 THREAD_INDEX thread-index: AcO7Y8iR61tzADqsRmmc5wNiFHEOig== 0.01 THREAD_TOPIC Thread-Topic: ...(Japanese Subject)... 0.12 CN_BODY_332 BODY: CN_BODY_332 0.00 BSF_SPF_SOFTFAIL Custom Rule SPF Softfail X-Virus-Checked: Checked by ClamAV on apache.org Not sure I understand your question (if there is one..) You are more than welcome to do CL ONE and assuming you have hadoop nodes in the right places on your ring things could work out very nicely. If you need to guarantee that you have all the data in your job then you'll need to use QUORUM. If you don't specify a CL in your job config it will default to ONE (at least that's what my read of the ConfigHelper source for 1.1.6 shows) On 10/18/12 1:29 PM, "Andrey Ilinykh" wrote: >On Thu, Oct 18, 2012 at 1:24 PM, Michael Kjellman > wrote: >> Well there is *some* data locality, it's just not guaranteed. My >> understanding (and someone correct me if I'm wrong) is that >> ColumnFamilyInputFormat implements InputSplit and the getLocations() >> method. >> >>=20 >>http://hadoop.apache.org/docs/mapreduce/current/api/org/apache/hadoop/map >>re >> duce/InputSplit.html >> >> ColumnFamilySplit.java contains logic to do it's best to determine what >> node that particular hadoop node contains the data for that mapper. >> >But no guarantee local data is in sync with other nodes. Which means >you have CL ONE. If you want CL QUORUM you have to make remote call, >no matter if data is local or not. 'Like' us on Facebook for exclusive content and other resources on all Barr= acuda Networks solutions.=0D Visit http://barracudanetworks.com/facebook=0D =0D