incubator-cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "dong.yajun" <dongt...@gmail.com>
Subject Re: Consistency QUORUM does not work anymore (hector:Could not fullfill request on this host)
Date Wed, 12 Dec 2012 07:12:23 GMT
hi aaron.

There is no problem with get_slices but get_range_slices failed. the way I
use this method is:

sliceRangeQuery.setFamily(family);
sliceRangeQuery.setKeys(rowkey,rowkey);
sliceRangeQuery.setRange(Character.Min_Value, Character.Max_Value, false,
Integer.Max_Value);
sliceRangeQuery.setRowCount(1);

current family has only abort 10000 rows, each row has ~10columns, the data
abort 25k.


On Wed, Dec 12, 2012 at 11:13 AM, dong.yajun <dongtalk@gmail.com> wrote:

> Thanks aaron.
>
> more information is I can read data  correctly using Aqulies with
> Local_Quorum.
>
> I just check the system.log which is normal on 172.16.74.31 and the number
> of RPC time out is 10s. the client exception occured on 2012-12-05.
>
> and the all log on server on 2012-12-05 were:
>
>  WARN [pool-1-thread-1] 2012-12-05 11:17:10,974 Memtable.java (line 169)
> setting live ratio to minimum of 1.0 instead of 0.09080883864721297
>  INFO [pool-1-thread-1] 2012-12-05 11:17:10,974 Memtable.java (line 179)
> CFS(Keyspace='APIPortal', ColumnFamily='Log') liveRatio is 1.0
> (just-counted was 1.0).  calculation took 4ms for 256 columns
>  INFO [pool-1-thread-1] 2012-12-05 17:48:25,988 Memtable.java (line 179)
> CFS(Keyspace='APIPortal', ColumnFamily='WebSite') liveRatio is
> 2.780009341429239 (just-counted was 2.780009341429239).  calculation took
> 1ms for 52 columns
>  INFO [pool-1-thread-1] 2012-12-05 17:48:27,944 Memtable.java (line 179)
> CFS(Keyspace='APIPortal', ColumnFamily='WebSite') liveRatio is
> 2.780009341429239 (just-counted was 2.3128462147190714).  calculation took
> 1ms for 75 columns
>  INFO [CompactionExecutor:153] 2012-12-05 18:11:27,718
> AutoSavingCache.java (line 269) Saved OpsCenter-rollups60-KeyCache (18
> items) in 2 ms
>  INFO [COMMIT-LOG-WRITER] 2012-12-05 20:31:10,025 CommitLogSegment.java
> (line 60) Creating new commitlog segment
> /data/cassandra/commitlog/CommitLog-1354768270025.log
>  INFO [ScheduledTasks:1] 2012-12-05 21:47:38,352 GCInspector.java (line
> 123) GC for ParNew: 437 ms for 1 collections, 1163185072 used; max is
> 8375238656
>  INFO [CompactionExecutor:163] 2012-12-05 22:11:27,679
> AutoSavingCache.java (line 269) Saved APIPortal-WebSite-KeyCache (1 items)
> in 2 ms
>
> On Wed, Dec 12, 2012 at 5:45 AM, aaron morton <aaron@thelastpickle.com>wrote:
>
>> > Caused by: TimedOutException()
>> Means the nodes involved in the request did not return to the co
>> ordinator before the rpc_timeout expired.
>>
>> Check the logs on the servers to see if they are overloaded and dropping
>> messages.
>>
>> Also check that you are not asking for too much data.
>>
>> Cheers
>>
>> -----------------
>> Aaron Morton
>> Freelance Cassandra Developer
>> New Zealand
>>
>> @aaronmorton
>> http://www.thelastpickle.com
>>
>> On 11/12/2012, at 10:13 PM, dong.yajun <dongtalk@gmail.com> wrote:
>>
>> > hi list,
>> > I am using Cassandra with 3 data centers, each DC has more than 10
>> nodes.
>> >
>> > the schema for a keyspace:
>> > {DC1:3, DC2:3, DC3:3}
>> >
>> > now, I put some rows using hector with CL Local_Quorum in DC1 ,and then
>> I get a row with the same CL Local_Quorum in DC1,some exceptions were
>> occured:
>> >
>> > Cassandra with dsc-1.0.5, and Hector with 1.1-2.
>> >
>> > 2012-12-05 21:26:49,667 - WARN [pool-1-thread-3:JCLLoggerAdapter@379]
>> - Could not fullfill request on this host CassandraClient172.16.74.31:9160-1
>> > 2012-12-05 21:26:49,668 - WARN [pool-1-thread-3:JCLLoggerAdapter@437]
>> - Exception:
>> > me.prettyprint.hector.api.exceptions.HTimedOutException:
>> TimedOutException()
>> > at
>> me.prettyprint.cassandra.service.ExceptionsTranslatorImpl.translate(ExceptionsTranslatorImpl.java:35)
>> > at
>> me.prettyprint.cassandra.service.KeyspaceServiceImpl$3.execute(KeyspaceServiceImpl.java:163)
>> > at
>> me.prettyprint.cassandra.service.KeyspaceServiceImpl$3.execute(KeyspaceServiceImpl.java:145)
>> > at
>> me.prettyprint.cassandra.service.Operation.executeAndSetResult(Operation.java:103)
>> > at
>> me.prettyprint.cassandra.connection.HConnectionManager.operateWithFailover(HConnectionManager.java:258)
>> > at
>> me.prettyprint.cassandra.service.KeyspaceServiceImpl.operateWithFailover(KeyspaceServiceImpl.java:131)
>> > at
>> me.prettyprint.cassandra.service.KeyspaceServiceImpl.getRangeSlices(KeyspaceServiceImpl.java:167)
>> > at
>> me.prettyprint.cassandra.model.thrift.ThriftRangeSlicesQuery$1.doInKeyspace(ThriftRangeSlicesQuery.java:66)
>> > at
>> me.prettyprint.cassandra.model.thrift.ThriftRangeSlicesQuery$1.doInKeyspace(ThriftRangeSlicesQuery.java:62)
>> > at
>> me.prettyprint.cassandra.model.KeyspaceOperationCallback.doInKeyspaceAndMeasure(KeyspaceOperationCallback.java:20)
>> > at
>> me.prettyprint.cassandra.model.ExecutingKeyspace.doExecute(ExecutingKeyspace.java:85)
>> > at
>> me.prettyprint.cassandra.model.thrift.ThriftRangeSlicesQuery.execute(ThriftRangeSlicesQuery.java:61)
>> >
>> org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:176)
>> > at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown
>> Source)
>> > at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
>> > at java.lang.Thread.run(Unknown Source)
>> > Caused by: TimedOutException()
>> > at
>> org.apache.cassandra.thrift.Cassandra$get_range_slices_result.read(Cassandra.java:12830)
>> > at
>> org.apache.cassandra.thrift.Cassandra$Client.recv_get_range_slices(Cassandra.java:762)
>> > at
>> org.apache.cassandra.thrift.Cassandra$Client.get_range_slices(Cassandra.java:734)
>> > at
>> me.prettyprint.cassandra.service.KeyspaceServiceImpl$3.execute(KeyspaceServiceImpl.java:151)
>> > ... 19 more
>> >
>> > any suggestion with this issue? thanks.
>> >
>> >
>> >
>>
>>
>
>
> --
> *Ric Dong *
> Newegg Ecommerce, MIS department
>
>
>


-- 
*Ric Dong *
Newegg Ecommerce, MIS department

Mime
View raw message