incubator-cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From VĂ­ctor Hugo Oliveira Molinar <vhmoli...@gmail.com>
Subject Re:
Date Mon, 04 Feb 2013 13:20:35 GMT
How do you establish the connection?
Are you closing and reopening it?
It's normal for cassandra slowing down after many insertions, but it would
only take more time to process your write, nothing more than that.

On Fri, Feb 1, 2013 at 5:53 PM, Marcelo Elias Del Valle
<mvallebr@gmail.com>wrote:

> Hello,
>
>      I am trying to figure out why the following behavior happened. Any
> help would be highly appreciated.
>      This graph shows the server resources allocation of my single
> cassandra machine (running at Amazon EC2):
> http://mvalle.com/downloads/cassandra_host1.png
>      I ran a hadoop process that reads a CSV file and writtes data to
> Cassandra. For about 1 h, the process ran fine, but taking about 100% of
> CPU. After 1 h, my hadoop process started to have its connection attempts
> refused by cassandra, as shown bellow.
>      Since them, it has been taking 100% of the machine IO. It has been 2
> h already since the IO is 100% on the machine running Cassandra.
>      I am running Cassandra under Amazon EBS, which is slow, but I didn't
> think it would be that slow. Just wondering, is it normal for Cassandra to
> use a high amount of CPU? I am guessing all the writes were going to the
> memtables and when it was time to flush the server went down.
>      Makes sense? I am still learning Cassandra as it's the first time I
> use it in production, so I am not sure if I am missing something really
> basic here.
>
>
> 2013-02-01 16:44:43,741 ERROR com.s1mbi0se.dmp.input.service.InputService (Thread-18):
EXCEPTION:PoolTimeoutException: [host=(10.84.65.108):9160, latency=5005(5005), attempts=1]
Timed out waiting for connection
> com.netflix.astyanax.connectionpool.exceptions.PoolTimeoutException: PoolTimeoutException:
[host=nosql1.s1mbi0se.com.br(10.84.65.108):9160, latency=5005(5005), attempts=1] Timed out
waiting for connection
> 	at com.netflix.astyanax.connectionpool.impl.SimpleHostConnectionPool.waitForConnection(SimpleHostConnectionPool.java:201)
> 	at com.netflix.astyanax.connectionpool.impl.SimpleHostConnectionPool.borrowConnection(SimpleHostConnectionPool.java:158)
> 	at com.netflix.astyanax.connectionpool.impl.RoundRobinExecuteWithFailover.borrowConnection(RoundRobinExecuteWithFailover.java:60)
> 	at com.netflix.astyanax.connectionpool.impl.AbstractExecuteWithFailoverImpl.tryOperation(AbstractExecuteWithFailoverImpl.java:50)
> 	at com.netflix.astyanax.connectionpool.impl.AbstractHostPartitionConnectionPool.executeWithFailover(AbstractHostPartitionConnectionPool.java:229)
> 	at com.netflix.astyanax.thrift.ThriftColumnFamilyQueryImpl$1.execute(ThriftColumnFamilyQueryImpl.java:186)
> 	at com.s1mbi0se.dmp.input.service.InputService.searchUserByKey(InputService.java:700)
>
> ...
> 	at com.s1mbi0se.dmp.importer.map.ImporterMapper.map(ImporterMapper.java:20)
> 	at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144)
> 	at org.apache.hadoop.mapreduce.lib.map.MultithreadedMapper$MapRunner.run(MultithreadedMapper.java:268)
> 2013-02-01 16:44:43,743 ERROR com.s1mbi0se.dmp.input.service.InputService (Thread-15):
EXCEPTION:PoolTimeoutException:
>
>
> Best regards,
> --
> Marcelo Elias Del Valle
> http://mvalle.com - @mvallebr
>

Mime
View raw message