cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Alain RODRIGUEZ <arodr...@gmail.com>
Subject Re: Getting error while issuing Cassandra stress
Date Sat, 23 Jan 2016 02:03:18 GMT
>
> Should I use nodetool repair utility
>

That wouldn't help, this an anti-entropy mechanism (see
https://docs.datastax.com/en/cassandra/2.1/cassandra/tools/toolsRepair.html#toolsRepair__description_unique_27
).

It is something really important too often left aside.

Yet, your issue here is not about consistency. The client can't find any
node in charge of the read / written tokens in the ring. This depends on
the topology, the replication factor and your network mainly. I think there
is something wrong in your setup. I would try this:

- Make sure connection / port are ok
- Try increasing the RF / Strategy in the stress tool
- Try with an other consistency level (not LOCAL_*, as mentioned here :
http://stackoverflow.com/questions/32055251/not-enough-replica-available-for-query-at-consistency-local-one-1-required-but
)

Good luck,

-----------------
Alain

The Last Pickle
http://www.thelastpickle.com

2016-01-22 23:02 GMT+01:00 Bhuvan Rawal <bhu1rawal@gmail.com>:

> Getting same exception again. Should I use nodetool repair utility?
>
> On Sat, Jan 23, 2016 at 3:10 AM, Sebastian Estevez <
> sebastian.estevez@datastax.com> wrote:
>
>> https://github.com/brianmhess/cassandra-loader
>>
>> All the best,
>>
>>
>> [image: datastax_logo.png] <http://www.datastax.com/>
>>
>> Sebastián Estévez
>>
>> Solutions Architect | 954 905 8615 | sebastian.estevez@datastax.com
>>
>> [image: linkedin.png] <https://www.linkedin.com/company/datastax> [image:
>> facebook.png] <https://www.facebook.com/datastax> [image: twitter.png]
>> <https://twitter.com/datastax> [image: g+.png]
>> <https://plus.google.com/+Datastax/about>
>> <http://feeds.feedburner.com/datastax>
>> <http://goog_410786983>
>>
>>
>> <http://www.datastax.com/gartner-magic-quadrant-odbms>
>>
>> DataStax is the fastest, most scalable distributed database technology,
>> delivering Apache Cassandra to the world’s most innovative enterprises.
>> Datastax is built to be agile, always-on, and predictably scalable to any
>> size. With more than 500 customers in 45 countries, DataStax is the
>> database technology and transactional backbone of choice for the worlds
>> most innovative companies such as Netflix, Adobe, Intuit, and eBay.
>>
>> On Fri, Jan 22, 2016 at 4:37 PM, Bhuvan Rawal <bhu1rawal@gmail.com>
>> wrote:
>>
>>> Yes im specifying -node parameter to stress, otherwise it throws network
>>> connection failed.
>>>
>>> Can you point me to a sample java application to test pushing data from
>>> external server? Let's see if that works
>>>
>>> On Sat, Jan 23, 2016 at 2:55 AM, Sebastian Estevez <
>>> sebastian.estevez@datastax.com> wrote:
>>>
>>>> when i opened my cassandra-rackdc.properties i saw that DC names were
>>>>> DC1 & DC2, rack name was RAC1 . Please note that this is the default
>>>>> configuration, I have not modified any file.
>>>>
>>>>
>>>> cassandra-rackdc.properties is only respected based on your snitch
>>>> <https://docs.datastax.com/en/cassandra/2.1/cassandra/architecture/architectureSnitchesAbout_c.html>
>>>> .
>>>>
>>>> $ cqlsh
>>>>> Connection error: ('Unable to connect to any servers', {'127.0.0.1':
>>>>> error(111, "Tried connecting to [('127.0.0.1', 9042)]. Last error:
>>>>> Connection refused")})
>>>>> whereas
>>>>> $ cqlsh <ip>
>>>>> works fine
>>>>> is that the reason why the cassandra-stress is not able to communicate
>>>>> with other replicas?
>>>>
>>>>
>>>> Are you providing the -node parameter to stress
>>>> <http://docs.datastax.com/en/cassandra/2.1/cassandra/tools/toolsCStress_t.html>
>>>> ?
>>>>
>>>>
>>>>
>>>> All the best,
>>>>
>>>>
>>>> [image: datastax_logo.png] <http://www.datastax.com/>
>>>>
>>>> Sebastián Estévez
>>>>
>>>> Solutions Architect | 954 905 8615 | sebastian.estevez@datastax.com
>>>>
>>>> [image: linkedin.png] <https://www.linkedin.com/company/datastax> [image:
>>>> facebook.png] <https://www.facebook.com/datastax> [image: twitter.png]
>>>> <https://twitter.com/datastax> [image: g+.png]
>>>> <https://plus.google.com/+Datastax/about>
>>>> <http://feeds.feedburner.com/datastax>
>>>> <http://goog_410786983>
>>>>
>>>>
>>>> <http://www.datastax.com/gartner-magic-quadrant-odbms>
>>>>
>>>> DataStax is the fastest, most scalable distributed database
>>>> technology, delivering Apache Cassandra to the world’s most innovative
>>>> enterprises. Datastax is built to be agile, always-on, and predictably
>>>> scalable to any size. With more than 500 customers in 45 countries, DataStax
>>>> is the database technology and transactional backbone of choice for the
>>>> worlds most innovative companies such as Netflix, Adobe, Intuit, and eBay.
>>>>
>>>> On Fri, Jan 22, 2016 at 4:07 PM, Bhuvan Rawal <bhu1rawal@gmail.com>
>>>> wrote:
>>>>
>>>>> I had a look at the jira below:
>>>>> https://issues.apache.org/jira/browse/CASSANDRA-7905
>>>>>
>>>>> when i opened my cassandra-rackdc.properties i saw that DC names were
>>>>> DC1 & DC2, rack name was RAC1 . Please note that this is the default
>>>>> configuration, I have not modified any file.
>>>>>
>>>>> There is another point of concern here which might be relevant to
>>>>> previous one as well, im not able to login to cqlsh directly, i.e. I
have
>>>>> to specify ip as well even when im logged in to that machine.
>>>>>
>>>>> $ cqlsh
>>>>> Connection error: ('Unable to connect to any servers', {'127.0.0.1':
>>>>> error(111, "Tried connecting to [('127.0.0.1', 9042)]. Last error:
>>>>> Connection refused")})
>>>>>
>>>>> whereas
>>>>> $ cqlsh <ip>
>>>>> works fine
>>>>>
>>>>> is that the reason why the cassandra-stress is not able to communicate
>>>>> with other replicas?
>>>>>
>>>>> On Sat, Jan 23, 2016 at 1:37 AM, Sebastian Estevez <
>>>>> sebastian.estevez@datastax.com> wrote:
>>>>>
>>>>>> Sorry I missed that.
>>>>>>
>>>>>> Both your nodetool status and keyspace replication settings say
>>>>>> Cassandra and Analytics for the DC names. I'm not sure where you're
seeing
>>>>>> DC1, DC2, etc. and why you suspect that is the problem.
>>>>>>
>>>>>> All the best,
>>>>>>
>>>>>>
>>>>>> [image: datastax_logo.png] <http://www.datastax.com/>
>>>>>>
>>>>>> Sebastián Estévez
>>>>>>
>>>>>> Solutions Architect | 954 905 8615 | sebastian.estevez@datastax.com
>>>>>>
>>>>>> [image: linkedin.png] <https://www.linkedin.com/company/datastax>
[image:
>>>>>> facebook.png] <https://www.facebook.com/datastax> [image:
>>>>>> twitter.png] <https://twitter.com/datastax> [image: g+.png]
>>>>>> <https://plus.google.com/+Datastax/about>
>>>>>> <http://feeds.feedburner.com/datastax>
>>>>>> <http://goog_410786983>
>>>>>>
>>>>>>
>>>>>> <http://www.datastax.com/gartner-magic-quadrant-odbms>
>>>>>>
>>>>>> DataStax is the fastest, most scalable distributed database
>>>>>> technology, delivering Apache Cassandra to the world’s most innovative
>>>>>> enterprises. Datastax is built to be agile, always-on, and predictably
>>>>>> scalable to any size. With more than 500 customers in 45 countries,
DataStax
>>>>>> is the database technology and transactional backbone of choice for
the
>>>>>> worlds most innovative companies such as Netflix, Adobe, Intuit,
and eBay.
>>>>>>
>>>>>> On Fri, Jan 22, 2016 at 1:45 PM, Bhuvan Rawal <bhu1rawal@gmail.com>
>>>>>> wrote:
>>>>>>
>>>>>>> Hi Sebastian,
>>>>>>>
>>>>>>> I had attached nodetool status output in previous mail, pasting
it
>>>>>>> again :
>>>>>>>
>>>>>>> $ nodetool status Datacenter: Analytics =====================
>>>>>>> Status=Up/Down |/ State=Normal/Leaving/Joining/Moving -- Address
>>>>>>> Load Tokens Owns Host ID Rack UN 10.41.55.17 428.5 KB 256 ?
>>>>>>> 39d6d585-e641-4046-9d0b-797356597b5e rack1 UN 10.41.55.19 404.44
KB 256 ?
>>>>>>> 69edf930-efd9-4d74-a798-f3d4ac02e516 rack1 UN 10.41.55.18 423.21
KB 256 ?
>>>>>>> b74bab13-09b2-4760-bce9-c8ef05e50f6d rack1 UN 10.41.55.20 683.23
KB 256 ?
>>>>>>> fb5c4fed-6e1e-4ea8-838d-358106906830 rack1 Datacenter: Cassandra
>>>>>>> ===================== Status=Up/Down |/
>>>>>>> State=Normal/Leaving/Joining/Moving -- Address Load Tokens Owns
>>>>>>> Host ID Rack UN 10.41.55.15 209.4 KB 256 ?
>>>>>>> ffc3b9a0-5d5c-4a3d-a99e-49d255731278 rack1 UN 10.41.55.21 227.44
KB 256 ?
>>>>>>> c68deba4-b9a2-43fc-bb13-6af74c88c210 rack1 UN 10.41.55.23 222.71
KB 256 ?
>>>>>>> 8229aa87-af00-48fa-ad6b-3066d3dc0e58 rack1 UN 10.41.55.22 218.72
KB 256 ?
>>>>>>> c7ba84fd-7992-41de-8c88-11574a72db99 rack1
>>>>>>>
>>>>>>> Regards,
>>>>>>> Bhuvan Rawal
>>>>>>>
>>>>>>> On Sat, Jan 23, 2016 at 12:11 AM, Sebastian Estevez <
>>>>>>> sebastian.estevez@datastax.com> wrote:
>>>>>>>
>>>>>>>> The output of `nodetool status` would help us diagnose.
>>>>>>>>
>>>>>>>> All the best,
>>>>>>>>
>>>>>>>>
>>>>>>>> [image: datastax_logo.png] <http://www.datastax.com/>
>>>>>>>>
>>>>>>>> Sebastián Estévez
>>>>>>>>
>>>>>>>> Solutions Architect | 954 905 8615 | sebastian.estevez@datastax.com
>>>>>>>>
>>>>>>>> [image: linkedin.png] <https://www.linkedin.com/company/datastax>
[image:
>>>>>>>> facebook.png] <https://www.facebook.com/datastax> [image:
>>>>>>>> twitter.png] <https://twitter.com/datastax> [image:
g+.png]
>>>>>>>> <https://plus.google.com/+Datastax/about>
>>>>>>>> <http://feeds.feedburner.com/datastax>
>>>>>>>> <http://goog_410786983>
>>>>>>>>
>>>>>>>>
>>>>>>>> <http://www.datastax.com/gartner-magic-quadrant-odbms>
>>>>>>>>
>>>>>>>> DataStax is the fastest, most scalable distributed database
>>>>>>>> technology, delivering Apache Cassandra to the world’s
most innovative
>>>>>>>> enterprises. Datastax is built to be agile, always-on, and
predictably
>>>>>>>> scalable to any size. With more than 500 customers in 45
countries, DataStax
>>>>>>>> is the database technology and transactional backbone of
choice for the
>>>>>>>> worlds most innovative companies such as Netflix, Adobe,
Intuit, and eBay.
>>>>>>>>
>>>>>>>> On Fri, Jan 22, 2016 at 1:39 PM, Bhuvan Rawal <bhu1rawal@gmail.com>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> Thanks for the response Alain,
>>>>>>>>>
>>>>>>>>> cqlsh> create keyspace mykeyspace WITH replication
=
>>>>>>>>> {'class':'NetworkTopologyStrategy', 'Analytics':2, 'Cassandra':3}
>>>>>>>>> cqlsh> use mykeyspace;
>>>>>>>>> cqlsh:mykeyspace>create table mytable (id int primary
key, name
>>>>>>>>> text, address text, phone text);
>>>>>>>>> cqlsh:mykeyspace> insert into mytable (id, name, address,
phone)
>>>>>>>>> values (1, 'Kiyu','Texas', '555-1212'); # and other similar
statement
>>>>>>>>> I then issued the below command from every node and found
>>>>>>>>> consistent results.
>>>>>>>>> cqlsh:mykeyspace> select * from mytable;
>>>>>>>>>
>>>>>>>>> // Then i repeated the above steps for NetworkTopologyStrategy
and
>>>>>>>>> found same results
>>>>>>>>>
>>>>>>>>> I ran basic cassandra stress
>>>>>>>>> seed1 - seed of datacenter 1
>>>>>>>>>  $ cassandra-stress write n=50000 -rate threads=4 -node
>>>>>>>>> any_random_ip
>>>>>>>>>  $ cassandra-stress write n=50000 -rate threads=4 -node
seed1
>>>>>>>>>  $ cassandra-stress write n=50000 -rate threads=4 -node
seed1,seed2
>>>>>>>>>  $ cassandra-stress write n=50000 -rate threads=4 -node
>>>>>>>>> all_8_ip_comma_seperated
>>>>>>>>>  $ cassandra-stress write n=1000000 cl=one -mode native
cql3
>>>>>>>>> -schema keyspace="keyspace1" -pop seq=1..1000000 -node
ip1,ip2,ip3,ip4
>>>>>>>>>
>>>>>>>>> All of them threw the exception
>>>>>>>>> *com.datastax.driver.core.exceptions.UnavailableException:
Not
>>>>>>>>> enough replica available for query at consistency LOCAL_ONE
(1 required but
>>>>>>>>> only 0 alive)*
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> I have a feeling that the issue is with datacenter name
for some
>>>>>>>>> reason, because in some config files I found DC name
to be like DC1/DC2/DC3
>>>>>>>>> in some it is like Cassandra/Analytics (The ones I had
specified while
>>>>>>>>> installation). Im unsure which yaml/property file to
look for correct
>>>>>>>>> inconsistency.
>>>>>>>>>
>>>>>>>>> (C*heers :) - im so tempted to copy that)
>>>>>>>>>
>>>>>>>>> Regards,
>>>>>>>>> Bhuvan
>>>>>>>>>
>>>>>>>>> On Fri, Jan 22, 2016 at 8:47 PM, Alain RODRIGUEZ <
>>>>>>>>> arodrime@gmail.com> wrote:
>>>>>>>>>
>>>>>>>>>> Hi,
>>>>>>>>>>
>>>>>>>>>> The the exact command you ran (stress-tool with options)
could be
>>>>>>>>>> useful to help you on that.
>>>>>>>>>>
>>>>>>>>>> However, Im able to create keyspace, tables and insert
data using
>>>>>>>>>>> cqlsh and it is replicating fine to all the nodes.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Having the schema might be useful too.
>>>>>>>>>>
>>>>>>>>>> Did you ran the cqlsh and the stress-tool from the
same server ?
>>>>>>>>>> If not, you might want to check the port you use
(9042/9160/...) are open.
>>>>>>>>>> Also, cqlsh uses local_one by default too. If both
commands were
>>>>>>>>>> run against the same DC, from the same machine they
should behave the same
>>>>>>>>>> way. Are they ?
>>>>>>>>>>
>>>>>>>>>> C*heers,
>>>>>>>>>>
>>>>>>>>>> -----------------
>>>>>>>>>> Alain
>>>>>>>>>>
>>>>>>>>>> The Last Pickle
>>>>>>>>>> http://www.thelastpickle.com
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> 2016-01-22 9:57 GMT+01:00 Bhuvan Rawal <bhu1rawal@gmail.com>:
>>>>>>>>>>
>>>>>>>>>>> Hi,
>>>>>>>>>>>
>>>>>>>>>>> i have created a POC cluster with 2 DC , each
having 4 nodes
>>>>>>>>>>> with DSE 4.8.1 installed.
>>>>>>>>>>>
>>>>>>>>>>> On issuing cassandra stress im getting an error
 and data is not
>>>>>>>>>>> being inserted:
>>>>>>>>>>> *com.datastax.driver.core.exceptions.UnavailableException:
Not
>>>>>>>>>>> enough replica available for query at consistency
LOCAL_ONE (1 required but
>>>>>>>>>>> only 0 alive)*
>>>>>>>>>>>
>>>>>>>>>>> However, Im able to create keyspace, tables and
insert data
>>>>>>>>>>> using cqlsh and it is replicating fine to all
the nodes.
>>>>>>>>>>>
>>>>>>>>>>> Details of the cluster can be found below (all
the nodes seem to
>>>>>>>>>>> be alive and kicking):
>>>>>>>>>>>
>>>>>>>>>>> $ nodetool status Datacenter: Analytics =====================
>>>>>>>>>>> Status=Up/Down |/ State=Normal/Leaving/Joining/Moving
-- Address
>>>>>>>>>>> Load Tokens Owns Host ID Rack UN 10.41.55.17
428.5 KB 256 ?
>>>>>>>>>>> 39d6d585-e641-4046-9d0b-797356597b5e rack1 UN
10.41.55.19 404.44 KB 256 ?
>>>>>>>>>>> 69edf930-efd9-4d74-a798-f3d4ac02e516 rack1 UN
10.41.55.18 423.21 KB 256 ?
>>>>>>>>>>> b74bab13-09b2-4760-bce9-c8ef05e50f6d rack1 UN
10.41.55.20 683.23 KB 256 ?
>>>>>>>>>>> fb5c4fed-6e1e-4ea8-838d-358106906830 rack1 Datacenter:
>>>>>>>>>>> Cassandra ===================== Status=Up/Down
|/
>>>>>>>>>>> State=Normal/Leaving/Joining/Moving -- Address
Load Tokens Owns
>>>>>>>>>>> Host ID Rack UN 10.41.55.15 209.4 KB 256 ?
>>>>>>>>>>> ffc3b9a0-5d5c-4a3d-a99e-49d255731278 rack1 UN
10.41.55.21 227.44 KB 256 ?
>>>>>>>>>>> c68deba4-b9a2-43fc-bb13-6af74c88c210 rack1 UN
10.41.55.23 222.71 KB 256 ?
>>>>>>>>>>> 8229aa87-af00-48fa-ad6b-3066d3dc0e58 rack1 UN
10.41.55.22 218.72 KB 256 ?
>>>>>>>>>>> c7ba84fd-7992-41de-8c88-11574a72db99 rack1
>>>>>>>>>>>
>>>>>>>>>>> Regards,
>>>>>>>>>>> Bhuvan Rawal
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>
>

Mime
View raw message