incubator-cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jonathan Ellis <jbel...@gmail.com>
Subject Re: Strange Data distribution in 0.4.2
Date Tue, 10 Nov 2009 17:11:19 GMT
... of course this will change over time, which is why we're adding
active load balancing in 0.5, but the closer you can get to a
reasonable starting point, the better.

On Tue, Nov 10, 2009 at 11:08 AM, Jonathan Ellis <jbellis@gmail.com> wrote:
> yup, so you need to decide what dates are actually going to appear in your data
>
> On Tue, Nov 10, 2009 at 10:59 AM, Richard grossman <richiesgr@gmail.com> wrote:
>> ok but If it's not defined ? for example 1 field is a date there is no end
>> for date ?
>>
>> On Tue, Nov 10, 2009 at 6:57 PM, Jonathan Ellis <jbellis@gmail.com> wrote:
>>>
>>> no.
>>>
>>> for randompartitioner, you use integers from 0 to 2**127, but for OPP
>>> you use strings from your key universe.
>>>
>>> On Tue, Nov 10, 2009 at 10:37 AM, Richard grossman <richiesgr@gmail.com>
>>> wrote:
>>> > If I understand good if I transform my String key <data>:<data>:<data>
>>> > to
>>> > some long value and If I've 3 server then I put on the first server
>>> > initialToken : 0
>>> >                                     
second : Long.max() /2
>>> >                                     
third : Long.max()
>>> >
>>> > Is it correct ?? or there is something better ?
>>> >
>>> > thanks
>>> >
>>> > On Tue, Nov 10, 2009 at 6:01 PM, Jonathan Ellis <jbellis@gmail.com>
>>> > wrote:
>>> >>
>>> >> for OPP, tokens are equivalent to keys so pick keys evenly spaced apart
>>> >>
>>> >> On Tue, Nov 10, 2009 at 9:55 AM, Richard grossman <richiesgr@gmail.com>
>>> >> wrote:
>>> >> > hi
>>> >> >
>>> >> > I've understand this but I don't know what to write into initialtoken
>>> >> > is
>>> >> > it
>>> >> > "1" or "a" or something else ?
>>> >> > as I've said in a previous post My keys are build like
>>> >> > <data>:<data>:<data>
>>> >> > Is there any link ?
>>> >> >
>>> >> > Thanks,
>>> >> >
>>> >> > On Tue, Nov 10, 2009 at 5:48 PM, Jonathan Ellis <jbellis@gmail.com>
>>> >> > wrote:
>>> >> >>
>>> >> >> if you're not specifying initialtoken, every time you wipe
your
>>> >> >> installation it will generate new tokens.  for a small number
of
>>> >> >> machines you'll definitely see some random tokens better balanced
>>> >> >> than
>>> >> >> others.
>>> >> >>
>>> >> >> On Tue, Nov 10, 2009 at 9:38 AM, Richard grossman
>>> >> >> <richiesgr@gmail.com>
>>> >> >> wrote:
>>> >> >> > Hi
>>> >> >> >
>>> >> >> > I've build the 0.4.2 from the tags in SVN.
>>> >> >> > I've made exactly the same cluster with same configuration
as
>>> >> >> > 0.4.1
>>> >> >> > I've delete all the data in all server
>>> >> >> >
>>> >> >> > Now I send the data to first server and the data is not
more
>>> >> >> > distributed
>>> >> >> > across the other server as previously.
>>> >> >> > I've configured replica to 1
>>> >> >> >
>>> >> >> > here is my storage-conf.xml
>>> >> >> >
>>> >> >> > <Storage>
>>> >> >> >
>>> >> >> >   <ClusterName>BeeCluster</ClusterName>
>>> >> >> >
>>> >> >> >
>>> >> >> >   <Keyspaces>
>>> >> >> >     <Keyspace Name="Keyspace1">
>>> >> >> >
>>> >> >> >       <ColumnFamily CompareWith="BytesType"
>>> >> >> >                     Name="channelShowLink"
>>> >> >> >                     FlushPeriodInMinutes="15"/>
>>> >> >> >
>>> >> >> >       <ColumnFamily CompareWith="BytesType"
>>> >> >> >                       Name="channelShow"
>>> >> >> >                       FlushPeriodInMinutes="15"/>
>>> >> >> >
>>> >> >> >       <ColumnFamily CompareWith="BytesType"
>>> >> >> >                        Name="userAction"
>>> >> >> >                        FlushPeriodInMinutes="15"/>
>>> >> >> >
>>> >> >> >       <ColumnFamily CompareWith="BytesType"
>>> >> >> >                        Name="headends"
>>> >> >> >                        FlushPeriodInMinutes="15"/>
>>> >> >> >
>>> >> >> >       <ColumnFamily CompareWith="BytesType"
>>> >> >> >                        Name="similarity"
>>> >> >> >                        FlushPeriodInMinutes="500"/>
>>> >> >> > </Keyspace>
>>> >> >> >   </Keyspaces>
>>> >> >> >
>>> >> >> >
>>> >> >> >
>>> >> >> >
>>> >> >> >
>>> >> >> >
>>> >> >> > <Partitioner>org.apache.cassandra.dht.OrderPreservingPartitioner</Partitioner>
>>> >> >> >
>>> >> >> >   <InitialToken></InitialToken>
>>> >> >> >
>>> >> >> >
>>> >> >> >
>>> >> >> >
>>> >> >> >
>>> >> >> >
>>> >> >> > <EndPointSnitch>org.apache.cassandra.locator.EndPointSnitch</EndPointSnitch>
>>> >> >> >
>>> >> >> >
>>> >> >> >
>>> >> >> >
>>> >> >> >
>>> >> >> >
>>> >> >> > <ReplicaPlacementStrategy>org.apache.cassandra.locator.RackUnawareStrategy</ReplicaPlacementStrategy>
>>> >> >> >
>>> >> >> >   <ReplicationFactor>1</ReplicationFactor>
>>> >> >> >
>>> >> >> >
>>> >> >> >
>>> >> >> >
>>> >> >> >
>>> >> >> >
>>> >> >> > <CommitLogDirectory>/home/beecloud/cassandrapart/commitlog</CommitLogDirectory>
>>> >> >> >   <DataFileDirectories>
>>> >> >> >
>>> >> >> >
>>> >> >> >
>>> >> >> > <DataFileDirectory>/home/beecloud/cassandrapart/data</DataFileDirectory>
>>> >> >> >   </DataFileDirectories>
>>> >> >> >
>>> >> >> >
>>> >> >> >
>>> >> >> > <CalloutLocation>/home/beecloud/cassandrapart/callouts</CalloutLocation>
>>> >> >> >
>>> >> >> >
>>> >> >> >
>>> >> >> >
>>> >> >> > <BootstrapFileDirectory>/home/beecloud/cassandrapart/bootstrap</BootstrapFileDirectory>
>>> >> >> >
>>> >> >> >
>>> >> >> >
>>> >> >> >
>>> >> >> > <StagingFileDirectory>/home/beecloud/cassandrapart/staging</StagingFileDirectory>
>>> >> >> >
>>> >> >> >
>>> >> >> >
>>> >> >> >   <Seeds>
>>> >> >> >       <Seed>192.168.249.200</Seed>
>>> >> >> >       <Seed>192.168.249.222</Seed>
>>> >> >> >       <Seed>192.168.249.95</Seed>
>>> >> >> >   </Seeds>
>>> >> >> >
>>> >> >> >
>>> >> >> >   <!-- Miscellaneous -->
>>> >> >> >
>>> >> >> >   <!-- Time to wait for a reply from other nodes before
failing
>>> >> >> > the
>>> >> >> > command
>>> >> >> > -->
>>> >> >> >   <RpcTimeoutInMillis>50000</RpcTimeoutInMillis>
>>> >> >> >   <!-- Size to allow commitlog to grow to before creating
a new
>>> >> >> > segment
>>> >> >> > -->
>>> >> >> >
>>> >> >> >
>>> >> >> > <CommitLogRotationThresholdInMB>128</CommitLogRotationThresholdInMB>
>>> >> >> >
>>> >> >> >
>>> >> >> >
>>> >> >> >   <ListenAddress>192.168.249.200</ListenAddress>
>>> >> >> >   <!-- TCP port, for commands and data -->
>>> >> >> >   <StoragePort>7000</StoragePort>
>>> >> >> >   <!-- UDP port, for membership communications (gossip)
-->
>>> >> >> >   <ControlPort>7001</ControlPort>
>>> >> >> >
>>> >> >> >
>>> >> >> >   <ThriftAddress>0.0.0.0</ThriftAddress>
>>> >> >> >   <!-- Thrift RPC port (the port clients connect to).
-->
>>> >> >> >   <ThriftPort>9160</ThriftPort>
>>> >> >> >   <!--
>>> >> >> >    ~ Whether or not to use a framed transport for Thrift.
If this
>>> >> >> > option
>>> >> >> >    ~ is set to true then you must also use a framed
transport on
>>> >> >> > the
>>> >> >> >    ~ client-side, (framed and non-framed transports
are not
>>> >> >> > compatible).
>>> >> >> >   -->
>>> >> >> >   <ThriftFramedTransport>false</ThriftFramedTransport>
>>> >> >> >
>>> >> >> >
>>> >> >> >
>>> >> >> >   <SlicedBufferSizeInKB>64</SlicedBufferSizeInKB>
>>> >> >> >
>>> >> >> >
>>> >> >> >   <FlushDataBufferSizeInMB>32</FlushDataBufferSizeInMB>
>>> >> >> >   <FlushIndexBufferSizeInMB>8</FlushIndexBufferSizeInMB>
>>> >> >> >
>>> >> >> >
>>> >> >> >   <ColumnIndexSizeInKB>64</ColumnIndexSizeInKB>
>>> >> >> >
>>> >> >> >
>>> >> >> >   <MemtableSizeInMB>64</MemtableSizeInMB>
>>> >> >> >
>>> >> >> >
>>> >> >> > <MemtableObjectCountInMillions>0.1</MemtableObjectCountInMillions>
>>> >> >> >
>>> >> >> >
>>> >> >> >   <ConcurrentReads>8</ConcurrentReads>
>>> >> >> >   <ConcurrentWrites>32</ConcurrentWrites>
>>> >> >> >
>>> >> >> >
>>> >> >> >   <CommitLogSync>periodic</CommitLogSync>
>>> >> >> >
>>> >> >> >   <CommitLogSyncPeriodInMS>1000</CommitLogSyncPeriodInMS>
>>> >> >> >
>>> >> >> >   <!--
>>> >> >> > <CommitLogSyncBatchWindowInMS>1</CommitLogSyncBatchWindowInMS>
>>> >> >> > -->
>>> >> >> >
>>> >> >> >
>>> >> >> >   <GCGraceSeconds>864000</GCGraceSeconds>
>>> >> >> >
>>> >> >> >
>>> >> >> >   <FlushMinThreads>1</FlushMinThreads>
>>> >> >> >   <FlushMaxThreads>1</FlushMaxThreads>
>>> >> >> >
>>> >> >> >
>>> >> >> >   <BinaryMemtableSizeInMB>256</BinaryMemtableSizeInMB>
>>> >> >> >
>>> >> >> > </Storage>
>>> >> >> >
>>> >> >> > Is there anyone with the same problem ??
>>> >> >> >
>>> >> >> > Thanks
>>> >> >> >
>>> >> >
>>> >> >
>>> >
>>> >
>>
>>
>

Mime
View raw message