cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "MaHaiyang (Commented) (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (CASSANDRA-3638) It may iterate the whole memtable while just query one row . This seriously affect the performance . of Cassandra
Date Mon, 19 Dec 2011 13:11:30 GMT

    [ https://issues.apache.org/jira/browse/CASSANDRA-3638?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13172252#comment-13172252
] 

MaHaiyang commented on CASSANDRA-3638:
--------------------------------------


{code:title=RowIteratorFactory.java|borderStyle=solid}
public IColumnIterator computeNext()
        {
            while (iter.hasNext())
            {
                Map.Entry<DecoratedKey, ColumnFamily> entry = iter.next();
                IColumnIterator ici = filter.getMemtableColumnIterator(entry.getValue(), entry.getKey(),
comparator);
                if (pred.apply(ici))  
                    return ici;
            }
            return endOfData();
{code} 
The 'iter' is a submap of memtable's columnFamilies  . If pred.apply(ici) == false , 'iter'
will iterate to the end .
Suppose 'iter' is  [ c ,d ,e ,f ] ,  pred.startKey is a ,and endKey is b , pred.apply(ici)
 will always be false . 
In fact ,in this case , 'iter' never need to iterate , and should directly return endOfData()
. This is the problem!
So , can anybody review the code too ?


                
> It may iterate the whole memtable while just query one row . This seriously affect the
 performance . of Cassandra
> ------------------------------------------------------------------------------------------------------------------
>
>                 Key: CASSANDRA-3638
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-3638
>             Project: Cassandra
>          Issue Type: Bug
>          Components: Core
>    Affects Versions: 1.0.0
>            Reporter: MaHaiyang
>
> RangeSliceVerbHandler may  just only query one row , but cassandra may iterate the whole
memtable .
> the problem is in ColumnFamilyStore.getRangeSlice() method .
> {color:red} // this iterator may iterate the whole memtable!!{color}
> {code:title=ColumnFamilyStore.java|borderStyle=solid}
>  public List<Row> getRangeSlice(ByteBuffer superColumn, final AbstractBounds range,
int maxResults, IFilter columnFilter)
>     throws ExecutionException, InterruptedException
>     {
>     ...
>         DecoratedKey startWith = new DecoratedKey(range.left, null);
>         DecoratedKey stopAt = new DecoratedKey(range.right, null);
>         QueryFilter filter = new QueryFilter(null, new QueryPath(columnFamily, superColumn,
null), columnFilter);
>         int gcBefore = (int)(System.currentTimeMillis() / 1000) - metadata.getGcGraceSeconds();
>         List<Row> rows;
>         ViewFragment view = markReferenced(startWith, stopAt);
>         try
>         {
>             CloseableIterator<Row> iterator = RowIteratorFactory.getIterator(view.memtables,
view.sstables, startWith, stopAt, filter, getComparator(), this);
>             rows = new ArrayList<Row>();
>             try
>             {
>                 // pull rows out of the iterator
>                 boolean first = true;
>                 while (iterator.hasNext()) // this iterator may iterate the whole memtable!!
              
>                {
>                     ....
>                 }
>             }
>           .....
>         }
>        .....
>         return rows;
>     }
> {code} 
> {color:red} // Just only query one row ,but returned a sublist of columnFamiles   {color}
> {code:title=Memtable.java|borderStyle=solid}
> // Just only query one row ,but returned a sublist of columnFamiles     
> public Iterator<Map.Entry<DecoratedKey, ColumnFamily>> getEntryIterator(DecoratedKey
startWith)
>     {
>         return columnFamilies.tailMap(startWith).entrySet().iterator();
>     }
> {code} 
> {color:red} // entry.getKey() will never bigger or equal to startKey, and then iterate
the whole sublist of memtable {color}             
> {code:title=RowIteratorFactory.java|borderStyle=solid}
>  public IColumnIterator computeNext()
>         {
>             while (iter.hasNext())
>             {
>                 Map.Entry<DecoratedKey, ColumnFamily> entry = iter.next();
>                 IColumnIterator ici = filter.getMemtableColumnIterator(entry.getValue(),
entry.getKey(), comparator);
>                 // entry.getKey() will never bigger or equal to startKey, and then iterate
the whole sublist of memtable             
>                 if (pred.apply(ici))  
>                     return ici;
>             }
>             return endOfData();
> {code} 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message