cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "MaHaiyang (Updated) (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (CASSANDRA-3638) It may iterate the whole memtable while just query one row . This seriously affect the performance . of Cassandra
Date Thu, 15 Dec 2011 13:14:30 GMT

     [ https://issues.apache.org/jira/browse/CASSANDRA-3638?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

MaHaiyang updated CASSANDRA-3638:
---------------------------------

    Description: 
RangeSliceVerbHandler may  just only query one row , but cassandra may iterate the whole memtable
.
the problem is in ColumnFamilyStore.getRangeSlice() method .
{code:title=ColumnFamilyStore.java|borderStyle=solid}
 public List<Row> getRangeSlice(ByteBuffer superColumn, final AbstractBounds range,
int maxResults, IFilter columnFilter)
    throws ExecutionException, InterruptedException
    {
    ...
        DecoratedKey startWith = new DecoratedKey(range.left, null);
        DecoratedKey stopAt = new DecoratedKey(range.right, null);

        QueryFilter filter = new QueryFilter(null, new QueryPath(columnFamily, superColumn,
null), columnFilter);
        int gcBefore = (int)(System.currentTimeMillis() / 1000) - metadata.getGcGraceSeconds();

        List<Row> rows;
        ViewFragment view = markReferenced(startWith, stopAt);
        try
        {
            CloseableIterator<Row> iterator = RowIteratorFactory.getIterator(view.memtables,
view.sstables, startWith, stopAt, filter, getComparator(), this);
            rows = new ArrayList<Row>();

            try
            {
                // pull rows out of the iterator
                boolean first = true;
                while (iterator.hasNext()) {color:red} // this iterator may iterate the whole
memtable!!{color}                 
               {
                    ....
                }
            }
          .....
        }
       .....
        return rows;
    }

{code} 

{code:title=Memtable.java|borderStyle=solid}
{color:red} // Just only query one row ,but returned a sublist of columnFamiles   {color}
 
public Iterator<Map.Entry<DecoratedKey, ColumnFamily>> getEntryIterator(DecoratedKey
startWith)
    {
        return columnFamilies.tailMap(startWith).entrySet().iterator();
    }
{code} 



{code:title=RowIteratorFactory.java|borderStyle=solid}
 public IColumnIterator computeNext()
        {
            while (iter.hasNext())
            {
                Map.Entry<DecoratedKey, ColumnFamily> entry = iter.next();
                IColumnIterator ici = filter.getMemtableColumnIterator(entry.getValue(), entry.getKey(),
comparator);
{color:red} // entry.getKey() will never bigger or equal to startKey, and then iterate the
whole sublist of memtable {color}             
                if (pred.apply(ici))  
                    return ici;
            }
            return endOfData();
{code} 







  was:
RangeSliceVerbHandler may  just only query one row , but cassandra may iterate the whole memtable
.
the problem is in ColumnFamilyStore.getRangeSlice() method .
{code:title=ColumnFamilyStore.java|borderStyle=solid}
 public List<Row> getRangeSlice(ByteBuffer superColumn, final AbstractBounds range,
int maxResults, IFilter columnFilter)
    throws ExecutionException, InterruptedException
    {
    ...
        DecoratedKey startWith = new DecoratedKey(range.left, null);
        DecoratedKey stopAt = new DecoratedKey(range.right, null);

        QueryFilter filter = new QueryFilter(null, new QueryPath(columnFamily, superColumn,
null), columnFilter);
        int gcBefore = (int)(System.currentTimeMillis() / 1000) - metadata.getGcGraceSeconds();

        List<Row> rows;
        ViewFragment view = markReferenced(startWith, stopAt);
        try
        {
            CloseableIterator<Row> iterator = RowIteratorFactory.getIterator(view.memtables,
view.sstables, startWith, stopAt, filter, getComparator(), this);
            rows = new ArrayList<Row>();

            try
            {
                // pull rows out of the iterator
                boolean first = true;
                while (iterator.hasNext()) {color:red} 
// this iterator may iterate the whole memtable!!
 
{color}                 
               {
                    ....
                }
            }
          .....
        }
       .....
        return rows;
    }

{code} 

{code:title=Memtable.java|borderStyle=solid}
{color:red} 
// Just only query one row ,but returned a sublist of columnFamiles   
{color}  
public Iterator<Map.Entry<DecoratedKey, ColumnFamily>> getEntryIterator(DecoratedKey
startWith)
    {
        return columnFamilies.tailMap(startWith).entrySet().iterator();
    }
{code} 



{code:title=RowIteratorFactory.java|borderStyle=solid}
 public IColumnIterator computeNext()
        {
            while (iter.hasNext())
            {
                Map.Entry<DecoratedKey, ColumnFamily> entry = iter.next();
                IColumnIterator ici = filter.getMemtableColumnIterator(entry.getValue(), entry.getKey(),
comparator);
{color:red} 
// entry.getKey() will never bigger or equal to startKey, and then iterate the whole sublist
of memtable 
{color}             
                if (pred.apply(ici))  
                    return ici;
            }
            return endOfData();
{code} 







    
> It may iterate the whole memtable while just query one row . This seriously affect the
 performance . of Cassandra
> ------------------------------------------------------------------------------------------------------------------
>
>                 Key: CASSANDRA-3638
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-3638
>             Project: Cassandra
>          Issue Type: Bug
>          Components: Core
>    Affects Versions: 1.0.0
>            Reporter: MaHaiyang
>
> RangeSliceVerbHandler may  just only query one row , but cassandra may iterate the whole
memtable .
> the problem is in ColumnFamilyStore.getRangeSlice() method .
> {code:title=ColumnFamilyStore.java|borderStyle=solid}
>  public List<Row> getRangeSlice(ByteBuffer superColumn, final AbstractBounds range,
int maxResults, IFilter columnFilter)
>     throws ExecutionException, InterruptedException
>     {
>     ...
>         DecoratedKey startWith = new DecoratedKey(range.left, null);
>         DecoratedKey stopAt = new DecoratedKey(range.right, null);
>         QueryFilter filter = new QueryFilter(null, new QueryPath(columnFamily, superColumn,
null), columnFilter);
>         int gcBefore = (int)(System.currentTimeMillis() / 1000) - metadata.getGcGraceSeconds();
>         List<Row> rows;
>         ViewFragment view = markReferenced(startWith, stopAt);
>         try
>         {
>             CloseableIterator<Row> iterator = RowIteratorFactory.getIterator(view.memtables,
view.sstables, startWith, stopAt, filter, getComparator(), this);
>             rows = new ArrayList<Row>();
>             try
>             {
>                 // pull rows out of the iterator
>                 boolean first = true;
>                 while (iterator.hasNext()) {color:red} // this iterator may iterate the
whole memtable!!{color}                 
>                {
>                     ....
>                 }
>             }
>           .....
>         }
>        .....
>         return rows;
>     }
> {code} 
> {code:title=Memtable.java|borderStyle=solid}
> {color:red} // Just only query one row ,but returned a sublist of columnFamiles   {color}
 
> public Iterator<Map.Entry<DecoratedKey, ColumnFamily>> getEntryIterator(DecoratedKey
startWith)
>     {
>         return columnFamilies.tailMap(startWith).entrySet().iterator();
>     }
> {code} 
> {code:title=RowIteratorFactory.java|borderStyle=solid}
>  public IColumnIterator computeNext()
>         {
>             while (iter.hasNext())
>             {
>                 Map.Entry<DecoratedKey, ColumnFamily> entry = iter.next();
>                 IColumnIterator ici = filter.getMemtableColumnIterator(entry.getValue(),
entry.getKey(), comparator);
> {color:red} // entry.getKey() will never bigger or equal to startKey, and then iterate
the whole sublist of memtable {color}             
>                 if (pred.apply(ici))  
>                     return ici;
>             }
>             return endOfData();
> {code} 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message