cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Milan Majercik (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (CASSANDRA-12796) Heap exhaustion when rebuilding secondary index over a table with wide partitions
Date Mon, 17 Oct 2016 09:30:58 GMT

     [ https://issues.apache.org/jira/browse/CASSANDRA-12796?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Milan Majercik updated CASSANDRA-12796:
---------------------------------------
    Description: 
We have a table with rather wide partition with a secondary index defined over it. As soon
as we try to rebuild the index we observed exhaustion of Java heap and eventual OOM error.
After a lengthy investigation we have managed to find a culprit which appears to be a wrong
granule of barrier issuances in method {{org.apache.cassandra.db.Keyspace.indexRow}}:
{code}
        *try (OpOrder.Group opGroup = cfs.keyspace.writeOrder.start())*
        {
            Set<SecondaryIndex> indexes = cfs.indexManager.getIndexesByNames(idxNames);

            Iterator<ColumnFamily> pager = QueryPagers.pageRowLocally(cfs, key.getKey(),
DEFAULT_PAGE_SIZE);
            while (pager.hasNext())
            {
                ColumnFamily cf = pager.next();
                ColumnFamily cf2 = cf.cloneMeShallow();
                for (Cell cell : cf)
                {
                    if (cfs.indexManager.indexes(cell.name(), indexes))
                        cf2.addColumn(cell);
                }
                cfs.indexManager.indexRow(key.getKey(), cf2, opGroup);
            }
        *}*
{code}


> Heap exhaustion when rebuilding secondary index over a table with wide partitions
> ---------------------------------------------------------------------------------
>
>                 Key: CASSANDRA-12796
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-12796
>             Project: Cassandra
>          Issue Type: Bug
>          Components: Core
>            Reporter: Milan Majercik
>            Priority: Critical
>
> We have a table with rather wide partition with a secondary index defined over it. As
soon as we try to rebuild the index we observed exhaustion of Java heap and eventual OOM error.
After a lengthy investigation we have managed to find a culprit which appears to be a wrong
granule of barrier issuances in method {{org.apache.cassandra.db.Keyspace.indexRow}}:
> {code}
>         *try (OpOrder.Group opGroup = cfs.keyspace.writeOrder.start())*
>         {
>             Set<SecondaryIndex> indexes = cfs.indexManager.getIndexesByNames(idxNames);
>             Iterator<ColumnFamily> pager = QueryPagers.pageRowLocally(cfs, key.getKey(),
DEFAULT_PAGE_SIZE);
>             while (pager.hasNext())
>             {
>                 ColumnFamily cf = pager.next();
>                 ColumnFamily cf2 = cf.cloneMeShallow();
>                 for (Cell cell : cf)
>                 {
>                     if (cfs.indexManager.indexes(cell.name(), indexes))
>                         cf2.addColumn(cell);
>                 }
>                 cfs.indexManager.indexRow(key.getKey(), cf2, opGroup);
>             }
>         *}*
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message