cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Tey Kar Shiang (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (CASSANDRA-2401) getColumnFamily() return null, which is not checked in ColumnFamilyStore.java scan() method, causing Timeout Exception in query
Date Tue, 05 Apr 2011 14:24:06 GMT

    [ https://issues.apache.org/jira/browse/CASSANDRA-2401?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13015932#comment-13015932
] 

Tey Kar Shiang commented on CASSANDRA-2401:
-------------------------------------------

hi, yes. it seems to me so. Here, we create a table "FileMap", in which we store columns e.g.
"content", "authorID", "Version", "Modified Time", "File Type", etc. Among them, sorted indices
are "authorID" (as UserIndex), "File Type", "Modified Time", and "CassType"; where CassType
means generally 'file type' here in our case. It is not used though.

03/30/2011  09:34 AM        11,366,878 FileMap-f-53-Data.db
03/30/2011  09:34 AM            78,496 FileMap-f-53-Filter.db
03/30/2011  09:34 AM           735,930 FileMap-f-53-Index.db
03/30/2011  09:34 AM             4,264 FileMap-f-53-Statistics.db
03/30/2011  05:37 PM             4,055 FileMap-f-54-Data.db
03/30/2011  05:37 PM                40 FileMap-f-54-Filter.db
03/30/2011  05:37 PM               270 FileMap-f-54-Index.db
03/30/2011  05:37 PM             4,264 FileMap-f-54-Statistics.db
04/04/2011  04:07 PM            24,068 FileMap-f-55-Data.db
04/04/2011  04:07 PM               200 FileMap-f-55-Filter.db
04/04/2011  04:07 PM             1,746 FileMap-f-55-Index.db
04/04/2011  04:07 PM             4,264 FileMap-f-55-Statistics.db
04/04/2011  04:07 PM           961,808 FileMap.CassTypeIndex-f-53-Data.db
04/04/2011  04:07 PM             1,936 FileMap.CassTypeIndex-f-53-Filter.db
04/04/2011  04:07 PM                11 FileMap.CassTypeIndex-f-53-Index.db
04/04/2011  04:07 PM             4,264 FileMap.CassTypeIndex-f-53-Statistics.db
03/29/2011  02:52 PM           961,386 FileMap.FileTypeIndex-f-50-Data.db
03/29/2011  02:52 PM             1,936 FileMap.FileTypeIndex-f-50-Filter.db
03/29/2011  02:52 PM                11 FileMap.FileTypeIndex-f-50-Index.db
03/29/2011  02:52 PM             4,264 FileMap.FileTypeIndex-f-50-Statistics.db
03/30/2011  05:37 PM               404 FileMap.FileTypeIndex-f-51-Data.db
03/30/2011  05:37 PM                16 FileMap.FileTypeIndex-f-51-Filter.db
03/30/2011  05:37 PM                11 FileMap.FileTypeIndex-f-51-Index.db
03/30/2011  05:37 PM             4,264 FileMap.FileTypeIndex-f-51-Statistics.db
04/04/2011  04:07 PM             2,358 FileMap.FileTypeIndex-f-52-Data.db
04/04/2011  04:07 PM                16 FileMap.FileTypeIndex-f-52-Filter.db
04/04/2011  04:07 PM                11 FileMap.FileTypeIndex-f-52-Index.db
04/04/2011  04:07 PM             4,264 FileMap.FileTypeIndex-f-52-Statistics.db
03/29/2011  02:52 PM         3,298,947 FileMap.ModifiedIndex-f-50-Data.db
03/29/2011  02:52 PM            78,016 FileMap.ModifiedIndex-f-50-Filter.db
03/29/2011  02:52 PM           731,106 FileMap.ModifiedIndex-f-50-Index.db
03/29/2011  02:52 PM             4,264 FileMap.ModifiedIndex-f-50-Statistics.db
03/30/2011  05:37 PM             2,065 FileMap.ModifiedIndex-f-51-Data.db
03/30/2011  05:37 PM                64 FileMap.ModifiedIndex-f-51-Filter.db
03/30/2011  05:37 PM               450 FileMap.ModifiedIndex-f-51-Index.db
03/30/2011  05:37 PM             4,264 FileMap.ModifiedIndex-f-51-Statistics.db
04/04/2011  04:07 PM            13,835 FileMap.ModifiedIndex-f-52-Data.db
04/04/2011  04:07 PM               328 FileMap.ModifiedIndex-f-52-Filter.db
04/04/2011  04:07 PM             3,006 FileMap.ModifiedIndex-f-52-Index.db
04/04/2011  04:07 PM             4,264 FileMap.ModifiedIndex-f-52-Statistics.db
04/04/2011  04:07 PM           962,874 FileMap.UserIndex-f-53-Data.db
04/04/2011  04:07 PM             1,936 FileMap.UserIndex-f-53-Filter.db
04/04/2011  04:07 PM               420 FileMap.UserIndex-f-53-Index.db
04/04/2011  04:07 PM             4,264 FileMap.UserIndex-f-53-Statistics.db

In the search, we are using IndexClause as:
		ByteBuffer field_author = ByteBuffer.wrap(new byte[]{'a'});
		ByteBuffer author_1 = IntegerSerializer.get().toByteBuffer(1);
		
		ByteBuffer file_type = ByteBuffer.wrap(new byte[]{'t'});
		ByteBuffer filetype_3 = ByteBuffer.wrap(new byte[]{3}); //file type 3
		
		IndexClause indexClause = new IndexClause();
		indexClause.setCount(3000);
		ArrayList<IndexExpression> expressions = new ArrayList();
		expressions.add(new IndexExpression(field_author, IndexOperator.EQ, author_1)); //user ID
= 1
		expressions.add(new IndexExpression(file_type, IndexOperator.EQ, filetype_3)); //file type
= 3
		
		indexClause.setExpressions(expressions);
		indexClause.setStart_key(new byte[]{});


In the search, it scans all the indices from "FileMap.UserIndex", within which there seems
having a key (index) which is not found in the table "FileMap"; and I roughly get that it
breaks at data retrieval with "FileMap-f-53-Data", when the position for the key is not found
/ available in "FileMap-f-53-Data".

> getColumnFamily() return null, which is not checked in ColumnFamilyStore.java scan()
method, causing Timeout Exception in query
> -------------------------------------------------------------------------------------------------------------------------------
>
>                 Key: CASSANDRA-2401
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-2401
>             Project: Cassandra
>          Issue Type: Bug
>    Affects Versions: 0.7.4
>         Environment: Hector 0.7.0-28, Cassandra 0.7.4, Windows 7, Eclipse
>            Reporter: Tey Kar Shiang
>
> ColumnFamilyStore.java, line near 1680, "ColumnFamily data = getColumnFamily(new QueryFilter(dk,
path, firstFilter))", the data is returned null, causing NULL exception in "satisfies(data,
clause, primary)" which is not captured. The callback got timeout and return a Timeout exception
to Hector.
> The data is empty, as I traced, I have the the columns Count as 0 in removeDeletedCF(),
which return the null there. (I am new and trying to understand the logics around still).
Instead of crash to NULL, could we bypass the data?
> About my test:
> A stress-test program to add, modify and delete data to keyspace. I have 30 threads simulate
concurrent users to perform the actions above, and do a query to all rows periodically. I
have Column Family with rows (as File) and columns as index (e.g. userID, fileType).
> No issue on the first day of test, and stopped for 3 days. I restart the test on 4th
day, 1 of the users failed to query the files (timeout exception received). Most of the users
are still okay with the query.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message