cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Brandon Williams (JIRA)" <j...@apache.org>
Subject [jira] Commented: (CASSANDRA-1898) json2sstable should support streaming
Date Wed, 12 Jan 2011 18:14:49 GMT

    [ https://issues.apache.org/jira/browse/CASSANDRA-1898?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12980844#action_12980844
] 

Brandon Williams commented on CASSANDRA-1898:
---------------------------------------------

Test fails:


    [junit] Testsuite: org.apache.cassandra.tools.SSTableExportTest
    [junit] Tests run: 4, Failures: 0, Errors: 1, Time elapsed: 1.008 sec
    [junit]
    [junit] Testcase: testRoundTripStandardCf(org.apache.cassandra.tools.SSTableExportTest):
   Caused an ERROR
    [junit] Unexpected character ('}' (code 125)): was expecting double-quote to start field
name
    [junit]  at [Source: /tmp/Standard18341291123979998324.json; line: 3, column: 2]
    [junit] org.codehaus.jackson.JsonParseException: Unexpected character ('}' (code 125)):
was expecting double-quote to start field name
    [junit]  at [Source: /tmp/Standard18341291123979998324.json; line: 3, column: 2]
    [junit]     at org.codehaus.jackson.JsonParser._constructError(JsonParser.java:929)
    [junit]     at org.codehaus.jackson.impl.JsonParserBase._reportError(JsonParserBase.java:632)
    [junit]     at org.codehaus.jackson.impl.JsonParserBase._reportUnexpectedChar(JsonParserBase.java:565)
    [junit]     at org.codehaus.jackson.impl.Utf8StreamParser._handleUnusualFieldName(Utf8StreamParser.java:561)
    [junit]     at org.codehaus.jackson.impl.Utf8StreamParser._parseFieldName(Utf8StreamParser.java:250)
    [junit]     at org.codehaus.jackson.impl.Utf8StreamParser.nextToken(Utf8StreamParser.java:140)
    [junit]     at org.codehaus.jackson.map.deser.MapDeserializer._readAndBind(MapDeserializer.java:186)
    [junit]     at org.codehaus.jackson.map.deser.MapDeserializer.deserialize(MapDeserializer.java:145)
    [junit]     at org.codehaus.jackson.map.deser.MapDeserializer.deserialize(MapDeserializer.java:23)
    [junit]     at org.codehaus.jackson.map.ObjectMapper._readValue(ObjectMapper.java:1261)
    [junit]     at org.codehaus.jackson.map.ObjectMapper.readValue(ObjectMapper.java:517)
    [junit]     at org.codehaus.jackson.JsonParser.readValueAs(JsonParser.java:897)
    [junit]     at org.apache.cassandra.tools.SSTableImport.importUnsorted(SSTableImport.java:207)
    [junit]     at org.apache.cassandra.tools.SSTableImport.importJson(SSTableImport.java:197)
    [junit]     at org.apache.cassandra.tools.SSTableExportTest.testRoundTripStandardCf(SSTableExportTest.java:206)


Also "DEPLICATED" should be "DEPRECATED"

> json2sstable should support streaming
> -------------------------------------
>
>                 Key: CASSANDRA-1898
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-1898
>             Project: Cassandra
>          Issue Type: Improvement
>          Components: Tools
>            Reporter: Nick Bailey
>            Assignee: Pavel Yaskevich
>             Fix For: 0.7.1
>
>         Attachments: CASSANDRA-1898.patch
>
>   Original Estimate: 8h
>          Time Spent: 8h
>  Remaining Estimate: 0h
>
> json2sstable loads the entire json file into memory. This is so it can sort the file
before creating an sstable. If the file was created using sstable2json and the partitioner
isn't changing, this isn't necessary.  For very large files this means json2sstable requires
a huge amount of memory.
> There should be an option to stream the file. A simple check for out of order keys will
prevent writing bad sstables.
> This should be possible with the SAX style parser available in our current json library.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message