Mailing-List: contact commits-help@cassandra.apache.org; run by ezmlm
Precedence: bulk
Date: Sat, 24 Jun 2017 05:53:00 +0000 (UTC)
From: "Michael Shuler (JIRA)" <jira@apache.org>
To: commits@cassandra.apache.org
Message-ID: <JIRA.13082145.1498228888000.97210.1498283580039@Atlassian.JIRA>
In-Reply-To: <JIRA.13082145.1498228888000@Atlassian.JIRA>
References: <JIRA.13082145.1498228888000@Atlassian.JIRA> <JIRA.13082145.1498228888781@jira-lw-us.apache.org>
Subject: [jira] [Updated] (CASSANDRA-13633) Data not loading to keyspace
 using sstable create via CQLSSTableWriter
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
archived-at: Sat, 24 Jun 2017 05:53:05 -0000


     [ https://issues.apache.org/jira/browse/CASSANDRA-13633?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Michael Shuler updated CASSANDRA-13633:
---------------------------------------
    Fix Version/s:     (was: 3.10)
                   3.11.x

> Data not loading to keyspace using sstable create via CQLSSTableWriter
> ----------------------------------------------------------------------
>
>                 Key: CASSANDRA-13633
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-13633
>             Project: Cassandra
>          Issue Type: Bug
>          Components: Tools
>         Environment: Linux
>            Reporter: Arpan Khandelwal
>             Fix For: 3.11.x
>
>         Attachments: dataloading_result.png
>
>
> Scenario : Need to read CSV, Write SSTable using CQLSSTableWriter and load it to the keyspace. [Explained here|https://stackoverflow.com/questions/44713777/json-cassandra-field-value-parsing-using-antlr4] . That was not working so tried a simple test case available here[https://github.com/apache/cassandra/blob/cassandra-3.11/test/unit/org/apache/cassandra/io/sstable/CQLSSTableWriterTest.java#L378]
> Which did not worked either. Following is what i tried.
> Created cql_keyspace3 with below table and types.
> {code:java}
> CREATE TYPE cql_keyspace3.tuple2 (a int, b int);
> CREATE TYPE cql_keyspace3.tuple3 (a int, b int, c int)
> CREATE TABLE cql_keyspace3.table3 ( k int, v1 list<frozen<tuple2>>, v2 frozen<tuple3>, PRIMARY KEY (k));
> {code}
> ran code 
> {code:java}
>         final String KS = "cql_keyspace3";
>         final String TABLE = "table3";
>         final String schema = "CREATE TABLE " + KS + "." + TABLE + " (" + "  k int," + "  v1 list<frozen<tuple2>>,"
>                 + "  v2 frozen<tuple3>," + "  PRIMARY KEY (k)" + ")";
>         File tempdir = Files.createTempDir();
>         File dataDir = new File(tempdir.getAbsolutePath() + File.separator + KS + File.separator + TABLE);
>         System.out.println(dataDir);
>         assert dataDir.mkdirs();
>         CQLSSTableWriter writer = CQLSSTableWriter.builder().inDirectory(dataDir)
>                 .withType("CREATE TYPE " + KS + ".tuple2 (a int, b int)")
>                 .withType("CREATE TYPE " + KS + ".tuple3 (a int, b int, c int)").forTable(schema)
>                 .using("INSERT INTO " + KS + "." + TABLE + " (k, v1, v2) " + "VALUES (?, ?, ?)").build();
>         
>         
>         UserType tuple2Type = writer.getUDType("tuple2");
>         UserType tuple3Type = writer.getUDType("tuple3");
>         for (int i = 0; i < 100; i++) {
>             writer.addRow(i,
>                     ImmutableList.builder().add(tuple2Type.newValue().setInt("a", i * 10).setInt("b", i * 20))
>                     .add(tuple2Type.newValue().setInt("a", i * 30).setInt("b", i * 40)).build(),
>                     tuple3Type.newValue().setInt("a", i * 100).setInt("b", i * 200).setInt("c", i * 300));
>         }
>         writer.close();
> {code}
> It generated sstable in "/tmp/1498224996687-0/cql_keyspace3" dir
> Loaded data using following command 
> {code:java}
> /tmp/1498224996687-0/cql_keyspace3 $ sstableloader -d localhost table3-e6e0fa61581911e78be6a72ebce4c745/
> Established connection to initial hosts
> Opening sstables and calculating sections to stream
> Streaming relevant part of /tmp/1498224996687-0/cql_keyspace3/table3-e6e0fa61581911e78be6a72ebce4c745/mc-2-big-Data.db to [localhost/127.0.0.1]
> progress: [localhost/127.0.0.1]0:1/1 100% total: 100% 1.060KiB/s (avg: 1.060KiB/s)
> progress: [localhost/127.0.0.1]0:1/1 100% total: 100% 0.000KiB/s (avg: 0.984KiB/s)
> Summary statistics: 
>    Connections per host    : 1         
>    Total files transferred : 1         
>    Total bytes transferred : 5.572KiB  
>    Total duration          : 5668 ms   
>    Average transfer rate   : 0.982KiB/s
>    Peak transfer rate      : 1.060KiB/s
> {code}
> ||k||v1||v2||
> |92|[{a:920,b:1840}, {a:2760,b:3680}]|
> find full result in attached snapshot.
> Please let me know which version of cassandra I should which can allow me to load  reseverd types, collections, UDT from sstable create using CQLSSTableWriter.


--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@cassandra.apache.org
For additional commands, e-mail: commits-help@cassandra.apache.org