kudu-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Todd Lipcon <t...@cloudera.com>
Subject Re: Issue with newInsert() in scala
Date Thu, 24 Nov 2016 03:53:29 GMT
Hi Yanlong,

Interesting. It looks like the issue is not Scala, but the fact that you
are using a REPL (the Spark shell). The REPL tries to print the result of
each line, and in this case it's trying to stringify the insert before
you've set any columns on it.

We should probably fix the toString() method here so that it doesn't throw
an exception when all of the fields are unset. Any interest in working on a
patch? (or any other contributor who might be interested?)

-Todd

On Wed, Nov 23, 2016 at 7:32 PM, Zheng Yanlong 郑延龙(IS,SGM) <
Yanlong_Zheng@saic-gm.com> wrote:

> Hi all,
>
>
>
> I’m trying java kudu api in scala. I can create table and read records.
> But met error when tried to insert data into kudu.
>
>
>
> In spark-shell:
>
>
>
> scala> import org.apache.kudu.ColumnSchema;
>
> import org.apache.kudu.ColumnSchema
>
>
>
> scala> import org.apache.kudu.Schema;
>
> import org.apache.kudu.Schema
>
>
>
> scala> import org.apache.kudu.Type;
>
> import org.apache.kudu.Type
>
>
>
> scala> import org.apache.kudu.client._
>
> import org.apache.kudu.client._
>
>
>
> scala> import java.util.ArrayList;
>
> import java.util.ArrayList
>
>
>
> scala> import java.util.List;
>
> import java.util.List
>
>
>
> scala> val kc = new KuduClient.KuduClientBuilder("
> XXX.XXX.XXX.XXX:7051").build()
>
> kc: org.apache.kudu.client.KuduClient = org.apache.kudu.client.
> KuduClient@2b852e9
>
>
>
> scala> val ktbl = kc.openTable("kudu_scala_test")
>
> ktbl: org.apache.kudu.client.KuduTable = org.apache.kudu.client.
> KuduTable@7e26fb98
>
>
>
> scala> val ksession = kc.newSession()
>
> ksession: org.apache.kudu.client.KuduSession = org.apache.kudu.client.
> KuduSession@cc1f71e
>
>
>
> scala> val kins = ktbl.newInsert()
>
> java.lang.IllegalStateException: Column id is not set      <== id is the
> first column in the table.
>
>      at org.apache.kudu.client.shaded.com.google.common.base.
> Preconditions.checkState(Preconditions.java:197)
>
>      at org.apache.kudu.client.PartialRow.appendDebugString(
> PartialRow.java:559)
>
>      at org.apache.kudu.client.PartialRow.stringifyRowKey(
> PartialRow.java:532)
>
>      at org.apache.kudu.client.Operation.toString(Operation.java:202)
>
>      at scala.runtime.ScalaRunTime$.scala$runtime$ScalaRunTime$$
> inner$1(ScalaRunTime.scala:324)
>
>      at scala.runtime.ScalaRunTime$.stringOf(ScalaRunTime.scala:329)
>
>      at scala.runtime.ScalaRunTime$.replStringOf(ScalaRunTime.scala:337)
>
>      at .<init>(<console>:10)
>
>      at .<clinit>(<console>)
>
>      at $print(<console>)
>
>      at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>
>      at sun.reflect.NativeMethodAccessorImpl.invoke(
> NativeMethodAccessorImpl.java:57)
>
>      at sun.reflect.DelegatingMethodAccessorImpl.invoke(
> DelegatingMethodAccessorImpl.java:43)
>
>      at java.lang.reflect.Method.invoke(Method.java:606)
>
>      at org.apache.spark.repl.SparkIMain$ReadEvalPrint.call(
> SparkIMain.scala:1045)
>
>      at org.apache.spark.repl.SparkIMain$Request.loadAndRun(
> SparkIMain.scala:1326)
>
>      at org.apache.spark.repl.SparkIMain.loadAndRunReq$1(
> SparkIMain.scala:821)
>
>      at org.apache.spark.repl.SparkIMain.interpret(SparkIMain.scala:852)
>
>      at org.apache.spark.repl.SparkIMain.interpret(SparkIMain.scala:800)
>
>      at org.apache.spark.repl.SparkILoop.reallyInterpret$1(
> SparkILoop.scala:857)
>
>      at org.apache.spark.repl.SparkILoop.interpretStartingWith(
> SparkILoop.scala:902)
>
>      at org.apache.spark.repl.SparkILoop.command(SparkILoop.scala:814)
>
>      at org.apache.spark.repl.SparkILoop.processLine$1(
> SparkILoop.scala:657)
>
>      at org.apache.spark.repl.SparkILoop.innerLoop$1(SparkILoop.scala:665)
>
>      at org.apache.spark.repl.SparkILoop.org$apache$spark$
> repl$SparkILoop$$loop(SparkILoop.scala:670)
>
>      at org.apache.spark.repl.SparkILoop$$anonfun$org$
> apache$spark$repl$SparkILoop$$process$1.apply$mcZ$sp(SparkILoop.scala:997)
>
>      at org.apache.spark.repl.SparkILoop$$anonfun$org$
> apache$spark$repl$SparkILoop$$process$1.apply(SparkILoop.scala:945)
>
>      at org.apache.spark.repl.SparkILoop$$anonfun$org$
> apache$spark$repl$SparkILoop$$process$1.apply(SparkILoop.scala:945)
>
>      at scala.tools.nsc.util.ScalaClassLoader$.savingContextLoader(
> ScalaClassLoader.scala:135)
>
>      at org.apache.spark.repl.SparkILoop.org$apache$spark$
> repl$SparkILoop$$process(SparkILoop.scala:945)
>
>      at org.apache.spark.repl.SparkILoop.process(SparkILoop.scala:1064)
>
>      at org.apache.spark.repl.Main$.main(Main.scala:31)
>
>      at org.apache.spark.repl.Main.main(Main.scala)
>
>      at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>
>      at sun.reflect.NativeMethodAccessorImpl.invoke(
> NativeMethodAccessorImpl.java:57)
>
>      at sun.reflect.DelegatingMethodAccessorImpl.invoke(
> DelegatingMethodAccessorImpl.java:43)
>
>      at java.lang.reflect.Method.invoke(Method.java:606)
>
>      at org.apache.spark.deploy.SparkSubmit$.org$apache$spark$
> deploy$SparkSubmit$$runMain(SparkSubmit.scala:731)
>
>      at org.apache.spark.deploy.SparkSubmit$.doRunMain$1(
> SparkSubmit.scala:181)
>
>      at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:206)
>
>      at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:121)
>
>      at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
>
>
>
> I have tried the java sample, it worked well in java. Could you help with
> this issue? Or any other ways to insert data in scala?
>
> Please let me know if you need other information.
>
>
>
> Best Regards,
>
> Yanlong
>
>
> *********************************************************************
> Email Disclaimer
> ----------------
> The information in this email is confidential and may be legally
> privileged. It is intended solely for the addressee. Access to this
> email by anyone else is unauthorised.
> If you are not the intended recipient, any disclosure, copying,
> distribution or any action taken or omitted to be taken in reliance
> on it, is prohibited and may be unlawful.
> Any opinions or advice contained in this email are subject to the
> terms and conditions expressed in the governing SGM client
> engagement letter and should not be relied upon unless they are
> confirmed in writing on SGM's letterhead.
> *********************************************************************
>
>
>
>


-- 
Todd Lipcon
Software Engineer, Cloudera

Mime
View raw message