kudu-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Zheng Yanlong 郑延龙(IS,SGM) <Yanlong_Zh...@SAIC-GM.com>
Subject RE: Issue with newInsert() in scala
Date Thu, 24 Nov 2016 10:00:26 GMT
Hi Todd,

I agree with you. It seems that the line in function appendCellValueDebugString in PartialRow.java
(line 595 in current source code) checks the column status and throws the exception:
         Preconditions.checkState(columnsBitSet.get(idx), "Column %s is not set", col.getName());

I think as the purpose of function appendCellValueDebugString is to append debug information,
this line may be inappropriate. But I’m not actually familiar with kudu source code and
not sure about it.
An issue has been created on Jira for it, I can try to set up a development environment and
work on it, but if anyone is interested, please go ahead and fix it:
https://issues.apache.org/jira/browse/KUDU-1757 .


Best Regards,
Yanlong
From: Todd Lipcon [mailto:todd@cloudera.com]
Sent: 2016年11月24日 11:53
To: user@kudu.apache.org
Subject: Re: Issue with newInsert() in scala

Hi Yanlong,

Interesting. It looks like the issue is not Scala, but the fact that you are using a REPL
(the Spark shell). The REPL tries to print the result of each line, and in this case it's
trying to stringify the insert before you've set any columns on it.

We should probably fix the toString() method here so that it doesn't throw an exception when
all of the fields are unset. Any interest in working on a patch? (or any other contributor
who might be interested?)

-Todd

On Wed, Nov 23, 2016 at 7:32 PM, Zheng Yanlong 郑延龙(IS,SGM) <Yanlong_Zheng@saic-gm.com<mailto:Yanlong_Zheng@saic-gm.com>>
wrote:
Hi all,

I’m trying java kudu api in scala. I can create table and read records. But met error when
tried to insert data into kudu.

In spark-shell:

scala> import org.apache.kudu.ColumnSchema;
import org.apache.kudu.ColumnSchema

scala> import org.apache.kudu.Schema;
import org.apache.kudu.Schema

scala> import org.apache.kudu.Type;
import org.apache.kudu.Type

scala> import org.apache.kudu.client._
import org.apache.kudu.client._

scala> import java.util.ArrayList;
import java.util.ArrayList

scala> import java.util.List;
import java.util.List

scala> val kc = new KuduClient.KuduClientBuilder("XXX.XXX.XXX.XXX:7051").build()
kc: org.apache.kudu.client.KuduClient = org.apache.kudu.client.KuduClient@2b852e9<mailto:org.apache.kudu.client.KuduClient@2b852e9>

scala> val ktbl = kc.openTable("kudu_scala_test")
ktbl: org.apache.kudu.client.KuduTable = org.apache.kudu.client.KuduTable@7e26fb98<mailto:org.apache.kudu.client.KuduTable@7e26fb98>

scala> val ksession = kc.newSession()
ksession: org.apache.kudu.client.KuduSession = org.apache.kudu.client.KuduSession@cc1f71e<mailto:org.apache.kudu.client.KuduSession@cc1f71e>

scala> val kins = ktbl.newInsert()
java.lang.IllegalStateException: Column id is not set      <== id is the first column in
the table.
     at org.apache.kudu.client.shaded.com.google.common.base.Preconditions.checkState(Preconditions.java:197)
     at org.apache.kudu.client.PartialRow.appendDebugString(PartialRow.java:559)
     at org.apache.kudu.client.PartialRow.stringifyRowKey(PartialRow.java:532)
     at org.apache.kudu.client.Operation.toString(Operation.java:202)
     at scala.runtime.ScalaRunTime$.scala$runtime$ScalaRunTime$$inner$1(ScalaRunTime.scala:324)
     at scala.runtime.ScalaRunTime$.stringOf(ScalaRunTime.scala:329)
     at scala.runtime.ScalaRunTime$.replStringOf(ScalaRunTime.scala:337)
     at .<init>(<console>:10)
     at .<clinit>(<console>)
     at $print(<console>)
     at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
     at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
     at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
     at java.lang.reflect.Method.invoke(Method.java:606)
     at org.apache.spark.repl.SparkIMain$ReadEvalPrint.call(SparkIMain.scala:1045)
     at org.apache.spark.repl.SparkIMain$Request.loadAndRun(SparkIMain.scala:1326)
     at org.apache.spark.repl.SparkIMain.loadAndRunReq$1(SparkIMain.scala:821)
     at org.apache.spark.repl.SparkIMain.interpret(SparkIMain.scala:852)
     at org.apache.spark.repl.SparkIMain.interpret(SparkIMain.scala:800)
     at org.apache.spark.repl.SparkILoop.reallyInterpret$1(SparkILoop.scala:857)
     at org.apache.spark.repl.SparkILoop.interpretStartingWith(SparkILoop.scala:902)
     at org.apache.spark.repl.SparkILoop.command(SparkILoop.scala:814)
     at org.apache.spark.repl.SparkILoop.processLine$1(SparkILoop.scala:657)
     at org.apache.spark.repl.SparkILoop.innerLoop$1(SparkILoop.scala:665)
     at org.apache.spark.repl.SparkILoop.org<http://org.apache.spark.repl.SparkILoop.org>$apache$spark$repl$SparkILoop$$loop(SparkILoop.scala:670)
     at org.apache.spark.repl.SparkILoop$$anonfun$org$apache$spark$repl$SparkILoop$$process$1.apply$mcZ$sp(SparkILoop.scala:997)
     at org.apache.spark.repl.SparkILoop$$anonfun$org$apache$spark$repl$SparkILoop$$process$1.apply(SparkILoop.scala:945)
     at org.apache.spark.repl.SparkILoop$$anonfun$org$apache$spark$repl$SparkILoop$$process$1.apply(SparkILoop.scala:945)
     at scala.tools.nsc.util.ScalaClassLoader$.savingContextLoader(ScalaClassLoader.scala:135)
     at org.apache.spark.repl.SparkILoop.org<http://org.apache.spark.repl.SparkILoop.org>$apache$spark$repl$SparkILoop$$process(SparkILoop.scala:945)
     at org.apache.spark.repl.SparkILoop.process(SparkILoop.scala:1064)
     at org.apache.spark.repl.Main$.main(Main.scala:31)
     at org.apache.spark.repl.Main.main(Main.scala)
     at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
     at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
     at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
     at java.lang.reflect.Method.invoke(Method.java:606)
     at org.apache.spark.deploy.SparkSubmit$.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:731)
     at org.apache.spark.deploy.SparkSubmit$.doRunMain$1(SparkSubmit.scala:181)
     at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:206)
     at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:121)
     at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)

I have tried the java sample, it worked well in java. Could you help with this issue? Or any
other ways to insert data in scala?
Please let me know if you need other information.

Best Regards,
Yanlong




*********************************************************************

Email Disclaimer

----------------

The information in this email is confidential and may be legally

privileged. It is intended solely for the addressee. Access to this

email by anyone else is unauthorised.

If you are not the intended recipient, any disclosure, copying,

distribution or any action taken or omitted to be taken in reliance

on it, is prohibited and may be unlawful.

Any opinions or advice contained in this email are subject to the

terms and conditions expressed in the governing SGM client

engagement letter and should not be relied upon unless they are

confirmed in writing on SGM's letterhead.

*********************************************************************






--
Todd Lipcon
Software Engineer, Cloudera
*********************************************************************
Email Disclaimer
----------------
The information in this email is confidential and may be legally 
privileged. It is intended solely for the addressee. Access to this 
email by anyone else is unauthorised.
If you are not the intended recipient, any disclosure, copying, 
distribution or any action taken or omitted to be taken in reliance 
on it, is prohibited and may be unlawful.
Any opinions or advice contained in this email are subject to the 
terms and conditions expressed in the governing SGM client 
engagement letter and should not be relied upon unless they are 
confirmed in writing on SGM's letterhead.
*********************************************************************
Mime
View raw message