hadoop-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From KayVajj <vajjalak...@gmail.com>
Subject Fwd: Sqoop Codegen Null String
Date Wed, 29 Jul 2015 17:37:02 GMT
Sorry the earlier email I sent didn't show up while I searched for it so


I have a question with the Sqoop CodeGen. I'm trying to load data from a
DB. I have used the codgen tool to generate the java code. I wanted to
treat the null-strings and null-non-strings as

--null-string '\\N'
--null-non-string '\\N'

Now the code that is generated looks like (The below code is excerpt from
__loadFromFields method in the generated code)

 __cur_str = __it.next();
if (__cur_str.equals("null")) { this.org_id = null; } else {
  this.org_id = __cur_str;

I was wondering even with the input options specifically provided it still
treats string "null" as the null string as if I did not provide. Then after
some code browsing, I saw the below code in org.apache.sqoop.orm.ClassWriter

private void parseNullVal(String javaType, String colName, StringBuilder
sb) {
    if (javaType.equals("String")) {
     sb.append("    if (__cur_str.equals(\""
         + this.options.getInNullStringValue() + "\")) { this.");
      sb.append(" = null; } else {\n");
    } else {
      sb.append("    if (__cur_str.equals(\""
         + this.options.getInNullNonStringValue());
      sb.append("\") || __cur_str.length() == 0) { this.");
      sb.append(" = null; } else {\n");

This tells me that the loadFromFields will be correct if I set the below

--input-null-string '\\N'
--input-null-non-string '\\N'

My understanding is these values are to be set only if we are writing to
the DB and not while reading. I'm not writing to the DB yet I ended up
setting both set of options which resulted in the below code in
__loadFromFields method in the new generated code

   __cur_str = __it.next();
    if (__cur_str.equals("\\N")) { this.org_id = null; } else {
      this.org_id = __cur_str;

Is this a bug?



View raw message