hadoop-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From KayVajj <vajjalak...@gmail.com>
Subject Fwd: Sqoop Codegen Null String
Date Wed, 29 Jul 2015 17:37:02 GMT
Sorry the earlier email I sent didn't show up while I searched for it so
resending.

Hi,

I have a question with the Sqoop CodeGen. I'm trying to load data from a
DB. I have used the codgen tool to generate the java code. I wanted to
treat the null-strings and null-non-strings as

--null-string '\\N'
--null-non-string '\\N'


Now the code that is generated looks like (The below code is excerpt from
__loadFromFields method in the generated code)

 __cur_str = __it.next();
if (__cur_str.equals("null")) { this.org_id = null; } else {
  this.org_id = __cur_str;
}

I was wondering even with the input options specifically provided it still
treats string "null" as the null string as if I did not provide. Then after
some code browsing, I saw the below code in org.apache.sqoop.orm.ClassWriter

private void parseNullVal(String javaType, String colName, StringBuilder
sb) {
    if (javaType.equals("String")) {
     sb.append("    if (__cur_str.equals(\""
         + this.options.getInNullStringValue() + "\")) { this.");
      sb.append(colName);
      sb.append(" = null; } else {\n");
    } else {
      sb.append("    if (__cur_str.equals(\""
         + this.options.getInNullNonStringValue());
      sb.append("\") || __cur_str.length() == 0) { this.");
      sb.append(colName);
      sb.append(" = null; } else {\n");
    }
  }

This tells me that the loadFromFields will be correct if I set the below
options

--input-null-string '\\N'
--input-null-non-string '\\N'

My understanding is these values are to be set only if we are writing to
the DB and not while reading. I'm not writing to the DB yet I ended up
setting both set of options which resulted in the below code in
__loadFromFields method in the new generated code

   __cur_str = __it.next();
    if (__cur_str.equals("\\N")) { this.org_id = null; } else {
      this.org_id = __cur_str;
    }

Is this a bug?

Thanks

Kay

Mime
View raw message