hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From 夏明超 <mingchao.xi...@alibaba-inc.com>
Subject specific character just as "\001" couldn't be solved in Configuration
Date Fri, 15 Jul 2011 11:04:40 GMT
Hi
         My  map job must split the record, so I just set the attribute “splitor” as the
bytes “\001\002” , just like these statements:

StringBuffer ret=new StringBuffer();
         ret.append((char)Integer.parseInt("001")).toString();


         while it seems that the map/reduce couldn’t resolve and store these characters.
When starting the job ,it goes with this issue

11/07/15 10:56:30 WARN mapred.JobClient: Use GenericOptionsParser for parsing the arguments.
Applications should implement Tool for the same.
11/07/15 10:56:30 INFO lzo.GPLNativeCodeLoader: Loaded native gpl library
11/07/15 10:56:30 INFO lzo.LzoCodec: Successfully loaded & initialized native-lzo library
[hadoop-lzo rev 927df213a90620c11333f9e254cb6816d5dc2b99]
11/07/15 10:56:30 INFO mapred.FileInputFormat: Total input paths to process : 2
11/07/15 10:56:30 INFO mapred.JobClient: Cleaning up the staging area hdfs://qabst146253:9857/tmp/hadoop-
hbase/mapred/staging/hbase/.staging/job_201107120947_0013
Exception in thread "main" org.apache.hadoop.ipc.RemoteException: java.io.IOException: java.lang.RuntimeException:
org.xml.sax.SAXParseException: Character
reference "&#1" is an invalid XML character.
        at org.apache.hadoop.mapred.JobTracker.submitJob(JobTracker.java:3759)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
        at java.lang.reflect.Method.invoke(Method.java:597)
        at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:557)
        at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1415)
        at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1411)
        at java.security.AccessController.doPrivileged(Native Method)
        at javax.security.auth.Subject.doAs(Subject.java:396)
        at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1115)
        at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1409)
Caused by: java.lang.RuntimeException: org.xml.sax.SAXParseException: Character reference
"&#1" is an invalid XML character.
        at org.apache.hadoop.conf.Configuration.loadResource(Configuration.java:1393)
        at org.apache.hadoop.conf.Configuration.loadResources(Configuration.java:1261)
        at org.apache.hadoop.conf.Configuration.getProps(Configuration.java:1192)
        at org.apache.hadoop.conf.Configuration.get(Configuration.java:415)
        at org.apache.hadoop.mapred.JobConf.checkAndWarnDeprecation(JobConf.java:1957)
        at org.apache.hadoop.mapred.JobConf.<init>(JobConf.java:386)
        at org.apache.hadoop.mapred.JobInProgress.<init>(JobInProgress.java:413)
        at org.apache.hadoop.mapred.JobTracker.submitJob(JobTracker.java:3757)
        ... 11 more
Caused by: org.xml.sax.SAXParseException: Character reference "&#1" is an invalid XML
character.
        at com.sun.org.apache.xerces.internal.parsers.DOMParser.parse(DOMParser.java:249)
        at com.sun.org.apache.xerces.internal.jaxp.DocumentBuilderImpl.parse(DocumentBuilderImpl.java:284)
        at javax.xml.parsers.DocumentBuilder.parse(DocumentBuilder.java:124)
        at org.apache.hadoop.conf.Configuration.loadResource(Configuration.java:1313)
        ... 18 more

        at org.apache.hadoop.ipc.Client.call(Client.java:1104)
        at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:226)
        at org.apache.hadoop.mapred.$Proxy1.submitJob(Unknown Source)
        at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:904)
        at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:833)
        at java.security.AccessController.doPrivileged(Native Method)
        at javax.security.auth.Subject.doAs(Subject.java:396)
        at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1115)
        at org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:833)
        at org.apache.hadoop.mapred.JobClient.submitJob(JobClient.java:807)
        at org.apache.hadoop.mapred.JobClient.runJob(JobClient.java:1242)
        at com.alibaba.idg.dumpdiff.dumpValidate.main(dumpValidate.java:454)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
        at java.lang.reflect.Method.invoke(Method.java:597)
        at org.apache.hadoop.util.RunJar.main(RunJar.java:186)

________________________________
This email (including any attachments) is confidential and may be legally privileged. If you
received this email in error, please delete it immediately and do not copy it or use it for
any purpose or disclose its contents to any other person. Thank you.

本电邮(包括任何附件)可能含有机密资料并受法律保护。如您不是正确的收件人,请您立即删除本邮件。请不要将本电邮进行复制并用作任何其他用途、或透露本邮件之内容。谢谢。
Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message