hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Devaraj Das" <d...@yahoo-inc.com>
Subject RE: Counters giving double values
Date Wed, 16 Apr 2008 10:52:42 GMT
Thanks for the detailed answer. Which hadoop version are you on? If you are
confident that it is not a problem with your app, pls raise a jira.


  _____  

From: rude@rosa.com [mailto:rude@rosa.com] 
Sent: Wednesday, April 16, 2008 3:25 PM
To: core-user@hadoop.apache.org
Subject: RE: Counters giving double values



Thanks so far. 

key and values are custom implementations. 

key implements WritableComparable 
value extends VersionedWritable 

btw. The only problem i encounter is that the Counter values are wrong. If i
check the records in the MapFile (re-read it) which is written as the output
of the mapred job, the amount of records is correct and represents the halve
of the reported counter value. 
the same applies for the results of the operations which are carried out in
the reduce(). everything is correct, except the counter values. 

the whole thing happens only sometimes. 


Key serializing / Deserializing, i guess you want to see this part of the
code: 

        public int language; 
        public String term; 

        public void readFields(DataInput in) throws IOException { 
                language = in.readInt(); 
                term =         Text.readString(in).toString(); 
        } 

        public void write(DataOutput out) throws IOException { 
                out.writeInt(language); 
                Text.writeString(out, term); 
        } 





Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message