hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Geoffry Roberts <geoffry.robe...@gmail.com>
Subject Re: Question About Passing Properties to a Mapper
Date Wed, 11 May 2011 16:50:34 GMT

Thanks Harsh for your response.  I have, however, solved the problem and I
shall now share.

The upshot is the property in question, when I did the get() was in fact not
null.  It contained the full XML document.  I thought it was either null or
a zero length string because I was logging the value like this:

LOG.info("scenarioText=" + scenarioText);

When I viewed the log file I would see:

... scenarioText=

This made me think the value was null.  But here's the kicker.  I was using
my browser to view the log, drilling down from
http://qq000:50070/dfshealth.jsp.  When you do this... well it can be
misleading.  If I cat the file from the command line I see what I would
expect to see,  i.e. the text of my document.

This now makes two things about viewing log files in the browser that I wish
were different. (1) the aforementioned, and (2) the wrapping of the logged
output as if there were no new line characters in the file.  This has to be
something that Hadoop is doing because if you simply open the same file with
a browser, the file displays normally--no wrap lines and displayed xml.

On 11 May 2011 09:15, Harsh J <harsh@cloudera.com> wrote:

> Hello Geoffry,
> On Wed, May 11, 2011 at 8:40 PM, Geoffry Roberts
> <geoffry.roberts@gmail.com> wrote:
> > All,
> >
> > I am attempting to pass a string value from my driver to each one of my
> > mappers and it is not working.  I can set the value, but when I read it
> back
> > it returns null.  the value is not null when I set() it and I am using
> the
> > correct key when I attempt to get() it.  This should be a simple,
> straight
> > forward thing but it isn't working.
> >
> > I am, however, passing another string value successfully. The difference
> > seems to be that the value that passes successfully is short and simple
> > whereas the other, the failure is long and in fact is an xml document
> > containing new lines, et al.
> >
> > Question: What must I do to pass an xml document? url encode it? strip
> the
> > new lines? other?
> Encoding would be a good idea for arbitrary strings containing
> possibly bad XML chars., etc..
> (I prefer doing Base64 enc., that one works well with XML I think)
> --
> Harsh J

Geoffry Roberts

View raw message