hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Aaron Kimball <aa...@cloudera.com>
Subject Re: How do I convert DataInput and ResultSet to array of String?
Date Wed, 03 Jun 2009 23:01:36 GMT
The text serializer will pull out an entire string by using a null
terminator at the end.

If you need to know the number of string objects, though, you'll have to
serialize that before the strings, then use a for loop to decode the rest of
them.
- Aaron

On Tue, Jun 2, 2009 at 6:01 PM, dealmaker <vinkhc@gmail.com> wrote:

>
> Thanks.  The number of elements in this array of String is unknown until
> run
> time.  If datainput treats it as a byte array, I still have to know the
> size
> of each String.  How do I do that?   Would you suggest some code samples or
> links that deal with similar situation like this?  The only examples I got
> are the ones about counting number of words which deal with integers.
> Thanks.
>
>
> Aaron Kimball-3 wrote:
> >
> > Hi,
> >
> > You can't just turn either of these two types into arrays of strings
> > automatically, because they are interfaces to underlying streams of data.
> > You are required to know what protocol you are implementing -- i.e., how
> > many fields you are transmitting -- and manually read through that many
> > fields yourself. For example, a DataInput object is effectively a pointer
> > into a byte array. There may be many records in that byte array, but you
> > only want to read the fields of the first record out.
> >
> > For DataInput / DataOutput, you can UTF8-decode the next field by calling
> > Text.readString(dataInput) and Text.writeString(dataOutput).
> > For ResultSet, you want resultSet.getString(fieldNum)
> >
> > As (yet another) shameless plug ( :smile: ), check out the tool we just
> > released, which automates database import tasks. It auto-generates the
> > classes necessary for your tables, too.
> > http://www.cloudera.com/blog/2009/06/01/introducing-sqoop/
> >
> > At the very least, you might want to play with it a bit and read its
> > source
> > code so you have a better idea of how to implement your own class (since
> > you're doing some more creative stuff like building up associative arrays
> > for each field).
> >
> > Cheers,
> > - Aaron
> >
> > On Mon, Jun 1, 2009 at 9:53 PM, dealmaker <vinkhc@gmail.com> wrote:
> >
> >>
> >> bump.  Does anyone know?
> >>
> >> I am using the following class of arraywritable:
> >>
> >>  private static class StringArrayWritable extends ArrayWritable {
> >>    private StringArrayWritable (String [] aSString) {
> >>      super (aSString);
> >>     }
> >>  }
> >>
> >>
> >> dealmaker wrote:
> >> >
> >> > Hi,
> >> >   How do I convert DataInput to array of String?
> >> >   How do I convert ResultSet to array of String?
> >> > Thanks.  Following is the code:
> >> >
> >> >   static class Record implements Writable, DBWritable {
> >> >     String [] aSAssoc;
> >> >
> >> >     public void write(DataOutput arg0) throws IOException {
> >> >       throw new UnsupportedOperationException("Not supported yet.");
> >> >     }
> >> >
> >> >     public void readFields(DataInput in) throws IOException {
> >> >       this.aSAssoc = // How to convert DataInput to String Array?
> >> >     }
> >> >
> >> >     public void write(PreparedStatement arg0) throws SQLException {
> >> >       throw new UnsupportedOperationException("Not supported yet.");
> >> >     }
> >> >
> >> >     public void readFields(ResultSet rs) throws SQLException {
> >> >       this.aSAssoc = // How to convert ResultSet to String Array?
> >> >     }
> >> >   }
> >> >
> >> >
> >>
> >> --
> >> View this message in context:
> >>
> http://www.nabble.com/How-do-I-convert-DataInput-and-ResultSet-to-array-of-String--tp23770747p23826464.html
> >> Sent from the Hadoop core-user mailing list archive at Nabble.com.
> >>
> >>
> >
> >
>
> --
> View this message in context:
> http://www.nabble.com/How-do-I-convert-DataInput-and-ResultSet-to-array-of-String--tp23770747p23843679.html
> Sent from the Hadoop core-user mailing list archive at Nabble.com.
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message