incubator-blur-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Aaron McCurry <amccu...@gmail.com>
Subject Re: custom fields with m/r
Date Tue, 12 Nov 2013 16:04:56 GMT
https://git-wip-us.apache.org/repos/asf?p=incubator-blur.git;a=blob;f=blur-mapred/src/main/java/org/apache/blur/mapreduce/lib/BlurOutputFormat.java;h=caa350450918003175978924a9478533850eb7c3;hb=blur-console-v2#l397

I think for the MR job to pickup the custom fields you will need to run the
following in order.

TD td1 = new TD();
client.create(td1);
TD td2 = client.describe(td1.name);
// Then if you use td2 for your MR job you will get the system fields I
think.

Let me know.

Aaron





On Tue, Nov 12, 2013 at 10:54 AM, Tim Williams <williamstw@gmail.com> wrote:

> I don't see where the TableContext created in outputformat could have
> access to the cluster's configuration or jar files.  Maybe when the
> cluster is initialized it should write the configuration and jars to
> HDFS, making it available to the output format?  If so, should the
> writer be arranged with zk?  Or is there another plan I'm not seeing
> in there?
>
> Thanks,
> --tim
>
>
> On Tue, Nov 12, 2013 at 10:34 AM, Aaron McCurry <amccurry@gmail.com>
> wrote:
> > It should be supported, this seems like a bug.
> >
> > Aaron
> >
> >
> > On Tue, Nov 12, 2013 at 10:29 AM, Tim Williams <williamstw@gmail.com>
> wrote:
> >
> >> It seems to be the case that when building a table with the
> >> BlurOutputFormat, we need to follow the per-table custom field
> >> definitions vs. cluster-wide.  If there's no plan to support
> >> cluster-wide custom defs with m/r we should just document the
> >> per-table version I reckon, thoughts?
> >>
> >> Thanks,
> >> --tim
> >>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message