incubator-blur-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From rahul challapalli <challapallira...@gmail.com>
Subject Re: custom fields with m/r
Date Tue, 12 Nov 2013 17:41:45 GMT
If my memory is correct it should actually merge the system properties with
td.properties before returning td
On Nov 12, 2013 9:36 AM, "Tim Williams" <williamstw@gmail.com> wrote:

> No joy.  I don't see how even that would work unless we're expecting
> the system properties to be merged with the tabledescriptor.properties
> when the tabledescriptor is returned?
>
> To check that, I changed TableAdmin.describe locally to merge in the
> system properties and it works fine.  I'm not sure if that's how you
> were thinking it should behave or not though?  Thoughts?
>
> Thanks,
> --tim
>
> On Tue, Nov 12, 2013 at 11:04 AM, Aaron McCurry <amccurry@gmail.com>
> wrote:
> >
> https://git-wip-us.apache.org/repos/asf?p=incubator-blur.git;a=blob;f=blur-mapred/src/main/java/org/apache/blur/mapreduce/lib/BlurOutputFormat.java;h=caa350450918003175978924a9478533850eb7c3;hb=blur-console-v2#l397
> >
> > I think for the MR job to pickup the custom fields you will need to run
> the
> > following in order.
> >
> > TD td1 = new TD();
> > client.create(td1);
> > TD td2 = client.describe(td1.name);
> > // Then if you use td2 for your MR job you will get the system fields I
> > think.
> >
> > Let me know.
> >
> > Aaron
> >
> >
> >
> >
> >
> > On Tue, Nov 12, 2013 at 10:54 AM, Tim Williams <williamstw@gmail.com>
> wrote:
> >
> >> I don't see where the TableContext created in outputformat could have
> >> access to the cluster's configuration or jar files.  Maybe when the
> >> cluster is initialized it should write the configuration and jars to
> >> HDFS, making it available to the output format?  If so, should the
> >> writer be arranged with zk?  Or is there another plan I'm not seeing
> >> in there?
> >>
> >> Thanks,
> >> --tim
> >>
> >>
> >> On Tue, Nov 12, 2013 at 10:34 AM, Aaron McCurry <amccurry@gmail.com>
> >> wrote:
> >> > It should be supported, this seems like a bug.
> >> >
> >> > Aaron
> >> >
> >> >
> >> > On Tue, Nov 12, 2013 at 10:29 AM, Tim Williams <williamstw@gmail.com>
> >> wrote:
> >> >
> >> >> It seems to be the case that when building a table with the
> >> >> BlurOutputFormat, we need to follow the per-table custom field
> >> >> definitions vs. cluster-wide.  If there's no plan to support
> >> >> cluster-wide custom defs with m/r we should just document the
> >> >> per-table version I reckon, thoughts?
> >> >>
> >> >> Thanks,
> >> >> --tim
> >> >>
> >>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message