incubator-blur-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Tim Williams <william...@gmail.com>
Subject Re: custom fields with m/r
Date Tue, 12 Nov 2013 17:35:11 GMT
No joy.  I don't see how even that would work unless we're expecting
the system properties to be merged with the tabledescriptor.properties
when the tabledescriptor is returned?

To check that, I changed TableAdmin.describe locally to merge in the
system properties and it works fine.  I'm not sure if that's how you
were thinking it should behave or not though?  Thoughts?

Thanks,
--tim

On Tue, Nov 12, 2013 at 11:04 AM, Aaron McCurry <amccurry@gmail.com> wrote:
> https://git-wip-us.apache.org/repos/asf?p=incubator-blur.git;a=blob;f=blur-mapred/src/main/java/org/apache/blur/mapreduce/lib/BlurOutputFormat.java;h=caa350450918003175978924a9478533850eb7c3;hb=blur-console-v2#l397
>
> I think for the MR job to pickup the custom fields you will need to run the
> following in order.
>
> TD td1 = new TD();
> client.create(td1);
> TD td2 = client.describe(td1.name);
> // Then if you use td2 for your MR job you will get the system fields I
> think.
>
> Let me know.
>
> Aaron
>
>
>
>
>
> On Tue, Nov 12, 2013 at 10:54 AM, Tim Williams <williamstw@gmail.com> wrote:
>
>> I don't see where the TableContext created in outputformat could have
>> access to the cluster's configuration or jar files.  Maybe when the
>> cluster is initialized it should write the configuration and jars to
>> HDFS, making it available to the output format?  If so, should the
>> writer be arranged with zk?  Or is there another plan I'm not seeing
>> in there?
>>
>> Thanks,
>> --tim
>>
>>
>> On Tue, Nov 12, 2013 at 10:34 AM, Aaron McCurry <amccurry@gmail.com>
>> wrote:
>> > It should be supported, this seems like a bug.
>> >
>> > Aaron
>> >
>> >
>> > On Tue, Nov 12, 2013 at 10:29 AM, Tim Williams <williamstw@gmail.com>
>> wrote:
>> >
>> >> It seems to be the case that when building a table with the
>> >> BlurOutputFormat, we need to follow the per-table custom field
>> >> definitions vs. cluster-wide.  If there's no plan to support
>> >> cluster-wide custom defs with m/r we should just document the
>> >> per-table version I reckon, thoughts?
>> >>
>> >> Thanks,
>> >> --tim
>> >>
>>

Mime
View raw message