incubator-blur-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Tim Williams <william...@gmail.com>
Subject Re: git commit: Let hdfs field manager ignore files he don't understand
Date Wed, 26 Feb 2014 01:53:21 GMT
It actually still does, because ultimately we do a tryToLoad() which
loads from storage based on the passed field name.  The best i've got
is to create a new one, loadfromstorage, the check
getFieldNames().contains(foo) - which fails.

On Tue, Feb 25, 2014 at 8:49 PM, Aaron McCurry <amccurry@gmail.com> wrote:
> To make it fail I believe that you will have to load a field with "type" in
> the beginning of the name.  Close the field manager and open a second one.
>  I think that the field with "type" as the prefix won't reload.
>
>
> On Tue, Feb 25, 2014 at 8:43 PM, Tim Williams <williamstw@gmail.com> wrote:
>
>> On Tue, Feb 25, 2014 at 7:55 PM, Tim Williams <williamstw@gmail.com>
>> wrote:
>> > On Tue, Feb 25, 2014 at 7:49 PM, Aaron McCurry <amccurry@gmail.com>
>> wrote:
>> >> @@ -116,7 +117,7 @@ public class HdfsFieldManager extends
>> BaseFieldManager {
>> >>        List<String> fieldNames = new ArrayList<String>();
>> >>        for (FileStatus fileStatus : listStatus) {
>> >>          if (!fileStatus.isDir()) {
>> >> -          fieldNames.add(fileStatus.getPath().getName());
>> >> +
>>  fieldNames.add(fileStatus.getPath().getName().replace(TYPE_FILE_EXT,
>> >> ""));
>> >>          }
>> >>        }
>> >>
>> >> Do you think that this will cause problems for fields that are <some
>> >> family>.type<any suffix> ?  Maybe we should check that the file
ends
>> with
>> >> the TYPE_FILE_EXT and substring up to the beginning of the suffix?
>> >
>> > Ahh, literally have 'type' as the prefix of their column... yeah,
>> > that'd cause problems, nice catch.  I'll get a test for that one...
>>
>> Interestingly enough, it actually still works just fine as is because
>> while we try to pull from the definitions map, we ultimately fall back
>> to storage anyway and since it's the correct fieldname coming in, it's
>> no problem.  I'll fix it, but I'm having a tough time crafting a test
>> that fails...
>>
>> --tim
>>

Mime
View raw message