incubator-blur-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Aaron McCurry <amccu...@gmail.com>
Subject Re: git commit: Let hdfs field manager ignore files he don't understand
Date Wed, 26 Feb 2014 01:49:36 GMT
To make it fail I believe that you will have to load a field with "type" in
the beginning of the name.  Close the field manager and open a second one.
 I think that the field with "type" as the prefix won't reload.


On Tue, Feb 25, 2014 at 8:43 PM, Tim Williams <williamstw@gmail.com> wrote:

> On Tue, Feb 25, 2014 at 7:55 PM, Tim Williams <williamstw@gmail.com>
> wrote:
> > On Tue, Feb 25, 2014 at 7:49 PM, Aaron McCurry <amccurry@gmail.com>
> wrote:
> >> @@ -116,7 +117,7 @@ public class HdfsFieldManager extends
> BaseFieldManager {
> >>        List<String> fieldNames = new ArrayList<String>();
> >>        for (FileStatus fileStatus : listStatus) {
> >>          if (!fileStatus.isDir()) {
> >> -          fieldNames.add(fileStatus.getPath().getName());
> >> +
>  fieldNames.add(fileStatus.getPath().getName().replace(TYPE_FILE_EXT,
> >> ""));
> >>          }
> >>        }
> >>
> >> Do you think that this will cause problems for fields that are <some
> >> family>.type<any suffix> ?  Maybe we should check that the file ends
> with
> >> the TYPE_FILE_EXT and substring up to the beginning of the suffix?
> >
> > Ahh, literally have 'type' as the prefix of their column... yeah,
> > that'd cause problems, nice catch.  I'll get a test for that one...
>
> Interestingly enough, it actually still works just fine as is because
> while we try to pull from the definitions map, we ultimately fall back
> to storage anyway and since it's the correct fieldname coming in, it's
> no problem.  I'll fix it, but I'm having a tough time crafting a test
> that fails...
>
> --tim
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message