asterixdb-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From abdullah alamoudi <bamou...@gmail.com>
Subject Re: inotify in HDFS for external indexing?
Date Tue, 16 Jun 2015 05:09:41 GMT
This can definitely be used to automate updates of external indexes. It is
however only available in 2.6 and above versions of HDFS.

Having it will make external indexes much more attractive since the user
will not need to worry about stale indexes. Will work on this once I am
done with things on my stack.

Abdullah.

On Tue, Jun 16, 2015 at 6:51 AM, abdullah alamoudi <bamousaa@gmail.com>
wrote:

> I will look into that :)
>
> On Tue, Jun 16, 2015 at 5:05 AM, Ted Dunning <ted.dunning@gmail.com>
> wrote:
>
>> Note that audit streams are often used for this as well.  These are
>> available for HDFS and MapR FS (at least).
>>
>> Also, for MapR DB at least, you can get full changed-data-records as a
>> call
>> back into your code even remote from the table.
>>
>>
>>
>> On Mon, Jun 15, 2015 at 5:40 PM, Mike Carey <dtabass@gmail.com> wrote:
>>
>> > Ooooohhhh.....  Cool!
>> > On Jun 15, 2015 5:31 PM, "Ian Maxon" <imaxon@uci.edu> wrote:
>> >
>> > > Stumbled across this new feature in HDFS:
>> > > https://issues.apache.org/jira/browse/HDFS-6634. Has anyone else seen
>> > > this? Basically it would let you be notified whenever files are
>> > > changed in HDFS, similar (but not exactly like) inotify. This sounds
>> > > like it could let us update externally indexed data on HDFS
>> > > incrementally doesn't it?...
>> > >
>> > > Just a random thought.
>> > >
>> > > -Ian
>> > >
>> >
>>
>
>
>
> --
> Amoudi, Abdullah.
>



-- 
Amoudi, Abdullah.

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message