asterixdb-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "abdullah alamoudi (Code Review)" <do-not-re...@asterix-gerrit.ics.uci.edu>
Subject Change in asterixdb[master]: Introduced Local Filesystem Feed Adapter
Date Thu, 03 Sep 2015 05:21:40 GMT
abdullah alamoudi has posted comments on this change.

Change subject: Introduced Local Filesystem Feed Adapter
......................................................................


Patch Set 3:

(2 comments)

That is correct. I have discussed this with Mike and Ian. file deletes are not of interest
to us and file modifications will most likely cause problems. The expected user behavior is
to specify a regular expression which will read files with names matching the expression.

For example the expression ".*\\.txt" will match files with the extension txt and so when
a user writes a new file, they are expected to create it with a name that doesn't end with
.txt and when it is written completely rename it to [filename].txt which will kick off the
ingestion.

https://asterix-gerrit.ics.uci.edu/#/c/376/3/asterix-app/data/local-dir/data/hello-world
File asterix-app/data/local-dir/data/hello-world:

Line 2: 
> I'm assuming the misspelling is for a purpose?
Haha. Can you believe I didn't notice. I will fix it (even though it doesn't matter for our
purpose)


https://asterix-gerrit.ics.uci.edu/#/c/376/3/asterix-external-data/src/main/java/org/apache/asterix/external/dataset/adapter/LocalFileSystemFeedAdapter.java
File asterix-external-data/src/main/java/org/apache/asterix/external/dataset/adapter/LocalFileSystemFeedAdapter.java:

Line 56:         // This needs to be fixed
> This comment isn't clear to me.
This needs to be fixed. The current stop mechanism is by flipping the boolean continue ingestion
doesn't actually stop the feed immediately but wait for one more record. What if that record
never came?!!

Right now, there is an easy but not so good looking way to do this by first disconnecting
the feed (which will call this method) and then add a dummy file with some text to cause the
parser to skip a record. and break out of the loop.

A better approach would be to have a different thread for waiting for new files and making
stop actually kill that thread and exit the loop waiting for events immediately.


-- 
To view, visit https://asterix-gerrit.ics.uci.edu/376
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I707756e3b4c9ffca4b55ec9817a08e5c16333010
Gerrit-PatchSet: 3
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: abdullah alamoudi <bamousaa@gmail.com>
Gerrit-Reviewer: Ian Maxon <imaxon@apache.org>
Gerrit-Reviewer: Jenkins <jenkins@fulliautomatix.ics.uci.edu>
Gerrit-Reviewer: Murtadha Hubail <hubailmor@gmail.com>
Gerrit-Reviewer: Steven Jacobs <sjaco002@ucr.edu>
Gerrit-Reviewer: abdullah alamoudi <bamousaa@gmail.com>
Gerrit-HasComments: Yes

Mime
View raw message