asterixdb-notifications mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "abdullah alamoudi (Code Review)" <do-not-re...@asterixdb.incubator.apache.org>
Subject Change in asterixdb[master]: Improve Error Handling in Feeds
Date Thu, 17 Mar 2016 06:17:47 GMT
Hello Jenkins,

I'd like you to reexamine a change.  Please visit

    https://asterix-gerrit.ics.uci.edu/720

to look at the new patch set (#2).

Change subject: Improve Error Handling in Feeds
......................................................................

Improve Error Handling in Feeds

This change separate parse exceptions from IOExceptions and
handle each differently. The responsibility of handling
IOExceptions falls on the datasource reader while records which
that couldn't be parsed end up added to the records log file.

In addition, some refactoring was performed on external data
to remove the stream provider layer of it. This will make
it consistent with the record reader.

Change-Id: I08d89229e33c91532b1038ba9f7a372f7ca1fdb5
---
M .gitattributes
M .gitignore
R asterix-external-data/src/main/java/org/apache/asterix/external/api/AsterixInputStream.java
M asterix-external-data/src/main/java/org/apache/asterix/external/api/IExternalIndexer.java
M asterix-external-data/src/main/java/org/apache/asterix/external/api/IIndexingDatasource.java
R asterix-external-data/src/main/java/org/apache/asterix/external/api/IInputStreamFactory.java
D asterix-external-data/src/main/java/org/apache/asterix/external/api/IInputStreamProvider.java
M asterix-external-data/src/main/java/org/apache/asterix/external/api/IRecordReader.java
M asterix-external-data/src/main/java/org/apache/asterix/external/dataflow/FeedRecordDataFlowController.java
M asterix-external-data/src/main/java/org/apache/asterix/external/dataflow/FeedStreamDataFlowController.java
M asterix-external-data/src/main/java/org/apache/asterix/external/dataflow/FeedTupleForwarder.java
M asterix-external-data/src/main/java/org/apache/asterix/external/indexing/FileOffsetIndexer.java
M asterix-external-data/src/main/java/org/apache/asterix/external/indexing/RecordColumnarIndexer.java
M asterix-external-data/src/main/java/org/apache/asterix/external/input/HDFSDataSourceFactory.java
M asterix-external-data/src/main/java/org/apache/asterix/external/input/record/reader/TestAsterixMembersReader.java
M asterix-external-data/src/main/java/org/apache/asterix/external/input/record/reader/hdfs/HDFSRecordReader.java
M asterix-external-data/src/main/java/org/apache/asterix/external/input/record/reader/kv/KVReader.java
M asterix-external-data/src/main/java/org/apache/asterix/external/input/record/reader/kv/KVTestReader.java
M asterix-external-data/src/main/java/org/apache/asterix/external/input/record/reader/rss/RSSRecordReader.java
M asterix-external-data/src/main/java/org/apache/asterix/external/input/record/reader/stream/AbstractStreamRecordReader.java
M asterix-external-data/src/main/java/org/apache/asterix/external/input/record/reader/stream/AbstractStreamRecordReaderFactory.java
M asterix-external-data/src/main/java/org/apache/asterix/external/input/record/reader/stream/EmptyLineSeparatedRecordReader.java
M asterix-external-data/src/main/java/org/apache/asterix/external/input/record/reader/stream/EmptyLineSeparatedRecordReaderFactory.java
M asterix-external-data/src/main/java/org/apache/asterix/external/input/record/reader/stream/LineRecordReader.java
M asterix-external-data/src/main/java/org/apache/asterix/external/input/record/reader/stream/LineRecordReaderFactory.java
M asterix-external-data/src/main/java/org/apache/asterix/external/input/record/reader/stream/QuotedLineRecordReader.java
M asterix-external-data/src/main/java/org/apache/asterix/external/input/record/reader/stream/SemiStructuredRecordReader.java
M asterix-external-data/src/main/java/org/apache/asterix/external/input/record/reader/stream/SemiStructuredRecordReaderFactory.java
M asterix-external-data/src/main/java/org/apache/asterix/external/input/record/reader/twitter/TwitterPullRecordReader.java
M asterix-external-data/src/main/java/org/apache/asterix/external/input/record/reader/twitter/TwitterPushRecordReader.java
R asterix-external-data/src/main/java/org/apache/asterix/external/input/stream/AsterixInputStreamReader.java
M asterix-external-data/src/main/java/org/apache/asterix/external/input/stream/BasicInputStream.java
A asterix-external-data/src/main/java/org/apache/asterix/external/input/stream/HDFSInputStream.java
R asterix-external-data/src/main/java/org/apache/asterix/external/input/stream/LocalFSInputStream.java
A asterix-external-data/src/main/java/org/apache/asterix/external/input/stream/SocketClientInputStream.java
M asterix-external-data/src/main/java/org/apache/asterix/external/input/stream/SocketServerInputStream.java
A asterix-external-data/src/main/java/org/apache/asterix/external/input/stream/TwitterFirehoseInputStream.java
R asterix-external-data/src/main/java/org/apache/asterix/external/input/stream/factory/LocalFSInputStreamFactory.java
R asterix-external-data/src/main/java/org/apache/asterix/external/input/stream/factory/SocketClientInputStreamFactory.java
R asterix-external-data/src/main/java/org/apache/asterix/external/input/stream/factory/SocketServerInputStreamFactory.java
R asterix-external-data/src/main/java/org/apache/asterix/external/input/stream/factory/TwitterFirehoseStreamFactory.java
D asterix-external-data/src/main/java/org/apache/asterix/external/input/stream/provider/HDFSInputStreamProvider.java
D asterix-external-data/src/main/java/org/apache/asterix/external/input/stream/provider/LocalFSInputStreamProvider.java
D asterix-external-data/src/main/java/org/apache/asterix/external/input/stream/provider/SocketClientInputStreamProvider.java
D asterix-external-data/src/main/java/org/apache/asterix/external/input/stream/provider/SocketServerInputStreamProvider.java
D asterix-external-data/src/main/java/org/apache/asterix/external/input/stream/provider/TwitterFirehoseInputStreamProvider.java
M asterix-external-data/src/main/java/org/apache/asterix/external/provider/DataflowControllerProvider.java
M asterix-external-data/src/main/java/org/apache/asterix/external/provider/DatasourceFactoryProvider.java
M asterix-external-data/src/main/java/org/apache/asterix/external/util/ExternalDataConstants.java
M asterix-external-data/src/main/java/org/apache/asterix/external/util/ExternalDataUtils.java
M asterix-external-data/src/main/java/org/apache/asterix/external/util/FileSystemWatcher.java
M asterix-external-data/src/main/java/org/apache/asterix/external/util/HDFSUtils.java
M asterix-external-data/src/test/java/org/apache/asterix/external/classad/test/ClassAdToADMTest.java
M asterix-external-data/src/test/java/org/apache/asterix/external/library/ClassAdParser.java
M asterix-external-data/src/test/java/org/apache/asterix/external/parser/test/RecordWithMetaTest.java
55 files changed, 845 insertions(+), 793 deletions(-)


  git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb refs/changes/20/720/2
-- 
To view, visit https://asterix-gerrit.ics.uci.edu/720
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I08d89229e33c91532b1038ba9f7a372f7ca1fdb5
Gerrit-PatchSet: 2
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: abdullah alamoudi <bamousaa@gmail.com>
Gerrit-Reviewer: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Mime
View raw message