Hello Jenkins,
I'd like you to reexamine a change. Please visit
https://asterix-gerrit.ics.uci.edu/585
to look at the new patch set (#6).
Change subject: Add flush() to IFrameWriter
......................................................................
Add flush() to IFrameWriter
This method is expected to be used with feeds to push
frames all the way to storage when needed. As of now, it is
needed in two cases:
1. No activities in ingestion node and need to push content
so it can be stored.
2. When the ingestion node needs to move the checkpoint ahead
if the at least once semantics are used.
Two feeds make use of this function. The filesystem feed and
couchbase feed which was introduced as well in this change.
Change-Id: Id862ce9e9b1360864c6976f2aea2137092f51203
---
M asterix-algebra/src/main/java/org/apache/asterix/algebra/operators/physical/CommitRuntime.java
M asterix-app/src/main/java/org/apache/asterix/feed/FeedMessageReceiver.java
M asterix-app/src/main/java/org/apache/asterix/hyracks/bootstrap/NCApplicationEntryPoint.java
M asterix-common/src/main/java/org/apache/asterix/common/config/AsterixPropertiesAccessor.java
M asterix-external-data/pom.xml
M asterix-external-data/src/main/java/org/apache/asterix/external/adapter/factory/LookupAdapterFactory.java
M asterix-external-data/src/main/java/org/apache/asterix/external/api/IDataParser.java
M asterix-external-data/src/main/java/org/apache/asterix/external/api/IRawRecord.java
M asterix-external-data/src/main/java/org/apache/asterix/external/api/IRecordReader.java
M asterix-external-data/src/main/java/org/apache/asterix/external/dataflow/AbstractFeedDataFlowController.java
M asterix-external-data/src/main/java/org/apache/asterix/external/dataflow/FeedRecordDataFlowController.java
M asterix-external-data/src/main/java/org/apache/asterix/external/dataflow/FeedTupleForwarder.java
M asterix-external-data/src/main/java/org/apache/asterix/external/dataset/adapter/LookupAdapter.java
M asterix-external-data/src/main/java/org/apache/asterix/external/feed/dataflow/CollectTransformFeedFrameWriter.java
M asterix-external-data/src/main/java/org/apache/asterix/external/feed/dataflow/DistributeFeedFrameWriter.java
M asterix-external-data/src/main/java/org/apache/asterix/external/feed/dataflow/FeedFrameCache.java
M asterix-external-data/src/main/java/org/apache/asterix/external/feed/dataflow/FeedFrameCollector.java
M asterix-external-data/src/main/java/org/apache/asterix/external/feed/dataflow/FeedFrameHandlers.java
M asterix-external-data/src/main/java/org/apache/asterix/external/feed/dataflow/FeedRuntimeInputHandler.java
M asterix-external-data/src/main/java/org/apache/asterix/external/feed/dataflow/FrameDistributor.java
M asterix-external-data/src/main/java/org/apache/asterix/external/feed/message/MessageReceiver.java
M asterix-external-data/src/main/java/org/apache/asterix/external/feed/watch/MonitoredBuffer.java
M asterix-external-data/src/main/java/org/apache/asterix/external/feed/watch/StorageSideMonitoredBuffer.java
M asterix-external-data/src/main/java/org/apache/asterix/external/input/HDFSDataSourceFactory.java
M asterix-external-data/src/main/java/org/apache/asterix/external/input/record/CharArrayRecord.java
M asterix-external-data/src/main/java/org/apache/asterix/external/input/record/GenericRecord.java
A asterix-external-data/src/main/java/org/apache/asterix/external/input/record/RecordWithMeta.java
M asterix-external-data/src/main/java/org/apache/asterix/external/input/record/reader/AbstractCharRecordLookupReader.java
M asterix-external-data/src/main/java/org/apache/asterix/external/input/record/reader/AbstractStreamRecordReader.java
A asterix-external-data/src/main/java/org/apache/asterix/external/input/record/reader/CouchbaseReader.java
M asterix-external-data/src/main/java/org/apache/asterix/external/input/record/reader/HDFSRecordReader.java
M asterix-external-data/src/main/java/org/apache/asterix/external/input/record/reader/RSSRecordReader.java
M asterix-external-data/src/main/java/org/apache/asterix/external/input/record/reader/TwitterPullRecordReader.java
M asterix-external-data/src/main/java/org/apache/asterix/external/input/record/reader/TwitterPushRecordReader.java
R asterix-external-data/src/main/java/org/apache/asterix/external/input/record/reader/factory/AbstractStreamRecordReaderFactory.java
A asterix-external-data/src/main/java/org/apache/asterix/external/input/record/reader/factory/CouchbaseReaderFactory.java
M asterix-external-data/src/main/java/org/apache/asterix/external/input/record/reader/factory/LineRecordReaderFactory.java
M asterix-external-data/src/main/java/org/apache/asterix/external/input/record/reader/factory/SemiStructuredRecordReaderFactory.java
M asterix-external-data/src/main/java/org/apache/asterix/external/input/stream/AInputStream.java
M asterix-external-data/src/main/java/org/apache/asterix/external/input/stream/AInputStreamReader.java
M asterix-external-data/src/main/java/org/apache/asterix/external/input/stream/LocalFileSystemInputStream.java
M asterix-external-data/src/main/java/org/apache/asterix/external/input/stream/factory/LocalFSInputStreamProviderFactory.java
M asterix-external-data/src/main/java/org/apache/asterix/external/input/stream/factory/SocketInputStreamProviderFactory.java
R asterix-external-data/src/main/java/org/apache/asterix/external/input/stream/provider/HDFSInputStreamProvider.java
R asterix-external-data/src/main/java/org/apache/asterix/external/input/stream/provider/LocalFSInputStreamProvider.java
R asterix-external-data/src/main/java/org/apache/asterix/external/input/stream/provider/SocketInputStreamProvider.java
R asterix-external-data/src/main/java/org/apache/asterix/external/input/stream/provider/TwitterFirehoseInputStreamProvider.java
M asterix-external-data/src/main/java/org/apache/asterix/external/operators/ExternalLookupOperatorDescriptor.java
M asterix-external-data/src/main/java/org/apache/asterix/external/operators/FeedMetaComputeNodePushable.java
M asterix-external-data/src/main/java/org/apache/asterix/external/operators/FeedMetaNodePushable.java
M asterix-external-data/src/main/java/org/apache/asterix/external/operators/FeedMetaStoreNodePushable.java
M asterix-external-data/src/main/java/org/apache/asterix/external/parser/ADMDataParser.java
M asterix-external-data/src/main/java/org/apache/asterix/external/parser/DelimitedDataParser.java
A asterix-external-data/src/main/java/org/apache/asterix/external/parser/RecordWithMetaParser.java
A asterix-external-data/src/main/java/org/apache/asterix/external/parser/factory/RecordWithMetaParserFactory.java
M asterix-external-data/src/main/java/org/apache/asterix/external/provider/DatasourceFactoryProvider.java
R asterix-external-data/src/main/java/org/apache/asterix/external/provider/LookupReaderFactoryProvider.java
M asterix-external-data/src/main/java/org/apache/asterix/external/provider/ParserFactoryProvider.java
M asterix-external-data/src/main/java/org/apache/asterix/external/util/ExternalDataConstants.java
M asterix-external-data/src/main/java/org/apache/asterix/external/util/ExternalDataUtils.java
M asterix-external-data/src/main/java/org/apache/asterix/external/util/FileSystemWatcher.java
M asterix-external-data/src/main/java/org/apache/asterix/external/util/HDFSUtils.java
M asterix-metadata/src/main/java/org/apache/asterix/metadata/feeds/FeedMetadataUtil.java
M asterix-runtime/src/main/java/org/apache/asterix/runtime/operators/std/NoTupleSourceRuntimeFactory.java
64 files changed, 1,138 insertions(+), 86 deletions(-)
git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb refs/changes/85/585/6
--
To view, visit https://asterix-gerrit.ics.uci.edu/585
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Id862ce9e9b1360864c6976f2aea2137092f51203
Gerrit-PatchSet: 6
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: abdullah alamoudi <bamousaa@gmail.com>
Gerrit-Reviewer: Jenkins <jenkins@fulliautomatix.ics.uci.edu>
|