asterixdb-notifications mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "abdullah alamoudi (Code Review)" <do-not-re...@asterixdb.incubator.apache.org>
Subject Change in asterixdb[master]: Add flush() to IFrameWriter
Date Fri, 29 Jan 2016 17:09:43 GMT
Hello Jenkins,

I'd like you to reexamine a change.  Please visit

    https://asterix-gerrit.ics.uci.edu/585

to look at the new patch set (#12).

Change subject: Add flush() to IFrameWriter
......................................................................

Add flush() to IFrameWriter

This method is expected to be used with feeds to push
frames all the way to storage when needed. As of now, it is
needed in two cases:
1. No activities in ingestion node and need to push content
   so it can be stored.
2. When the ingestion node needs to move the checkpoint ahead
   if the at least once semantics are used.

Two feeds make use of this function. The filesystem feed and
couchbase feed which was introduced as well in this change.

Change-Id: Id862ce9e9b1360864c6976f2aea2137092f51203
---
M asterix-algebra/src/main/java/org/apache/asterix/algebra/operators/physical/CommitRuntime.java
M asterix-app/src/main/java/org/apache/asterix/feed/FeedMessageReceiver.java
M asterix-app/src/main/java/org/apache/hyracks/dataflow/std/misc/ConstantTupleSourceOperatorNodePushable.java
M asterix-external-data/pom.xml
M asterix-external-data/src/main/java/org/apache/asterix/external/adapter/factory/LookupAdapterFactory.java
M asterix-external-data/src/main/java/org/apache/asterix/external/api/IDataParser.java
M asterix-external-data/src/main/java/org/apache/asterix/external/api/IExternalDataSourceFactory.java
M asterix-external-data/src/main/java/org/apache/asterix/external/api/IRawRecord.java
M asterix-external-data/src/main/java/org/apache/asterix/external/api/IRecordReader.java
M asterix-external-data/src/main/java/org/apache/asterix/external/dataflow/AbstractFeedDataFlowController.java
M asterix-external-data/src/main/java/org/apache/asterix/external/dataflow/FeedRecordDataFlowController.java
M asterix-external-data/src/main/java/org/apache/asterix/external/dataflow/FeedTupleForwarder.java
M asterix-external-data/src/main/java/org/apache/asterix/external/dataset/adapter/LookupAdapter.java
M asterix-external-data/src/main/java/org/apache/asterix/external/feed/dataflow/CollectTransformFeedFrameWriter.java
M asterix-external-data/src/main/java/org/apache/asterix/external/feed/dataflow/DistributeFeedFrameWriter.java
M asterix-external-data/src/main/java/org/apache/asterix/external/feed/dataflow/FeedFrameCache.java
M asterix-external-data/src/main/java/org/apache/asterix/external/feed/dataflow/FeedFrameCollector.java
M asterix-external-data/src/main/java/org/apache/asterix/external/feed/dataflow/FeedFrameHandlers.java
M asterix-external-data/src/main/java/org/apache/asterix/external/feed/dataflow/FeedRuntimeInputHandler.java
M asterix-external-data/src/main/java/org/apache/asterix/external/feed/dataflow/FrameDistributor.java
M asterix-external-data/src/main/java/org/apache/asterix/external/feed/message/MessageReceiver.java
M asterix-external-data/src/main/java/org/apache/asterix/external/feed/watch/MonitoredBuffer.java
M asterix-external-data/src/main/java/org/apache/asterix/external/feed/watch/StorageSideMonitoredBuffer.java
M asterix-external-data/src/main/java/org/apache/asterix/external/indexing/FileOffsetIndexer.java
M asterix-external-data/src/main/java/org/apache/asterix/external/indexing/RecordColumnarIndexer.java
M asterix-external-data/src/main/java/org/apache/asterix/external/input/HDFSDataSourceFactory.java
M asterix-external-data/src/main/java/org/apache/asterix/external/input/record/CharArrayRecord.java
M asterix-external-data/src/main/java/org/apache/asterix/external/input/record/GenericRecord.java
A asterix-external-data/src/main/java/org/apache/asterix/external/input/record/RecordWithMetadata.java
A asterix-external-data/src/main/java/org/apache/asterix/external/input/record/reader/couchbase/CouchbaseReader.java
A asterix-external-data/src/main/java/org/apache/asterix/external/input/record/reader/couchbase/CouchbaseReaderFactory.java
R asterix-external-data/src/main/java/org/apache/asterix/external/input/record/reader/hdfs/AbstractCharRecordLookupReader.java
R asterix-external-data/src/main/java/org/apache/asterix/external/input/record/reader/hdfs/AbstractHDFSLookupRecordReader.java
R asterix-external-data/src/main/java/org/apache/asterix/external/input/record/reader/hdfs/HDFSLookupReaderFactory.java
R asterix-external-data/src/main/java/org/apache/asterix/external/input/record/reader/hdfs/HDFSRecordReader.java
R asterix-external-data/src/main/java/org/apache/asterix/external/input/record/reader/hdfs/HDFSTextLineReader.java
R asterix-external-data/src/main/java/org/apache/asterix/external/input/record/reader/hdfs/RCLookupReader.java
R asterix-external-data/src/main/java/org/apache/asterix/external/input/record/reader/hdfs/SequenceLookupReader.java
R asterix-external-data/src/main/java/org/apache/asterix/external/input/record/reader/hdfs/TextLookupReader.java
R asterix-external-data/src/main/java/org/apache/asterix/external/input/record/reader/rss/RSSRecordReader.java
R asterix-external-data/src/main/java/org/apache/asterix/external/input/record/reader/rss/RSSRecordReaderFactory.java
R asterix-external-data/src/main/java/org/apache/asterix/external/input/record/reader/stream/AbstractStreamRecordReader.java
R asterix-external-data/src/main/java/org/apache/asterix/external/input/record/reader/stream/AbstractStreamRecordReaderFactory.java
R asterix-external-data/src/main/java/org/apache/asterix/external/input/record/reader/stream/LineRecordReader.java
R asterix-external-data/src/main/java/org/apache/asterix/external/input/record/reader/stream/LineRecordReaderFactory.java
R asterix-external-data/src/main/java/org/apache/asterix/external/input/record/reader/stream/QuotedLineRecordReader.java
R asterix-external-data/src/main/java/org/apache/asterix/external/input/record/reader/stream/SemiStructuredRecordReader.java
R asterix-external-data/src/main/java/org/apache/asterix/external/input/record/reader/stream/SemiStructuredRecordReaderFactory.java
R asterix-external-data/src/main/java/org/apache/asterix/external/input/record/reader/twitter/TwitterPullRecordReader.java
R asterix-external-data/src/main/java/org/apache/asterix/external/input/record/reader/twitter/TwitterPushRecordReader.java
R asterix-external-data/src/main/java/org/apache/asterix/external/input/record/reader/twitter/TwitterRecordReaderFactory.java
M asterix-external-data/src/main/java/org/apache/asterix/external/input/stream/AInputStream.java
M asterix-external-data/src/main/java/org/apache/asterix/external/input/stream/AInputStreamReader.java
M asterix-external-data/src/main/java/org/apache/asterix/external/input/stream/LocalFileSystemInputStream.java
M asterix-external-data/src/main/java/org/apache/asterix/external/input/stream/factory/LocalFSInputStreamProviderFactory.java
M asterix-external-data/src/main/java/org/apache/asterix/external/input/stream/factory/SocketInputStreamProviderFactory.java
R asterix-external-data/src/main/java/org/apache/asterix/external/input/stream/provider/HDFSInputStreamProvider.java
R asterix-external-data/src/main/java/org/apache/asterix/external/input/stream/provider/LocalFSInputStreamProvider.java
R asterix-external-data/src/main/java/org/apache/asterix/external/input/stream/provider/SocketInputStreamProvider.java
R asterix-external-data/src/main/java/org/apache/asterix/external/input/stream/provider/TwitterFirehoseInputStreamProvider.java
M asterix-external-data/src/main/java/org/apache/asterix/external/operators/ExternalLookupOperatorDescriptor.java
M asterix-external-data/src/main/java/org/apache/asterix/external/operators/FeedMetaComputeNodePushable.java
M asterix-external-data/src/main/java/org/apache/asterix/external/operators/FeedMetaNodePushable.java
M asterix-external-data/src/main/java/org/apache/asterix/external/operators/FeedMetaStoreNodePushable.java
M asterix-external-data/src/main/java/org/apache/asterix/external/parser/ADMDataParser.java
M asterix-external-data/src/main/java/org/apache/asterix/external/parser/DelimitedDataParser.java
A asterix-external-data/src/main/java/org/apache/asterix/external/parser/RecordWithMetadataParser.java
A asterix-external-data/src/main/java/org/apache/asterix/external/parser/factory/RecordWithMetadataParserFactory.java
M asterix-external-data/src/main/java/org/apache/asterix/external/provider/DatasourceFactoryProvider.java
R asterix-external-data/src/main/java/org/apache/asterix/external/provider/LookupReaderFactoryProvider.java
M asterix-external-data/src/main/java/org/apache/asterix/external/provider/ParserFactoryProvider.java
M asterix-external-data/src/main/java/org/apache/asterix/external/util/DataflowUtils.java
M asterix-external-data/src/main/java/org/apache/asterix/external/util/ExternalDataConstants.java
M asterix-external-data/src/main/java/org/apache/asterix/external/util/ExternalDataUtils.java
M asterix-external-data/src/main/java/org/apache/asterix/external/util/FileSystemWatcher.java
M asterix-external-data/src/main/java/org/apache/asterix/external/util/HDFSUtils.java
M asterix-metadata/src/main/java/org/apache/asterix/metadata/declared/AqlMetadataProvider.java
M asterix-metadata/src/main/java/org/apache/asterix/metadata/feeds/FeedMetadataUtil.java
M asterix-om/src/main/java/org/apache/asterix/om/util/AsterixClusterProperties.java
79 files changed, 1,081 insertions(+), 177 deletions(-)


  git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb refs/changes/85/585/12
-- 
To view, visit https://asterix-gerrit.ics.uci.edu/585
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Id862ce9e9b1360864c6976f2aea2137092f51203
Gerrit-PatchSet: 12
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: abdullah alamoudi <bamousaa@gmail.com>
Gerrit-Reviewer: Ian Maxon <imaxon@apache.org>
Gerrit-Reviewer: Jenkins <jenkins@fulliautomatix.ics.uci.edu>
Gerrit-Reviewer: Murtadha Hubail <hubailmor@gmail.com>
Gerrit-Reviewer: Till Westmann <tillw@apache.org>
Gerrit-Reviewer: abdullah alamoudi <bamousaa@gmail.com>

Mime
View raw message