asterixdb-notifications mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Till Westmann (Code Review)" <do-not-re...@asterixdb.incubator.apache.org>
Subject Change in asterixdb[master]: Enabled Feed Tests and Added External Library tests
Date Fri, 12 Feb 2016 03:22:14 GMT
Till Westmann has posted comments on this change.

Change subject: Enabled Feed Tests and Added External Library tests
......................................................................


Patch Set 11:

(40 comments)

Did first round of reviews (up to IRecordReader).
I also had to rebase this change to master in order to be able to build it.

https://asterix-gerrit.ics.uci.edu/#/c/625/11/asterix-app/src/main/java/org/apache/asterix/app/external/ExternalLibraryUtils.java
File asterix-app/src/main/java/org/apache/asterix/app/external/ExternalLibraryUtils.java:

Line 82:                         // install if needed (i,e, add the functions, adapters, datasources,
parsers to the metadata) <Not required for use>
fix linebreak


Line 92:      * @return a map from dataverse -> list of uninstalled libraries. 
WS


Line 127:      * TODO Currently, external libraries only include functions and adapters. we
need to extend this to include:
Could you file a JIRA for this?


Line 143:             // make sure dataverse exists
Do we need this check? Won't this happen automatically when checking for the library?


Line 210:                 LOGGER.info("Added library " + libraryName + "to Metadata");
add space between quote and "to"


Line 226:                 //throw new Exception("No library descriptor defined");
remove commented code?


https://asterix-gerrit.ics.uci.edu/#/c/625/11/asterix-app/src/test/java/org/apache/asterix/test/common/TestLibrarian.java
File asterix-app/src/test/java/org/apache/asterix/test/common/TestLibrarian.java:

Line 51:         source.getName();
Why do we call this?


Line 60:         } catch (Exception e) {
Would be enough to catch ZipException here?


Line 62:             throw new Exception("Couldn't unzip the file: " + source, e);
remove empty line above?


Line 67:         // install if needed (i,e, add the functions, adapters, datasources, parsers
to the metadata) <Not required for use>
fix line break?


https://asterix-gerrit.ics.uci.edu/#/c/625/11/asterix-app/src/test/resources/runtimets/queries/external-library/typed_adapter/typed_adapter.1.ddl.aql
File asterix-app/src/test/resources/runtimets/queries/external-library/typed_adapter/typed_adapter.1.ddl.aql:

Line 22:                   Associate with the feed an external user-defined function. The
UDF 
WS


Line 23:                   finds topics in each tweet. A topic is identified by a #. 
WS


https://asterix-gerrit.ics.uci.edu/#/c/625/11/asterix-app/src/test/resources/runtimets/queries/external-library/typed_adapter/typed_adapter.3.ddl.aql
File asterix-app/src/test/resources/runtimets/queries/external-library/typed_adapter/typed_adapter.3.ddl.aql:

Line 22:                   Associate with the feed an external user-defined function. The
UDF 
WS


Line 23:                   finds topics in each tweet. A topic is identified by a #. 
WS


https://asterix-gerrit.ics.uci.edu/#/c/625/11/asterix-app/src/test/resources/runtimets/queries/external-library/typed_adapter/typed_adapter.4.update.aql
File asterix-app/src/test/resources/runtimets/queries/external-library/typed_adapter/typed_adapter.4.update.aql:

Line 22:                   Associate with the feed an external user-defined function. The
UDF 
WS


Line 23:                   finds topics in each tweet. A topic is identified by a #. 
WS


https://asterix-gerrit.ics.uci.edu/#/c/625/11/asterix-app/src/test/resources/runtimets/queries/external-library/typed_adapter/typed_adapter.5.query.aql
File asterix-app/src/test/resources/runtimets/queries/external-library/typed_adapter/typed_adapter.5.query.aql:

Line 22:                   Associate with the feed an external user-defined function. The
UDF 
WS


Line 23:                   finds topics in each tweet. A topic is identified by a #. 
WS


https://asterix-gerrit.ics.uci.edu/#/c/625/11/asterix-app/src/test/resources/runtimets/queries/feeds/feeds_06/feeds_06.3.sleep.aql
File asterix-app/src/test/resources/runtimets/queries/feeds/feeds_06/feeds_06.3.sleep.aql:

Line 1: 3000
We should not remove the header if the file ends in .aql ...


https://asterix-gerrit.ics.uci.edu/#/c/625/11/asterix-app/src/test/resources/runtimets/queries/feeds/feeds_09/feeds_09.4.ddl.aql
File asterix-app/src/test/resources/runtimets/queries/feeds/feeds_09/feeds_09.4.ddl.aql:

Line 20:  * Description  : Create a feed using the synthetic feed simulator adapter. 
WS


Line 22:                   The synthetic feed simulator uses the Social-Data generator to
generate data and simulate a feed. 
WS


Line 23:                   The feed lasts a configured duration with data arriving at a configured
rate (tweets per second). 
WS


Line 25:  * Issue        : 711                 
WS


https://asterix-gerrit.ics.uci.edu/#/c/625/11/asterix-app/src/test/resources/runtimets/queries/feeds/feeds_10/feeds_10.4.ddl.aql
File asterix-app/src/test/resources/runtimets/queries/feeds/feeds_10/feeds_10.4.ddl.aql:

Line 20:  * Description  : Create a feed using the synthetic feed simulator adapter. 
WS


Line 22:                   The synthetic feed simulator uses the Social-Data generator to
generate data and simulate a feed. 
WS


Line 23:                   The feed lasts a configured duration with data arriving at a configured
rate (tweets per second). 
WS


Line 25:  * Issue        : 711                 
WS


https://asterix-gerrit.ics.uci.edu/#/c/625/11/asterix-app/src/test/resources/runtimets/queries/feeds/feeds_12/feeds_12.4.ddl.aql
File asterix-app/src/test/resources/runtimets/queries/feeds/feeds_12/feeds_12.4.ddl.aql:

Line 20:  * Description  : Create a feed using the synthetic feed simulator adapter. 
WS


Line 22:                   The synthetic feed simulator uses the Social-Data generator to
generate data and simulate a feed. 
WS


Line 23:                   The feed lasts a configured duration with data arriving at a configured
rate (tweets per second). 
WS


Line 25:  * Issue        : 711                 
WS


https://asterix-gerrit.ics.uci.edu/#/c/625/11/asterix-app/src/test/resources/runtimets/queries/feeds/issue_230_feeds/issue_230_feeds.4.ddl.aql
File asterix-app/src/test/resources/runtimets/queries/feeds/issue_230_feeds/issue_230_feeds.4.ddl.aql:

Line 20:  * Description  : Create a feed using the synthetic feed simulator adapter. 
WS


Line 22:                   The synthetic feed simulator uses the Social-Data generator to
generate data and simulate a feed. 
WS


Line 23:                   The feed lasts a configured duration with data arriving at a configured
rate (tweets per second). 
WS


Line 25:  * Issue        : 711                 
WS


https://asterix-gerrit.ics.uci.edu/#/c/625/11/asterix-app/src/test/resources/runtimets/testsuite.xml
File asterix-app/src/test/resources/runtimets/testsuite.xml:

Line 103: <!-- 
WS


https://asterix-gerrit.ics.uci.edu/#/c/625/11/asterix-external-data/src/main/java/org/apache/asterix/external/adapter/factory/GenericAdapterFactory.java
File asterix-external-data/src/main/java/org/apache/asterix/external/adapter/factory/GenericAdapterFactory.java:

Line 86:     private void restoreExternalObjects() throws Exception {
Why is this methods called "restore..."? It seems that it's initializing stuff.


https://asterix-gerrit.ics.uci.edu/#/c/625/11/asterix-external-data/src/main/java/org/apache/asterix/external/api/IInputStreamProvider.java
File asterix-external-data/src/main/java/org/apache/asterix/external/api/IInputStreamProvider.java:

Line 31:     public void setFeedLogManager(FeedLogManager feedLogManager);
Why do we have these methods here? Couldn't we pass the configuration in when we create the
IInputStreamProvider instance?

(Also, I'm not sure why every InputStream would get a "FeedLogManager".)


https://asterix-gerrit.ics.uci.edu/#/c/625/11/asterix-external-data/src/main/java/org/apache/asterix/external/api/IRecordFlowController.java
File asterix-external-data/src/main/java/org/apache/asterix/external/api/IRecordFlowController.java:

Line 21: public interface IRecordFlowController<T> extends IDataFlowController {
Remove the whole interface?


https://asterix-gerrit.ics.uci.edu/#/c/625/11/asterix-external-data/src/main/java/org/apache/asterix/external/api/IRecordReader.java
File asterix-external-data/src/main/java/org/apache/asterix/external/api/IRecordReader.java:

Line 74:     public void setFeedLogManager(FeedLogManager feedLogManager);;
Why would every RecordReader get a FeedLogManager? Is this really Feeds-specific?


-- 
To view, visit https://asterix-gerrit.ics.uci.edu/625
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: Idd1fccd136fa2645b2707bbf7c04e60991ae8d4a
Gerrit-PatchSet: 11
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: abdullah alamoudi <bamousaa@gmail.com>
Gerrit-Reviewer: Ian Maxon <imaxon@apache.org>
Gerrit-Reviewer: Jenkins <jenkins@fulliautomatix.ics.uci.edu>
Gerrit-Reviewer: Michael Blow <michael.blow@couchbase.com>
Gerrit-Reviewer: Till Westmann <tillw@apache.org>
Gerrit-HasComments: Yes

Mime
View raw message