flume-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Kaifeng Huang (Jira)" <j...@apache.org>
Subject [jira] [Created] (FLUME-3357) Inconsistent library versions notice.
Date Sun, 01 Mar 2020 12:28:00 GMT
Kaifeng Huang created FLUME-3357:
------------------------------------

             Summary: Inconsistent library versions notice.
                 Key: FLUME-3357
                 URL: https://issues.apache.org/jira/browse/FLUME-3357
             Project: Flume
          Issue Type: Improvement
            Reporter: Kaifeng Huang
         Attachments: apache flume.pdf

Hi. I have implemented a tool to detect library version inconsistencies. Your project have
1 inconsistent library.

Take com.google.guava:guava for example, this library is declared as version 11.0.2 in flume-ng-sinks/flume-ng-elasticsearch-sink,
18.0 in flume-ng-sinks/flume-http-sink and etc... Such version inconsistencies may cause unnecessary
maintenance effort in the long run. For example, if two modules become inter-dependent, library
version conflict may happen. It has already become a common issue and hinders development
progress. Thus a version harmonization is necessary.

Provided we applied a version harmonization, I calculated the cost it may have to harmonize
to all upper versions including an up-to-date one. The cost refers to POM config changes and
API invocation changes. Take com.google.guava:guava for example, if we harmonize all the library
versions into 28.2-jre. The concern is, how much should the project code adapt to the newer
library version. We list an effort table to quantify the harmonization cost.

The effort table is listed below. It shows the overall harmonization effort by modules. The
columns represents the number of library APIs and API calls(NA,NAC), deleted APIs and API
calls(NDA,NDAC) as well as modified API and API calls(NMA,NMAC). Modified APIs refers to those
APIs whose call graph is not the same as previous version. Take the first row for example,
if upgrading the library into version 28.2-jre. Given that 82 APIs is used in module flume-ng-core,
19 of them is deleted in a recommended version(which will throw a NoMethodFoundError unless
re-compiling the project), 32 of them is regarded as modified which could break the former
API contract.


||Index||Module||NA(NAC)||NDA(NDAC)||NMA(NMAC)||
|1|flume-ng-core|82(299)|19(49)|32(95)|
|2|flume-ng-sinks/flume-hdfs-sink|12(48)|2(14)|4(4)|
|3|flume-ng-node|14(20)|6(10)|7(9)|
|4|..|..|..|..|


Also we provided another table to show the potential files that may be affected due to library
API change, which could help to spot the concerned API usage and rerun the test cases. The
table is listed below.




||Module||File||Type||API||
|flume-ng-sinks/flume-ng-elasticsearch-sink|flume-ng-sinks/flume-ng-elasticsearch-sink/src/test/java/org/apache/flume/sink/elasticsearch/TimestampedEventTest.java|delete|com.google.common.collect.Maps.newHashMap()|
|flume-ng-sinks/flume-ng-elasticsearch-sink|flume-ng-sinks/flume-ng-elasticsearch-sink/src/test/java/org/apache/flume/sink/elasticsearch/client/TestElasticSearchRestClient.java|modify|com.google.common.base.Splitter.split(java.lang.CharSequence)|
|flume-ng-sinks/flume-ng-elasticsearch-sink|flume-ng-sinks/flume-ng-elasticsearch-sink/src/test/java/org/apache/flume/sink/elasticsearch/client/TestElasticSearchRestClient.java|modify|com.google.common.base.Splitter.on(java.lang.String)|
|4|..|..|..|



If you are interested, you can have a more complete and detailed report in the attached PDF
file.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@flume.apache.org
For additional commands, e-mail: issues-help@flume.apache.org


Mime
View raw message