chukwa-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Patrick Hunt <ph...@apache.org>
Subject Re: releasing Chukwa? [VOTE]
Date Wed, 21 Oct 2009 18:35:34 GMT
-1, m5sum is ok however the RAT tool picked up a large number of 
compliance issues. Also "ant test" failed.

TEST

     [junit] Running 
org.apache.hadoop.chukwa.datacollection.adaptor.filetailer.TestFileTailingAdaptors
     [junit] Tests run: 3, Failures: 0, Errors: 1, Time elapsed: 29.185 sec


RAT

Probably not all of this is a real problem, but typically *.xml, .sql, 
and especially .sh, .js and .java should have apache headers (must for 
.sh, .js and .java, unless generated/lib.) Probably your .pig files as 
well, if you allow comments.

I also noticed some missing license files for the jars you are 
including. You typically want a license file for each jar included in 
the release. See avro for a good example of this:
http://svn.apache.org/repos/asf/hadoop/avro/trunk/lib/

junit is a type b license btw (requires notice, see apache on compliance 
issues re licensing)

you can easily run the rat tool yourself as:
  java -jar <rat>/rat-app/target/rat-app-0.6-SNAPSHOT.jar .

from here: http://incubator.apache.org/rat/


here's the list of questions found by rat:

  !????? ./CHANGES.txt
  !????? ./ivy.xml
  !????? ./bin/VERSION
  !????? ./bin/netstat.sh
  !????? ./bin/shutdown.sh
  !????? ./bin/startWatchingFile.sh
  !????? ./bin/startup.sh
  !????? ./conf/aggregator.sql
  !????? ./conf/alert
  !????? ./conf/chukwa-agent-conf.xml
  !????? ./conf/chukwa-collector-conf.xml
  !????? ./conf/chukwa-demux-conf.xml
  !????? ./conf/chukwa-env.sh
  !????? ./conf/chukwa-log4j.properties
  !????? ./conf/commons-logging.properties
  !????? ./conf/configuration.xsl
  !????? ./conf/database_create_tables.sql
  !????? ./conf/database_upgrade_0.1.1-0.1.2.sql
  !????? ./conf/hadoop-log4j.properties
  !????? ./conf/hadoop-metrics-log4j.properties
  !????? ./conf/jdbc.conf.template
  !????? ./conf/log4j.properties
  !????? ./conf/mdl.xml
  !????? ./conf/nagios.properties
  !????? ./conf/nodeActivity.properties
  !????? ./conf/queueinfo.properties
  !????? ./conf/system-data-loader.properties
  !????? 
./contrib/chukwa-pig/src/java/org/apache/hadoop/chukwa/ChukwaArchive.java
  !????? 
./contrib/xtrace/src/java/edu/berkeley/chukwa_xtrace/CausalGraph.java
  !????? ./docs/admin.html
  !????? ./docs/agent.html
  !????? ./docs/broken-links.xml
  !????? ./docs/collector.html
  !????? ./docs/design.html
  !????? ./docs/index.html
  !????? ./docs/linkmap.html
  !????? ./docs/programming.html
  !????? ./docs/releasenotes.html
  !????? ./docs/api/package-list
  !????? ./docs/skin/note.txt
  !????? ./docs/skin/profile.css
  !????? ./ivy/ivysettings.xml
  !????? ./ivy/libraries.properties
  !????? ./lib/asm-3.1-LICENSE.txt
  !????? ./lib/jersey-bundle-LICENSE
  !????? ./lib/json-LICENSE.txt
  !????? ./lib/json-README.txt
  !????? ./lib/jsr311-api-1.0-LICENSE
  !????? ./script/pig/Df.pig
  !????? ./script/pig/Hadoop_dfs_FSDirectory.pig
  !????? ./script/pig/Hadoop_dfs_FSNamesystem.pig
  !????? ./script/pig/Hadoop_dfs_datanode.pig
  !????? ./script/pig/Hadoop_dfs_namenode.pig
  !????? ./script/pig/Hadoop_jvm_metrics.pig
  !????? ./script/pig/Hadoop_mapred_jobtracker.pig
  !????? ./script/pig/Hadoop_rpc_metrics.pig
  !????? ./script/pig/SystemMetrics.pig
  !????? ./script/pig/UserDailySummary.pig
  !????? ./src/docs/releasenotes.html
  !????? ./src/java/org/apache/hadoop/chukwa/ChukwaArchiveKey.java
  !????? ./src/java/org/apache/hadoop/chukwa/database/Macro.java
  !????? 
./src/java/org/apache/hadoop/chukwa/database/MetricsAggregation.java
  !????? 
./src/java/org/apache/hadoop/chukwa/datacollection/ChunkReceiver.java
  !????? 
./src/java/org/apache/hadoop/chukwa/datacollection/adaptor/AdaptorShutdownPolicy.java
  !????? 
./src/java/org/apache/hadoop/chukwa/datacollection/adaptor/filetailer/TerminatorThread.java
  !????? 
./src/java/org/apache/hadoop/chukwa/datacollection/agent/metrics/AgentMetrics.java
  !????? 
./src/java/org/apache/hadoop/chukwa/datacollection/agent/metrics/ChunkQueueMetrics.java
  !????? 
./src/java/org/apache/hadoop/chukwa/datacollection/sender/ChukwaSender.java
  !????? 
./src/java/org/apache/hadoop/chukwa/datacollection/sender/metrics/HttpSenderMetrics.java
  !????? 
./src/java/org/apache/hadoop/chukwa/datacollection/writer/NullWriter.java
  !????? 
./src/java/org/apache/hadoop/chukwa/datacollection/writer/WriterException.java
  !????? 
./src/java/org/apache/hadoop/chukwa/dataloader/MetricDataLoaderPool.java
  !????? 
./src/java/org/apache/hadoop/chukwa/extraction/archive/ChukwaArchiveMerger.java
  !????? 
./src/java/org/apache/hadoop/chukwa/extraction/demux/processor/ChukwaOutputCollector.java
  !????? 
./src/java/org/apache/hadoop/chukwa/extraction/demux/processor/mapper/DefaultProcessor.java
  !????? 
./src/java/org/apache/hadoop/chukwa/extraction/demux/processor/mapper/JobConfProcessor.java
  !????? 
./src/java/org/apache/hadoop/chukwa/extraction/demux/processor/mapper/LogEntry.java
  !????? 
./src/java/org/apache/hadoop/chukwa/extraction/demux/processor/mapper/TsProcessor.java
  !????? 
./src/java/org/apache/hadoop/chukwa/extraction/demux/processor/reducer/IdentityReducer.java
  !????? 
./src/java/org/apache/hadoop/chukwa/extraction/demux/processor/reducer/MRJobReduceProcessor.java
  !????? 
./src/java/org/apache/hadoop/chukwa/extraction/engine/ChukwaRecordKey.java
  !????? 
./src/java/org/apache/hadoop/chukwa/extraction/engine/RecordUtil.java
  !????? ./src/java/org/apache/hadoop/chukwa/extraction/engine/Token.java
  !????? 
./src/java/org/apache/hadoop/chukwa/extraction/engine/datasource/record/ChukwaDSInternalResult.java
  !????? 
./src/java/org/apache/hadoop/chukwa/extraction/engine/datasource/record/ChukwaRecordDataSource.java
  !????? ./src/java/org/apache/hadoop/chukwa/hicc/HiccWebServer.java
  !????? ./src/java/org/apache/hadoop/chukwa/hicc/Iframe.java
  !????? 
./src/java/org/apache/hadoop/chukwa/inputtools/log4j/ChukwaDailyRollingFileAppender.java
  !????? 
./src/java/org/apache/hadoop/chukwa/inputtools/plugin/metrics/Exec.java
  !????? 
./src/java/org/apache/hadoop/chukwa/inputtools/plugin/metrics/ExecHelper.java
  !????? 
./src/java/org/apache/hadoop/chukwa/inputtools/plugin/metrics/Log4JMetricsContext.java
  !????? 
./src/java/org/apache/hadoop/chukwa/inputtools/plugin/pbsnode/PbsNodePlugin.java
  !????? 
./src/java/org/apache/hadoop/chukwa/tools/backfilling/BackfillingLoader.java
  !????? ./src/java/org/apache/hadoop/chukwa/util/DaemonWatcher.java
  !????? ./src/java/org/apache/hadoop/chukwa/util/DriverManagerUtil.java
  !????? ./src/java/org/apache/hadoop/chukwa/util/DumpRecord.java
  !????? ./src/java/org/apache/hadoop/chukwa/util/NagiosHelper.java
  !????? ./src/java/org/apache/hadoop/chukwa/util/WatchDog.java
  !????? ./src/test/org/apache/hadoop/chukwa/ChunkImplTest.java
  !????? 
./src/test/org/apache/hadoop/chukwa/datacollection/adaptor/filetailer/TestCharFileTailingAdaptorUTF8.java
  !????? 
./src/test/org/apache/hadoop/chukwa/datacollection/agent/TestAgentConfig.java
  !????? 
./src/test/org/apache/hadoop/chukwa/datacollection/collector/TestAdaptorTimeout.java
  !????? 
./src/test/org/apache/hadoop/chukwa/datacollection/collector/TestBackpressure.java
  !????? 
./src/test/org/apache/hadoop/chukwa/datacollection/collector/TestFailedCollectorAck.java
  !????? 
./src/test/org/apache/hadoop/chukwa/datacollection/writer/TestClientAck.java
  !????? 
./src/test/org/apache/hadoop/chukwa/extraction/demux/processor/mapper/ChukwaTestOutputCollector.java
  !????? 
./src/test/org/apache/hadoop/chukwa/extraction/demux/processor/mapper/Hadoop18JobHistoryLog.txt
  !????? 
./src/test/org/apache/hadoop/chukwa/extraction/demux/processor/mapper/TestAbtractProcessor.java
  !????? 
./src/test/org/apache/hadoop/chukwa/extraction/demux/processor/mapper/TestJobLog.txt
  !????? 
./src/test/org/apache/hadoop/chukwa/inputtools/log4j/TestChukwaDailyRollingFileAppender.java
  !????? 
./src/test/org/apache/hadoop/chukwa/inputtools/log4j/late-log4j.properties
  !????? 
./src/test/org/apache/hadoop/chukwa/inputtools/log4j/tasklog-log4j.properties
  !????? 
./src/test/org/apache/hadoop/chukwa/tools/backfilling/TestBackfillingLoader.java
  !????? 
./src/test/org/apache/hadoop/chukwa/validationframework/interceptor/ChunkDumper.java
  !????? 
./src/test/org/apache/hadoop/chukwa/validationframework/interceptor/ChunkQueueInterceptor.java
  !????? 
./src/test/org/apache/hadoop/chukwa/validationframework/interceptor/SetupTestClasses.java
  !????? 
./src/test/org/apache/hadoop/chukwa/validationframework/util/DataOperations.java
  !????? 
./src/test/org/apache/hadoop/chukwa/validationframework/util/MD5.java
  !????? ./src/web/collector/WEB-INF/web.xml
  !????? ./src/web/hicc/iphone.jsp
  !????? ./src/web/hicc/WEB-INF/jetty.xml
  !????? ./src/web/hicc/WEB-INF/web.xml
  !????? ./src/web/hicc/css/default.css
  !????? ./src/web/hicc/css/iframe.css
  !????? ./src/web/hicc/css/iui.css
  !????? ./src/web/hicc/css/menu.css
  !????? ./src/web/hicc/css/tab.css
  !????? ./src/web/hicc/css/timeframe.css
  !????? ./src/web/hicc/css/timeline.css
  !????? ./src/web/hicc/css/flexigrid/flexigrid.css
  !????? ./src/web/hicc/descriptors/client_trace.descriptor
  !????? ./src/web/hicc/descriptors/cluster_disk.descriptor
  !????? ./src/web/hicc/descriptors/cluster_disk_pcnt.descriptor
  !????? ./src/web/hicc/descriptors/cluster_metrics.descriptor
  !????? ./src/web/hicc/descriptors/cluster_metrics_pcnt.descriptor
  !????? ./src/web/hicc/descriptors/cluster_selector.descriptor
  !????? ./src/web/hicc/descriptors/debug.descriptor
  !????? ./src/web/hicc/descriptors/dfs_datanode_counter.descriptor
  !????? ./src/web/hicc/descriptors/dfs_datanode_rate.descriptor
  !????? ./src/web/hicc/descriptors/dfs_fsnamesystem.descriptor
  !????? ./src/web/hicc/descriptors/dfs_namenode_counter.descriptor
  !????? ./src/web/hicc/descriptors/dfs_namenode_rate.descriptor
  !????? ./src/web/hicc/descriptors/dfs_throughput.descriptor
  !????? ./src/web/hicc/descriptors/disk.descriptor
  !????? ./src/web/hicc/descriptors/disk_pcnt.descriptor
  !????? ./src/web/hicc/descriptors/event_viewer.descriptor
  !????? ./src/web/hicc/descriptors/hadoop_jvm_counter.descriptor
  !????? ./src/web/hicc/descriptors/hadoop_jvm_rate.descriptor
  !????? ./src/web/hicc/descriptors/hadoop_mapred.descriptor
  !????? ./src/web/hicc/descriptors/heatmap-static.descriptor
  !????? ./src/web/hicc/descriptors/heatmap_datanode.descriptor
  !????? ./src/web/hicc/descriptors/host_selector.descriptor
  !????? ./src/web/hicc/descriptors/node_activity_chart.descriptor
  !????? ./src/web/hicc/descriptors/rpc_metrics.descriptor
  !????? ./src/web/hicc/descriptors/sql_client.descriptor
  !????? ./src/web/hicc/descriptors/swimlanes-static.descriptor
  !????? ./src/web/hicc/descriptors/swimlanes.descriptor
  !????? ./src/web/hicc/descriptors/system_metrics.descriptor
  !????? ./src/web/hicc/descriptors/system_metrics_pcnt.descriptor
  !????? ./src/web/hicc/descriptors/time.descriptor
  !????? ./src/web/hicc/descriptors/utilization.descriptor
  !????? ./src/web/hicc/js/ajaxtree.js
  !????? ./src/web/hicc/js/base64.js
  !????? ./src/web/hicc/js/behaviour.js
  !????? ./src/web/hicc/js/canvas2image.js
  !????? ./src/web/hicc/js/excanvas.pack.js
  !????? ./src/web/hicc/js/flexigrid.js
  !????? ./src/web/hicc/js/flexigrid.pack.js
  !????? ./src/web/hicc/js/flot.extend.js
  !????? ./src/web/hicc/js/gsv.js
  !????? ./src/web/hicc/js/iui.js
  !????? ./src/web/hicc/js/jquery-1.2.6.min.js
  !????? ./src/web/hicc/js/jquery.flot.pack.js
  !????? ./src/web/hicc/js/json.js
  !????? ./src/web/hicc/js/timeframe.js
  !????? ./src/web/hicc/js/treeview-min.js
  !????? ./src/web/hicc/js/yahoo-dom-event.js
  !????? ./src/web/hicc/js/org/tool-man/cookies.js
  !????? ./src/web/hicc/js/org/tool-man/coordinates.js
  !????? ./src/web/hicc/js/org/tool-man/core.js
  !????? ./src/web/hicc/js/org/tool-man/core.js.old
  !????? ./src/web/hicc/js/org/tool-man/css.js
  !????? ./src/web/hicc/js/org/tool-man/drag.js
  !????? ./src/web/hicc/js/org/tool-man/dragsort.js
  !????? ./src/web/hicc/js/org/tool-man/events.js
  !????? ./src/web/hicc/js/workspace/builder.js
  !????? ./src/web/hicc/js/workspace/controls.js
  !????? ./src/web/hicc/js/workspace/dragdrop.js
  !????? ./src/web/hicc/js/workspace/effects.js
  !????? ./src/web/hicc/js/workspace/prototype.js
  !????? ./src/web/hicc/js/workspace/scriptaculous.js
  !????? ./src/web/hicc/js/workspace/slider.js
  !????? ./src/web/hicc/js/workspace/sound.js
  !????? ./src/web/hicc/jsp/help.jsp
  !????? ./src/web/hicc/lib/timeline/bundle.css
  !????? ./src/web/hicc/lib/timeline/bundle.js
  !????? ./src/web/hicc/lib/timeline/search.js
  !????? ./src/web/hicc/lib/timeline/timeline-api.js
  !????? ./src/web/hicc/lib/timeline/ext/geochrono/geochrono-api.js
  !????? ./src/web/hicc/lib/timeline/ext/geochrono/scripts/ether-painters.js
  !????? ./src/web/hicc/lib/timeline/ext/geochrono/scripts/geochrono.js
  !????? ./src/web/hicc/lib/timeline/ext/geochrono/scripts/labellers.js
  !????? ./src/web/hicc/lib/timeline/ext/geochrono/scripts/units.js
  !????? 
./src/web/hicc/lib/timeline/ext/geochrono/scripts/l10n/en/labellers.js
  !????? ./src/web/hicc/lib/timeline/ext/planning/planning-api.js
  !????? ./src/web/hicc/lib/timeline/ext/planning/scripts/ether-painters.js
  !????? ./src/web/hicc/lib/timeline/ext/planning/scripts/labellers.js
  !????? ./src/web/hicc/lib/timeline/ext/planning/scripts/planning.js
  !????? ./src/web/hicc/lib/timeline/ext/planning/scripts/units.js
  !????? 
./src/web/hicc/lib/timeline/ext/planning/scripts/l10n/en/labellers.js
  !????? ./src/web/hicc/lib/timeline/scripts/decorators.js
  !????? ./src/web/hicc/lib/timeline/scripts/ether-painters.js
  !????? ./src/web/hicc/lib/timeline/scripts/ethers.js
  !????? ./src/web/hicc/lib/timeline/scripts/labellers.js
  !????? ./src/web/hicc/lib/timeline/scripts/layouts.js
  !????? ./src/web/hicc/lib/timeline/scripts/painters.js
  !????? ./src/web/hicc/lib/timeline/scripts/sources.js
  !????? ./src/web/hicc/lib/timeline/scripts/themes.js
  !????? ./src/web/hicc/lib/timeline/scripts/timeline.js
  !????? ./src/web/hicc/lib/timeline/scripts/units.js
  !????? ./src/web/hicc/lib/timeline/scripts/ext/japanese-eras.js
  !????? ./src/web/hicc/lib/timeline/scripts/l10n/cs/labellers.js
  !????? ./src/web/hicc/lib/timeline/scripts/l10n/cs/timeline.js
  !????? ./src/web/hicc/lib/timeline/scripts/l10n/de/labellers.js
  !????? ./src/web/hicc/lib/timeline/scripts/l10n/de/timeline.js
  !????? ./src/web/hicc/lib/timeline/scripts/l10n/en/labellers.js
  !????? ./src/web/hicc/lib/timeline/scripts/l10n/en/timeline.js
  !????? ./src/web/hicc/lib/timeline/scripts/l10n/es/labellers.js
  !????? ./src/web/hicc/lib/timeline/scripts/l10n/es/timeline.js
  !????? ./src/web/hicc/lib/timeline/scripts/l10n/fr/labellers.js
  !????? ./src/web/hicc/lib/timeline/scripts/l10n/fr/timeline.js
  !????? ./src/web/hicc/lib/timeline/scripts/l10n/it/labellers.js
  !????? ./src/web/hicc/lib/timeline/scripts/l10n/it/timeline.js
  !????? ./src/web/hicc/lib/timeline/scripts/l10n/ru/labellers.js
  !????? ./src/web/hicc/lib/timeline/scripts/l10n/ru/timeline.js
  !????? ./src/web/hicc/lib/timeline/scripts/l10n/se/labellers.js
  !????? ./src/web/hicc/lib/timeline/scripts/l10n/se/timeline.js
  !????? ./src/web/hicc/lib/timeline/scripts/l10n/vi/labellers.js
  !????? ./src/web/hicc/lib/timeline/scripts/l10n/vi/timeline.js
  !????? ./src/web/hicc/lib/timeline/scripts/l10n/zh/labellers.js
  !????? ./src/web/hicc/lib/timeline/scripts/l10n/zh/timeline.js
  !????? ./src/web/hicc/lib/timeline/scripts/util/data-structure.js
  !????? ./src/web/hicc/lib/timeline/scripts/util/date-time.js
  !????? ./src/web/hicc/lib/timeline/scripts/util/debug.js
  !????? ./src/web/hicc/lib/timeline/scripts/util/dom.js
  !????? ./src/web/hicc/lib/timeline/scripts/util/graphics.js
  !????? ./src/web/hicc/lib/timeline/scripts/util/html.js
  !????? ./src/web/hicc/lib/timeline/scripts/util/platform.js
  !????? ./src/web/hicc/lib/timeline/scripts/util/xmlhttp.js
  !????? ./src/web/hicc/lib/timeline/styles/ethers.css
  !????? ./src/web/hicc/lib/timeline/styles/events.css
  !????? ./src/web/hicc/lib/timeline/styles/timeline.css
  !????? ./src/web/hicc/views/default.view
  !????? ./src/web/hicc/views/newview.tpl
  !????? ./src/web/hicc/views/view.permission
  !????? ./src/web/hicc/views/workspace_view_list.cache
  !????? ./test/samples/ClientTrace.log
  !????? ./test/samples/Iostat.log
  !????? ./test/samples/JobHistory.log
  !????? ./tools/service/chukwa-agent/run
  !????? ./tools/service/chukwa-collector/run
  !????? ./tools/service/chukwa-df/run
  !????? ./tools/service/chukwa-hdfsusage/run
  !????? ./tools/service/chukwa-iostat/run
  !????? ./tools/service/chukwa-netstat/run
  !????? ./tools/service/chukwa-pbsnodes/run
  !????? ./tools/service/chukwa-ps/run
  !????? ./tools/service/chukwa-sar/run
  !????? ./tools/service/chukwa-top/run
  !????? ./tools/service/chukwa-torque/run
  !????? ./var/descriptors/client_trace.descriptor
  !????? ./var/descriptors/cluster_disk.descriptor
  !????? ./var/descriptors/cluster_disk_pcnt.descriptor
  !????? ./var/descriptors/cluster_metrics.descriptor
  !????? ./var/descriptors/cluster_metrics_pcnt.descriptor
  !????? ./var/descriptors/cluster_selector.descriptor
  !????? ./var/descriptors/debug.descriptor
  !????? ./var/descriptors/dfs_datanode_counter.descriptor
  !????? ./var/descriptors/dfs_datanode_rate.descriptor
  !????? ./var/descriptors/dfs_fsnamesystem.descriptor
  !????? ./var/descriptors/dfs_namenode_counter.descriptor
  !????? ./var/descriptors/dfs_namenode_rate.descriptor
  !????? ./var/descriptors/dfs_throughput.descriptor
  !????? ./var/descriptors/disk.descriptor
  !????? ./var/descriptors/disk_pcnt.descriptor
  !????? ./var/descriptors/event_viewer.descriptor
  !????? ./var/descriptors/hadoop_jvm_counter.descriptor
  !????? ./var/descriptors/hadoop_jvm_rate.descriptor
  !????? ./var/descriptors/hadoop_mapred.descriptor
  !????? ./var/descriptors/heatmap-static.descriptor
  !????? ./var/descriptors/heatmap_datanode.descriptor
  !????? ./var/descriptors/host_selector.descriptor
  !????? ./var/descriptors/node_activity_chart.descriptor
  !????? ./var/descriptors/rpc_metrics.descriptor
  !????? ./var/descriptors/sql_client.descriptor
  !????? ./var/descriptors/swimlanes-static.descriptor
  !????? ./var/descriptors/swimlanes.descriptor
  !????? ./var/descriptors/system_metrics.descriptor
  !????? ./var/descriptors/system_metrics_pcnt.descriptor
  !????? ./var/descriptors/time.descriptor
  !????? ./var/descriptors/utilization.descriptor
  !????? ./var/views/default.view
  !????? ./var/views/newview.tpl
  !????? ./var/views/view.permission
  !????? ./var/views/workspace_view_list.cache


Ariel Rabkin wrote:
> Dear Hadoop PMC:
> 
> The Chukwa team would like to do a release. Eric, Jerome, and I all
> think that http://people.apache.org/~asrabkin/chukwa-0.3.0-candidate-3/
> is ready to go. Release notes and license information are in place.
> It's signed under my key [http://people.apache.org/~asrabkin/KEYS].
> 
> Are you comfortable releasing this?  If not, what do we need to fix?
> 
> --Ari
> 
> PS: I uploaded my signing key to the keyserver network, but it isn't
> yet in the Apache web of trust. I don't quite know how to deal with
> that?
> 

Mime
View raw message