bigtop-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Bruno Mahé <bm...@apache.org>
Subject Re: [VOTE] Bigtop 0.7.0 RC0
Date Mon, 28 Oct 2013 09:47:42 GMT
On 10/18/2013 09:54 PM, Roman Shaposhnik wrote:
> This is the seventh release for Apache Bigtop, version 0.7.0
>
> It fixes the following issues:
>    http://s.apache.org/Pkp
>
> *** Please download, test and vote by Fri 10/25 noon PST
>
> Note that we are voting upon the source (tag):
>     release-0.7.0-RC0
>
> Source and binary files:
>    https://repository.apache.org/content/repositories/orgapachebigtop-194/org/apache/bigtop/bigtop/0.7.0/
>
> Binary convenience artifacts:
>     http://bigtop01.cloudera.org:8080/view/Releases/job/Bigtop-0.7.0/
>
> Documentation on how to install (just make sure to adjust the repos for 0.7.0):
>   https://cwiki.apache.org/confluence/display/BIGTOP/How+to+install+Hadoop+distribution+from+Bigtop+0.6.0
>
> Maven staging repo:
>     https://repository.apache.org/content/repositories/orgapachebigtop-194/
>
> The tag to be voted upon:
>     https://git-wip-us.apache.org/repos/asf?p=bigtop.git;a=commit;h=fb628180d289335dcf95641b44482fb680f11573
>
> Bigtop's KEYS file containing PGP keys we use to sign the release:
>     http://svn.apache.org/repos/asf/bigtop/dist/KEYS
>
> Thanks,
> Roman.
>


I am not voting yet since I still have some time, but so far I am 
leaning toward a -1.

I am learning toward a -1 because of 
https://issues.apache.org/jira/browse/BIGTOP-1129 and my issues with Hue.
Other than that, everything I tested either just works out of the box or 
is nitpick.
But BIGTOP-1129 is what I would consider a blocker since it is part of 
the basic use case of Apache Bigtop.

Things I tested:
* Apache Hadoop and some basic jobs
* Apache HBase and Phoenix. Just basic testing
* Apache Flume sending Apache Hadoop and Apache HBase logs to an 
Elasticsearch instance and visualized through Kibana
* Apache Hue smoke tests
* Everything running on OpenJDK 6 on ec2 instances

Things I still want to test (or rather, things I hope I can test by 
Tuesday evening):
* Apache Pig and datafu
* Apache Solr
* Load more data into Phoenix


Things we could do better:
* As described on BIGTOP-1129, I could not stop datanode/namenode 
through init scripts.
* We could provide some templates for Apache Hadoop. I wasted a few 
hours just to get the pi job running. Thankfully we have the init script 
for hdfs (which needs some tweaks for the staging directory) and 
templates for the configuration files in our puppet modules
* I enabled short-circuit in Apache HBase. Not sure if I missed 
something, but I got some 
"org.apache.hadoop.security.AccessControlException: Can't continue with 
getBlockLocalPathInfo() authorization" exceptions. From reading 
http://www.spaggiari.org/index.php/hbase/how-to-activate-hbase-shortcircuit 
it seems there are a few things we could do to make it work out of the box
* Not sure what I did wrong but although I could access Hue UI, most 
apps I tried were not working. Ex: all shells give me the error "value 
222 for UID is less than the minimum UID allowed (500)". And the file 
browser gives me the error "Cannot access: /. Note: You are a Hue admin 
but not a HDFS superuser (which is "hdfs").". Note that the first user I 
created was a user named "ec2-user". Although it is not an hdfs super 
user, I would expect to have a working equivalent of what I can browse 
with the "hdfs -ls" command. Also creating a hue user named "hdfs" 
yields the same result. Note that I did not have time to dig further.
* Phoenix directly embeds Apache Hadoop, Apache HBase and Apache 
Zookeeper jars. These jars should be symlinks.
* Phoenix required me to delete some old Apache lucene jars from Apache 
Flume installation directory. From the output of the command "mvn 
dependency:tree" on the flume project, it appears these jars are only 
needed for the ElasticSearch and MorphlineSolrSink plugins. but Flume 
documentation for both of these plugin explicitly ask users to provide 
jars of Apache Lucene and Apache Solr/ElasticSearch themselves (since 
they may use a different version of Apache Lucene). So the dependency on 
Apache Lucene by Apache Flume should probably be marked as "provided" 
and we should probably provide some packages to manage these dependencies.
* I still need to figure out why my instance of Hue needs access to 
google-analytics.com


Other than that, it was an enjoyable experience to use Apache Bigtop 
0.7.0RC0.
Doing SQL queries through Phoenix was pretty impressive and did not 
require much work to setup.
Also seeing Apache Hadoop and Apache HBase logs being shipped by flume 
to ElasticSearch and then being able to query events and create some 
dynamic charts on kibana was exciting!


Also, since I am about to test Apache Solr, is there an equivalent to 
Kibana I can use for visualizing my indexed logs?


Thanks,
Bruno

Mime
View raw message