cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Tomas Salfischberger (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (CASSANDRA-3061) Optionally skip log4j configuration
Date Wed, 31 Aug 2011 06:11:10 GMT

    [ https://issues.apache.org/jira/browse/CASSANDRA-3061?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13094334#comment-13094334
] 

Tomas Salfischberger commented on CASSANDRA-3061:
-------------------------------------------------

For me the log4j problem is solved with this patch

I ran into the same problem as Aaron did (which is fixed in CASSANDRA-3044), but worked around
it by attaching a debugger and manually changing the endpoint to be correct. In Brisk there
is another issue, because rpc_address is set to "0.0.0.0" which is not a valid endpoint to
connect to. Also posted that here: http://www.datastax.com/support-forums/topic/building-brisk-from-source-to-update-cassandra-to-082?replies=1#post-409


> Optionally skip log4j configuration
> -----------------------------------
>
>                 Key: CASSANDRA-3061
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-3061
>             Project: Cassandra
>          Issue Type: Improvement
>          Components: Core
>    Affects Versions: 0.8.4
>            Reporter: Aaron Morton
>            Assignee: Aaron Morton
>            Priority: Minor
>             Fix For: 0.8.5
>
>         Attachments: 0001-3061.patch, 3061_v2.txt
>
>
> from this thread http://groups.google.com/group/brisk-users/browse_thread/thread/3a18f4679673bea8
> When brisk accesses cassandra classes inside of a Hadoop Task JVM the AbstractCassandraDaemon
uses a log4j PropertyConfigurator to setup cassandra logging. This closes all the existing
appenders, including the TaskLogAppender for the hadoop task. They are not opened again because
they are not in the config. 
> log4j has Logger Repositories to handle multiple configs in the same process, but there
is a bit of suck involved in making a RepositorySelector. 
> Two examples...
> http://www.mail-archive.com/log4j-dev@jakarta.apache.org/msg02972.html
> http://docs.redhat.com/docs/en-US/JBoss_Enterprise_Application_Platform/4.2/html/Getting_Started_Guide/logging.log4j.reposelect.html
> Basically all the selector has access to thread local storage, and it looks like normally
people get the class loader from the current thread. A thread will inherit it's class loader
from the thread that created it, unless otherwise specified. 
> We have code in the same thread the uses hadoop and cassandra classes, so this could
be a dead end.  
> As a work around i've added cassandra.log4j.configure JVM param and made the AbstractCassandraServer
skip the log4j config if it's false. My job completes and I can see the cassandra code logging
an extra message I put in into the Hadoop task log file...
> 2011-08-19 15:56:06,442 WARN org.apache.hadoop.metrics2.impl.MetricsSystemImpl: Metrics
system not started: Cannot locate configuration: tried hadoop-metrics2-maptask.properties,
hadoop-metrics2.properties
> 2011-08-19 15:56:06,776 INFO org.apache.cassandra.service.AbstractCassandraDaemon: Logging
initialized externally
> 2011-08-19 15:56:07,332 INFO org.apache.hadoop.mapred.MapTask: numReduceTasks: 0
>  
> The param has to be passed to the task JVM, so need to modify Haddop mapred-site.xml
as follows 
> <property>
>   <name>mapred.child.java.opts</name>
>   <value>-Xmx256m -Dcassandra.log4j.configure=false</value>
>   <description>
>     Tune your mapred jvm arguments for best performance. 
>     Also see documentation from jvm vendor.
>   </description>
> </property>
> It's not pretty but it works. In my extra log4j logging I can see the second reset()
call is gone.  
> Change the to Hadoop TaskLogAppender also stops the NPE but there may also be some lost
log messages 
> https://issues.apache.org/jira/browse/HADOOP-7556

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message