chukwa-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jie Huang (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (CHUKWA-650) Re-configure Demux ReduceNumber without re-starting the DemuxManager service
Date Mon, 30 Jul 2012 07:28:33 GMT

    [ https://issues.apache.org/jira/browse/CHUKWA-650?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13424725#comment-13424725
] 

Jie Huang commented on CHUKWA-650:
----------------------------------

@Ari, I have verified the patch on a single-node deployment. The following Demux log to demonstrate
the final effect.
{noformat}
2012-07-30 14:38:22,395 INFO main DemuxManager - old reduce number: 5
2012-07-30 14:38:22,415 INFO main DemuxManager - new reduce number: 10
2012-07-30 14:38:22,424 INFO main ChukwaConfiguration - chukwaConf is /home/Grace/work/chukwa-incubating-0.6.0/etc/chukwa
2012-07-30 14:38:22,453 WARN main JobClient - Use GenericOptionsParser for parsing the arguments.
Applications should implement Tool for the same.
2012-07-30 14:38:22,544 INFO main FileInputFormat - Total input paths to process : 1
2012-07-30 14:38:22,678 INFO main JobClient - Running job: job_201207291428_0009
{noformat}

Any comment, please let me know. thanks for your time.
                
> Re-configure Demux ReduceNumber without re-starting the DemuxManager service
> ----------------------------------------------------------------------------
>
>                 Key: CHUKWA-650
>                 URL: https://issues.apache.org/jira/browse/CHUKWA-650
>             Project: Chukwa
>          Issue Type: Improvement
>          Components: Data Processors
>    Affects Versions: 0.6.0
>            Reporter: Jie Huang
>            Assignee: Jie Huang
>         Attachments: chukwa-650.patch
>
>
> Currently, DemuxManager controls the Demux's reduce number by reading "CHUKWA_DEMUX_REDUCER_COUNT_FIELD"
at the initialization part. It is impossible to tune that reduce number before the next round
of demux happens. The only way is to restart that DemuxManager service. I wonder if it is
OK to parse that configuration item in  runDemux() function.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message