hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Vinod K V (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-6788) [Herriot] Exception exclusion functionality is not working correctly.
Date Fri, 04 Jun 2010 03:57:53 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-6788?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12875457#action_12875457
] 

Vinod K V commented on HADOOP-6788:
-----------------------------------

bq. Well, I'm not sure about this. Herriot never was approved/discussed by the community as
an official name of the framework. Let's think about this first.
Sure Kos. My main concern is that the titles for most of these issues are fairly generic and
on a surface look aren't very reflective of what they are really about, and which component.

So, yes, I'll stop messing with the titles till the name issue is resolved.

> [Herriot] Exception exclusion functionality is not working correctly.
> ---------------------------------------------------------------------
>
>                 Key: HADOOP-6788
>                 URL: https://issues.apache.org/jira/browse/HADOOP-6788
>             Project: Hadoop Common
>          Issue Type: Bug
>          Components: test
>    Affects Versions: 0.21.0
>            Reporter: Vinay Kumar Thota
>            Assignee: Balaji Rajagopalan
>         Attachments: 6788-ydist-security.patch, 6788-ydist-security.patch, HADOOP-6788.patch,
HADOOP-6788.patch
>
>
> Exception exclusion functionality is not working correctly because of that tests are
failing by not matching the error count.
> I debugged the issue and found that the problem with shell command which is generating
in the getNumberOfMatchesInLogFile function.
> Currently building the shell command in the following way. 
> if(list != null){
>   for(int i =0; i < list.length; ++i)
>   {
>     filePattern.append(" | grep -v " + list[i] );
>   }
> }
>     String[] cmd =
>         new String[] {
>             "bash",
>             "-c",
>             "grep -c "
>                 + pattern + " " + filePattern
>                 + " | awk -F: '{s+=$2} END {print s}'" };    
> However, The above commnad won't work correctly because you are counting the exceptions
in the file before excluding the known exceptions.
> In this case it gives the mismatch error counts everytime.The shell command should be
in the following way to work correctly.
> if (list != null) {
>   int index = 0;
>   for (String excludeExp : list) {
>     filePattern.append((++index < list.length)? "| grep -v " : 
>             "| grep -vc " + list[i] );  
>   }
> }
> String[] cmd =
>    new String[] {
>        "bash",
>        "-c",
>        "grep "
>            + pattern + " " + filePattern
>            + " | awk -F: '{s+=$2} END {print s}'" };  

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message