hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "leyli javid (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-1228) Eclipse project files
Date Thu, 17 May 2012 08:24:11 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-1228?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13277655#comment-13277655
] 

leyli javid commented on HADOOP-1228:
-------------------------------------

I have created Map/reduce project in Eclipse. My program is working properly. I measure the
time takes to do the job by putting Two long variable start and finish before and after job.waitForCompletion(true)
as follow:
start=System.currentTimeMilli
boolean b = job.waitForCompletion(true); 
finish=System.currentTimeMillis();
But the problem is that the job takes more that the normal program which I wrote without Map/reduce.
I do not know if I measure the time correctly. And also how could I reach to history file
of job when I am running program from eclipse? 
I have Three text files each contain about 80,000 binary string records. Given a string like
A=(1001001001111......), I would like to search among all text files and check weather the
zero bits in a A, are also zero in a record. I also would like to know wether Map/reduce job
always has to work on some text files and in String format or not?
 while (itr.hasMoreTokens()) 
      {
    		word.set(itr.nextToken());
    		for (int i=0;i<queryIndex.length() && match ;i++ )
    			{
	    			if (queryIndex.charAt(i)=='0')
	    			{
		    			if (word.charAt(i)!=queryIndex.charAt(i))
			    			{
			    				match=false;
			    			}
		    		}
    			}
    		if (match)
    		{
    			Text temp=new Text("user"+counter);
    			context.write(temp, one);
    		}	
      }
      counter++;
    }

Thanks  a lot
                
> Eclipse project files
> ---------------------
>
>                 Key: HADOOP-1228
>                 URL: https://issues.apache.org/jira/browse/HADOOP-1228
>             Project: Hadoop Common
>          Issue Type: Improvement
>          Components: build
>            Reporter: Albert Strasheim
>            Assignee: Tom White
>            Priority: Minor
>             Fix For: 0.17.0
>
>         Attachments: .classpath, .project, eclipse.patch, hadoop-1228-v2.patch, hadoop-1228.patch,
hadoop-eclipse.zip
>
>
> I've created Eclipse project files for Hadoop (to be attached). I've found them very
useful for exploring Hadoop and running the unit tests.
> The project files can be included in the source repository to make it easy to import
Hadoop into Eclipse.
> A few features:
> - Eclipse automatically calls the Ant build to generate some of the necessary source
files
> - Single unit tests can be run from inside Eclipse
> - Basic Java code style formatter settings for the Hadoop conventions (still needs some
work)
> The following VM arguments must be specified in the run configuration to get unit tests
to run:
> -Xms256m -Xmx256m -Dtest.build.data=${project_loc}\build\test\data
> Some of the unit tests don't run yet, possibly due to some missing VM flags, the fact
that I'm running Windows, or some other reason(s).
> TODO:
> - Specify native library location(s) once I investigate building of Hadoop's native library
> - Get all the unit tests to run

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message