giraph-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Anna Xenaki <axen...@mail.ntua.gr>
Subject Re: giraph-netbeans -simple question
Date Fri, 22 May 2015 16:02:44 GMT
Finaly I got a maven project up and running Depedencies and all. I added 
a test class this is what I get
public class KatzReduceTest {

     public KatzReduceTest() {
     }

     @BeforeClass
     public static void setUpClass() {
     }

     @AfterClass
     public static void tearDownClass() {
     }

     @Before
     public void setUp() {
     }

     @After
     public void tearDown() {
     }

     /**
      * Test of compute method, of class KatzReduce.
      */
     @Test
     public void testCompute() throws Exception {
         System.out.println("compute");
         String[] tinyGraph = 
{"1#2,3#54,78","2#1,3,4#78,63","3#1,2,4#77,125","4#2,3#77,63" };
         // This is where you configure your job
         GiraphConfiguration conf = new GiraphConfiguration();
         conf.setComputationClass(KatzReduce.class);
   conf.setMasterComputeClass();
          conf.setEdgeInputFormatClass();
     conf.setVertexInputFormatClass();

         conf.setVertexInputFormatClass(FbGraphInputFormat.class);
         conf.setVertexOutputFormatClass(FbGraphOutputFormat.class);
         // Run and print results
         Iterable<String> results = InternalVertexRunner.run(conf, 
tinyGraph, tinyGraph);
         for (String result : results) {

         }
     }
would you kindly provide me with an example  as to what to add in 
MasterComputeClass (difference with simple computationclass) and 
EdgeInputFormat , VertexInputFormat (should those two be totally 
seperate? and If so in Edge input format how do I add the edge 
(Source-Target ValueId) if I dont have a vertex created when I read the 
input

Thanks in advance, Anna
On 22/5/2015 5:53 μμ, Anna Xenaki wrote:
> Hi, I dont have a maven project, I have never created one before so I 
> dont know the process. I have a few questions:
> 1. If I compile it from terminal how do I do that? Do I have to create 
> a FbGraphRunner class? make the whole project a jar? and then run " 
> $HADOOP_HOME/bin/hadoop jar my.jar my.project.folder.myFbGraphRunner 
> my.project.folder.KatzReduce -vif my.project.folder.FbGraphInputFormat 
> -vip /input/fb0.txt -vof my.project.folder.FbGraphOutputFormat -op 
> /output/katz1.txt -w 1 -ca giraph.SplitMasterWorker=false" ?
> 2. Do I need to have a class that extends MasterCompute class?
> 3.When I create a Maven project for giraph what are my dependencies? 
> do I only put the one you mentioned below? Do I have any libraries 
> (like the ones I added in my java project)?
> 4.In your suggestion you mentioned   EdgeInputFormatClass and 
> VertexInputFormatClass in order to import a Graph in Giraph do I need 
> to have both?To  Import edges and vertexes separately I currently have 
> only one InputFormat and add edges as a List with createEdgeFactory
>
> Best Regards, Anna
>
> On 22/5/2015 4:16 μμ, Panagiotis Eustratiadis wrote:
>> Hello Anna,
>>
>> I use two ways to run a Giraph job for testing. The one is to setup a 
>> single node cluster (there are many tutorials on that) and run your 
>> job via the command line.
>>
>> The other way (through NetBeans) is Junit testing. Add those lines to 
>> your pom:
>>
>> <dependency>
>>     <groupId>junit</groupId>
>>     <artifactId>junit</artifactId>
>>     <version>4.11</version>
>> </dependency>
>>
>> Then create a test package in your netbeans project. Normally there 
>> are two directories under your_root/src called "main" and "test". If 
>> there isn't one named "test", you should create and structure it 
>> normally, like the other one (add "java" and "resources" directories, 
>> packages, etc.). After that you will be able to see it through your 
>> IDE as well, and manage it like the main package. Finally, you can 
>> write Junit tests. Create a class called "Tests.java" or whatever 
>> else you want, and write methods in it using the @Test annotation. 
>> For example:
>>
>> @Test
>> public void yourTestFunction() throws Exception {
>>
>>     String[] tinyGraph = {
>>         "1 2", "1 3",
>>         "2 1", "2 4",
>>         "3 1", "3 4",
>>         "4 2", "4 3", "4 5", "4 6",
>>         "5 4", "5 7",
>>         "6 4", "6 7",
>>         "7 5", "7 6"
>>     };
>>
>>     // This is where you configure your job
>> GiraphConfiguration conf = new GiraphConfiguration();
>>     conf.setComputationClass();
>>     conf.setMasterComputeClass();
>>     conf.setEdgeInputFormatClass();
>>     conf.setVertexInputFormatClass();
>>     conf.setVertexOutputFormatClass();
>>
>>     // Run and print results
>> Iterable<String> results = InternalVertexRunner.run(conf, tinyGraph, 
>> tinyGraph);
>>     for (String result : results) {
>>         LOGGER.debug(result);
>>     }
>>
>> }
>>
>> Note that the tests will run every time you compile it, unless you 
>> specify -DskipTests in the mvn command.
>> Regards,
>> Panagiotis Eustratiadis.
>>
>>
>> On 22 May 2015 at 15:43, Anna Xenaki <axenaki@mail.ntua.gr 
>> <mailto:axenaki@mail.ntua.gr>> wrote:
>>
>>     Hi, sorry for the inconvenience I found your mail via giraph
>>     mailing list. Im new in Giraph.
>>     I want to run a giraph job via netbeans. I have imported all libs
>>     in project , and installed hadoop 2.5.2 Giraph 1.1.0 in ubuntu 14.04.
>>     I created Java classes FbGraphInputFormat, FbGraphMessage,
>>     FbGraphOutputFormat,FbGraphState, KatzReduce
>>
>>     public class KatzReduce extends BasicComputation<Text,
>>     FbGraphState, DoubleWritable, FbGraphMessage> {
>>     public class FbGraphState implements Writable {
>>     public class FbGraphOutputFormat  extends
>>     TextVertexOutputFormat<Text,FbGraphState,DoubleWritable>{
>>     public class FbGraphInputFormat extends
>>     TextVertexInputFormat<Text, FbGraphState, DoubleWritable> {
>>     class FbGraphMessage implements Writable{
>>
>>
>>     MY QUESTION is simple I want to run my java project from netbeans
>>     I dont know how . I dont have a main class and do not know to
>>     start a giraph job that does exactly what KatzReduce.java says.
>>
>>     I added in the run of the project the arguments
>>     infile_location_path  outfile_location_path
>>
>>     Do I create a main method saying what? I cannot find any
>>     documentation on deploying giraph projects from scratch so any
>>     help would be of great assistance.
>>
>>     Thanks in advance, Anna
>>
>>
>


Mime
View raw message