giraph-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Carlo Sartiani <sarti...@gmail.com>
Subject Re: Problems when running Giraph1.0 on Hadoop 2.0.5alpha.gphd2.1.0
Date Mon, 17 Mar 2014 12:29:34 GMT
The cluster uses Pivotal own version of Hadoop 2.0.5alpha. We are 
submitting the Giraph job by using the standard GIRAPH api. The cluster 
tries to activate a mapreducev2 job, but it seems failing in launching 
the web server. The strange thing is that, if we use a "standard" 
version of Hadoop 2.0.5alpha, everything works fine.

This is the piece of code we use to configure and launch the job:

GiraphConfiguration conf = new GiraphConfiguration();


         conf.setEdgeInputFormatClass(VertigoTextEdgeInputFormat.class);
         conf.setVertexClass(VertigoVertex.class); // vertex class
         conf.VERTEX_EDGES_CLASS.set(conf, VertigoEdges.class);

         conf.MASTER_COMPUTE_CLASS.set(conf, 
VertigoExtendedMasterComputation.class);

         conf.SPLIT_MASTER_WORKER.set(conf, false);
         conf.STATIC_GRAPH.set(conf, true);
         //conf.setLocalTestMode(true);
         conf.setLocalTestMode(false);
         conf.setWorkerConfiguration(1,1, 100);
         conf.NUM_COMPUTE_THREADS.set(conf, 4);
         String splitValue = conf.get(LINE_TOKENIZE_VALUE, 
LINE_TOKENIZE_VALUE_DEFAULT);
         System.out.println("MAX_WORKERS: " + conf.getMaxWorkers());
         System.out.println("LINE_TOKENIZE_VALUE: " + splitValue);
         conf.setZooKeeperConfiguration(new 
String("node0522:2181,node0722:2181,node0922:2181"));



         GiraphFileInputFormat.setEdgeInputPath(conf, this.inputFile);
         GiraphJob job = new GiraphJob(conf, "corejob");


         // prepare dir for final results
         try {
             // create a directory where superstep results will be stored

             Path resultDir = new 
org.apache.hadoop.fs.Path(this.fileSystem.getHomeDirectory(), "results");
             if (this.fileSystem.exists(resultDir)) {
                 this.fileSystem.delete(resultDir, true);
             }
             boolean res = this.fileSystem.mkdirs(resultDir);

         }
         catch(Exception e) {
             e.printStackTrace();
         }


         long start = System.currentTimeMillis();
         boolean result = job.run(true);
         long stop = System.currentTimeMillis();

We have not tried to use Giraph1.1.0, as we were not able to compile it 
on Hadoop2.0.5alpha.

Carlo


> Eli Reisman <mailto:apache.mailbox@gmail.com>
> 16 Mar 2014 21:01
> it looks like the Pivotal cluster is running on YARN or trying to 
> launch Giraph on YARN from the stack trace?
>
>
>
> Carlo Sartiani <mailto:sartiani@gmail.com>
> 13 Mar 2014 12:34
> Hi all.
>
> I am encountering a strange problem when running a Giraph application 
> on a cluster with Hadoop 2.0.5alpha.gphd2.1.0 (Pivotal Hadoop 
> distribution).
>
> Our application is based on Giraph 1.0 and is executed in mapreduce 
> compatibility mode  rather than in yarn mode. The application works 
> smoothly on a standard Hadoop 2.0.5alpha distribution in our local 
> cluster. However, when moved to an external cluster with Pivotal 
> Hadoop 2.0.5alpha, it returns the  error described by the following log:
>
> 2014-03-12 17:33:21,489 INFO [main] 
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster: Created MRAppMaster 
> for application appattempt_1388715708455_1597_000001
> 2014-03-1217:33:21,722 WARN [main] 
> org.apache.hadoop.conf.Configuration: job.xml:an attempt to override 
> final parameter: mapreduce.job.end-notification.max.retry.interval;  
> Ignoring.
> 2014-03-12 17:33:21,723 WARN [main] 
> org.apache.hadoop.conf.Configuration: job.xml:an attempt to override 
> final parameter: mapreduce.job.end-notification.max.attempts;  Ignoring.
> 2014-03-12 17:33:21,989 WARN [main] 
> org.apache.hadoop.conf.Configuration: job.xml:an attempt to override 
> final parameter: mapreduce.job.end-notification.max.retry.interval;  
> Ignoring.
> 2014-03-12 17:33:21,989 WARN [main] 
> org.apache.hadoop.conf.Configuration: job.xml:an attempt to override 
> final parameter: mapreduce.job.end-notification.max.attempts;  Ignoring.
> 2014-03-12 17:33:22,005 INFO [main] 
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster: 
> jobSubmitDir=file:/data5/phd/yarn/nm-local-dir/usercache/csartiani/appcache/application_1388715708455_1597/container_1388715708455_1597_01_000001/jobSubmitDir

> jobTokenFile=file:/data5/phd/yarn/nm-local-dir/usercache/csartiani/appcache/application_1388715708455_1597/container_1388715708455_1597_01_000001/jobSubmitDir/appTokens

>
> 2014-03-12 17:33:22,005 INFO [main] 
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster: AM Retries: 1 attempt 
> num: 1 is last retry: true
> 2014-03-12 17:33:22,013 INFO [main] 
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster: Using mapred 
> newApiCommitter.
> 2014-03-12 17:33:22,315 INFO [main] 
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster: OutputCommitter set in 
> config null
> 2014-03-12 17:33:22,377 WARN [main] 
> org.apache.giraph.bsp.BspOutputFormat: getOutputCommitter: Returning 
> ImmutableOutputCommiter (does nothing).
> 2014-03-12 17:33:22,377 INFO [main] 
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster: OutputCommitter is 
> org.apache.giraph.bsp.ImmutableOutputCommitter
> 2014-03-12 17:33:22,377 INFO [main] 
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster: Not starting 
> RecoveryService: recoveryEnabled: true recoverySupportedByCommitter: 
> false ApplicationAttemptID: 1
> 2014-03-1217:33:22,394 INFO [main] 
> org.apache.hadoop.yarn.event.AsyncDispatcher: Registering class 
> org.apache.hadoop.mapreduce.jobhistory.EventType for class 
> org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler
> 2014-03-12 17:33:22,395 INFO [main] 
> org.apache.hadoop.yarn.event.AsyncDispatcher: Registering class 
> org.apache.hadoop.mapreduce.v2.app.job.event.JobEventType for class 
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster$JobEventDispatcher
> 2014-03-12 17:33:22,396 INFO [main] 
> org.apache.hadoop.yarn.event.AsyncDispatcher: Registering class 
> org.apache.hadoop.mapreduce.v2.app.job.event.TaskEventType for class 
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster$TaskEventDispatcher
> 2014-03-12 17:33:22,396 INFO [main] 
> org.apache.hadoop.yarn.event.AsyncDispatcher: Registering class 
> org.apache.hadoop.mapreduce.v2.app.job.event.TaskAttemptEventType for 
> class 
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster$TaskAttemptEventDispatcher
> 2014-03-12 17:33:22,397 INFO [main] 
> org.apache.hadoop.yarn.event.AsyncDispatcher: Registering class 
> org.apache.hadoop.mapreduce.v2.app.commit.CommitterEventType for class 
> org.apache.hadoop.mapreduce.v2.app.commit.CommitterEventHandler
> 2014-03-12 17:33:22,401 INFO [main] 
> org.apache.hadoop.yarn.event.AsyncDispatcher: Registering class 
> org.apache.hadoop.mapreduce.v2.app.speculate.Speculator$EventType for 
> class 
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster$SpeculatorEventDispatcher
> 2014-03-12 17:33:22,402 INFO [main] 
> org.apache.hadoop.yarn.event.AsyncDispatcher: Registering class 
> org.apache.hadoop.mapreduce.v2.app.rm.ContainerAllocator$EventType for 
> class 
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster$ContainerAllocatorRouter
> 2014-03-12 17:33:22,403 INFO [main] 
> org.apache.hadoop.yarn.event.AsyncDispatcher: Registering class 
> org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncher$EventType for 
> class 
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster$ContainerLauncherRouter
> 2014-03-12 17:33:22,403 INFO [main] 
> org.apache.hadoop.yarn.service.AbstractService: Service:Dispatcher is 
> inited.
> 2014-03-12 17:33:22,403 INFO [main] 
> org.apache.hadoop.yarn.service.AbstractService: 
> Service:MRClientService is inited.
> 2014-03-12 17:33:22,403 INFO [main] 
> org.apache.hadoop.yarn.service.AbstractService: 
> Service:CommitterEventHandler is inited.
> 2014-03-12 17:33:22,405 INFO [main] 
> org.apache.hadoop.yarn.service.AbstractService: 
> Service:TaskHeartbeatHandler is inited.
> 2014-03-12 17:33:22,405 INFO [main] 
> org.apache.hadoop.yarn.service.AbstractService: 
> Service:org.apache.hadoop.mapred.TaskAttemptListenerImpl is inited.
> 2014-03-12 17:33:22,405 INFO [main] 
> org.apache.hadoop.yarn.service.AbstractService: 
> Service:org.apache.hadoop.mapreduce.v2.app.speculate.DefaultSpeculator 
> is inited.
> 2014-03-12 17:33:22,405 INFO [main] 
> org.apache.hadoop.yarn.service.AbstractService: 
> Service:org.apache.hadoop.mapreduce.v2.app.MRAppMaster$ContainerAllocatorRouter 
> is inited.
> 2014-03-12 17:33:22,405 INFO [main] 
> org.apache.hadoop.yarn.service.AbstractService: 
> Service:org.apache.hadoop.mapreduce.v2.app.MRAppMaster$ContainerLauncherRouter 
> is inited.
> 2014-03-12 17:33:22,405 INFO [main] 
> org.apache.hadoop.yarn.service.AbstractService: 
> Service:org.apache.hadoop.mapreduce.v2.app.MRAppMaster$StagingDirCleaningService 
> is inited.
> 2014-03-12 17:33:22,411 INFO [main] 
> org.apache.hadoop.yarn.service.AbstractService: 
> Service:JobHistoryEventHandler is inited.
> 2014-03-12 17:33:22,411 INFO [main] 
> org.apache.hadoop.yarn.service.AbstractService: 
> Service:org.apache.hadoop.mapreduce.v2.app.MRAppMaster is inited.
> 2014-03-12 17:33:22,460 INFO [main] 
> org.apache.hadoop.yarn.event.AsyncDispatcher: Registering class 
> org.apache.hadoop.mapreduce.v2.app.job.event.JobFinishEvent$Type for 
> class 
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster$JobFinishEventHandler
> 2014-03-12 17:33:22,653 INFO [main] 
> org.apache.hadoop.metrics2.impl.MetricsConfig: loaded properties from 
> hadoop-metrics2.properties
> 2014-03-12 17:33:22,696 INFO [main] 
> org.apache.hadoop.metrics2.impl.MetricsSystemImpl: Scheduled snapshot 
> period at 10 second(s).
> 2014-03-12 17:33:22,696 INFO [main] 
> org.apache.hadoop.metrics2.impl.MetricsSystemImpl: MRAppMaster metrics 
> system started
> 2014-03-12 17:33:22,704 INFO [main] 
> org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl: Adding job token 
> for job_1388715708455_1597 to jobTokenSecretManager
> 2014-03-12 17:33:22,785 INFO [main] 
> org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl: Not uberizing 
> job_1388715708455_1597 because: not enabled; too much RAM;
> 2014-03-12 17:33:22,799 INFO [main] 
> org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl: Input size for 
> job job_1388715708455_1597 = 0. Number of splits = 1
> 2014-03-1217:33:22,799 INFO [main] 
> org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl: Number of reduces 
> for job job_1388715708455_1597 = 0
> 2014-03-12 17:33:22,799 INFO [main] 
> org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl: 
> job_1388715708455_1597Job Transitioned from NEW to INITED
> 2014-03-12 17:33:22,799 INFO [main] 
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster: MRAppMaster launching 
> normal, non-uberized, multi-container job job_1388715708455_1597.
> 2014-03-12 17:33:22,799 INFO [main] 
> org.apache.hadoop.yarn.service.AbstractService: Service:Dispatcher is 
> started.
> 2014-03-12 17:33:22,801 INFO [AsyncDispatcher event handler] 
> org.apache.hadoop.mapreduce.v2.app.speculate.DefaultSpeculator: 
> JOB_CREATE job_1388715708455_1597
> 2014-03-12 17:33:22,833 INFO [Socket Reader #1 for port 43578] 
> org.apache.hadoop.ipc.Server: Starting Socket Reader #1 for port 43578
> 2014-03-12 17:33:22,846 INFO [main] 
> org.apache.hadoop.yarn.factories.impl.pb.RpcServerFactoryPBImpl: 
> Adding protocol org.apache.hadoop.mapreduce.v2.api.MRClientProtocolPB 
> to the server
> 2014-03-12 17:33:22,847 INFO [IPC Server Responder] 
> org.apache.hadoop.ipc.Server: IPC Server Responder: starting
> 2014-03-12 17:33:22,847 INFO [IPC Server listener on 43578] 
> org.apache.hadoop.ipc.Server: IPC Server listener on 43578: starting
> 2014-03-12 17:33:22,847 INFO [main] 
> org.apache.hadoop.mapreduce.v2.app.client.MRClientService: 
> Instantiated MRClientService at node0801/10.1.41.1:43578
> 2014-03-12 17:33:22,867 ERROR [main] 
> org.apache.hadoop.yarn.service.CompositeService: Error starting 
> services org.apache.hadoop.mapreduce.v2.app.MRAppMaster
> java.lang.NoSuchMethodError: 
> org.apache.hadoop.http.HttpServer.<init>(Ljava/lang/String;Ljava/lang/String;IZLorg/apache/hadoop/conf/Configuration;Lorg/apache/hadoop/security/authorize/AccessControlList;Lorg/mortbay/jetty/Connector;[Ljava/lang/String;)V

>
>     at 
> org.apache.hadoop.yarn.webapp.WebApps$Builder.start(WebApps.java:196)
>     at 
> org.apache.hadoop.mapreduce.v2.app.client.MRClientService.start(MRClientService.java:149)

>
>     at 
> org.apache.hadoop.yarn.service.CompositeService.start(CompositeService.java:68) 
>
>     at 
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster.start(MRAppMaster.java:1022) 
>
>     at 
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster$1.run(MRAppMaster.java:1276) 
>
>     at java.security.AccessController.doPrivileged(Native Method)
>     at javax.security.auth.Subject.doAs(Subject.java:396)
>     at 
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1478)

>
>     at 
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster.initAndStartAppMaster(MRAppMaster.java:1272)

>
>     at 
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster.main(MRAppMaster.java:1228) 
>
> 2014-03-12 17:33:22,868 INFO [main] org.apache.hadoop.ipc.Server: 
> Stopping server on 43578
> 2014-03-12 17:33:22,869 INFO [IPC Server listener on 43578] 
> org.apache.hadoop.ipc.Server: Stopping IPC Server listener on 43578
> 2014-03-12 17:33:22,869 INFO [main] 
> org.apache.hadoop.yarn.service.AbstractService: Service:Dispatcher is 
> stopped.
> 2014-03-12 17:33:22,869 INFO [IPC Server Responder] 
> org.apache.hadoop.ipc.Server: Stopping IPC Server Responder
> 2014-03-12 17:33:22,869 FATAL [main] 
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster: Error starting 
> MRAppMaster
> org.apache.hadoop.yarn.YarnException: Failed to Start 
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster
>     at 
> org.apache.hadoop.yarn.service.CompositeService.start(CompositeService.java:78) 
>
>     at 
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster.start(MRAppMaster.java:1022) 
>
>     at 
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster$1.run(MRAppMaster.java:1276) 
>
>     at java.security.AccessController.doPrivileged(Native Method)
>     at javax.security.auth.Subject.doAs(Subject.java:396)
>     at 
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1478)

>
>     at 
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster.initAndStartAppMaster(MRAppMaster.java:1272)

>
>     at 
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster.main(MRAppMaster.java:1228) 
>
> Caused by: java.lang.NoSuchMethodError: 
> org.apache.hadoop.http.HttpServer.<init>(Ljava/lang/String;Ljava/lang/String;IZLorg/apache/hadoop/conf/Configuration;Lorg/apache/hadoop/security/authorize/AccessControlList;Lorg/mortbay/jetty/Connector;[Ljava/lang/String;)V

>
>     at 
> org.apache.hadoop.yarn.webapp.WebApps$Builder.start(WebApps.java:196)
>     at 
> org.apache.hadoop.mapreduce.v2.app.client.MRClientService.start(MRClientService.java:149)

>
>     at 
> org.apache.hadoop.yarn.service.CompositeService.start(CompositeService.java:68) 
>
>     ... 7 more
> 2014-03-12 17:33:22,870 INFO [Thread-1] 
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster: MRAppMaster received a 
> signal. Signaling RMCommunicator and JobHistoryEventHandler.
> 2014-03-12 17:33:22,871 WARN [Thread-1] 
> org.apache.hadoop.util.ShutdownHookManager: ShutdownHook 
> 'MRAppMasterShutdownHook' failed, java.lang.NullPointerException
> java.lang.NullPointerException
>     at 
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster$ContainerAllocatorRouter.setSignalled(MRAppMaster.java:811)

>
>     at 
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster$MRAppMasterShutdownHook.run(MRAppMaster.java:1249)

>
>     at 
> org.apache.hadoop.util.ShutdownHookManager$1.run(ShutdownHookManager.java:54) 
>
>
> The error seems quite strange, as the system is reporting the lack of  
> a given constructor for org.apache.hadoop.http.HttpServer.
>
> Other applications that we are running on this cluster and that do not 
> use Giraph work perfectly. We have no control on this cluster, so we 
> cannot modify any cluster setting.
>
> Do you any idea on how to solve this issue?
>
> Thanks in advance.
>
> Best Regards
>
> Carlo Sartiani
> ------------------------------------------------------------------------


-- 
"Humanity is overrated", Gregory House, MD

Mime
View raw message