Return-Path: X-Original-To: apmail-airavata-dev-archive@www.apache.org Delivered-To: apmail-airavata-dev-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 27FFF18A94 for ; Mon, 17 Aug 2015 13:48:47 +0000 (UTC) Received: (qmail 97955 invoked by uid 500); 17 Aug 2015 13:48:47 -0000 Delivered-To: apmail-airavata-dev-archive@airavata.apache.org Received: (qmail 97916 invoked by uid 500); 17 Aug 2015 13:48:47 -0000 Mailing-List: contact dev-help@airavata.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@airavata.apache.org Delivered-To: mailing list dev@airavata.apache.org Received: (qmail 97708 invoked by uid 99); 17 Aug 2015 13:48:46 -0000 Received: from mail-relay.apache.org (HELO mail-relay.apache.org) (140.211.11.15) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 17 Aug 2015 13:48:46 +0000 Received: from mail-ig0-f175.google.com (mail-ig0-f175.google.com [209.85.213.175]) by mail-relay.apache.org (ASF Mail Server at mail-relay.apache.org) with ESMTPSA id 76F371A009B for ; Mon, 17 Aug 2015 13:48:46 +0000 (UTC) Received: by igui7 with SMTP id i7so57307759igu.1 for ; Mon, 17 Aug 2015 06:48:45 -0700 (PDT) X-Received: by 10.50.143.101 with SMTP id sd5mr15682480igb.44.1439819325648; Mon, 17 Aug 2015 06:48:45 -0700 (PDT) MIME-Version: 1.0 References: <8F2DF782-2305-4FBF-BD6A-9EDA82572B82@illinois.edu> In-Reply-To: <8F2DF782-2305-4FBF-BD6A-9EDA82572B82@illinois.edu> From: Suresh Marru Date: Mon, 17 Aug 2015 13:48:36 +0000 Message-ID: Subject: Re: Permission error at submitting Gaussian jobs to Airavata To: "dev@airavata.apache.org" Content-Type: multipart/alternative; boundary=001a1134bad072383e051d820fb7 --001a1134bad072383e051d820fb7 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Hi Sudhakar, Both PGA and command line invocation should use same account. But you are right that scigap account is not added, I will get that fixed now. Suresh On Mon, Aug 17, 2015 at 8:48 AM Pamidighantam, Sudhakar V < spamidig@illinois.edu> wrote: > From PGA it may be going to ogce login on Comet and I noticed here it is > using scigap login which does not appear to be in gaussian user group. > > Sudhakar. > > On Aug 17, 2015, at 6:41 AM, DImuthu Upeksha > wrote: > > Hi Sudhakar, > > I'm using my user 'dimuthu' to submit the job. However from PGA I was abl= e > to submit using same user. > > On Mon, Aug 17, 2015 at 4:34 PM, Pamidighantam, Sudhakar V < > spamidig@illinois.edu> wrote: > >> Dimuthu: >> Are you using ind123 for the account to submit the job on comet. Also >> scigap user need to be added to gaussian group on comet for this gaussia= n >> job to execute. >> >> Thanks, >> Sudhakar. >> On Aug 17, 2015, at 5:54 AM, DImuthu Upeksha >> wrote: >> >> Hi all, >> >> $subject please. What could be the reason for this? Hope the issue is >> with this >> >> org.apache.airavata.gsi.ssh.impl.GSISSHAbstractCluster - Standard Error >> output : sbatch: error: bank_limit plugin: expired user, can't submit jo= b >> >> Here is the full log >> -------------------- >> >> 2015-08-17 06:48:07,601 [Thread-9] INFO >> org.apache.airavata.gfac.monitor.email.EmailBasedMonitor - [EJM]: Job >> Monitor Map is empty, no need to retrieve emails >> >> 2015-08-17 06:48:17,602 [Thread-9] INFO >> org.apache.airavata.gfac.monitor.email.EmailBasedMonitor - [EJM]: Job >> Monitor Map is empty, no need to retrieve emails >> >> 2015-08-17 06:48:27,603 [Thread-9] INFO >> org.apache.airavata.gfac.monitor.email.EmailBasedMonitor - [EJM]: Job >> Monitor Map is empty, no need to retrieve emails >> >> 2015-08-17 06:48:37,603 [Thread-9] INFO >> org.apache.airavata.gfac.monitor.email.EmailBasedMonitor - [EJM]: Job >> Monitor Map is empty, no need to retrieve emails >> >> 2015-08-17 06:48:43,612 [pool-3-thread-3] INFO >> org.apache.airavata.gfac.core.utils.GFacUtils - This is a new Job, so >> creating all the experiment docs from the scratch >> >> 2015-08-17 06:48:43,679 [pool-3-thread-3] INFO >> org.apache.airavata.gfac.core.utils.GFacUtils - Created the node: >> /gfac-experiments/gfac-node0/dimuthu_experiment_gau_11_6e6ffc70-5dd2-4a9= f-94b6-8cd6e672ec08/state >> successfully ! >> >> 2015-08-17 06:48:43,692 [pool-3-thread-3] INFO >> org.apache.airavata.gfac.server.GfacServerHandler - >> -----------------------------------------------------92-----------------= ------------------------------------ >> >> 2015-08-17 06:48:43,692 [pool-3-thread-3] INFO >> org.apache.airavata.gfac.server.GfacServerHandler - >> Id:dimuthu_experiment_gau_11_6e6ffc70-5dd2-4a9f-94b6-8cd6e672ec08 : GFac >> Received submit job request for the Experiment: >> dimuthu_experiment_gau_11_6e6ffc70-5dd2-4a9f-94b6-8cd6e672ec08 TaskId: >> tempNode_56f41083-10be-4161-8803-ee1106bd3f54 >> >> 2015-08-17 06:48:44,595 [pool-8-thread-56] ERROR >> org.apache.airavata.gfac.core.monitor.MonitorID - There is not job crea= ted >> at this point >> >> 2015-08-17 06:48:44,599 [pool-8-thread-56] INFO >> org.apache.airavata.gfac.core.monitor.GfacInternalStatusUpdator - Gfac >> internal state changed to: ACCEPTED >> >> 2015-08-17 06:48:44,790 [pool-8-thread-56] ERROR >> org.apache.airavata.gfac.core.monitor.MonitorID - There is not job crea= ted >> at this point >> >> 2015-08-17 06:48:44,790 [pool-8-thread-56] INFO >> org.apache.airavata.gfac.core.monitor.GfacInternalStatusUpdator - Gfac >> internal state changed to: INHANDLERSINVOKING >> >> 2015-08-17 06:48:44,882 [pool-8-thread-56] INFO >> org.apache.airavata.gfac.core.handler.AppDescriptorCheckHandler - Invok= ing >> ApplicationDescriptorCheckHandler ... >> >> 2015-08-17 06:48:44,905 [pool-8-thread-56] INFO >> org.apache.airavata.gfac.core.handler.AppDescriptorCheckHandler - >> Recoverable data is saving to zk: >> /oasis/scratch/comet/scigap/temp_project/airavata-workdirs,/oasis/scratc= h/comet/scigap/temp_project/airavata-workdirs/dimuthu_experiment_gau_11_6e6= ffc70-5dd2-4a9f-94b6-8cd6e672ec08,/oasis/scratch/comet/scigap/temp_project/= airavata-workdirs/dimuthu_experiment_gau_11_6e6ffc70-5dd2-4a9f-94b6-8cd6e67= 2ec08,/oasis/scratch/comet/scigap/temp_project/airavata-workdirs/dimuthu_ex= periment_gau_11_6e6ffc70-5dd2-4a9f-94b6-8cd6e672ec08,/oasis/scratch/comet/s= cigap/temp_project/airavata-workdirs/dimuthu_experiment_gau_11_6e6ffc70-5dd= 2-4a9f-94b6-8cd6e672ec08/Gaussian.stdout,/oasis/scratch/comet/scigap/temp_p= roject/airavata-workdirs/dimuthu_experiment_gau_11_6e6ffc70-5dd2-4a9f-94b6-= 8cd6e672ec08/Gaussian.stderr >> >> 2015-08-17 06:48:45,216 [pool-8-thread-56] INFO >> org.apache.airavata.gsi.ssh.impl.GSISSHAbstractCluster - Listing >> directory: comet.sdsc.edu >> >> :~/ >> >> 2015-08-17 06:48:45,424 [pool-8-thread-56] INFO >> org.apache.airavata.gfac.ssh.util.GFACSSHUtils - Re-using the same >> connection used with the connection string:scigapcomet.sdsc.edu22 >> >> 2015-08-17 06:48:45,424 [pool-8-thread-56] INFO >> org.apache.airavata.gfac.ssh.handler.SSHDirectorySetupHandler - Setup S= SH >> job directorties >> >> 2015-08-17 06:48:45,441 [pool-8-thread-56] INFO >> org.apache.airavata.gfac.ssh.handler.SSHDirectorySetupHandler - >> Successfully retrieved the Security Context >> >> 2015-08-17 06:48:45,441 [pool-8-thread-56] INFO >> org.apache.airavata.gsi.ssh.impl.GSISSHAbstractCluster - Creating >> directory: comet.sdsc.edu >> >> :/oasis/scratch/comet/scigap/temp_project/airavata-workdirs/dimuthu_expe= riment_gau_11_6e6ffc70-5dd2-4a9f-94b6-8cd6e672ec08 >> >> 2015-08-17 06:48:45,924 [pool-8-thread-56] ERROR >> org.apache.airavata.gfac.ssh.handler.AdvancedSCPInputHandler - Previous= ly >> stored data is wrong so we continue the operations >> >> 2015-08-17 06:48:46,251 [pool-8-thread-56] INFO >> org.apache.airavata.gsi.ssh.impl.GSISSHAbstractCluster - Transfering >> from:localhost:/home/airavata/oodt/archive/ghqcgrdl57hmhbhd37p11qqjnl/c4= b4nhtwbs3.inp >> To:file://home/airavata/inputData/dimuthu_experiment_gau_11_6e6ffc70-5dd= 2-4a9f-94b6-8cd6e672ec08/tempNode_56f41083-10be-4161-8803-ee1106bd3f54 >> >> 2015-08-17 06:48:47,604 [Thread-9] INFO >> org.apache.airavata.gfac.monitor.email.EmailBasedMonitor - [EJM]: Job >> Monitor Map is empty, no need to retrieve emails >> >> 2015-08-17 06:48:48,263 [pool-8-thread-56] INFO >> org.apache.airavata.gfac.ssh.handler.SSHInputHandler - Successfully >> retrieved the Security Context >> >> 2015-08-17 06:48:48,265 [pool-8-thread-56] INFO >> org.apache.airavata.gfac.ssh.handler.SSHInputHandler - Invoking >> SCPInputHandler >> >> 2015-08-17 06:48:48,333 [pool-8-thread-56] INFO >> org.apache.airavata.gsi.ssh.impl.GSISSHAbstractCluster - Transfering >> file:////home/airavata/inputData/dimuthu_experiment_gau_11_6e6ffc70-5dd2= -4a9f-94b6-8cd6e672ec08/tempNode_56f41083-10be-4161-8803-ee1106bd3f54/c4b4n= htwbs3.inp >> To:comet.sdsc.edu: >> /oasis/scratch/comet/scigap/temp_project/airavata-workdirs/dimuthu_exper= iment_gau_11_6e6ffc70-5dd2-4a9f-94b6-8cd6e672ec08/c4b4nhtwbs3.inp >> >> 2015-08-17 06:48:49,125 [pool-8-thread-56] ERROR >> org.apache.airavata.gfac.core.monitor.MonitorID - There is not job crea= ted >> at this point >> >> 2015-08-17 06:48:49,125 [pool-8-thread-56] INFO >> org.apache.airavata.gfac.core.monitor.GfacInternalStatusUpdator - Gfac >> internal state changed to: INHANDLERSINVOKED >> >> 2015-08-17 06:48:49,199 [pool-8-thread-56] ERROR >> org.apache.airavata.gfac.core.monitor.MonitorID - There is not job crea= ted >> at this point >> >> 2015-08-17 06:48:49,200 [pool-8-thread-56] INFO >> org.apache.airavata.gfac.core.monitor.GfacInternalStatusUpdator - Gfac >> internal state changed to: PROVIDERINVOKING >> >> 2015-08-17 06:48:49,299 [pool-8-thread-56] INFO >> org.apache.airavata.gfac.ssh.provider.impl.SSHProvider - Successfully >> retrieved the Security Context >> >> 2015-08-17 06:48:49,346 [pool-8-thread-56] INFO >> org.apache.airavata.gfac.ssh.util.GFACSSHUtils - Email list: >> gw77jobs@scigap.org,jobnotif@scigap.org >> >> 2015-08-17 06:48:49,363 [pool-8-thread-56] INFO >> org.apache.airavata.gfac.ssh.provider.impl.SSHProvider - > xmlns:ns=3D"http://airavata.apache.org/gsi/ssh/2012/12 >> >> ">scigap/bin/bashcomputeA516147113trueabe >> gw77jobs@scigap.org,jobnotif@scigap.orgsds12800:30:00/oasis/scratch/comet/scigap/temp_project/airavata-workdirs/dimut= hu_experiment_gau_11_6e6ffc70-5dd2-4a9f-94b6-8cd6e672ec08/Gaussian.stdout/oasis/scratch/comet/scigap/temp_p= roject/airavata-workdirs/dimuthu_experiment_gau_11_6e6ffc70-5dd2-4a9f-94b6-= 8cd6e672ec08/Gaussian.stderr/oas= is/scratch/comet/scigap/temp_project/airavata-workdirs/dimuthu_experiment_g= au_11_6e6ffc70-5dd2-4a9f-94b6-8cd6e672ec08/oasis/scratch/comet/scigap/temp_project/airavata-workdirs/dimuthu_e= xperiment_gau_11_6e6ffc70-5dd2-4a9f-94b6-8cd6e672ec0811616/oasis/scratch/comet/scigap/temp_pr= oject/airavata-workdirs/dimuthu_experiment_gau_11_6e6ffc70-5dd2-4a9f-94b6-8= cd6e672ec08/opt/gaussian/g09/g09gaussian.out<= ns:owner>scigap100module >> load >> gaussian149.165.228.= 428950 >> >> 2015-08-17 06:48:49,443 [pool-8-thread-56] INFO >> org.apache.airavata.gsi.ssh.impl.GSISSHAbstractCluster - Transfering >> file://home/airavata/production-deployment/gfac-instance1/apache-airavat= a-server-0.15-SNAPSHOT/bin/1029102677.slurm >> To:comet.sdsc.edu: >> /oasis/scratch/comet/scigap/temp_project/airavata-workdirs/dimuthu_exper= iment_gau_11_6e6ffc70-5dd2-4a9f-94b6-8cd6e672ec08 >> >> 2015-08-17 06:48:49,897 [pool-8-thread-56] INFO >> org.apache.airavata.gsi.ssh.impl.GSISSHAbstractCluster - Executing >> RawCommand : /usr/bin/sbatch >> /oasis/scratch/comet/scigap/temp_project/airavata-workdirs/dimuthu_exper= iment_gau_11_6e6ffc70-5dd2-4a9f-94b6-8cd6e672ec08/1029102677.slurm >> >> 2015-08-17 06:48:50,195 [pool-8-thread-56] INFO >> org.apache.airavata.gsi.ssh.impl.GSISSHAbstractCluster - StandardOutput >> Returned: >> >> 2015-08-17 06:48:50,196 [pool-8-thread-56] INFO >> org.apache.airavata.gsi.ssh.impl.GSISSHAbstractCluster - StandardError >> Returned:sbatch: error: bank_limit plugin: expired user, can't submit jo= b >> >> sbatch: error: Batch job submission failed: Invalid account or >> account/partition combination specified >> >> >> 2015-08-17 06:48:50,196 [pool-8-thread-56] ERROR >> org.apache.airavata.gsi.ssh.impl.GSISSHAbstractCluster - Standard Error >> output : sbatch: error: bank_limit plugin: expired user, can't submit jo= b >> >> sbatch: error: Batch job submission failed: Invalid account or >> account/partition combination specified >> >> >> 2015-08-17 06:48:55,197 [pool-8-thread-56] ERROR >> org.apache.airavata.gsi.ssh.impl.GSISSHAbstractCluster - Error occured >> during job submission but doing a retry >> >> 2015-08-17 06:48:55,197 [pool-8-thread-56] INFO >> org.apache.airavata.gsi.ssh.impl.GSISSHAbstractCluster - Transfering >> file://home/airavata/production-deployment/gfac-instance1/apache-airavat= a-server-0.15-SNAPSHOT/bin/1029102677.slurm >> To:comet.sdsc.edu: >> /oasis/scratch/comet/scigap/temp_project/airavata-workdirs/dimuthu_exper= iment_gau_11_6e6ffc70-5dd2-4a9f-94b6-8cd6e672ec08 >> >> 2015-08-17 06:48:55,641 [pool-8-thread-56] INFO >> org.apache.airavata.gsi.ssh.impl.GSISSHAbstractCluster - Executing >> RawCommand : /usr/bin/sbatch >> /oasis/scratch/comet/scigap/temp_project/airavata-workdirs/dimuthu_exper= iment_gau_11_6e6ffc70-5dd2-4a9f-94b6-8cd6e672ec08/1029102677.slurm >> >> 2015-08-17 06:48:55,905 [pool-8-thread-56] INFO >> org.apache.airavata.gsi.ssh.impl.GSISSHAbstractCluster - StandardOutput >> Returned: >> >> 2015-08-17 06:48:55,905 [pool-8-thread-56] INFO >> org.apache.airavata.gsi.ssh.impl.GSISSHAbstractCluster - StandardError >> Returned:sbatch: error: bank_limit plugin: expired user, can't submit jo= b >> >> sbatch: error: Batch job submission failed: Invalid account or >> account/partition combination specified >> >> >> 2015-08-17 06:48:55,905 [pool-8-thread-56] ERROR >> org.apache.airavata.gsi.ssh.impl.GSISSHAbstractCluster - Standard Error >> output : sbatch: error: bank_limit plugin: expired user, can't submit jo= b >> >> sbatch: error: Batch job submission failed: Invalid account or >> account/partition combination specified >> >> >> 2015-08-17 06:48:57,612 [Thread-9] INFO >> org.apache.airavata.gfac.monitor.email.EmailBasedMonitor - [EJM]: Job >> Monitor Map is empty, no need to retrieve emails >> >> 2015-08-17 06:49:00,906 [pool-8-thread-56] ERROR >> org.apache.airavata.gsi.ssh.impl.GSISSHAbstractCluster - Error occured >> during job submission but doing a retry >> >> 2015-08-17 06:49:00,906 [pool-8-thread-56] INFO >> org.apache.airavata.gsi.ssh.impl.GSISSHAbstractCluster - Transfering >> file://home/airavata/production-deployment/gfac-instance1/apache-airavat= a-server-0.15-SNAPSHOT/bin/1029102677.slurm >> To:comet.sdsc.edu: >> /oasis/scratch/comet/scigap/temp_project/airavata-workdirs/dimuthu_exper= iment_gau_11_6e6ffc70-5dd2-4a9f-94b6-8cd6e672ec08 >> >> 2015-08-17 06:49:01,321 [pool-8-thread-56] INFO >> org.apache.airavata.gsi.ssh.impl.GSISSHAbstractCluster - Executing >> RawCommand : /usr/bin/sbatch >> /oasis/scratch/comet/scigap/temp_project/airavata-workdirs/dimuthu_exper= iment_gau_11_6e6ffc70-5dd2-4a9f-94b6-8cd6e672ec08/1029102677.slurm >> >> 2015-08-17 06:49:01,587 [pool-8-thread-56] INFO >> org.apache.airavata.gsi.ssh.impl.GSISSHAbstractCluster - StandardOutput >> Returned: >> >> 2015-08-17 06:49:01,587 [pool-8-thread-56] INFO >> org.apache.airavata.gsi.ssh.impl.GSISSHAbstractCluster - StandardError >> Returned:sbatch: error: bank_limit plugin: expired user, can't submit jo= b >> >> sbatch: error: Batch job submission failed: Invalid account or >> account/partition combination specified >> >> >> 2015-08-17 06:49:01,587 [pool-8-thread-56] ERROR >> org.apache.airavata.gsi.ssh.impl.GSISSHAbstractCluster - Standard Error >> output : sbatch: error: bank_limit plugin: expired user, can't submit jo= b >> >> sbatch: error: Batch job submission failed: Invalid account or >> account/partition combination specified >> >> >> 2015-08-17 06:49:01,588 [pool-8-thread-56] ERROR >> org.apache.airavata.gfac.ssh.provider.impl.SSHProvider - Error submitti= ng >> the job to host comet.sdsc.edu >> >> message: Error reading output of job submission >> >> StandardOutput: >> >> StandardError: sbatch: error: bank_limit plugin: expired user, can't >> submit job >> >> sbatch: error: Batch job submission failed: Invalid account or >> account/partition combination specified >> >> >> 2015-08-17 06:49:01,660 [pool-8-thread-56] ERROR >> org.apache.airavata.persistance.registry.jpa.resources.JobDetailResource= - >> The transaction has been rolled back. See the nested exceptions for >> details on the errors that occurred. >> >> >> org.apache.openjpa.persistence.RollbackException: The transaction has be= en >> rolled back. See the nested exceptions for details on the errors that >> occurred. >> >> FailedObject: >> org.apache.airavata.persistance.registry.jpa.model.JobDetail@6f1d32ff >> >> at >> org.apache.openjpa.persistence.EntityManagerImpl.commit(EntityManagerImp= l.java:594) >> >> at >> org.apache.airavata.persistance.registry.jpa.resources.JobDetailResource= .save(JobDetailResource.java:308) >> >> at >> org.apache.airavata.persistance.registry.jpa.impl.ExperimentRegistry.add= JobDetails(ExperimentRegistry.java:1203) >> >> at >> org.apache.airavata.persistance.registry.jpa.impl.RegistryImpl.add(Regis= tryImpl.java:157) >> >> at >> org.apache.airavata.gfac.core.utils.GFacUtils.saveJobStatus(GFacUtils.ja= va:232) >> >> at >> org.apache.airavata.gfac.ssh.provider.impl.SSHProvider.execute(SSHProvid= er.java:215) >> >> at >> org.apache.airavata.gfac.core.cpi.BetterGfacImpl.executeProvider(BetterG= facImpl.java:842) >> >> at >> org.apache.airavata.gfac.core.cpi.BetterGfacImpl.invokeProviderExecute(B= etterGfacImpl.java:753) >> >> at >> org.apache.airavata.gfac.core.cpi.BetterGfacImpl.launch(BetterGfacImpl.j= ava:703) >> >> at >> org.apache.airavata.gfac.core.cpi.BetterGfacImpl.submitJob(BetterGfacImp= l.java:492) >> >> at >> org.apache.airavata.gfac.core.cpi.BetterGfacImpl.submitJob(BetterGfacImp= l.java:183) >> >> at >> org.apache.airavata.gfac.core.utils.InputHandlerWorker.run(InputHandlerW= orker.java:47) >> >> at >> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.jav= a:1142) >> >> at >> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.ja= va:617) >> >> at java.lang.Thread.run(Thread.java:745) >> >> Caused by: >> org.apache.openjpa.persistence.EntityExistsException: The transaction ha= s >> been rolled back. See the nested exceptions for details on the errors t= hat >> occurred. >> >> FailedObject: >> org.apache.airavata.persistance.registry.jpa.model.JobDetail@6f1d32ff >> >> at >> org.apache.openjpa.kernel.BrokerImpl.newFlushException(BrokerImpl.java:2= 337) >> >> at org.apache.openjpa.kernel.BrokerImpl.flush(BrokerImpl.java:2174) >> >> at org.apache.openjpa.kernel.BrokerImpl.flushSafe(BrokerImpl.java:2072) >> >> at >> org.apache.openjpa.kernel.BrokerImpl.beforeCompletion(BrokerImpl.java:19= 90) >> >> at >> org.apache.openjpa.kernel.LocalManagedRuntime.commit(LocalManagedRuntime= .java:81) >> >> at org.apache.openjpa.kernel.BrokerImpl.commit(BrokerImpl.java:1514) >> >> at >> org.apache.openjpa.kernel.DelegatingBroker.commit(DelegatingBroker.java:= 933) >> >> at >> org.apache.openjpa.persistence.EntityManagerImpl.commit(EntityManagerImp= l.java:570) >> >> ... 14 more >> >> Caused by: >> org.apache.openjpa.persistence.EntityExistsException: Column >> 'JOB_DESCRIPTION' cannot be null {prepstmnt 1744538295 >> >> INSERT INTO JOB_DETAIL (JOB_ID, TASK_ID, COMPUTE_RESOURCE_CONSUMED, >> >> CREATION_TIME, JOB_DESCRIPTION, JOBNAME, WORKING_DIR) >> >> VALUES (?, ?, ?, ?, ?, ?, ?) >> >> [params=3D(String) none, (String) >> tempNode_56f41083-10be-4161-8803-ee1106bd3f54, (null) null, (Timestamp) >> 2015-08-17 06:49:01.613, (null) null, (String) A516147113, (null) null]} >> [code=3D1048, state=3D23000] >> >> FailedObject: >> org.apache.airavata.persistance.registry.jpa.model.JobDetail@6f1d32ff >> >> at org.apache.openjpa.jdbc.sql.DBDictionary.narrow(DBDictionary.java:490= 3) >> >> at >> org.apache.openjpa.jdbc.sql.DBDictionary.newStoreException(DBDictionary.= java:4878) >> >> at >> org.apache.openjpa.jdbc.sql.SQLExceptions.getStore(SQLExceptions.java:13= 6) >> >> at >> org.apache.openjpa.jdbc.sql.SQLExceptions.getStore(SQLExceptions.java:78= ) >> >> at >> org.apache.openjpa.jdbc.kernel.PreparedStatementManagerImpl.flushAndUpda= te(PreparedStatementManagerImpl.java:143) >> >> at >> org.apache.openjpa.jdbc.kernel.BatchingPreparedStatementManagerImpl.flus= hAndUpdate(BatchingPreparedStatementManagerImpl.java:79) >> >> at >> org.apache.openjpa.jdbc.kernel.PreparedStatementManagerImpl.flushInterna= l(PreparedStatementManagerImpl.java:99) >> >> at >> org.apache.openjpa.jdbc.kernel.PreparedStatementManagerImpl.flush(Prepar= edStatementManagerImpl.java:87) >> >> at >> org.apache.openjpa.jdbc.kernel.ConstraintUpdateManager.flush(ConstraintU= pdateManager.java:550) >> >> at >> org.apache.openjpa.jdbc.kernel.ConstraintUpdateManager.flush(ConstraintU= pdateManager.java:106) >> >> at >> org.apache.openjpa.jdbc.kernel.BatchingConstraintUpdateManager.flush(Bat= chingConstraintUpdateManager.java:59) >> >> at >> org.apache.openjpa.jdbc.kernel.AbstractUpdateManager.flush(AbstractUpdat= eManager.java:103) >> >> at >> org.apache.openjpa.jdbc.kernel.AbstractUpdateManager.flush(AbstractUpdat= eManager.java:76) >> >> at >> org.apache.openjpa.jdbc.kernel.JDBCStoreManager.flush(JDBCStoreManager.j= ava:713) >> >> at >> org.apache.openjpa.kernel.DelegatingStoreManager.flush(DelegatingStoreMa= nager.java:131) >> >> ... 21 more >> >> Caused by: org.apache.openjpa.lib.jdbc.ReportingSQLException: Column >> 'JOB_DESCRIPTION' cannot be null {prepstmnt 1744538295 >> >> INSERT INTO JOB_DETAIL (JOB_ID, TASK_ID, COMPUTE_RESOURCE_CONSUMED, >> >> CREATION_TIME, JOB_DESCRIPTION, JOBNAME, WORKING_DIR) >> >> VALUES (?, ?, ?, ?, ?, ?, ?) >> >> [params=3D(String) none, (String) >> tempNode_56f41083-10be-4161-8803-ee1106bd3f54, (null) null, (Timestamp) >> 2015-08-17 06:49:01.613, (null) null, (String) A516147113, (null) null]} >> [code=3D1048, state=3D23000] >> >> at >> org.apache.openjpa.lib.jdbc.LoggingConnectionDecorator.wrap(LoggingConne= ctionDecorator.java:247) >> >> at >> org.apache.openjpa.lib.jdbc.LoggingConnectionDecorator.wrap(LoggingConne= ctionDecorator.java:223) >> >> at >> org.apache.openjpa.lib.jdbc.LoggingConnectionDecorator.access$1000(Loggi= ngConnectionDecorator.java:72) >> >> at >> org.apache.openjpa.lib.jdbc.LoggingConnectionDecorator$LoggingConnection= $LoggingPreparedStatement.executeUpdate(LoggingConnectionDecorator.java:116= 2) >> >> at >> org.apache.openjpa.lib.jdbc.DelegatingPreparedStatement.executeUpdate(De= legatingPreparedStatement.java:272) >> >> at >> org.apache.openjpa.jdbc.kernel.JDBCStoreManager$CancelPreparedStatement.= executeUpdate(JDBCStoreManager.java:1741) >> >> at >> org.apache.openjpa.jdbc.kernel.PreparedStatementManagerImpl.executeUpdat= e(PreparedStatementManagerImpl.java:267) >> >> at >> org.apache.openjpa.jdbc.kernel.PreparedStatementManagerImpl.flushAndUpda= te(PreparedStatementManagerImpl.java:118) >> >> ... 31 more >> >> 2015-08-17 06:49:01,664 [pool-8-thread-56] ERROR >> org.apache.airavata.persistance.registry.jpa.impl.ExperimentRegistry - >> Error while adding job details... >> >> org.apache.airavata.registry.cpi.RegistryException: >> >> org.apache.openjpa.persistence.RollbackException: The transaction has be= en >> rolled back. See the nested exceptions for details on the errors that >> occurred. >> >> FailedObject: >> org.apache.airavata.persistance.registry.jpa.model.JobDetail@6f1d32ff >> >> at >> org.apache.airavata.persistance.registry.jpa.resources.JobDetailResource= .save(JobDetailResource.java:312) >> >> at >> org.apache.airavata.persistance.registry.jpa.impl.ExperimentRegistry.add= JobDetails(ExperimentRegistry.java:1203) >> >> at >> org.apache.airavata.persistance.registry.jpa.impl.RegistryImpl.add(Regis= tryImpl.java:157) >> >> at >> org.apache.airavata.gfac.core.utils.GFacUtils.saveJobStatus(GFacUtils.ja= va:232) >> >> at >> org.apache.airavata.gfac.ssh.provider.impl.SSHProvider.execute(SSHProvid= er.java:215) >> >> at >> org.apache.airavata.gfac.core.cpi.BetterGfacImpl.executeProvider(BetterG= facImpl.java:842) >> >> at >> org.apache.airavata.gfac.core.cpi.BetterGfacImpl.invokeProviderExecute(B= etterGfacImpl.java:753) >> >> at >> org.apache.airavata.gfac.core.cpi.BetterGfacImpl.launch(BetterGfacImpl.j= ava:703) >> >> at >> org.apache.airavata.gfac.core.cpi.BetterGfacImpl.submitJob(BetterGfacImp= l.java:492) >> >> at >> org.apache.airavata.gfac.core.cpi.BetterGfacImpl.submitJob(BetterGfacImp= l.java:183) >> >> at >> org.apache.airavata.gfac.core.utils.InputHandlerWorker.run(InputHandlerW= orker.java:47) >> >> at >> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.jav= a:1142) >> >> at >> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.ja= va:617) >> >> at java.lang.Thread.run(Thread.java:745) >> >> Caused by: >> org.apache.openjpa.persistence.RollbackException: The transaction has be= en >> rolled back. See the nested exceptions for details on the errors that >> occurred. >> >> FailedObject: >> org.apache.airavata.persistance.registry.jpa.model.JobDetail@6f1d32ff >> >> at >> org.apache.openjpa.persistence.EntityManagerImpl.commit(EntityManagerImp= l.java:594) >> >> at >> org.apache.airavata.persistance.registry.jpa.resources.JobDetailResource= .save(JobDetailResource.java:308) >> >> ... 13 more >> >> Caused by: >> org.apache.openjpa.persistence.EntityExistsException: The transaction ha= s >> been rolled back. See the nested exceptions for details on the errors t= hat >> occurred. >> >> FailedObject: >> org.apache.airavata.persistance.registry.jpa.model.JobDetail@6f1d32ff >> >> at >> org.apache.openjpa.kernel.BrokerImpl.newFlushException(BrokerImpl.java:2= 337) >> >> at org.apache.openjpa.kernel.BrokerImpl.flush(BrokerImpl.java:2174) >> >> at org.apache.openjpa.kernel.BrokerImpl.flushSafe(BrokerImpl.java:2072) >> >> at >> org.apache.openjpa.kernel.BrokerImpl.beforeCompletion(BrokerImpl.java:19= 90) >> >> at >> org.apache.openjpa.kernel.LocalManagedRuntime.commit(LocalManagedRuntime= .java:81) >> >> at org.apache.openjpa.kernel.BrokerImpl.commit(BrokerImpl.java:1514) >> >> at >> org.apache.openjpa.kernel.DelegatingBroker.commit(DelegatingBroker.java:= 933) >> >> at >> org.apache.openjpa.persistence.EntityManagerImpl.commit(EntityManagerImp= l.java:570) >> >> ... 14 more >> >> Caused by: >> org.apache.openjpa.persistence.EntityExistsException: Column >> 'JOB_DESCRIPTION' cannot be null {prepstmnt 1744538295 >> >> INSERT INTO JOB_DETAIL (JOB_ID, TASK_ID, COMPUTE_RESOURCE_CONSUMED, >> >> CREATION_TIME, JOB_DESCRIPTION, JOBNAME, WORKING_DIR) >> >> VALUES (?, ?, ?, ?, ?, ?, ?) >> >> [params=3D(String) none, (String) >> tempNode_56f41083-10be-4161-8803-ee1106bd3f54, (null) null, (Timestamp) >> 2015-08-17 06:49:01.613, (null) null, (String) A516147113, (null) null]} >> [code=3D1048, state=3D23000] >> >> FailedObject: >> org.apache.airavata.persistance.registry.jpa.model.JobDetail@6f1d32ff >> >> at org.apache.openjpa.jdbc.sql.DBDictionary.narrow(DBDictionary.java:490= 3) >> >> at >> org.apache.openjpa.jdbc.sql.DBDictionary.newStoreException(DBDictionary.= java:4878) >> >> at >> org.apache.openjpa.jdbc.sql.SQLExceptions.getStore(SQLExceptions.java:13= 6) >> >> at >> org.apache.openjpa.jdbc.sql.SQLExceptions.getStore(SQLExceptions.java:78= ) >> >> at >> org.apache.openjpa.jdbc.kernel.PreparedStatementManagerImpl.flushAndUpda= te(PreparedStatementManagerImpl.java:143) >> >> at >> org.apache.openjpa.jdbc.kernel.BatchingPreparedStatementManagerImpl.flus= hAndUpdate(BatchingPreparedStatementManagerImpl.java:79) >> >> at >> org.apache.openjpa.jdbc.kernel.PreparedStatementManagerImpl.flushInterna= l(PreparedStatementManagerImpl.java:99) >> >> at >> org.apache.openjpa.jdbc.kernel.PreparedStatementManagerImpl.flush(Prepar= edStatementManagerImpl.java:87) >> >> at >> org.apache.openjpa.jdbc.kernel.ConstraintUpdateManager.flush(ConstraintU= pdateManager.java:550) >> >> at >> org.apache.openjpa.jdbc.kernel.ConstraintUpdateManager.flush(ConstraintU= pdateManager.java:106) >> >> at >> org.apache.openjpa.jdbc.kernel.BatchingConstraintUpdateManager.flush(Bat= chingConstraintUpdateManager.java:59) >> >> at >> org.apache.openjpa.jdbc.kernel.AbstractUpdateManager.flush(AbstractUpdat= eManager.java:103) >> >> at >> org.apache.openjpa.jdbc.kernel.AbstractUpdateManager.flush(AbstractUpdat= eManager.java:76) >> >> at >> org.apache.openjpa.jdbc.kernel.JDBCStoreManager.flush(JDBCStoreManager.j= ava:713) >> >> at >> org.apache.openjpa.kernel.DelegatingStoreManager.flush(DelegatingStoreMa= nager.java:131) >> >> ... 21 more >> >> Caused by: org.apache.openjpa.lib.jdbc.ReportingSQLException: Column >> 'JOB_DESCRIPTION' cannot be null {prepstmnt 1744538295 >> >> INSERT INTO JOB_DETAIL (JOB_ID, TASK_ID, COMPUTE_RESOURCE_CONSUMED, >> >> CREATION_TIME, JOB_DESCRIPTION, JOBNAME, WORKING_DIR) >> >> VALUES (?, ?, ?, ?, ?, ?, ?) >> >> [params=3D(String) none, (String) >> tempNode_56f41083-10be-4161-8803-ee1106bd3f54, (null) null, (Timestamp) >> 2015-08-17 06:49:01.613, (null) null, (String) A516147113, (null) null]} >> [code=3D1048, state=3D23000] >> >> at >> org.apache.openjpa.lib.jdbc.LoggingConnectionDecorator.wrap(LoggingConne= ctionDecorator.java:247) >> >> at >> org.apache.openjpa.lib.jdbc.LoggingConnectionDecorator.wrap(LoggingConne= ctionDecorator.java:223) >> >> at >> org.apache.openjpa.lib.jdbc.LoggingConnectionDecorator.access$1000(Loggi= ngConnectionDecorator.java:72) >> >> at >> org.apache.openjpa.lib.jdbc.LoggingConnectionDecorator$LoggingConnection= $LoggingPreparedStatement.executeUpdate(LoggingConnectionDecorator.java:116= 2) >> >> at >> org.apache.openjpa.lib.jdbc.DelegatingPreparedStatement.executeUpdate(De= legatingPreparedStatement.java:272) >> >> at >> org.apache.openjpa.jdbc.kernel.JDBCStoreManager$CancelPreparedStatement.= executeUpdate(JDBCStoreManager.java:1741) >> >> at >> org.apache.openjpa.jdbc.kernel.PreparedStatementManagerImpl.executeUpdat= e(PreparedStatementManagerImpl.java:267) >> >> at >> org.apache.openjpa.jdbc.kernel.PreparedStatementManagerImpl.flushAndUpda= te(PreparedStatementManagerImpl.java:118) >> >> ... 31 more >> >> 2015-08-17 06:49:01,666 [pool-8-thread-56] ERROR >> org.apache.airavata.persistance.registry.jpa.impl.RegistryImpl - Error >> while adding JOB_DETAIL >> >> org.apache.airavata.registry.cpi.RegistryException: >> org.apache.airavata.registry.cpi.RegistryException: >> org.apache.airavata.registry.cpi.RegistryException: >> >> org.apache.openjpa.persistence.RollbackException: The transaction has be= en >> rolled back. See the nested exceptions for details on the errors that >> occurred. >> >> FailedObject: >> org.apache.airavata.persistance.registry.jpa.model.JobDetail@6f1d32ff >> >> at >> org.apache.airavata.persistance.registry.jpa.impl.RegistryImpl.add(Regis= tryImpl.java:181) >> >> at >> org.apache.airavata.gfac.core.utils.GFacUtils.saveJobStatus(GFacUtils.ja= va:232) >> >> at >> org.apache.airavata.gfac.ssh.provider.impl.SSHProvider.execute(SSHProvid= er.java:215) >> >> at >> org.apache.airavata.gfac.core.cpi.BetterGfacImpl.executeProvider(BetterG= facImpl.java:842) >> >> at >> org.apache.airavata.gfac.core.cpi.BetterGfacImpl.invokeProviderExecute(B= etterGfacImpl.java:753) >> >> at >> org.apache.airavata.gfac.core.cpi.BetterGfacImpl.launch(BetterGfacImpl.j= ava:703) >> >> at >> org.apache.airavata.gfac.core.cpi.BetterGfacImpl.submitJob(BetterGfacImp= l.java:492) >> >> at >> org.apache.airavata.gfac.core.cpi.BetterGfacImpl.submitJob(BetterGfacImp= l.java:183) >> >> at >> org.apache.airavata.gfac.core.utils.InputHandlerWorker.run(InputHandlerW= orker.java:47) >> >> at >> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.jav= a:1142) >> >> at >> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.ja= va:617) >> >> at java.lang.Thread.run(Thread.java:745) >> >> Caused by: org.apache.airavata.registry.cpi.RegistryException: >> org.apache.airavata.registry.cpi.RegistryException: >> >> org.apache.openjpa.persistence.RollbackException: The transaction has be= en >> rolled back. See the nested exceptions for details on the errors that >> occurred. >> >> FailedObject: >> org.apache.airavata.persistance.registry.jpa.model.JobDetail@6f1d32ff >> >> at >> org.apache.airavata.persistance.registry.jpa.impl.ExperimentRegistry.add= JobDetails(ExperimentRegistry.java:1231) >> >> at >> org.apache.airavata.persistance.registry.jpa.impl.RegistryImpl.add(Regis= tryImpl.java:157) >> >> ... 11 more >> >> Caused by: org.apache.airavata.registry.cpi.RegistryException: >> >> org.apache.openjpa.persistence.RollbackException: The transaction has be= en >> rolled back. See the nested exceptions for details on the errors that >> occurred. >> >> FailedObject: >> org.apache.airavata.persistance.registry.jpa.model.JobDetail@6f1d32ff >> >> at >> org.apache.airavata.persistance.registry.jpa.resources.JobDetailResource= .save(JobDetailResource.java:312) >> >> at >> org.apache.airavata.persistance.registry.jpa.impl.ExperimentRegistry.add= JobDetails(ExperimentRegistry.java:1203) >> >> ... 12 more >> >> Caused by: >> org.apache.openjpa.persistence.RollbackException: The transaction has be= en >> rolled back. See the nested exceptions for details on the errors that >> occurred. >> >> FailedObject: >> org.apache.airavata.persistance.registry.jpa.model.JobDetail@6f1d32ff >> >> at >> org.apache.openjpa.persistence.EntityManagerImpl.commit(EntityManagerImp= l.java:594) >> >> at >> org.apache.airavata.persistance.registry.jpa.resources.JobDetailResource= .save(JobDetailResource.java:308) >> >> ... 13 more >> >> Caused by: >> org.apache.openjpa.persistence.EntityExistsException: The transaction ha= s >> been rolled back. See the nested exceptions for details on the errors t= hat >> occurred. >> >> FailedObject: >> org.apache.airavata.persistance.registry.jpa.model.JobDetail@6f1d32ff >> >> at >> org.apache.openjpa.kernel.BrokerImpl.newFlushException(BrokerImpl.java:2= 337) >> >> at org.apache.openjpa.kernel.BrokerImpl.flush(BrokerImpl.java:2174) >> >> at org.apache.openjpa.kernel.BrokerImpl.flushSafe(BrokerImpl.java:2072) >> >> at >> org.apache.openjpa.kernel.BrokerImpl.beforeCompletion(BrokerImpl.java:19= 90) >> >> at >> org.apache.openjpa.kernel.LocalManagedRuntime.commit(LocalManagedRuntime= .java:81) >> >> at org.apache.openjpa.kernel.BrokerImpl.commit(BrokerImpl.java:1514) >> >> at >> org.apache.openjpa.kernel.DelegatingBroker.commit(DelegatingBroker.java:= 933) >> >> at >> org.apache.openjpa.persistence.EntityManagerImpl.commit(EntityManagerImp= l.java:570) >> >> ... 14 more >> >> Caused by: >> org.apache.openjpa.persistence.EntityExistsException: Column >> 'JOB_DESCRIPTION' cannot be null {prepstmnt 1744538295 >> >> INSERT INTO JOB_DETAIL (JOB_ID, TASK_ID, COMPUTE_RESOURCE_CONSUMED, >> >> CREATION_TIME, JOB_DESCRIPTION, JOBNAME, WORKING_DIR) >> >> VALUES (?, ?, ?, ?, ?, ?, ?) >> >> [params=3D(String) none, (String) >> tempNode_56f41083-10be-4161-8803-ee1106bd3f54, (null) null, (Timestamp) >> 2015-08-17 06:49:01.613, (null) null, (String) A516147113, (null) null]} >> [code=3D1048, state=3D23000] >> >> FailedObject: >> org.apache.airavata.persistance.registry.jpa.model.JobDetail@6f1d32ff >> >> at org.apache.openjpa.jdbc.sql.DBDictionary.narrow(DBDictionary.java:490= 3) >> >> at >> org.apache.openjpa.jdbc.sql.DBDictionary.newStoreException(DBDictionary.= java:4878) >> >> at >> org.apache.openjpa.jdbc.sql.SQLExceptions.getStore(SQLExceptions.java:13= 6) >> >> at >> org.apache.openjpa.jdbc.sql.SQLExceptions.getStore(SQLExceptions.java:78= ) >> >> at >> org.apache.openjpa.jdbc.kernel.PreparedStatementManagerImpl.flushAndUpda= te(PreparedStatementManagerImpl.java:143) >> >> at >> org.apache.openjpa.jdbc.kernel.BatchingPreparedStatementManagerImpl.flus= hAndUpdate(BatchingPreparedStatementManagerImpl.java:79) >> >> at >> org.apache.openjpa.jdbc.kernel.PreparedStatementManagerImpl.flushInterna= l(PreparedStatementManagerImpl.java:99) >> >> at >> org.apache.openjpa.jdbc.kernel.PreparedStatementManagerImpl.flush(Prepar= edStatementManagerImpl.java:87) >> >> at >> org.apache.openjpa.jdbc.kernel.ConstraintUpdateManager.flush(ConstraintU= pdateManager.java:550) >> >> at >> org.apache.openjpa.jdbc.kernel.ConstraintUpdateManager.flush(ConstraintU= pdateManager.java:106) >> >> at >> org.apache.openjpa.jdbc.kernel.BatchingConstraintUpdateManager.flush(Bat= chingConstraintUpdateManager.java:59) >> >> at >> org.apache.openjpa.jdbc.kernel.AbstractUpdateManager.flush(AbstractUpdat= eManager.java:103) >> >> at >> org.apache.openjpa.jdbc.kernel.AbstractUpdateManager.flush(AbstractUpdat= eManager.java:76) >> >> at >> org.apache.openjpa.jdbc.kernel.JDBCStoreManager.flush(JDBCStoreManager.j= ava:713) >> >> at >> org.apache.openjpa.kernel.DelegatingStoreManager.flush(DelegatingStoreMa= nager.java:131) >> >> ... 21 more >> >> Caused by: org.apache.openjpa.lib.jdbc.ReportingSQLException: Column >> 'JOB_DESCRIPTION' cannot be null {prepstmnt 1744538295 >> >> INSERT INTO JOB_DETAIL (JOB_ID, TASK_ID, COMPUTE_RESOURCE_CONSUMED, >> >> CREATION_TIME, JOB_DESCRIPTION, JOBNAME, WORKING_DIR) >> >> VALUES (?, ?, ?, ?, ?, ?, ?) >> >> [params=3D(String) none, (String) >> tempNode_56f41083-10be-4161-8803-ee1106bd3f54, (null) null, (Timestamp) >> 2015-08-17 06:49:01.613, (null) null, (String) A516147113, (null) null]} >> [code=3D1048, state=3D23000] >> >> at >> org.apache.openjpa.lib.jdbc.LoggingConnectionDecorator.wrap(LoggingConne= ctionDecorator.java:247) >> >> at >> org.apache.openjpa.lib.jdbc.LoggingConnectionDecorator.wrap(LoggingConne= ctionDecorator.java:223) >> >> at >> org.apache.openjpa.lib.jdbc.LoggingConnectionDecorator.access$1000(Loggi= ngConnectionDecorator.java:72) >> >> at >> org.apache.openjpa.lib.jdbc.LoggingConnectionDecorator$LoggingConnection= $LoggingPreparedStatement.executeUpdate(LoggingConnectionDecorator.java:116= 2) >> >> at >> org.apache.openjpa.lib.jdbc.DelegatingPreparedStatement.executeUpdate(De= legatingPreparedStatement.java:272) >> >> at >> org.apache.openjpa.jdbc.kernel.JDBCStoreManager$CancelPreparedStatement.= executeUpdate(JDBCStoreManager.java:1741) >> >> at >> org.apache.openjpa.jdbc.kernel.PreparedStatementManagerImpl.executeUpdat= e(PreparedStatementManagerImpl.java:267) >> >> at >> org.apache.openjpa.jdbc.kernel.PreparedStatementManagerImpl.flushAndUpda= te(PreparedStatementManagerImpl.java:118) >> >> ... 31 more >> >> 2015-08-17 06:49:01,690 [pool-8-thread-56] ERROR >> org.apache.airavata.gfac.GFacException - Error persisting job statusErr= or >> while adding JOB_DETAIL >> >> org.apache.airavata.registry.cpi.RegistryException: Error while adding >> JOB_DETAIL >> >> at >> org.apache.airavata.persistance.registry.jpa.impl.RegistryImpl.add(Regis= tryImpl.java:182) >> >> at >> org.apache.airavata.gfac.core.utils.GFacUtils.saveJobStatus(GFacUtils.ja= va:232) >> >> at >> org.apache.airavata.gfac.ssh.provider.impl.SSHProvider.execute(SSHProvid= er.java:215) >> >> at >> org.apache.airavata.gfac.core.cpi.BetterGfacImpl.executeProvider(BetterG= facImpl.java:842) >> >> at >> org.apache.airavata.gfac.core.cpi.BetterGfacImpl.invokeProviderExecute(B= etterGfacImpl.java:753) >> >> at >> org.apache.airavata.gfac.core.cpi.BetterGfacImpl.launch(BetterGfacImpl.j= ava:703) >> >> at >> org.apache.airavata.gfac.core.cpi.BetterGfacImpl.submitJob(BetterGfacImp= l.java:492) >> >> at >> org.apache.airavata.gfac.core.cpi.BetterGfacImpl.submitJob(BetterGfacImp= l.java:183) >> >> at >> org.apache.airavata.gfac.core.utils.InputHandlerWorker.run(InputHandlerW= orker.java:47) >> >> at >> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.jav= a:1142) >> >> at >> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.ja= va:617) >> >> at java.lang.Thread.run(Thread.java:745) >> >> Caused by: org.apache.airavata.registry.cpi.RegistryException: >> org.apache.airavata.registry.cpi.RegistryException: >> >> org.apache.openjpa.persistence.RollbackException: The transaction has be= en >> rolled back. See the nested exceptions for details on the errors that >> occurred. >> >> FailedObject: >> org.apache.airavata.persistance.registry.jpa.model.JobDetail@6f1d32ff >> >> at >> org.apache.airavata.persistance.registry.jpa.impl.ExperimentRegistry.add= JobDetails(ExperimentRegistry.java:1231) >> >> at >> org.apache.airavata.persistance.registry.jpa.impl.RegistryImpl.add(Regis= tryImpl.java:157) >> >> ... 11 more >> >> Caused by: org.apache.airavata.registry.cpi.RegistryException: >> >> org.apache.openjpa.persistence.RollbackException: The transaction has be= en >> rolled back. See the nested exceptions for details on the errors that >> occurred. >> >> FailedObject: >> org.apache.airavata.persistance.registry.jpa.model.JobDetail@6f1d32ff >> >> at >> org.apache.airavata.persistance.registry.jpa.resources.JobDetailResource= .save(JobDetailResource.java:312) >> >> at >> org.apache.airavata.persistance.registry.jpa.impl.ExperimentRegistry.add= JobDetails(ExperimentRegistry.java:1203) >> >> ... 12 more >> >> Caused by: >> org.apache.openjpa.persistence.RollbackException: The transaction has be= en >> rolled back. See the nested exceptions for details on the errors that >> occurred. >> >> FailedObject: >> org.apache.airavata.persistance.registry.jpa.model.JobDetail@6f1d32ff >> >> at >> org.apache.openjpa.persistence.EntityManagerImpl.commit(EntityManagerImp= l.java:594) >> >> at >> org.apache.airavata.persistance.registry.jpa.resources.JobDetailResource= .save(JobDetailResource.java:308) >> >> ... 13 more >> >> Caused by: >> org.apache.openjpa.persistence.EntityExistsException: The transaction ha= s >> been rolled back. See the nested exceptions for details on the errors t= hat >> occurred. >> >> FailedObject: >> org.apache.airavata.persistance.registry.jpa.model.JobDetail@6f1d32ff >> >> at >> org.apache.openjpa.kernel.BrokerImpl.newFlushException(BrokerImpl.java:2= 337) >> >> at org.apache.openjpa.kernel.BrokerImpl.flush(BrokerImpl.java:2174) >> >> at org.apache.openjpa.kernel.BrokerImpl.flushSafe(BrokerImpl.java:2072) >> >> at >> org.apache.openjpa.kernel.BrokerImpl.beforeCompletion(BrokerImpl.java:19= 90) >> >> at >> org.apache.openjpa.kernel.LocalManagedRuntime.commit(LocalManagedRuntime= .java:81) >> >> at org.apache.openjpa.kernel.BrokerImpl.commit(BrokerImpl.java:1514) >> >> at >> org.apache.openjpa.kernel.DelegatingBroker.commit(DelegatingBroker.java:= 933) >> >> at >> org.apache.openjpa.persistence.EntityManagerImpl.commit(EntityManagerImp= l.java:570) >> >> ... 14 more >> >> Caused by: >> org.apache.openjpa.persistence.EntityExistsException: Column >> 'JOB_DESCRIPTION' cannot be null {prepstmnt 1744538295 >> >> INSERT INTO JOB_DETAIL (JOB_ID, TASK_ID, COMPUTE_RESOURCE_CONSUMED, >> >> CREATION_TIME, JOB_DESCRIPTION, JOBNAME, WORKING_DIR) >> >> VALUES (?, ?, ?, ?, ?, ?, ?) >> >> [params=3D(String) none, (String) >> tempNode_56f41083-10be-4161-8803-ee1106bd3f54, (null) null, (Timestamp) >> 2015-08-17 06:49:01.613, (null) null, (String) A516147113, (null) null]} >> [code=3D1048, state=3D23000] >> >> FailedObject: >> org.apache.airavata.persistance.registry.jpa.model.JobDetail@6f1d32ff >> >> at org.apache.openjpa.jdbc.sql.DBDictionary.narrow(DBDictionary.java:490= 3) >> >> at >> org.apache.openjpa.jdbc.sql.DBDictionary.newStoreException(DBDictionary.= java:4878) >> >> at >> org.apache.openjpa.jdbc.sql.SQLExceptions.getStore(SQLExceptions.java:13= 6) >> >> at >> org.apache.openjpa.jdbc.sql.SQLExceptions.getStore(SQLExceptions.java:78= ) >> >> at >> org.apache.openjpa.jdbc.kernel.PreparedStatementManagerImpl.flushAndUpda= te(PreparedStatementManagerImpl.java:143) >> >> at >> org.apache.openjpa.jdbc.kernel.BatchingPreparedStatementManagerImpl.flus= hAndUpdate(BatchingPreparedStatementManagerImpl.java:79) >> >> at >> org.apache.openjpa.jdbc.kernel.PreparedStatementManagerImpl.flushInterna= l(PreparedStatementManagerImpl.java:99) >> >> at >> org.apache.openjpa.jdbc.kernel.PreparedStatementManagerImpl.flush(Prepar= edStatementManagerImpl.java:87) >> >> at >> org.apache.openjpa.jdbc.kernel.ConstraintUpdateManager.flush(ConstraintU= pdateManager.java:550) >> >> at >> org.apache.openjpa.jdbc.kernel.ConstraintUpdateManager.flush(ConstraintU= pdateManager.java:106) >> >> at >> org.apache.openjpa.jdbc.kernel.BatchingConstraintUpdateManager.flush(Bat= chingConstraintUpdateManager.java:59) >> >> at >> org.apache.openjpa.jdbc.kernel.AbstractUpdateManager.flush(AbstractUpdat= eManager.java:103) >> >> at >> org.apache.openjpa.jdbc.kernel.AbstractUpdateManager.flush(AbstractUpdat= eManager.java:76) >> >> at >> org.apache.openjpa.jdbc.kernel.JDBCStoreManager.flush(JDBCStoreManager.j= ava:713) >> >> at >> org.apache.openjpa.kernel.DelegatingStoreManager.flush(DelegatingStoreMa= nager.java:131) >> >> ... 21 more >> >> Caused by: org.apache.openjpa.lib.jdbc.ReportingSQLException: Column >> 'JOB_DESCRIPTION' cannot be null {prepstmnt 1744538295 >> >> INSERT INTO JOB_DETAIL (JOB_ID, TASK_ID, COMPUTE_RESOURCE_CONSUMED, >> >> CREATION_TIME, JOB_DESCRIPTION, JOBNAME, WORKING_DIR) >> >> VALUES (?, ?, ?, ?, ?, ?, ?) >> >> [params=3D(String) none, (String) >> tempNode_56f41083-10be-4161-8803-ee1106bd3f54, (null) null, (Timestamp) >> 2015-08-17 06:49:01.613, (null) null, (String) A516147113, (null) null]} >> [code=3D1048, state=3D23000] >> >> at >> org.apache.openjpa.lib.jdbc.LoggingConnectionDecorator.wrap(LoggingConne= ctionDecorator.java:247) >> >> at >> org.apache.openjpa.lib.jdbc.LoggingConnectionDecorator.wrap(LoggingConne= ctionDecorator.java:223) >> >> at >> org.apache.openjpa.lib.jdbc.LoggingConnectionDecorator.access$1000(Loggi= ngConnectionDecorator.java:72) >> >> at >> org.apache.openjpa.lib.jdbc.LoggingConnectionDecorator$LoggingConnection= $LoggingPreparedStatement.executeUpdate(LoggingConnectionDecorator.java:116= 2) >> >> at >> org.apache.openjpa.lib.jdbc.DelegatingPreparedStatement.executeUpdate(De= legatingPreparedStatement.java:272) >> >> at >> org.apache.openjpa.jdbc.kernel.JDBCStoreManager$CancelPreparedStatement.= executeUpdate(JDBCStoreManager.java:1741) >> >> at >> org.apache.openjpa.jdbc.kernel.PreparedStatementManagerImpl.executeUpdat= e(PreparedStatementManagerImpl.java:267) >> >> at >> org.apache.openjpa.jdbc.kernel.PreparedStatementManagerImpl.flushAndUpda= te(PreparedStatementManagerImpl.java:118) >> >> ... 31 more >> >> 2015-08-17 06:49:01,691 [pool-8-thread-56] INFO >> org.apache.airavata.gfac.ssh.provider.impl.SSHProvider - Saving data fo= r >> future recovery: >> >> 2015-08-17 06:49:01,691 [pool-8-thread-56] INFO >> org.apache.airavata.gfac.ssh.provider.impl.SSHProvider - >> >> 2015-08-17 06:49:01,721 [pool-8-thread-56] ERROR >> org.apache.airavata.gfac.core.provider.GFacProviderException - Error >> persisting job statusError while adding JOB_DETAIL >> >> org.apache.airavata.gfac.GFacException: Error persisting job statusError >> while adding JOB_DETAIL >> >> at >> org.apache.airavata.gfac.core.utils.GFacUtils.saveJobStatus(GFacUtils.ja= va:242) >> >> at >> org.apache.airavata.gfac.ssh.provider.impl.SSHProvider.execute(SSHProvid= er.java:215) >> >> at >> org.apache.airavata.gfac.core.cpi.BetterGfacImpl.executeProvider(BetterG= facImpl.java:842) >> >> at >> org.apache.airavata.gfac.core.cpi.BetterGfacImpl.invokeProviderExecute(B= etterGfacImpl.java:753) >> >> at >> org.apache.airavata.gfac.core.cpi.BetterGfacImpl.launch(BetterGfacImpl.j= ava:703) >> >> at >> org.apache.airavata.gfac.core.cpi.BetterGfacImpl.submitJob(BetterGfacImp= l.java:492) >> >> at >> org.apache.airavata.gfac.core.cpi.BetterGfacImpl.submitJob(BetterGfacImp= l.java:183) >> >> at >> org.apache.airavata.gfac.core.utils.InputHandlerWorker.run(InputHandlerW= orker.java:47) >> >> at >> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.jav= a:1142) >> >> at >> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.ja= va:617) >> >> at java.lang.Thread.run(Thread.java:745) >> >> Caused by: org.apache.airavata.registry.cpi.RegistryException: Error >> while adding JOB_DETAIL >> >> at >> org.apache.airavata.persistance.registry.jpa.impl.RegistryImpl.add(Regis= tryImpl.java:182) >> >> at >> org.apache.airavata.gfac.core.utils.GFacUtils.saveJobStatus(GFacUtils.ja= va:232) >> >> ... 10 more >> >> Caused by: org.apache.airavata.registry.cpi.RegistryException: >> org.apache.airavata.registry.cpi.RegistryException: >> >> org.apache.openjpa.persistence.RollbackException: The transaction has be= en >> rolled back. See the nested exceptions for details on the errors that >> occurred. >> >> FailedObject: >> org.apache.airavata.persistance.registry.jpa.model.JobDetail@6f1d32ff >> >> at >> org.apache.airavata.persistance.registry.jpa.impl.ExperimentRegistry.add= JobDetails(ExperimentRegistry.java:1231) >> >> at >> org.apache.airavata.persistance.registry.jpa.impl.RegistryImpl.add(Regis= tryImpl.java:157) >> >> ... 11 more >> >> Caused by: org.apache.airavata.registry.cpi.RegistryException: >> >> org.apache.openjpa.persistence.RollbackException: The transaction has be= en >> rolled back. See the nested exceptions for details on the errors that >> occurred. >> >> FailedObject: >> org.apache.airavata.persistance.registry.jpa.model.JobDetail@6f1d32ff >> >> at >> org.apache.airavata.persistance.registry.jpa.resources.JobDetailResource= .save(JobDetailResource.java:312) >> >> at >> org.apache.airavata.persistance.registry.jpa.impl.ExperimentRegistry.add= JobDetails(ExperimentRegistry.java:1203) >> >> ... 12 more >> >> Caused by: >> org.apache.openjpa.persistence.RollbackException: The transaction has be= en >> rolled back. See the nested exceptions for details on the errors that >> occurred. >> >> FailedObject: >> org.apache.airavata.persistance.registry.jpa.model.JobDetail@6f1d32ff >> >> at >> org.apache.openjpa.persistence.EntityManagerImpl.commit(EntityManagerImp= l.java:594) >> >> at >> org.apache.airavata.persistance.registry.jpa.resources.JobDetailResource= .save(JobDetailResource.java:308) >> >> ... 13 more >> >> Caused by: >> org.apache.openjpa.persistence.EntityExistsException: The transaction ha= s >> been rolled back. See the nested exceptions for details on the errors t= hat >> occurred. >> >> FailedObject: >> org.apache.airavata.persistance.registry.jpa.model.JobDetail@6f1d32ff >> >> at >> org.apache.openjpa.kernel.BrokerImpl.newFlushException(BrokerImpl.java:2= 337) >> >> at org.apache.openjpa.kernel.BrokerImpl.flush(BrokerImpl.java:2174) >> >> at org.apache.openjpa.kernel.BrokerImpl.flushSafe(BrokerImpl.java:2072) >> >> at >> org.apache.openjpa.kernel.BrokerImpl.beforeCompletion(BrokerImpl.java:19= 90) >> >> at >> org.apache.openjpa.kernel.LocalManagedRuntime.commit(LocalManagedRuntime= .java:81) >> >> at org.apache.openjpa.kernel.BrokerImpl.commit(BrokerImpl.java:1514) >> >> at >> org.apache.openjpa.kernel.DelegatingBroker.commit(DelegatingBroker.java:= 933) >> >> at >> org.apache.openjpa.persistence.EntityManagerImpl.commit(EntityManagerImp= l.java:570) >> >> ... 14 more >> >> Caused by: >> org.apache.openjpa.persistence.EntityExistsException: Column >> 'JOB_DESCRIPTION' cannot be null {prepstmnt 1744538295 >> >> INSERT INTO JOB_DETAIL (JOB_ID, TASK_ID, COMPUTE_RESOURCE_CONSUMED, >> >> CREATION_TIME, JOB_DESCRIPTION, JOBNAME, WORKING_DIR) >> >> VALUES (?, ?, ?, ?, ?, ?, ?) >> >> [params=3D(String) none, (String) >> tempNode_56f41083-10be-4161-8803-ee1106bd3f54, (null) null, (Timestamp) >> 2015-08-17 06:49:01.613, (null) null, (String) A516147113, (null) null]} >> [code=3D1048, state=3D23000] >> >> FailedObject: >> org.apache.airavata.persistance.registry.jpa.model.JobDetail@6f1d32ff >> >> at org.apache.openjpa.jdbc.sql.DBDictionary.narrow(DBDictionary.java:490= 3) >> >> at >> org.apache.openjpa.jdbc.sql.DBDictionary.newStoreException(DBDictionary.= java:4878) >> >> at >> org.apache.openjpa.jdbc.sql.SQLExceptions.getStore(SQLExceptions.java:13= 6) >> >> at >> org.apache.openjpa.jdbc.sql.SQLExceptions.getStore(SQLExceptions.java:78= ) >> >> at >> org.apache.openjpa.jdbc.kernel.PreparedStatementManagerImpl.flushAndUpda= te(PreparedStatementManagerImpl.java:143) >> >> at >> org.apache.openjpa.jdbc.kernel.BatchingPreparedStatementManagerImpl.flus= hAndUpdate(BatchingPreparedStatementManagerImpl.java:79) >> >> at >> org.apache.openjpa.jdbc.kernel.PreparedStatementManagerImpl.flushInterna= l(PreparedStatementManagerImpl.java:99) >> >> at >> org.apache.openjpa.jdbc.kernel.PreparedStatementManagerImpl.flush(Prepar= edStatementManagerImpl.java:87) >> >> at >> org.apache.openjpa.jdbc.kernel.ConstraintUpdateManager.flush(ConstraintU= pdateManager.java:550) >> >> at >> org.apache.openjpa.jdbc.kernel.ConstraintUpdateManager.flush(ConstraintU= pdateManager.java:106) >> >> at >> org.apache.openjpa.jdbc.kernel.BatchingConstraintUpdateManager.flush(Bat= chingConstraintUpdateManager.java:59) >> >> at >> org.apache.openjpa.jdbc.kernel.AbstractUpdateManager.flush(AbstractUpdat= eManager.java:103) >> >> at >> org.apache.openjpa.jdbc.kernel.AbstractUpdateManager.flush(AbstractUpdat= eManager.java:76) >> >> at >> org.apache.openjpa.jdbc.kernel.JDBCStoreManager.flush(JDBCStoreManager.j= ava:713) >> >> at >> org.apache.openjpa.kernel.DelegatingStoreManager.flush(DelegatingStoreMa= nager.java:131) >> >> ... 21 more >> >> Caused by: org.apache.openjpa.lib.jdbc.ReportingSQLException: Column >> 'JOB_DESCRIPTION' cannot be null {prepstmnt 1744538295 >> >> INSERT INTO JOB_DETAIL (JOB_ID, TASK_ID, COMPUTE_RESOURCE_CONSUMED, >> >> CREATION_TIME, JOB_DESCRIPTION, JOBNAME, WORKING_DIR) >> >> VALUES (?, ?, ?, ?, ?, ?, ?) >> >> [params=3D(String) none, (String) >> tempNode_56f41083-10be-4161-8803-ee1106bd3f54, (null) null, (Timestamp) >> 2015-08-17 06:49:01.613, (null) null, (String) A516147113, (null) null]} >> [code=3D1048, state=3D23000] >> >> at >> org.apache.openjpa.lib.jdbc.LoggingConnectionDecorator.wrap(LoggingConne= ctionDecorator.java:247) >> >> at >> org.apache.openjpa.lib.jdbc.LoggingConnectionDecorator.wrap(LoggingConne= ctionDecorator.java:223) >> >> at >> org.apache.openjpa.lib.jdbc.LoggingConnectionDecorator.access$1000(Loggi= ngConnectionDecorator.java:72) >> >> at >> org.apache.openjpa.lib.jdbc.LoggingConnectionDecorator$LoggingConnection= $LoggingPreparedStatement.executeUpdate(LoggingConnectionDecorator.java:116= 2) >> >> at >> org.apache.openjpa.lib.jdbc.DelegatingPreparedStatement.executeUpdate(De= legatingPreparedStatement.java:272) >> >> at >> org.apache.openjpa.jdbc.kernel.JDBCStoreManager$CancelPreparedStatement.= executeUpdate(JDBCStoreManager.java:1741) >> >> at >> org.apache.openjpa.jdbc.kernel.PreparedStatementManagerImpl.executeUpdat= e(PreparedStatementManagerImpl.java:267) >> >> at >> org.apache.openjpa.jdbc.kernel.PreparedStatementManagerImpl.flushAndUpda= te(PreparedStatementManagerImpl.java:118) >> >> ... 31 more >> >> 2015-08-17 06:49:01,723 [pool-8-thread-56] ERROR >> org.apache.airavata.gfac.GFacException - Error while executing provider >> org.apache.airavata.gfac.ssh.provider.impl.SSHProvider functionality. >> >> org.apache.airavata.gfac.core.provider.GFacProviderException: Error >> persisting job statusError while adding JOB_DETAIL >> >> at >> org.apache.airavata.gfac.ssh.provider.impl.SSHProvider.execute(SSHProvid= er.java:231) >> >> at >> org.apache.airavata.gfac.core.cpi.BetterGfacImpl.executeProvider(BetterG= facImpl.java:842) >> >> at >> org.apache.airavata.gfac.core.cpi.BetterGfacImpl.invokeProviderExecute(B= etterGfacImpl.java:753) >> >> at >> org.apache.airavata.gfac.core.cpi.BetterGfacImpl.launch(BetterGfacImpl.j= ava:703) >> >> at >> org.apache.airavata.gfac.core.cpi.BetterGfacImpl.submitJob(BetterGfacImp= l.java:492) >> >> at >> org.apache.airavata.gfac.core.cpi.BetterGfacImpl.submitJob(BetterGfacImp= l.java:183) >> >> at >> org.apache.airavata.gfac.core.utils.InputHandlerWorker.run(InputHandlerW= orker.java:47) >> >> at >> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.jav= a:1142) >> >> at >> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.ja= va:617) >> >> at java.lang.Thread.run(Thread.java:745) >> >> Caused by: org.apache.airavata.gfac.GFacException: Error persisting job >> statusError while adding JOB_DETAIL >> >> at >> org.apache.airavata.gfac.core.utils.GFacUtils.saveJobStatus(GFacUtils.ja= va:242) >> >> at >> org.apache.airavata.gfac.ssh.provider.impl.SSHProvider.execute(SSHProvid= er.java:215) >> >> ... 9 more >> >> Caused by: org.apache.airavata.registry.cpi.RegistryException: Error >> while adding JOB_DETAIL >> >> at >> org.apache.airavata.persistance.registry.jpa.impl.RegistryImpl.add(Regis= tryImpl.java:182) >> >> at >> org.apache.airavata.gfac.core.utils.GFacUtils.saveJobStatus(GFacUtils.ja= va:232) >> >> ... 10 more >> >> Caused by: org.apache.airavata.registry.cpi.RegistryException: >> org.apache.airavata.registry.cpi.RegistryException: >> >> org.apache.openjpa.persistence.RollbackException: The transaction has be= en >> rolled back. See the nested exceptions for details on the errors that >> occurred. >> >> FailedObject: >> org.apache.airavata.persistance.registry.jpa.model.JobDetail@6f1d32ff >> >> at >> org.apache.airavata.persistance.registry.jpa.impl.ExperimentRegistry.add= JobDetails(ExperimentRegistry.java:1231) >> >> at >> org.apache.airavata.persistance.registry.jpa.impl.RegistryImpl.add(Regis= tryImpl.java:157) >> >> ... 11 more >> >> Caused by: org.apache.airavata.registry.cpi.RegistryException: >> >> org.apache.openjpa.persistence.RollbackException: The transaction has be= en >> rolled back. See the nested exceptions for details on the errors that >> occurred. >> >> FailedObject: >> org.apache.airavata.persistance.registry.jpa.model.JobDetail@6f1d32ff >> >> at >> org.apache.airavata.persistance.registry.jpa.resources.JobDetailResource= .save(JobDetailResource.java:312) >> >> at >> org.apache.airavata.persistance.registry.jpa.impl.ExperimentRegistry.add= JobDetails(ExperimentRegistry.java:1203) >> >> ... 12 more >> >> Caused by: >> org.apache.openjpa.persistence.RollbackException: The transaction has be= en >> rolled back. See the nested exceptions for details on the errors that >> occurred. >> >> FailedObject: >> org.apache.airavata.persistance.registry.jpa.model.JobDetail@6f1d32ff >> >> at >> org.apache.openjpa.persistence.EntityManagerImpl.commit(EntityManagerImp= l.java:594) >> >> at >> org.apache.airavata.persistance.registry.jpa.resources.JobDetailResource= .save(JobDetailResource.java:308) >> >> ... 13 more >> >> Caused by: >> org.apache.openjpa.persistence.EntityExistsException: The transaction ha= s >> been rolled back. See the nested exceptions for details on the errors t= hat >> occurred. >> >> FailedObject: >> org.apache.airavata.persistance.registry.jpa.model.JobDetail@6f1d32ff >> >> at >> org.apache.openjpa.kernel.BrokerImpl.newFlushException(BrokerImpl.java:2= 337) >> >> at org.apache.openjpa.kernel.BrokerImpl.flush(BrokerImpl.java:2174) >> >> at org.apache.openjpa.kernel.BrokerImpl.flushSafe(BrokerImpl.java:2072) >> >> at >> org.apache.openjpa.kernel.BrokerImpl.beforeCompletion(BrokerImpl.java:19= 90) >> >> at >> org.apache.openjpa.kernel.LocalManagedRuntime.commit(LocalManagedRuntime= .java:81) >> >> at org.apache.openjpa.kernel.BrokerImpl.commit(BrokerImpl.java:1514) >> >> at >> org.apache.openjpa.kernel.DelegatingBroker.commit(DelegatingBroker.java:= 933) >> >> at >> org.apache.openjpa.persistence.EntityManagerImpl.commit(EntityManagerImp= l.java:570) >> >> ... 14 more >> >> Caused by: >> org.apache.openjpa.persistence.EntityExistsException: Column >> 'JOB_DESCRIPTION' cannot be null {prepstmnt 1744538295 >> >> INSERT INTO JOB_DETAIL (JOB_ID, TASK_ID, COMPUTE_RESOURCE_CONSUMED, >> >> CREATION_TIME, JOB_DESCRIPTION, JOBNAME, WORKING_DIR) >> >> VALUES (?, ?, ?, ?, ?, ?, ?) >> >> [params=3D(String) none, (String) >> tempNode_56f41083-10be-4161-8803-ee1106bd3f54, (null) null, (Timestamp) >> 2015-08-17 06:49:01.613, (null) null, (String) A516147113, (null) null]} >> [code=3D1048, state=3D23000] >> >> FailedObject: >> org.apache.airavata.persistance.registry.jpa.model.JobDetail@6f1d32ff >> >> at org.apache.openjpa.jdbc.sql.DBDictionary.narrow(DBDictionary.java:490= 3) >> >> at >> org.apache.openjpa.jdbc.sql.DBDictionary.newStoreException(DBDictionary.= java:4878) >> >> at >> org.apache.openjpa.jdbc.sql.SQLExceptions.getStore(SQLExceptions.java:13= 6) >> >> at >> org.apache.openjpa.jdbc.sql.SQLExceptions.getStore(SQLExceptions.java:78= ) >> >> at >> org.apache.openjpa.jdbc.kernel.PreparedStatementManagerImpl.flushAndUpda= te(PreparedStatementManagerImpl.java:143) >> >> at >> org.apache.openjpa.jdbc.kernel.BatchingPreparedStatementManagerImpl.flus= hAndUpdate(BatchingPreparedStatementManagerImpl.java:79) >> >> at >> org.apache.openjpa.jdbc.kernel.PreparedStatementManagerImpl.flushInterna= l(PreparedStatementManagerImpl.java:99) >> >> at >> org.apache.openjpa.jdbc.kernel.PreparedStatementManagerImpl.flush(Prepar= edStatementManagerImpl.java:87) >> >> at >> org.apache.openjpa.jdbc.kernel.ConstraintUpdateManager.flush(ConstraintU= pdateManager.java:550) >> >> at >> org.apache.openjpa.jdbc.kernel.ConstraintUpdateManager.flush(ConstraintU= pdateManager.java:106) >> >> at >> org.apache.openjpa.jdbc.kernel.BatchingConstraintUpdateManager.flush(Bat= chingConstraintUpdateManager.java:59) >> >> at >> org.apache.openjpa.jdbc.kernel.AbstractUpdateManager.flush(AbstractUpdat= eManager.java:103) >> >> at >> org.apache.openjpa.jdbc.kernel.AbstractUpdateManager.flush(AbstractUpdat= eManager.java:76) >> >> at >> org.apache.openjpa.jdbc.kernel.JDBCStoreManager.flush(JDBCStoreManager.j= ava:713) >> >> at >> org.apache.openjpa.kernel.DelegatingStoreManager.flush(DelegatingStoreMa= nager.java:131) >> >> ... 21 more >> >> Caused by: org.apache.openjpa.lib.jdbc.ReportingSQLException: Column >> 'JOB_DESCRIPTION' cannot be null {prepstmnt 1744538295 >> >> INSERT INTO JOB_DETAIL (JOB_ID, TASK_ID, COMPUTE_RESOURCE_CONSUMED, >> >> CREATION_TIME, JOB_DESCRIPTION, JOBNAME, WORKING_DIR) >> >> VALUES (?, ?, ?, ?, ?, ?, ?) >> >> [params=3D(String) none, (String) >> tempNode_56f41083-10be-4161-8803-ee1106bd3f54, (null) null, (Timestamp) >> 2015-08-17 06:49:01.613, (null) null, (String) A516147113, (null) null]} >> [code=3D1048, state=3D23000] >> >> at >> org.apache.openjpa.lib.jdbc.LoggingConnectionDecorator.wrap(LoggingConne= ctionDecorator.java:247) >> >> at >> org.apache.openjpa.lib.jdbc.LoggingConnectionDecorator.wrap(LoggingConne= ctionDecorator.java:223) >> >> at >> org.apache.openjpa.lib.jdbc.LoggingConnectionDecorator.access$1000(Loggi= ngConnectionDecorator.java:72) >> >> at >> org.apache.openjpa.lib.jdbc.LoggingConnectionDecorator$LoggingConnection= $LoggingPreparedStatement.executeUpdate(LoggingConnectionDecorator.java:116= 2) >> >> at >> org.apache.openjpa.lib.jdbc.DelegatingPreparedStatement.executeUpdate(De= legatingPreparedStatement.java:272) >> >> at >> org.apache.openjpa.jdbc.kernel.JDBCStoreManager$CancelPreparedStatement.= executeUpdate(JDBCStoreManager.java:1741) >> >> at >> org.apache.openjpa.jdbc.kernel.PreparedStatementManagerImpl.executeUpdat= e(PreparedStatementManagerImpl.java:267) >> >> at >> org.apache.openjpa.jdbc.kernel.PreparedStatementManagerImpl.flushAndUpda= te(PreparedStatementManagerImpl.java:118) >> >> ... 31 more >> >> 2015-08-17 06:49:01,724 [pool-8-thread-56] ERROR >> org.apache.airavata.gfac.core.monitor.MonitorID - There is not job crea= ted >> at this point >> >> 2015-08-17 06:49:01,724 [pool-8-thread-56] INFO >> org.apache.airavata.gfac.core.monitor.GfacInternalStatusUpdator - Gfac >> internal state changed to: FAILED >> >> 2015-08-17 06:49:01,724 [pool-8-thread-56] INFO >> org.apache.airavata.gfac.core.monitor.GfacInternalStatusUpdator - >> Experiment Failed, So removing the ZK entry for the >> experimentdimuthu_experiment_gau_11_6e6ffc70-5dd2-4a9f-94b6-8cd6e672ec08 >> >> 2015-08-17 06:49:01,724 [pool-8-thread-56] INFO >> org.apache.airavata.gfac.core.monitor.GfacInternalStatusUpdator - >> Zookeeper experiment Path: >> /gfac-experiments/gfac-node0/dimuthu_experiment_gau_11_6e6ffc70-5dd2-4a9= f-94b6-8cd6e672ec08 >> >> 2015-08-17 06:49:01,725 [pool-8-thread-56] ERROR >> org.apache.airavata.gfac.core.cpi.BetterGfacImpl - Error occured during >> updating the statuses of Experiments,tasks or Job statuses to failed, >> NullPointerException occurred because at this point there might not have >> Job Created >> >> org.apache.airavata.gfac.GFacException: Error while executing provider >> org.apache.airavata.gfac.ssh.provider.impl.SSHProvider functionality. >> >> at >> org.apache.airavata.gfac.core.cpi.BetterGfacImpl.executeProvider(BetterG= facImpl.java:844) >> >> at >> org.apache.airavata.gfac.core.cpi.BetterGfacImpl.invokeProviderExecute(B= etterGfacImpl.java:753) >> >> at >> org.apache.airavata.gfac.core.cpi.BetterGfacImpl.launch(BetterGfacImpl.j= ava:703) >> >> at >> org.apache.airavata.gfac.core.cpi.BetterGfacImpl.submitJob(BetterGfacImp= l.java:492) >> >> at >> org.apache.airavata.gfac.core.cpi.BetterGfacImpl.submitJob(BetterGfacImp= l.java:183) >> >> at >> org.apache.airavata.gfac.core.utils.InputHandlerWorker.run(InputHandlerW= orker.java:47) >> >> at >> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.jav= a:1142) >> >> at >> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.ja= va:617) >> >> at java.lang.Thread.run(Thread.java:745) >> >> Caused by: org.apache.airavata.gfac.core.provider.GFacProviderException: >> Error persisting job statusError while adding JOB_DETAIL >> >> at >> org.apache.airavata.gfac.ssh.provider.impl.SSHProvider.execute(SSHProvid= er.java:231) >> >> at >> org.apache.airavata.gfac.core.cpi.BetterGfacImpl.executeProvider(BetterG= facImpl.java:842) >> >> ... 8 more >> >> Caused by: org.apache.airavata.gfac.GFacException: Error persisting job >> statusError while adding JOB_DETAIL >> >> at >> org.apache.airavata.gfac.core.utils.GFacUtils.saveJobStatus(GFacUtils.ja= va:242) >> >> at >> org.apache.airavata.gfac.ssh.provider.impl.SSHProvider.execute(SSHProvid= er.java:215) >> >> ... 9 more >> >> Caused by: org.apache.airavata.registry.cpi.RegistryException: Error >> while adding JOB_DETAIL >> >> at >> org.apache.airavata.persistance.registry.jpa.impl.RegistryImpl.add(Regis= tryImpl.java:182) >> >> at >> org.apache.airavata.gfac.core.utils.GFacUtils.saveJobStatus(GFacUtils.ja= va:232) >> >> ... 10 more >> >> Caused by: org.apache.airavata.registry.cpi.RegistryException: >> org.apache.airavata.registry.cpi.RegistryException: >> >> org.apache.openjpa.persistence.RollbackException: The transaction has be= en >> rolled back. See the nested exceptions for details on the errors that >> occurred. >> >> FailedObject: >> org.apache.airavata.persistance.registry.jpa.model.JobDetail@6f1d32ff >> >> at >> org.apache.airavata.persistance.registry.jpa.impl.ExperimentRegistry.add= JobDetails(ExperimentRegistry.java:1231) >> >> at >> org.apache.airavata.persistance.registry.jpa.impl.RegistryImpl.add(Regis= tryImpl.java:157) >> >> ... 11 more >> >> Caused by: org.apache.airavata.registry.cpi.RegistryException: >> >> org.apache.openjpa.persistence.RollbackException: The transaction has be= en >> rolled back. See the nested exceptions for details on the errors that >> occurred. >> >> FailedObject: >> org.apache.airavata.persistance.registry.jpa.model.JobDetail@6f1d32ff >> >> at >> org.apache.airavata.persistance.registry.jpa.resources.JobDetailResource= .save(JobDetailResource.java:312) >> >> at >> org.apache.airavata.persistance.registry.jpa.impl.ExperimentRegistry.add= JobDetails(ExperimentRegistry.java:1203) >> >> ... 12 more >> >> Caused by: >> org.apache.openjpa.persistence.RollbackException: The transaction has be= en >> rolled back. See the nested exceptions for details on the errors that >> occurred. >> >> FailedObject: >> org.apache.airavata.persistance.registry.jpa.model.JobDetail@6f1d32ff >> >> at >> org.apache.openjpa.persistence.EntityManagerImpl.commit(EntityManagerImp= l.java:594) >> >> at >> org.apache.airavata.persistance.registry.jpa.resources.JobDetailResource= .save(JobDetailResource.java:308) >> >> ... 13 more >> >> Caused by: >> org.apache.openjpa.persistence.EntityExistsException: The transaction ha= s >> been rolled back. See the nested exceptions for details on the errors t= hat >> occurred. >> >> FailedObject: >> org.apache.airavata.persistance.registry.jpa.model.JobDetail@6f1d32ff >> >> at >> org.apache.openjpa.kernel.BrokerImpl.newFlushException(BrokerImpl.java:2= 337) >> >> at org.apache.openjpa.kernel.BrokerImpl.flush(BrokerImpl.java:2174) >> >> at org.apache.openjpa.kernel.BrokerImpl.flushSafe(BrokerImpl.java:2072) >> >> at >> org.apache.openjpa.kernel.BrokerImpl.beforeCompletion(BrokerImpl.java:19= 90) >> >> at >> org.apache.openjpa.kernel.LocalManagedRuntime.commit(LocalManagedRuntime= .java:81) >> >> at org.apache.openjpa.kernel.BrokerImpl.commit(BrokerImpl.java:1514) >> >> at >> org.apache.openjpa.kernel.DelegatingBroker.commit(DelegatingBroker.java:= 933) >> >> at >> org.apache.openjpa.persistence.EntityManagerImpl.commit(EntityManagerImp= l.java:570) >> >> ... 14 more >> >> Caused by: >> org.apache.openjpa.persistence.EntityExistsException: Column >> 'JOB_DESCRIPTION' cannot be null {prepstmnt 1744538295 >> >> INSERT INTO JOB_DETAIL (JOB_ID, TASK_ID, COMPUTE_RESOURCE_CONSUMED, >> >> CREATION_TIME, JOB_DESCRIPTION, JOBNAME, WORKING_DIR) >> >> VALUES (?, ?, ?, ?, ?, ?, ?) >> >> [params=3D(String) none, (String) >> tempNode_56f41083-10be-4161-8803-ee1106bd3f54, (null) null, (Timestamp) >> 2015-08-17 06:49:01.613, (null) null, (String) A516147113, (null) null]} >> [code=3D1048, state=3D23000] >> >> FailedObject: >> org.apache.airavata.persistance.registry.jpa.model.JobDetail@6f1d32ff >> >> at org.apache.openjpa.jdbc.sql.DBDictionary.narrow(DBDictionary.java:490= 3) >> >> at >> org.apache.openjpa.jdbc.sql.DBDictionary.newStoreException(DBDictionary.= java:4878) >> >> at >> org.apache.openjpa.jdbc.sql.SQLExceptions.getStore(SQLExceptions.java:13= 6) >> >> at >> org.apache.openjpa.jdbc.sql.SQLExceptions.getStore(SQLExceptions.java:78= ) >> >> at >> org.apache.openjpa.jdbc.kernel.PreparedStatementManagerImpl.flushAndUpda= te(PreparedStatementManagerImpl.java:143) >> >> at >> org.apache.openjpa.jdbc.kernel.BatchingPreparedStatementManagerImpl.flus= hAndUpdate(BatchingPreparedStatementManagerImpl.java:79) >> >> at >> org.apache.openjpa.jdbc.kernel.PreparedStatementManagerImpl.flushInterna= l(PreparedStatementManagerImpl.java:99) >> >> at >> org.apache.openjpa.jdbc.kernel.PreparedStatementManagerImpl.flush(Prepar= edStatementManagerImpl.java:87) >> >> at >> org.apache.openjpa.jdbc.kernel.ConstraintUpdateManager.flush(ConstraintU= pdateManager.java:550) >> >> at >> org.apache.openjpa.jdbc.kernel.ConstraintUpdateManager.flush(ConstraintU= pdateManager.java:106) >> >> at >> org.apache.openjpa.jdbc.kernel.BatchingConstraintUpdateManager.flush(Bat= chingConstraintUpdateManager.java:59) >> >> at >> org.apache.openjpa.jdbc.kernel.AbstractUpdateManager.flush(AbstractUpdat= eManager.java:103) >> >> at >> org.apache.openjpa.jdbc.kernel.AbstractUpdateManager.flush(AbstractUpdat= eManager.java:76) >> >> at >> org.apache.openjpa.jdbc.kernel.JDBCStoreManager.flush(JDBCStoreManager.j= ava:713) >> >> at >> org.apache.openjpa.kernel.DelegatingStoreManager.flush(DelegatingStoreMa= nager.java:131) >> >> ... 21 more >> >> Caused by: org.apache.openjpa.lib.jdbc.ReportingSQLException: Column >> 'JOB_DESCRIPTION' cannot be null {prepstmnt 1744538295 >> >> INSERT INTO JOB_DETAIL (JOB_ID, TASK_ID, COMPUTE_RESOURCE_CONSUMED, >> >> CREATION_TIME, JOB_DESCRIPTION, JOBNAME, WORKING_DIR) >> >> VALUES (?, ?, ?, ?, ?, ?, ?) >> >> [params=3D(String) none, (String) >> tempNode_56f41083-10be-4161-8803-ee1106bd3f54, (null) null, (Timestamp) >> 2015-08-17 06:49:01.613, (null) null, (String) A516147113, (null) null]} >> [code=3D1048, state=3D23000] >> >> at >> org.apache.openjpa.lib.jdbc.LoggingConnectionDecorator.wrap(LoggingConne= ctionDecorator.java:247) >> >> at >> org.apache.openjpa.lib.jdbc.LoggingConnectionDecorator.wrap(LoggingConne= ctionDecorator.java:223) >> >> at >> org.apache.openjpa.lib.jdbc.LoggingConnectionDecorator.access$1000(Loggi= ngConnectionDecorator.java:72) >> >> at >> org.apache.openjpa.lib.jdbc.LoggingConnectionDecorator$LoggingConnection= $LoggingPreparedStatement.executeUpdate(LoggingConnectionDecorator.java:116= 2) >> >> at >> org.apache.openjpa.lib.jdbc.DelegatingPreparedStatement.executeUpdate(De= legatingPreparedStatement.java:272) >> >> at >> org.apache.openjpa.jdbc.kernel.JDBCStoreManager$CancelPreparedStatement.= executeUpdate(JDBCStoreManager.java:1741) >> >> at >> org.apache.openjpa.jdbc.kernel.PreparedStatementManagerImpl.executeUpdat= e(PreparedStatementManagerImpl.java:267) >> >> at >> org.apache.openjpa.jdbc.kernel.PreparedStatementManagerImpl.flushAndUpda= te(PreparedStatementManagerImpl.java:118) >> >> ... 31 more >> >> 2015-08-17 06:49:01,726 [pool-8-thread-56] INFO >> org.apache.airavata.gfac.core.monitor.AiravataWorkflowNodeStatusUpdator = - >> expId - dimuthu_experiment_gau_11_6e6ffc70-5dd2-4a9f-94b6-8cd6e672ec08: >> Updating workflow node status for >> tempNode_20c62522-62c4-4342-81d8-9541dbf13862:FAILED >> >> 2015-08-17 06:49:02,129 [pool-8-thread-56] INFO >> org.apache.airavata.messaging.core.impl.RabbitMQStatusPublisher - >> Publishing status to rabbitmq... >> >> 2015-08-17 06:49:03,300 [pool-8-thread-56] INFO >> org.apache.airavata.api.server.listener.AiravataExperimentStatusUpdator = - >> Updating the experiment status of experiment: >> dimuthu_experiment_gau_11_6e6ffc70-5dd2-4a9f-94b6-8cd6e672ec08 to FAILED >> >> 2015-08-17 06:49:03,368 [pool-8-thread-56] INFO >> org.apache.airavata.messaging.core.impl.RabbitMQStatusPublisher - >> Publishing status to rabbitmq... >> >> 2015-08-17 06:49:03,376 [pool-8-thread-56] INFO >> org.apache.airavata.api.server.listener.AiravataExperimentStatusUpdator = - >> Clean up task - Acknowledging delivery Tag - 93 >> >> 2015-08-17 06:49:03,611 [pool-8-thread-56] ERROR >> org.apache.airavata.gfac.GFacException - Error while executing provider >> org.apache.airavata.gfac.ssh.provider.impl.SSHProvider functionality. >> >> org.apache.airavata.gfac.GFacException: Error while executing provider >> org.apache.airavata.gfac.ssh.provider.impl.SSHProvider functionality. >> >> at >> org.apache.airavata.gfac.core.cpi.BetterGfacImpl.executeProvider(BetterG= facImpl.java:844) >> >> at >> org.apache.airavata.gfac.core.cpi.BetterGfacImpl.invokeProviderExecute(B= etterGfacImpl.java:753) >> >> at >> org.apache.airavata.gfac.core.cpi.BetterGfacImpl.launch(BetterGfacImpl.j= ava:703) >> >> at >> org.apache.airavata.gfac.core.cpi.BetterGfacImpl.submitJob(BetterGfacImp= l.java:492) >> >> at >> org.apache.airavata.gfac.core.cpi.BetterGfacImpl.submitJob(BetterGfacImp= l.java:183) >> >> at >> org.apache.airavata.gfac.core.utils.InputHandlerWorker.run(InputHandlerW= orker.java:47) >> >> at >> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.jav= a:1142) >> >> at >> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.ja= va:617) >> >> at java.lang.Thread.run(Thread.java:745) >> >> Caused by: org.apache.airavata.gfac.core.provider.GFacProviderException: >> Error persisting job statusError while adding JOB_DETAIL >> >> at >> org.apache.airavata.gfac.ssh.provider.impl.SSHProvider.execute(SSHProvid= er.java:231) >> >> at >> org.apache.airavata.gfac.core.cpi.BetterGfacImpl.executeProvider(BetterG= facImpl.java:842) >> >> ... 8 more >> >> Caused by: org.apache.airavata.gfac.GFacException: Error persisting job >> statusError while adding JOB_DETAIL >> >> at >> org.apache.airavata.gfac.core.utils.GFacUtils.saveJobStatus(GFacUtils.ja= va:242) >> >> at >> org.apache.airavata.gfac.ssh.provider.impl.SSHProvider.execute(SSHProvid= er.java:215) >> >> ... 9 more >> >> Caused by: org.apache.airavata.registry.cpi.RegistryException: Error >> while adding JOB_DETAIL >> >> at >> org.apache.airavata.persistance.registry.jpa.impl.RegistryImpl.add(Regis= tryImpl.java:182) >> >> at >> org.apache.airavata.gfac.core.utils.GFacUtils.saveJobStatus(GFacUtils.ja= va:232) >> >> ... 10 more >> >> Caused by: org.apache.airavata.registry.cpi.RegistryException: >> org.apache.airavata.registry.cpi.RegistryException: >> >> org.apache.openjpa.persistence.RollbackException: The transaction has be= en >> rolled back. See the nested exceptions for details on the errors that >> occurred. >> >> FailedObject: >> org.apache.airavata.persistance.registry.jpa.model.JobDetail@6f1d32ff >> >> at >> org.apache.airavata.persistance.registry.jpa.impl.ExperimentRegistry.add= JobDetails(ExperimentRegistry.java:1231) >> >> at >> org.apache.airavata.persistance.registry.jpa.impl.RegistryImpl.add(Regis= tryImpl.java:157) >> >> ... 11 more >> >> Caused by: org.apache.airavata.registry.cpi.RegistryException: >> >> org.apache.openjpa.persistence.RollbackException: The transaction has be= en >> rolled back. See the nested exceptions for details on the errors that >> occurred. >> >> FailedObject: >> org.apache.airavata.persistance.registry.jpa.model.JobDetail@6f1d32ff >> >> at >> org.apache.airavata.persistance.registry.jpa.resources.JobDetailResource= .save(JobDetailResource.java:312) >> >> at >> org.apache.airavata.persistance.registry.jpa.impl.ExperimentRegistry.add= JobDetails(ExperimentRegistry.java:1203) >> >> ... 12 more >> >> Caused by: >> org.apache.openjpa.persistence.RollbackException: The transaction has be= en >> rolled back. See the nested exceptions for details on the errors that >> occurred. >> >> FailedObject: >> org.apache.airavata.persistance.registry.jpa.model.JobDetail@6f1d32ff >> >> at >> org.apache.openjpa.persistence.EntityManagerImpl.commit(EntityManagerImp= l.java:594) >> >> at >> org.apache.airavata.persistance.registry.jpa.resources.JobDetailResource= .save(JobDetailResource.java:308) >> >> ... 13 more >> >> Caused by: >> org.apache.openjpa.persistence.EntityExistsException: The transaction ha= s >> been rolled back. See the nested exceptions for details on the errors t= hat >> occurred. >> >> FailedObject: >> org.apache.airavata.persistance.registry.jpa.model.JobDetail@6f1d32ff >> >> at >> org.apache.openjpa.kernel.BrokerImpl.newFlushException(BrokerImpl.java:2= 337) >> >> at org.apache.openjpa.kernel.BrokerImpl.flush(BrokerImpl.java:2174) >> >> at org.apache.openjpa.kernel.BrokerImpl.flushSafe(BrokerImpl.java:2072) >> >> at >> org.apache.openjpa.kernel.BrokerImpl.beforeCompletion(BrokerImpl.java:19= 90) >> >> at >> org.apache.openjpa.kernel.LocalManagedRuntime.commit(LocalManagedRuntime= .java:81) >> >> at org.apache.openjpa.kernel.BrokerImpl.commit(BrokerImpl.java:1514) >> >> at >> org.apache.openjpa.kernel.DelegatingBroker.commit(DelegatingBroker.java:= 933) >> >> at >> org.apache.openjpa.persistence.EntityManagerImpl.commit(EntityManagerImp= l.java:570) >> >> ... 14 more >> >> Caused by: >> org.apache.openjpa.persistence.EntityExistsException: Column >> 'JOB_DESCRIPTION' cannot be null {prepstmnt 1744538295 >> >> INSERT INTO JOB_DETAIL (JOB_ID, TASK_ID, COMPUTE_RESOURCE_CONSUMED, >> >> CREATION_TIME, JOB_DESCRIPTION, JOBNAME, WORKING_DIR) >> >> VALUES (?, ?, ?, ?, ?, ?, ?) >> >> [params=3D(String) none, (String) >> tempNode_56f41083-10be-4161-8803-ee1106bd3f54, (null) null, (Timestamp) >> 2015-08-17 06:49:01.613, (null) null, (String) A516147113, (null) null]} >> [code=3D1048, state=3D23000] >> >> FailedObject: >> org.apache.airavata.persistance.registry.jpa.model.JobDetail@6f1d32ff >> >> at org.apache.openjpa.jdbc.sql.DBDictionary.narrow(DBDictionary.java:490= 3) >> >> at >> org.apache.openjpa.jdbc.sql.DBDictionary.newStoreException(DBDictionary.= java:4878) >> >> at >> org.apache.openjpa.jdbc.sql.SQLExceptions.getStore(SQLExceptions.java:13= 6) >> >> at >> org.apache.openjpa.jdbc.sql.SQLExceptions.getStore(SQLExceptions.java:78= ) >> >> at >> org.apache.openjpa.jdbc.kernel.PreparedStatementManagerImpl.flushAndUpda= te(PreparedStatementManagerImpl.java:143) >> >> at >> org.apache.openjpa.jdbc.kernel.BatchingPreparedStatementManagerImpl.flus= hAndUpdate(BatchingPreparedStatementManagerImpl.java:79) >> >> at >> org.apache.openjpa.jdbc.kernel.PreparedStatementManagerImpl.flushInterna= l(PreparedStatementManagerImpl.java:99) >> >> at >> org.apache.openjpa.jdbc.kernel.PreparedStatementManagerImpl.flush(Prepar= edStatementManagerImpl.java:87) >> >> at >> org.apache.openjpa.jdbc.kernel.ConstraintUpdateManager.flush(ConstraintU= pdateManager.java:550) >> >> at >> org.apache.openjpa.jdbc.kernel.ConstraintUpdateManager.flush(ConstraintU= pdateManager.java:106) >> >> at >> org.apache.openjpa.jdbc.kernel.BatchingConstraintUpdateManager.flush(Bat= chingConstraintUpdateManager.java:59) >> >> at >> org.apache.openjpa.jdbc.kernel.AbstractUpdateManager.flush(AbstractUpdat= eManager.java:103) >> >> at >> org.apache.openjpa.jdbc.kernel.AbstractUpdateManager.flush(AbstractUpdat= eManager.java:76) >> >> at >> org.apache.openjpa.jdbc.kernel.JDBCStoreManager.flush(JDBCStoreManager.j= ava:713) >> >> at >> org.apache.openjpa.kernel.DelegatingStoreManager.flush(DelegatingStoreMa= nager.java:131) >> >> ... 21 more >> >> Caused by: org.apache.openjpa.lib.jdbc.ReportingSQLException: Column >> 'JOB_DESCRIPTION' cannot be null {prepstmnt 1744538295 >> >> INSERT INTO JOB_DETAIL (JOB_ID, TASK_ID, COMPUTE_RESOURCE_CONSUMED, >> >> CREATION_TIME, JOB_DESCRIPTION, JOBNAME, WORKING_DIR) >> >> VALUES (?, ?, ?, ?, ?, ?, ?) >> >> [params=3D(String) none, (String) >> tempNode_56f41083-10be-4161-8803-ee1106bd3f54, (null) null, (Timestamp) >> 2015-08-17 06:49:01.613, (null) null, (String) A516147113, (null) null]} >> [code=3D1048, state=3D23000] >> >> at >> org.apache.openjpa.lib.jdbc.LoggingConnectionDecorator.wrap(LoggingConne= ctionDecorator.java:247) >> >> at >> org.apache.openjpa.lib.jdbc.LoggingConnectionDecorator.wrap(LoggingConne= ctionDecorator.java:223) >> >> at >> org.apache.openjpa.lib.jdbc.LoggingConnectionDecorator.access$1000(Loggi= ngConnectionDecorator.java:72) >> >> at >> org.apache.openjpa.lib.jdbc.LoggingConnectionDecorator$LoggingConnection= $LoggingPreparedStatement.executeUpdate(LoggingConnectionDecorator.java:116= 2) >> >> at >> org.apache.openjpa.lib.jdbc.DelegatingPreparedStatement.executeUpdate(De= legatingPreparedStatement.java:272) >> >> at >> org.apache.openjpa.jdbc.kernel.JDBCStoreManager$CancelPreparedStatement.= executeUpdate(JDBCStoreManager.java:1741) >> >> at >> org.apache.openjpa.jdbc.kernel.PreparedStatementManagerImpl.executeUpdat= e(PreparedStatementManagerImpl.java:267) >> >> at >> org.apache.openjpa.jdbc.kernel.PreparedStatementManagerImpl.flushAndUpda= te(PreparedStatementManagerImpl.java:118) >> >> ... 31 more >> >> 2015-08-17 06:49:03,692 [pool-8-thread-56] ERROR >> org.apache.airavata.gfac.GFacException - Error launching the Job >> >> org.apache.airavata.gfac.GFacException: Error while executing provider >> org.apache.airavata.gfac.ssh.provider.impl.SSHProvider functionality. >> >> at >> org.apache.airavata.gfac.core.cpi.BetterGfacImpl.launch(BetterGfacImpl.j= ava:743) >> >> at >> org.apache.airavata.gfac.core.cpi.BetterGfacImpl.submitJob(BetterGfacImp= l.java:492) >> >> at >> org.apache.airavata.gfac.core.cpi.BetterGfacImpl.submitJob(BetterGfacImp= l.java:183) >> >> --001a1134bad072383e051d820fb7 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable

Hi Sudhakar,

Both PGA and command line invocation sho= uld use same account. But you are right that scigap account is not added, I= will get that fixed now.

Suresh


On Mon, Aug 17, 2015 at 8:48 AM Pamidighanta= m, Sudhakar V <spamidig@illinoi= s.edu> wrote:
>From PGA it may be going to ogce login on Comet and I noticed here it is us= ing scigap login which does not appear to be in gaussian user group.=C2=A0

Sudhakar.

On Aug 17, 2015, at 6:41 AM, DImuthu Upeksha <dimuthu.upeksha2@gmail.com&g= t; wrote:

Hi Sudhakar,

I'm using my user 'dimuthu' to submit the job. However fro= m PGA I was able to submit using same user.

On Mon, Aug 17, 2015 at 4:34 PM, Pamidighantam, = Sudhakar V <spamidig@illinois.edu> wrote:
Dimuthu:
Are you using ind123 for the account to submit the job on comet. Also sciga= p user need to be added to gaussian group on comet for this gaussian job to= execute.=C2=A0

Thanks,
Sudhakar.
On Aug 17, 2015, at 5:54 AM, DImuthu Upeksha <dimuthu.upeksha2@gmail.com&g= t; wrote:

Hi all,

$subject please. What could be the reason for this? Hope the issue is = with this=C2=A0

org.apache.airavata.gsi.ssh.impl.GSISSHAbstractCluster=C2=A0=C2=A0- St= andard Error output : sbatch: error: bank_limit plugin: expired user, can&#= 39;t submit job

Here is the full log
--------------------

2015-08-17 06:48:07,601 [Thread-9] INFO=C2=A0 org.apache.airavata.= gfac.monitor.email.EmailBasedMonitor=C2=A0 - [EJM]: Job Monitor Map is empt= y, no need to retrieve emails

2015-08-17 06:48:17,602 [Thread-9] INFO=C2=A0 org.apache.airavata.= gfac.monitor.email.EmailBasedMonitor=C2=A0 - [EJM]: Job Monitor Map is empt= y, no need to retrieve emails

2015-08-17 06:48:27,603 [Thread-9] INFO=C2=A0 org.apache.airavata.= gfac.monitor.email.EmailBasedMonitor=C2=A0 - [EJM]: Job Monitor Map is empt= y, no need to retrieve emails

2015-08-17 06:48:37,603 [Thread-9] INFO=C2=A0 org.apache.airavata.= gfac.monitor.email.EmailBasedMonitor=C2=A0 - [EJM]: Job Monitor Map is empt= y, no need to retrieve emails

2015-08-17 06:48:43,612 [pool-3-thread-3] INFO=C2=A0 org.apache.ai= ravata.gfac.core.utils.GFacUtils=C2=A0 - This is a new Job, so creating all= the experiment docs from the scratch

2015-08-17 06:48:43,679 [pool-3-thread-3] INFO=C2=A0 org.apache.ai= ravata.gfac.core.utils.GFacUtils=C2=A0 - Created the node: /gfac-experiment= s/gfac-node0/dimuthu_experiment_gau_11_6e6ffc70-5dd2-4a9f-94b6-8cd6e672ec08= /state successfully !

2015-08-17 06:48:43,692 [pool-3-thread-3] INFO=C2=A0 org.apache.ai= ravata.gfac.server.GfacServerHandler=C2=A0 - ------------------------------= -----------------------92--------------------------------------------------= ---

2015-08-17 06:48:43,692 [pool-3-thread-3] INFO=C2=A0 org.apache.ai= ravata.gfac.server.GfacServerHandler=C2=A0 - Id:dimuthu_experiment_gau_11_6= e6ffc70-5dd2-4a9f-94b6-8cd6e672ec08 : GFac Received submit job request for = the Experiment: dimuthu_experiment_gau_11_6e6ffc70-5dd2-4a9f-94b6-8cd6e672e= c08 TaskId: tempNode_56f41083-10be-4161-8803-ee1106bd3f54

2015-08-17 06:48:44,595 [pool-8-thread-56] ERROR org.apache.airava= ta.gfac.core.monitor.MonitorID=C2=A0 - There is not job created at this poi= nt

2015-08-17 06:48:44,599 [pool-8-thread-56] INFO=C2=A0 org.apache.a= iravata.gfac.core.monitor.GfacInternalStatusUpdator=C2=A0 - Gfac internal s= tate changed to: ACCEPTED

2015-08-17 06:48:44,790 [pool-8-thread-56] ERROR org.apache.airava= ta.gfac.core.monitor.MonitorID=C2=A0 - There is not job created at this poi= nt

2015-08-17 06:48:44,790 [pool-8-thread-56] INFO=C2=A0 org.apache.a= iravata.gfac.core.monitor.GfacInternalStatusUpdator=C2=A0 - Gfac internal s= tate changed to: INHANDLERSINVOKING

2015-08-17 06:48:44,882 [pool-8-thread-56] INFO=C2=A0 org.apache.a= iravata.gfac.core.handler.AppDescriptorCheckHandler=C2=A0 - Invoking Applic= ationDescriptorCheckHandler ...

2015-08-17 06:48:44,905 [pool-8-thread-56] INFO=C2=A0 org.apache.a= iravata.gfac.core.handler.AppDescriptorCheckHandler=C2=A0 - Recoverable dat= a is saving to zk: /oasis/scratch/comet/scigap/temp_project/airavata-workdi= rs,/oasis/scratch/comet/scigap/temp_project/airavata-workdirs/dimuthu_exper= iment_gau_11_6e6ffc70-5dd2-4a9f-94b6-8cd6e672ec08,/oasis/scratch/comet/scig= ap/temp_project/airavata-workdirs/dimuthu_experiment_gau_11_6e6ffc70-5dd2-4= a9f-94b6-8cd6e672ec08,/oasis/scratch/comet/scigap/temp_project/airavata-wor= kdirs/dimuthu_experiment_gau_11_6e6ffc70-5dd2-4a9f-94b6-8cd6e672ec08,/oasis= /scratch/comet/scigap/temp_project/airavata-workdirs/dimuthu_experiment_gau= _11_6e6ffc70-5dd2-4a9f-94b6-8cd6e672ec08/Gaussian.stdout,/oasis/scratch/com= et/scigap/temp_project/airavata-workdirs/dimuthu_experiment_gau_11_6e6ffc70= -5dd2-4a9f-94b6-8cd6e672ec08/Gaussian.stderr

2015-08-17 06:48:45,216 [pool-8-thread-56] INFO=C2=A0 org.apache.a= iravata.gsi.ssh.impl.GSISSHAbstractCluster=C2=A0 - Listing directory: comet.sdsc.edu:~/

2015-08-17 06:48:45,424 [pool-8-thread-56] INFO=C2=A0 org.apache.a= iravata.gfac.ssh.util.GFACSSHUtils=C2=A0 - Re-using the same connection use= d with the connection string:scigapcomet.sdsc.edu22

2015-08-17 06:48:45,424 [pool-8-thread-56] INFO=C2=A0 org.apache.a= iravata.gfac.ssh.handler.SSHDirectorySetupHandler=C2=A0 - Setup SSH job dir= ectorties

2015-08-17 06:48:45,441 [pool-8-thread-56] INFO=C2=A0 org.apache.a= iravata.gfac.ssh.handler.SSHDirectorySetupHandler=C2=A0 - Successfully retr= ieved the Security Context

2015-08-17 06:48:45,441 [pool-8-thread-56] INFO=C2=A0 org.apache.a= iravata.gsi.ssh.impl.GSISSHAbstractCluster=C2=A0 - Creating directory: comet.sdsc.edu:/oasis/scratch/comet/scigap/temp_project/airavata-workdi= rs/dimuthu_experiment_gau_11_6e6ffc70-5dd2-4a9f-94b6-8cd6e672ec08

2015-08-17 06:48:45,924 [pool-8-thread-56] ERROR org.apache.airava= ta.gfac.ssh.handler.AdvancedSCPInputHandler=C2=A0 - Previously stored data= =C2=A0 is wrong so we continue the operations

2015-08-17 06:48:46,251 [pool-8-thread-56] INFO=C2=A0 org.apache.a= iravata.gsi.ssh.impl.GSISSHAbstractCluster=C2=A0 - Transfering from:localho= st:/home/airavata/oodt/archive/ghqcgrdl57hmhbhd37p11qqjnl/c4b4nhtwbs3.inp T= o:file://home/airavata/inputData/dimuthu_experiment_gau_11_6e6ffc70-5dd2-4a= 9f-94b6-8cd6e672ec08/tempNode_56f41083-10be-4161-8803-ee1106bd3f54

2015-08-17 06:48:47,604 [Thread-9] INFO=C2=A0 org.apache.airavata.= gfac.monitor.email.EmailBasedMonitor=C2=A0 - [EJM]: Job Monitor Map is empt= y, no need to retrieve emails

2015-08-17 06:48:48,263 [pool-8-thread-56] INFO=C2=A0 org.apache.a= iravata.gfac.ssh.handler.SSHInputHandler=C2=A0 - Successfully retrieved the= Security Context

2015-08-17 06:48:48,265 [pool-8-thread-56] INFO=C2=A0 org.apache.a= iravata.gfac.ssh.handler.SSHInputHandler=C2=A0 - Invoking SCPInputHandler

2015-08-17 06:48:48,333 [pool-8-thread-56] INFO=C2=A0 org.apache.a= iravata.gsi.ssh.impl.GSISSHAbstractCluster=C2=A0 - Transfering file:////home/airavata/inputData/dimuthu_experiment_gau_11_6e6ffc70-5dd2= -4a9f-94b6-8cd6e672ec08/tempNode_56f41083-10be-4161-8803-ee1106bd3f54/c4b4n= htwbs3.inp To:comet.sdsc.edu:/oasis/scratch/comet/scigap/temp_project/a= iravata-workdirs/dimuthu_experiment_gau_11_6e6ffc70-5dd2-4a9f-94b6-8cd6e672= ec08/c4b4nhtwbs3.inp

2015-08-17 06:48:49,125 [pool-8-thread-56] ERROR org.apache.airava= ta.gfac.core.monitor.MonitorID=C2=A0 - There is not job created at this poi= nt

2015-08-17 06:48:49,125 [pool-8-thread-56] INFO=C2=A0 org.apache.a= iravata.gfac.core.monitor.GfacInternalStatusUpdator=C2=A0 - Gfac internal s= tate changed to: INHANDLERSINVOKED

2015-08-17 06:48:49,199 [pool-8-thread-56] ERROR org.apache.airava= ta.gfac.core.monitor.MonitorID=C2=A0 - There is not job created at this poi= nt

2015-08-17 06:48:49,200 [pool-8-thread-56] INFO=C2=A0 org.apache.a= iravata.gfac.core.monitor.GfacInternalStatusUpdator=C2=A0 - Gfac internal s= tate changed to: PROVIDERINVOKING

2015-08-17 06:48:49,299 [pool-8-thread-56] INFO=C2=A0 org.apache.a= iravata.gfac.ssh.provider.impl.SSHProvider=C2=A0 - Successfully retrieved t= he Security Context

2015-08-17 06:48:49,346 [pool-8-thread-56] INFO=C2=A0 org.apache.a= iravata.gfac.ssh.util.GFACSSHUtils=C2=A0 - Email list: gw77jobs@scigap.or= g,jobnotif@sci= gap.org

2015-08-17 06:48:49,363 [pool-8-thread-56] INFO=C2=A0 org.apache.a= iravata.gfac.ssh.provider.impl.SSHProvider=C2=A0 - <ns:JobDescriptor xml= ns:ns=3D"http://airavata.apache.org/gsi= /ssh/2012/12"><ns:userName>scigap</ns:userName><= ns:shellName>/bin/bash</ns:shellName><ns:queueName>compute&l= t;/ns:queueName><ns:jobName>A516147113</ns:jobName><ns:al= lEnvExport>true</ns:allEnvExport><ns:mailOptions>abe</ns:= mailOptions><ns:mailAddress>gw77jobs@scigap.org,jobnotif@scigap.org</ns:mailAddress><= ns:acountString>sds128</ns:acountString><ns:maxWallTime>00:3= 0:00</ns:maxWallTime><ns:standardOutFile>/oasis/scratch/comet/s= cigap/temp_project/airavata-workdirs/dimuthu_experiment_gau_11_6e6ffc70-5dd= 2-4a9f-94b6-8cd6e672ec08/Gaussian.stdout</ns:standardOutFile><ns:s= tandardErrorFile>/oasis/scratch/comet/scigap/temp_project/airavata-workd= irs/dimuthu_experiment_gau_11_6e6ffc70-5dd2-4a9f-94b6-8cd6e672ec08/Gaussian= .stderr</ns:standardErrorFile><ns:outputDirectory>/oasis/scratc= h/comet/scigap/temp_project/airavata-workdirs/dimuthu_experiment_gau_11_6e6= ffc70-5dd2-4a9f-94b6-8cd6e672ec08</ns:outputDirectory><ns:inputDir= ectory>/oasis/scratch/comet/scigap/temp_project/airavata-workdirs/dimuth= u_experiment_gau_11_6e6ffc70-5dd2-4a9f-94b6-8cd6e672ec08</ns:inputDirect= ory><ns:nodes>1</ns:nodes><ns:processesPerNode>16</= ns:processesPerNode><ns:cpuCount>16</ns:cpuCount><ns:work= ingDirectory>/oasis/scratch/comet/scigap/temp_project/airavata-workdirs/= dimuthu_experiment_gau_11_6e6ffc70-5dd2-4a9f-94b6-8cd6e672ec08</ns:worki= ngDirectory><ns:executablePath>/opt/gaussian/g09/g09</ns:execut= ablePath><ns:inputs><ns:input>gaussian.out</ns:input>&= lt;/ns:inputs><ns:owner>scigap</ns:owner><ns:usedMem>1= 00</ns:usedMem><ns:moduleLoadCommands><ns:command>module load gaussian</ns:command></ns:moduleLoadCommands><ns:callB= ackIp>149.165.228.42</ns:callBackIp><ns:callBackPort>8950<= ;/ns:callBackPort></ns:JobDescriptor>

2015-08-17 06:48:49,443 [pool-8-thread-56] INFO=C2=A0 org.apache.a= iravata.gsi.ssh.impl.GSISSHAbstractCluster=C2=A0 - Transfering file://home/airavata/production-deployment/gfac-instance1/apache-airavat= a-server-0.15-SNAPSHOT/bin/1029102677.slurm To:comet.sdsc.edu:/oasis/sc= ratch/comet/scigap/temp_project/airavata-workdirs/dimuthu_experiment_gau_11= _6e6ffc70-5dd2-4a9f-94b6-8cd6e672ec08

2015-08-17 06:48:49,897 [pool-8-thread-56] INFO=C2=A0 org.apache.a= iravata.gsi.ssh.impl.GSISSHAbstractCluster=C2=A0 - Executing RawCommand : /= usr/bin/sbatch /oasis/scratch/comet/scigap/temp_project/airavata-workdirs/d= imuthu_experiment_gau_11_6e6ffc70-5dd2-4a9f-94b6-8cd6e672ec08/1029102677.sl= urm

2015-08-17 06:48:50,195 [pool-8-thread-56] INFO=C2=A0 org.apache.a= iravata.gsi.ssh.impl.GSISSHAbstractCluster=C2=A0 - StandardOutput Returned:=

2015-08-17 06:48:50,196 [pool-8-thread-56] INFO=C2=A0 org.apache.a= iravata.gsi.ssh.impl.GSISSHAbstractCluster=C2=A0 - StandardError=C2=A0 Retu= rned:sbatch: error: bank_limit plugin: expired user, can't submit job

sbatch: error: Batch job submission failed: Invalid account or acc= ount/partition combination specified


2015-08-17 06:48:50,196 [pool-8-thread-56] ERROR org.apache.airava= ta.gsi.ssh.impl.GSISSHAbstractCluster=C2=A0 - Standard Error output : sbatc= h: error: bank_limit plugin: expired user, can't submit job

sbatch: error: Batch job submission failed: Invalid account or acc= ount/partition combination specified


2015-08-17 06:48:55,197 [pool-8-thread-56] ERROR org.apache.airava= ta.gsi.ssh.impl.GSISSHAbstractCluster=C2=A0 - Error occured during job subm= ission but doing a retry

2015-08-17 06:48:55,197 [pool-8-thread-56] INFO=C2=A0 org.apache.a= iravata.gsi.ssh.impl.GSISSHAbstractCluster=C2=A0 - Transfering file://home/airavata/production-deployment/gfac-instance1/apache-airavat= a-server-0.15-SNAPSHOT/bin/1029102677.slurm To:comet.sdsc.edu:/oasis/sc= ratch/comet/scigap/temp_project/airavata-workdirs/dimuthu_experiment_gau_11= _6e6ffc70-5dd2-4a9f-94b6-8cd6e672ec08

2015-08-17 06:48:55,641 [pool-8-thread-56] INFO=C2=A0 org.apache.a= iravata.gsi.ssh.impl.GSISSHAbstractCluster=C2=A0 - Executing RawCommand : /= usr/bin/sbatch /oasis/scratch/comet/scigap/temp_project/airavata-workdirs/d= imuthu_experiment_gau_11_6e6ffc70-5dd2-4a9f-94b6-8cd6e672ec08/1029102677.sl= urm

2015-08-17 06:48:55,905 [pool-8-thread-56] INFO=C2=A0 org.apache.a= iravata.gsi.ssh.impl.GSISSHAbstractCluster=C2=A0 - StandardOutput Returned:=

2015-08-17 06:48:55,905 [pool-8-thread-56] INFO=C2=A0 org.apache.a= iravata.gsi.ssh.impl.GSISSHAbstractCluster=C2=A0 - StandardError=C2=A0 Retu= rned:sbatch: error: bank_limit plugin: expired user, can't submit job

sbatch: error: Batch job submission failed: Invalid account or acc= ount/partition combination specified


2015-08-17 06:48:55,905 [pool-8-thread-56] ERROR org.apache.airava= ta.gsi.ssh.impl.GSISSHAbstractCluster=C2=A0 - Standard Error output : sbatc= h: error: bank_limit plugin: expired user, can't submit job

sbatch: error: Batch job submission failed: Invalid account or acc= ount/partition combination specified


2015-08-17 06:48:57,612 [Thread-9] INFO=C2=A0 org.apache.airavata.= gfac.monitor.email.EmailBasedMonitor=C2=A0 - [EJM]: Job Monitor Map is empt= y, no need to retrieve emails

2015-08-17 06:49:00,906 [pool-8-thread-56] ERROR org.apache.airava= ta.gsi.ssh.impl.GSISSHAbstractCluster=C2=A0 - Error occured during job subm= ission but doing a retry

2015-08-17 06:49:00,906 [pool-8-thread-56] INFO=C2=A0 org.apache.a= iravata.gsi.ssh.impl.GSISSHAbstractCluster=C2=A0 - Transfering file://home/airavata/production-deployment/gfac-instance1/apache-airavat= a-server-0.15-SNAPSHOT/bin/1029102677.slurm To:comet.sdsc.edu:/oasis/sc= ratch/comet/scigap/temp_project/airavata-workdirs/dimuthu_experiment_gau_11= _6e6ffc70-5dd2-4a9f-94b6-8cd6e672ec08

2015-08-17 06:49:01,321 [pool-8-thread-56] INFO=C2=A0 org.apache.a= iravata.gsi.ssh.impl.GSISSHAbstractCluster=C2=A0 - Executing RawCommand : /= usr/bin/sbatch /oasis/scratch/comet/scigap/temp_project/airavata-workdirs/d= imuthu_experiment_gau_11_6e6ffc70-5dd2-4a9f-94b6-8cd6e672ec08/1029102677.sl= urm

2015-08-17 06:49:01,587 [pool-8-thread-56] INFO=C2=A0 org.apache.a= iravata.gsi.ssh.impl.GSISSHAbstractCluster=C2=A0 - StandardOutput Returned:=

2015-08-17 06:49:01,587 [pool-8-thread-56] INFO=C2=A0 org.apache.a= iravata.gsi.ssh.impl.GSISSHAbstractCluster=C2=A0 - StandardError=C2=A0 Retu= rned:sbatch: error: bank_limit plugin: expired user, can't submit job

sbatch: error: Batch job submission failed: Invalid account or acc= ount/partition combination specified


2015-08-17 06:49:01,587 [pool-8-thread-56] ERROR org.apache.airava= ta.gsi.ssh.impl.GSISSHAbstractCluster=C2=A0 - Standard Error output : sbatc= h: error: bank_limit plugin: expired user, can't submit job

sbatch: error: Batch job submission failed: Invalid account or acc= ount/partition combination specified


2015-08-17 06:49:01,588 [pool-8-thread-56] ERROR org.apache.airava= ta.gfac.ssh.provider.impl.SSHProvider=C2=A0 - Error submitting the job to h= ost comet.sdsc.edu message: Error reading output of job submission

=C2=A0StandardOutput:=C2=A0

=C2=A0StandardError: sbatch: error: bank_limit plugin: expired use= r, can't submit job

sbatch: error: Batch job submission failed: Invalid account or acc= ount/partition combination specified


2015-08-17 06:49:01,660 [pool-8-thread-56] ERROR org.apache.airava= ta.persistance.registry.jpa.resources.JobDetailResource=C2=A0 - The transac= tion has been rolled back.=C2=A0 See the nested exceptions for details on t= he errors that occurred.

<openjpa-2.2.0-r422266:1244990 fatal store error> org.apache= .openjpa.persistence.RollbackException: The transaction has been rolled bac= k.=C2=A0 See the nested exceptions for details on the errors that occurred.=

FailedObject: org.apache.airavata.persistance.registry.jpa.model.J= obDetail@6f1d32ff

at org.apache.openjpa.persistence.EntityManagerImpl.c= ommit(EntityManagerImpl.java:594)

at org.apache.airavata.persistance.registry.jpa.resou= rces.JobDetailResource.save(JobDetailResource.java:308)

at org.apache.airavata.persistance.registry.jpa.impl.= ExperimentRegistry.addJobDetails(ExperimentRegistry.java:1203)

at org.apache.airavata.persistance.registry.jpa.impl.= RegistryImpl.add(RegistryImpl.java:157)

at org.apache.airavata.gfac.core.utils.GFacUtils.save= JobStatus(GFacUtils.java:232)

at org.apache.airavata.gfac.ssh.provider.impl.SSHProv= ider.execute(SSHProvider.java:215)

at org.apache.airavata.gfac.core.cpi.BetterGfacImpl.e= xecuteProvider(BetterGfacImpl.java:842)

at org.apache.airavata.gfac.core.cpi.BetterGfacImpl.i= nvokeProviderExecute(BetterGfacImpl.java:753)

at org.apache.airavata.gfac.core.cpi.BetterGfacImpl.l= aunch(BetterGfacImpl.java:703)

at org.apache.airavata.gfac.core.cpi.BetterGfacImpl.s= ubmitJob(BetterGfacImpl.java:492)

at org.apache.airavata.gfac.core.cpi.BetterGfacImpl.s= ubmitJob(BetterGfacImpl.java:183)

at org.apache.airavata.gfac.core.utils.InputHandlerWo= rker.run(InputHandlerWorker.java:47)

at java.util.concurrent.ThreadPoolExecutor.runWorker(= ThreadPoolExecutor.java:1142)

at java.util.concurrent.ThreadPoolExecutor$Worker.run= (ThreadPoolExecutor.java:617)

at java.lang.Thread.run(Thread.java:745)

Caused by: <openjpa-2.2.0-r422266:1244990 fatal store error>= org.apache.openjpa.persistence.EntityExistsException: The transaction has = been rolled back.=C2=A0 See the nested exceptions for details on the errors= that occurred.

FailedObject: org.apache.airavata.persistance.registry.jpa.model.J= obDetail@6f1d32ff

at org.apache.openjpa.kernel.BrokerImpl.newFlushExcep= tion(BrokerImpl.java:2337)

at org.apache.openjpa.kernel.BrokerImpl.flush(BrokerI= mpl.java:2174)

at org.apache.openjpa.kernel.BrokerImpl.flushSafe(Bro= kerImpl.java:2072)

at org.apache.openjpa.kernel.BrokerImpl.beforeComplet= ion(BrokerImpl.java:1990)

at org.apache.openjpa.kernel.LocalManagedRuntime.comm= it(LocalManagedRuntime.java:81)

at org.apache.openjpa.kernel.BrokerImpl.commit(Broker= Impl.java:1514)

at org.apache.openjpa.kernel.DelegatingBroker.commit(= DelegatingBroker.java:933)

at org.apache.openjpa.persistence.EntityManagerImpl.c= ommit(EntityManagerImpl.java:570)

... 14 more

Caused by: <openjpa-2.2.0-r422266:1244990 fatal store error>= org.apache.openjpa.persistence.EntityExistsException: Column 'JOB_DESC= RIPTION' cannot be null {prepstmnt 1744538295=C2=A0

INSERT INTO JOB_DETAIL (JOB_ID, TASK_ID, COMPUTE_RESOURCE_CONSUMED= ,=C2=A0

=C2=A0 =C2=A0 =C2=A0 =C2=A0 CREATION_TIME, JOB_DESCRIPTION, JOBNAM= E, WORKING_DIR)=C2=A0

=C2=A0 =C2=A0 VALUES (?, ?, ?, ?, ?, ?, ?)=C2=A0

[params=3D(String) none, (String) tempNode_56f41083-10be-4161-8803= -ee1106bd3f54, (null) null, (Timestamp) 2015-08-17 06:49:01.613, (null) nul= l, (String) A516147113, (null) null]} [code=3D1048, state=3D23000]

FailedObject: org.apache.airavata.persistance.registry.jpa.model.J= obDetail@6f1d32ff

at org.apache.openjpa.jdbc.sql.DBDictionary.narrow(DB= Dictionary.java:4903)

at org.apache.openjpa.jdbc.sql.DBDictionary.newStoreE= xception(DBDictionary.java:4878)

at org.apache.openjpa.jdbc.sql.SQLExceptions.getStore= (SQLExceptions.java:136)

at org.apache.openjpa.jdbc.sql.SQLExceptions.getStore= (SQLExceptions.java:78)

at org.apache.openjpa.jdbc.kernel.PreparedStatementMa= nagerImpl.flushAndUpdate(PreparedStatementManagerImpl.java:143)

at org.apache.openjpa.jdbc.kernel.BatchingPreparedSta= tementManagerImpl.flushAndUpdate(BatchingPreparedStatementManagerImpl.java:= 79)

at org.apache.openjpa.jdbc.kernel.PreparedStatementMa= nagerImpl.flushInternal(PreparedStatementManagerImpl.java:99)

at org.apache.openjpa.jdbc.kernel.PreparedStatementMa= nagerImpl.flush(PreparedStatementManagerImpl.java:87)

at org.apache.openjpa.jdbc.kernel.ConstraintUpdateMan= ager.flush(ConstraintUpdateManager.java:550)

at org.apache.openjpa.jdbc.kernel.ConstraintUpdateMan= ager.flush(ConstraintUpdateManager.java:106)

at org.apache.openjpa.jdbc.kernel.BatchingConstraintU= pdateManager.flush(BatchingConstraintUpdateManager.java:59)

at org.apache.openjpa.jdbc.kernel.AbstractUpdateManag= er.flush(AbstractUpdateManager.java:103)

at org.apache.openjpa.jdbc.kernel.AbstractUpdateManag= er.flush(AbstractUpdateManager.java:76)

at org.apache.openjpa.jdbc.kernel.JDBCStoreManager.fl= ush(JDBCStoreManager.java:713)

at org.apache.openjpa.kernel.DelegatingStoreManager.f= lush(DelegatingStoreManager.java:131)

... 21 more

Caused by: org.apache.openjpa.lib.jdbc.ReportingSQLException: Colu= mn 'JOB_DESCRIPTION' cannot be null {prepstmnt 1744538295=C2=A0

INSERT INTO JOB_DETAIL (JOB_ID, TASK_ID, COMPUTE_RESOURCE_CONSUMED= ,=C2=A0

=C2=A0 =C2=A0 =C2=A0 =C2=A0 CREATION_TIME, JOB_DESCRIPTION, JOBNAM= E, WORKING_DIR)=C2=A0

=C2=A0 =C2=A0 VALUES (?, ?, ?, ?, ?, ?, ?)=C2=A0

[params=3D(String) none, (String) tempNode_56f41083-10be-4161-8803= -ee1106bd3f54, (null) null, (Timestamp) 2015-08-17 06:49:01.613, (null) nul= l, (String) A516147113, (null) null]} [code=3D1048, state=3D23000]

at org.apache.openjpa.lib.jdbc.LoggingConnectionDecor= ator.wrap(LoggingConnectionDecorator.java:247)

at org.apache.openjpa.lib.jdbc.LoggingConnectionDecor= ator.wrap(LoggingConnectionDecorator.java:223)

at org.apache.openjpa.lib.jdbc.LoggingConnectionDecor= ator.access$1000(LoggingConnectionDecorator.java:72)

at org.apache.openjpa.lib.jdbc.LoggingConnectionDecor= ator$LoggingConnection$LoggingPreparedStatement.executeUpdate(LoggingConnec= tionDecorator.java:1162)

at org.apache.openjpa.lib.jdbc.DelegatingPreparedStat= ement.executeUpdate(DelegatingPreparedStatement.java:272)

at org.apache.openjpa.jdbc.kernel.JDBCStoreManager$Ca= ncelPreparedStatement.executeUpdate(JDBCStoreManager.java:1741)

at org.apache.openjpa.jdbc.kernel.PreparedStatementMa= nagerImpl.executeUpdate(PreparedStatementManagerImpl.java:267)

at org.apache.openjpa.jdbc.kernel.PreparedStatementMa= nagerImpl.flushAndUpdate(PreparedStatementManagerImpl.java:118)

... 31 more

2015-08-17 06:49:01,664 [pool-8-thread-56] ERROR org.apache.airava= ta.persistance.registry.jpa.impl.ExperimentRegistry=C2=A0 - Error while add= ing job details...

org.apache.airavata.registry.cpi.RegistryException: <openjpa-2.= 2.0-r422266:1244990 fatal store error> org.apache.openjpa.persistence.Ro= llbackException: The transaction has been rolled back.=C2=A0 See the nested= exceptions for details on the errors that occurred.

FailedObject: org.apache.airavata.persistance.registry.jpa.model.J= obDetail@6f1d32ff

at org.apache.airavata.persistance.registry.jpa.resou= rces.JobDetailResource.save(JobDetailResource.java:312)

at org.apache.airavata.persistance.registry.jpa.impl.= ExperimentRegistry.addJobDetails(ExperimentRegistry.java:1203)

at org.apache.airavata.persistance.registry.jpa.impl.= RegistryImpl.add(RegistryImpl.java:157)

at org.apache.airavata.gfac.core.utils.GFacUtils.save= JobStatus(GFacUtils.java:232)

at org.apache.airavata.gfac.ssh.provider.impl.SSHProv= ider.execute(SSHProvider.java:215)

at org.apache.airavata.gfac.core.cpi.BetterGfacImpl.e= xecuteProvider(BetterGfacImpl.java:842)

at org.apache.airavata.gfac.core.cpi.BetterGfacImpl.i= nvokeProviderExecute(BetterGfacImpl.java:753)

at org.apache.airavata.gfac.core.cpi.BetterGfacImpl.l= aunch(BetterGfacImpl.java:703)

at org.apache.airavata.gfac.core.cpi.BetterGfacImpl.s= ubmitJob(BetterGfacImpl.java:492)

at org.apache.airavata.gfac.core.cpi.BetterGfacImpl.s= ubmitJob(BetterGfacImpl.java:183)

at org.apache.airavata.gfac.core.utils.InputHandlerWo= rker.run(InputHandlerWorker.java:47)

at java.util.concurrent.ThreadPoolExecutor.runWorker(= ThreadPoolExecutor.java:1142)

at java.util.concurrent.ThreadPoolExecutor$Worker.run= (ThreadPoolExecutor.java:617)

at java.lang.Thread.run(Thread.java:745)

Caused by: <openjpa-2.2.0-r422266:1244990 fatal store error>= org.apache.openjpa.persistence.RollbackException: The transaction has been= rolled back.=C2=A0 See the nested exceptions for details on the errors tha= t occurred.

FailedObject: org.apache.airavata.persistance.registry.jpa.model.J= obDetail@6f1d32ff

at org.apache.openjpa.persistence.EntityManagerImpl.c= ommit(EntityManagerImpl.java:594)

at org.apache.airavata.persistance.registry.jpa.resou= rces.JobDetailResource.save(JobDetailResource.java:308)

... 13 more

Caused by: <openjpa-2.2.0-r422266:1244990 fatal store error>= org.apache.openjpa.persistence.EntityExistsException: The transaction has = been rolled back.=C2=A0 See the nested exceptions for details on the errors= that occurred.

FailedObject: org.apache.airavata.persistance.registry.jpa.model.J= obDetail@6f1d32ff

at org.apache.openjpa.kernel.BrokerImpl.newFlushExcep= tion(BrokerImpl.java:2337)

at org.apache.openjpa.kernel.BrokerImpl.flush(BrokerI= mpl.java:2174)

at org.apache.openjpa.kernel.BrokerImpl.flushSafe(Bro= kerImpl.java:2072)

at org.apache.openjpa.kernel.BrokerImpl.beforeComplet= ion(BrokerImpl.java:1990)

at org.apache.openjpa.kernel.LocalManagedRuntime.comm= it(LocalManagedRuntime.java:81)

at org.apache.openjpa.kernel.BrokerImpl.commit(Broker= Impl.java:1514)

at org.apache.openjpa.kernel.DelegatingBroker.commit(= DelegatingBroker.java:933)

at org.apache.openjpa.persistence.EntityManagerImpl.c= ommit(EntityManagerImpl.java:570)

... 14 more

Caused by: <openjpa-2.2.0-r422266:1244990 fatal store error>= org.apache.openjpa.persistence.EntityExistsException: Column 'JOB_DESC= RIPTION' cannot be null {prepstmnt 1744538295=C2=A0

INSERT INTO JOB_DETAIL (JOB_ID, TASK_ID, COMPUTE_RESOURCE_CONSUMED= ,=C2=A0

=C2=A0 =C2=A0 =C2=A0 =C2=A0 CREATION_TIME, JOB_DESCRIPTION, JOBNAM= E, WORKING_DIR)=C2=A0

=C2=A0 =C2=A0 VALUES (?, ?, ?, ?, ?, ?, ?)=C2=A0

[params=3D(String) none, (String) tempNode_56f41083-10be-4161-8803= -ee1106bd3f54, (null) null, (Timestamp) 2015-08-17 06:49:01.613, (null) nul= l, (String) A516147113, (null) null]} [code=3D1048, state=3D23000]

FailedObject: org.apache.airavata.persistance.registry.jpa.model.J= obDetail@6f1d32ff

at org.apache.openjpa.jdbc.sql.DBDictionary.narrow(DB= Dictionary.java:4903)

at org.apache.openjpa.jdbc.sql.DBDictionary.newStoreE= xception(DBDictionary.java:4878)

at org.apache.openjpa.jdbc.sql.SQLExceptions.getStore= (SQLExceptions.java:136)

at org.apache.openjpa.jdbc.sql.SQLExceptions.getStore= (SQLExceptions.java:78)

at org.apache.openjpa.jdbc.kernel.PreparedStatementMa= nagerImpl.flushAndUpdate(PreparedStatementManagerImpl.java:143)

at org.apache.openjpa.jdbc.kernel.BatchingPreparedSta= tementManagerImpl.flushAndUpdate(BatchingPreparedStatementManagerImpl.java:= 79)

at org.apache.openjpa.jdbc.kernel.PreparedStatementMa= nagerImpl.flushInternal(PreparedStatementManagerImpl.java:99)

at org.apache.openjpa.jdbc.kernel.PreparedStatementMa= nagerImpl.flush(PreparedStatementManagerImpl.java:87)

at org.apache.openjpa.jdbc.kernel.ConstraintUpdateMan= ager.flush(ConstraintUpdateManager.java:550)

at org.apache.openjpa.jdbc.kernel.ConstraintUpdateMan= ager.flush(ConstraintUpdateManager.java:106)

at org.apache.openjpa.jdbc.kernel.BatchingConstraintU= pdateManager.flush(BatchingConstraintUpdateManager.java:59)

at org.apache.openjpa.jdbc.kernel.AbstractUpdateManag= er.flush(AbstractUpdateManager.java:103)

at org.apache.openjpa.jdbc.kernel.AbstractUpdateManag= er.flush(AbstractUpdateManager.java:76)

at org.apache.openjpa.jdbc.kernel.JDBCStoreManager.fl= ush(JDBCStoreManager.java:713)

at org.apache.openjpa.kernel.DelegatingStoreManager.f= lush(DelegatingStoreManager.java:131)

... 21 more

Caused by: org.apache.openjpa.lib.jdbc.ReportingSQLException: Colu= mn 'JOB_DESCRIPTION' cannot be null {prepstmnt 1744538295=C2=A0

INSERT INTO JOB_DETAIL (JOB_ID, TASK_ID, COMPUTE_RESOURCE_CONSUMED= ,=C2=A0

=C2=A0 =C2=A0 =C2=A0 =C2=A0 CREATION_TIME, JOB_DESCRIPTION, JOBNAM= E, WORKING_DIR)=C2=A0

=C2=A0 =C2=A0 VALUES (?, ?, ?, ?, ?, ?, ?)=C2=A0

[params=3D(String) none, (String) tempNode_56f41083-10be-4161-8803= -ee1106bd3f54, (null) null, (Timestamp) 2015-08-17 06:49:01.613, (null) nul= l, (String) A516147113, (null) null]} [code=3D1048, state=3D23000]

at org.apache.openjpa.lib.jdbc.LoggingConnectionDecor= ator.wrap(LoggingConnectionDecorator.java:247)

at org.apache.openjpa.lib.jdbc.LoggingConnectionDecor= ator.wrap(LoggingConnectionDecorator.java:223)

at org.apache.openjpa.lib.jdbc.LoggingConnectionDecor= ator.access$1000(LoggingConnectionDecorator.java:72)

at org.apache.openjpa.lib.jdbc.LoggingConnectionDecor= ator$LoggingConnection$LoggingPreparedStatement.executeUpdate(LoggingConnec= tionDecorator.java:1162)

at org.apache.openjpa.lib.jdbc.DelegatingPreparedStat= ement.executeUpdate(DelegatingPreparedStatement.java:272)

at org.apache.openjpa.jdbc.kernel.JDBCStoreManager$Ca= ncelPreparedStatement.executeUpdate(JDBCStoreManager.java:1741)

at org.apache.openjpa.jdbc.kernel.PreparedStatementMa= nagerImpl.executeUpdate(PreparedStatementManagerImpl.java:267)

at org.apache.openjpa.jdbc.kernel.PreparedStatementMa= nagerImpl.flushAndUpdate(PreparedStatementManagerImpl.java:118)

... 31 more

2015-08-17 06:49:01,666 [pool-8-thread-56] ERROR org.apache.airava= ta.persistance.registry.jpa.impl.RegistryImpl=C2=A0 - Error while adding JO= B_DETAIL

org.apache.airavata.registry.cpi.RegistryException: org.apache.air= avata.registry.cpi.RegistryException: org.apache.airavata.registry.cpi.Regi= stryException: <openjpa-2.2.0-r422266:1244990 fatal store error> org.= apache.openjpa.persistence.RollbackException: The transaction has been rolled back.=C2=A0 See the nested exceptions for = details on the errors that occurred.

FailedObject: org.apache.airavata.persistance.registry.jpa.model.J= obDetail@6f1d32ff

at org.apache.airavata.persistance.registry.jpa.impl.= RegistryImpl.add(RegistryImpl.java:181)

at org.apache.airavata.gfac.core.utils.GFacUtils.save= JobStatus(GFacUtils.java:232)

at org.apache.airavata.gfac.ssh.provider.impl.SSHProv= ider.execute(SSHProvider.java:215)

at org.apache.airavata.gfac.core.cpi.BetterGfacImpl.e= xecuteProvider(BetterGfacImpl.java:842)

at org.apache.airavata.gfac.core.cpi.BetterGfacImpl.i= nvokeProviderExecute(BetterGfacImpl.java:753)

at org.apache.airavata.gfac.core.cpi.BetterGfacImpl.l= aunch(BetterGfacImpl.java:703)

at org.apache.airavata.gfac.core.cpi.BetterGfacImpl.s= ubmitJob(BetterGfacImpl.java:492)

at org.apache.airavata.gfac.core.cpi.BetterGfacImpl.s= ubmitJob(BetterGfacImpl.java:183)

at org.apache.airavata.gfac.core.utils.InputHandlerWo= rker.run(InputHandlerWorker.java:47)

at java.util.concurrent.ThreadPoolExecutor.runWorker(= ThreadPoolExecutor.java:1142)

at java.util.concurrent.ThreadPoolExecutor$Worker.run= (ThreadPoolExecutor.java:617)

at java.lang.Thread.run(Thread.java:745)

Caused by: org.apache.airavata.registry.cpi.RegistryException: org= .apache.airavata.registry.cpi.RegistryException: <openjpa-2.2.0-r422266:= 1244990 fatal store error> org.apache.openjpa.persistence.RollbackExcept= ion: The transaction has been rolled back.=C2=A0 See the nested exceptions for details on the errors that occurred.<= /p>

FailedObject: org.apache.airavata.persistance.registry.jpa.model.J= obDetail@6f1d32ff

at org.apache.airavata.persistance.registry.jpa.impl.= ExperimentRegistry.addJobDetails(ExperimentRegistry.java:1231)

at org.apache.airavata.persistance.registry.jpa.impl.= RegistryImpl.add(RegistryImpl.java:157)

... 11 more

Caused by: org.apache.airavata.registry.cpi.RegistryException: <= ;openjpa-2.2.0-r422266:1244990 fatal store error> org.apache.openjpa.per= sistence.RollbackException: The transaction has been rolled back.=C2=A0 See= the nested exceptions for details on the errors that occurred.

FailedObject: org.apache.airavata.persistance.registry.jpa.model.J= obDetail@6f1d32ff

at org.apache.airavata.persistance.registry.jpa.resou= rces.JobDetailResource.save(JobDetailResource.java:312)

at org.apache.airavata.persistance.registry.jpa.impl.= ExperimentRegistry.addJobDetails(ExperimentRegistry.java:1203)

... 12 more

Caused by: <openjpa-2.2.0-r422266:1244990 fatal store error>= org.apache.openjpa.persistence.RollbackException: The transaction has been= rolled back.=C2=A0 See the nested exceptions for details on the errors tha= t occurred.

FailedObject: org.apache.airavata.persistance.registry.jpa.model.J= obDetail@6f1d32ff

at org.apache.openjpa.persistence.EntityManagerImpl.c= ommit(EntityManagerImpl.java:594)

at org.apache.airavata.persistance.registry.jpa.resou= rces.JobDetailResource.save(JobDetailResource.java:308)

... 13 more

Caused by: <openjpa-2.2.0-r422266:1244990 fatal store error>= org.apache.openjpa.persistence.EntityExistsException: The transaction has = been rolled back.=C2=A0 See the nested exceptions for details on the errors= that occurred.

FailedObject: org.apache.airavata.persistance.registry.jpa.model.J= obDetail@6f1d32ff

at org.apache.openjpa.kernel.BrokerImpl.newFlushExcep= tion(BrokerImpl.java:2337)

at org.apache.openjpa.kernel.BrokerImpl.flush(BrokerI= mpl.java:2174)

at org.apache.openjpa.kernel.BrokerImpl.flushSafe(Bro= kerImpl.java:2072)

at org.apache.openjpa.kernel.BrokerImpl.beforeComplet= ion(BrokerImpl.java:1990)

at org.apache.openjpa.kernel.LocalManagedRuntime.comm= it(LocalManagedRuntime.java:81)

at org.apache.openjpa.kernel.BrokerImpl.commit(Broker= Impl.java:1514)

at org.apache.openjpa.kernel.DelegatingBroker.commit(= DelegatingBroker.java:933)

at org.apache.openjpa.persistence.EntityManagerImpl.c= ommit(EntityManagerImpl.java:570)

... 14 more

Caused by: <openjpa-2.2.0-r422266:1244990 fatal store error>= org.apache.openjpa.persistence.EntityExistsException: Column 'JOB_DESC= RIPTION' cannot be null {prepstmnt 1744538295=C2=A0

INSERT INTO JOB_DETAIL (JOB_ID, TASK_ID, COMPUTE_RESOURCE_CONSUMED= ,=C2=A0

=C2=A0 =C2=A0 =C2=A0 =C2=A0 CREATION_TIME, JOB_DESCRIPTION, JOBNAM= E, WORKING_DIR)=C2=A0

=C2=A0 =C2=A0 VALUES (?, ?, ?, ?, ?, ?, ?)=C2=A0

[params=3D(String) none, (String) tempNode_56f41083-10be-4161-8803= -ee1106bd3f54, (null) null, (Timestamp) 2015-08-17 06:49:01.613, (null) nul= l, (String) A516147113, (null) null]} [code=3D1048, state=3D23000]

FailedObject: org.apache.airavata.persistance.registry.jpa.model.J= obDetail@6f1d32ff

at org.apache.openjpa.jdbc.sql.DBDictionary.narrow(DB= Dictionary.java:4903)

at org.apache.openjpa.jdbc.sql.DBDictionary.newStoreE= xception(DBDictionary.java:4878)

at org.apache.openjpa.jdbc.sql.SQLExceptions.getStore= (SQLExceptions.java:136)

at org.apache.openjpa.jdbc.sql.SQLExceptions.getStore= (SQLExceptions.java:78)

at org.apache.openjpa.jdbc.kernel.PreparedStatementMa= nagerImpl.flushAndUpdate(PreparedStatementManagerImpl.java:143)

at org.apache.openjpa.jdbc.kernel.BatchingPreparedSta= tementManagerImpl.flushAndUpdate(BatchingPreparedStatementManagerImpl.java:= 79)

at org.apache.openjpa.jdbc.kernel.PreparedStatementMa= nagerImpl.flushInternal(PreparedStatementManagerImpl.java:99)

at org.apache.openjpa.jdbc.kernel.PreparedStatementMa= nagerImpl.flush(PreparedStatementManagerImpl.java:87)

at org.apache.openjpa.jdbc.kernel.ConstraintUpdateMan= ager.flush(ConstraintUpdateManager.java:550)

at org.apache.openjpa.jdbc.kernel.ConstraintUpdateMan= ager.flush(ConstraintUpdateManager.java:106)

at org.apache.openjpa.jdbc.kernel.BatchingConstraintU= pdateManager.flush(BatchingConstraintUpdateManager.java:59)

at org.apache.openjpa.jdbc.kernel.AbstractUpdateManag= er.flush(AbstractUpdateManager.java:103)

at org.apache.openjpa.jdbc.kernel.AbstractUpdateManag= er.flush(AbstractUpdateManager.java:76)

at org.apache.openjpa.jdbc.kernel.JDBCStoreManager.fl= ush(JDBCStoreManager.java:713)

at org.apache.openjpa.kernel.DelegatingStoreManager.f= lush(DelegatingStoreManager.java:131)

... 21 more

Caused by: org.apache.openjpa.lib.jdbc.ReportingSQLException: Colu= mn 'JOB_DESCRIPTION' cannot be null {prepstmnt 1744538295=C2=A0

INSERT INTO JOB_DETAIL (JOB_ID, TASK_ID, COMPUTE_RESOURCE_CONSUMED= ,=C2=A0

=C2=A0 =C2=A0 =C2=A0 =C2=A0 CREATION_TIME, JOB_DESCRIPTION, JOBNAM= E, WORKING_DIR)=C2=A0

=C2=A0 =C2=A0 VALUES (?, ?, ?, ?, ?, ?, ?)=C2=A0

[params=3D(String) none, (String) tempNode_56f41083-10be-4161-8803= -ee1106bd3f54, (null) null, (Timestamp) 2015-08-17 06:49:01.613, (null) nul= l, (String) A516147113, (null) null]} [code=3D1048, state=3D23000]

at org.apache.openjpa.lib.jdbc.LoggingConnectionDecor= ator.wrap(LoggingConnectionDecorator.java:247)

at org.apache.openjpa.lib.jdbc.LoggingConnectionDecor= ator.wrap(LoggingConnectionDecorator.java:223)

at org.apache.openjpa.lib.jdbc.LoggingConnectionDecor= ator.access$1000(LoggingConnectionDecorator.java:72)

at org.apache.openjpa.lib.jdbc.LoggingConnectionDecor= ator$LoggingConnection$LoggingPreparedStatement.executeUpdate(LoggingConnec= tionDecorator.java:1162)

at org.apache.openjpa.lib.jdbc.DelegatingPreparedStat= ement.executeUpdate(DelegatingPreparedStatement.java:272)

at org.apache.openjpa.jdbc.kernel.JDBCStoreManager$Ca= ncelPreparedStatement.executeUpdate(JDBCStoreManager.java:1741)

at org.apache.openjpa.jdbc.kernel.PreparedStatementMa= nagerImpl.executeUpdate(PreparedStatementManagerImpl.java:267)

at org.apache.openjpa.jdbc.kernel.PreparedStatementMa= nagerImpl.flushAndUpdate(PreparedStatementManagerImpl.java:118)

... 31 more

2015-08-17 06:49:01,690 [pool-8-thread-56] ERROR org.apache.airava= ta.gfac.GFacException=C2=A0 - Error persisting job statusError while adding= JOB_DETAIL

org.apache.airavata.registry.cpi.RegistryException: Error while ad= ding JOB_DETAIL

at org.apache.airavata.persistance.registry.jpa.impl.= RegistryImpl.add(RegistryImpl.java:182)

at org.apache.airavata.gfac.core.utils.GFacUtils.save= JobStatus(GFacUtils.java:232)

at org.apache.airavata.gfac.ssh.provider.impl.SSHProv= ider.execute(SSHProvider.java:215)

at org.apache.airavata.gfac.core.cpi.BetterGfacImpl.e= xecuteProvider(BetterGfacImpl.java:842)

at org.apache.airavata.gfac.core.cpi.BetterGfacImpl.i= nvokeProviderExecute(BetterGfacImpl.java:753)

at org.apache.airavata.gfac.core.cpi.BetterGfacImpl.l= aunch(BetterGfacImpl.java:703)

at org.apache.airavata.gfac.core.cpi.BetterGfacImpl.s= ubmitJob(BetterGfacImpl.java:492)

at org.apache.airavata.gfac.core.cpi.BetterGfacImpl.s= ubmitJob(BetterGfacImpl.java:183)

at org.apache.airavata.gfac.core.utils.InputHandlerWo= rker.run(InputHandlerWorker.java:47)

at java.util.concurrent.ThreadPoolExecutor.runWorker(= ThreadPoolExecutor.java:1142)

at java.util.concurrent.ThreadPoolExecutor$Worker.run= (ThreadPoolExecutor.java:617)

at java.lang.Thread.run(Thread.java:745)

Caused by: org.apache.airavata.registry.cpi.RegistryException: org= .apache.airavata.registry.cpi.RegistryException: <openjpa-2.2.0-r422266:= 1244990 fatal store error> org.apache.openjpa.persistence.RollbackExcept= ion: The transaction has been rolled back.=C2=A0 See the nested exceptions for details on the errors that occurred.<= /p>

FailedObject: org.apache.airavata.persistance.registry.jpa.model.J= obDetail@6f1d32ff

at org.apache.airavata.persistance.registry.jpa.impl.= ExperimentRegistry.addJobDetails(ExperimentRegistry.java:1231)

at org.apache.airavata.persistance.registry.jpa.impl.= RegistryImpl.add(RegistryImpl.java:157)

... 11 more

Caused by: org.apache.airavata.registry.cpi.RegistryException: <= ;openjpa-2.2.0-r422266:1244990 fatal store error> org.apache.openjpa.per= sistence.RollbackException: The transaction has been rolled back.=C2=A0 See= the nested exceptions for details on the errors that occurred.

FailedObject: org.apache.airavata.persistance.registry.jpa.model.J= obDetail@6f1d32ff

at org.apache.airavata.persistance.registry.jpa.resou= rces.JobDetailResource.save(JobDetailResource.java:312)

at org.apache.airavata.persistance.registry.jpa.impl.= ExperimentRegistry.addJobDetails(ExperimentRegistry.java:1203)

... 12 more

Caused by: <openjpa-2.2.0-r422266:1244990 fatal store error>= org.apache.openjpa.persistence.RollbackException: The transaction has been= rolled back.=C2=A0 See the nested exceptions for details on the errors tha= t occurred.

FailedObject: org.apache.airavata.persistance.registry.jpa.model.J= obDetail@6f1d32ff

at org.apache.openjpa.persistence.EntityManagerImpl.c= ommit(EntityManagerImpl.java:594)

at org.apache.airavata.persistance.registry.jpa.resou= rces.JobDetailResource.save(JobDetailResource.java:308)

... 13 more

Caused by: <openjpa-2.2.0-r422266:1244990 fatal store error>= org.apache.openjpa.persistence.EntityExistsException: The transaction has = been rolled back.=C2=A0 See the nested exceptions for details on the errors= that occurred.

FailedObject: org.apache.airavata.persistance.registry.jpa.model.J= obDetail@6f1d32ff

at org.apache.openjpa.kernel.BrokerImpl.newFlushExcep= tion(BrokerImpl.java:2337)

at org.apache.openjpa.kernel.BrokerImpl.flush(BrokerI= mpl.java:2174)

at org.apache.openjpa.kernel.BrokerImpl.flushSafe(Bro= kerImpl.java:2072)

at org.apache.openjpa.kernel.BrokerImpl.beforeComplet= ion(BrokerImpl.java:1990)

at org.apache.openjpa.kernel.LocalManagedRuntime.comm= it(LocalManagedRuntime.java:81)

at org.apache.openjpa.kernel.BrokerImpl.commit(Broker= Impl.java:1514)

at org.apache.openjpa.kernel.DelegatingBroker.commit(= DelegatingBroker.java:933)

at org.apache.openjpa.persistence.EntityManagerImpl.c= ommit(EntityManagerImpl.java:570)

... 14 more

Caused by: <openjpa-2.2.0-r422266:1244990 fatal store error>= org.apache.openjpa.persistence.EntityExistsException: Column 'JOB_DESC= RIPTION' cannot be null {prepstmnt 1744538295=C2=A0

INSERT INTO JOB_DETAIL (JOB_ID, TASK_ID, COMPUTE_RESOURCE_CONSUMED= ,=C2=A0

=C2=A0 =C2=A0 =C2=A0 =C2=A0 CREATION_TIME, JOB_DESCRIPTION, JOBNAM= E, WORKING_DIR)=C2=A0

=C2=A0 =C2=A0 VALUES (?, ?, ?, ?, ?, ?, ?)=C2=A0

[params=3D(String) none, (String) tempNode_56f41083-10be-4161-8803= -ee1106bd3f54, (null) null, (Timestamp) 2015-08-17 06:49:01.613, (null) nul= l, (String) A516147113, (null) null]} [code=3D1048, state=3D23000]

FailedObject: org.apache.airavata.persistance.registry.jpa.model.J= obDetail@6f1d32ff

at org.apache.openjpa.jdbc.sql.DBDictionary.narrow(DB= Dictionary.java:4903)

at org.apache.openjpa.jdbc.sql.DBDictionary.newStoreE= xception(DBDictionary.java:4878)

at org.apache.openjpa.jdbc.sql.SQLExceptions.getStore= (SQLExceptions.java:136)

at org.apache.openjpa.jdbc.sql.SQLExceptions.getStore= (SQLExceptions.java:78)

at org.apache.openjpa.jdbc.kernel.PreparedStatementMa= nagerImpl.flushAndUpdate(PreparedStatementManagerImpl.java:143)

at org.apache.openjpa.jdbc.kernel.BatchingPreparedSta= tementManagerImpl.flushAndUpdate(BatchingPreparedStatementManagerImpl.java:= 79)

at org.apache.openjpa.jdbc.kernel.PreparedStatementMa= nagerImpl.flushInternal(PreparedStatementManagerImpl.java:99)

at org.apache.openjpa.jdbc.kernel.PreparedStatementMa= nagerImpl.flush(PreparedStatementManagerImpl.java:87)

at org.apache.openjpa.jdbc.kernel.ConstraintUpdateMan= ager.flush(ConstraintUpdateManager.java:550)

at org.apache.openjpa.jdbc.kernel.ConstraintUpdateMan= ager.flush(ConstraintUpdateManager.java:106)

at org.apache.openjpa.jdbc.kernel.BatchingConstraintU= pdateManager.flush(BatchingConstraintUpdateManager.java:59)

at org.apache.openjpa.jdbc.kernel.AbstractUpdateManag= er.flush(AbstractUpdateManager.java:103)

at org.apache.openjpa.jdbc.kernel.AbstractUpdateManag= er.flush(AbstractUpdateManager.java:76)

at org.apache.openjpa.jdbc.kernel.JDBCStoreManager.fl= ush(JDBCStoreManager.java:713)

at org.apache.openjpa.kernel.DelegatingStoreManager.f= lush(DelegatingStoreManager.java:131)

... 21 more

Caused by: org.apache.openjpa.lib.jdbc.ReportingSQLException: Colu= mn 'JOB_DESCRIPTION' cannot be null {prepstmnt 1744538295=C2=A0

INSERT INTO JOB_DETAIL (JOB_ID, TASK_ID, COMPUTE_RESOURCE_CONSUMED= ,=C2=A0

=C2=A0 =C2=A0 =C2=A0 =C2=A0 CREATION_TIME, JOB_DESCRIPTION, JOBNAM= E, WORKING_DIR)=C2=A0

=C2=A0 =C2=A0 VALUES (?, ?, ?, ?, ?, ?, ?)=C2=A0

[params=3D(String) none, (String) tempNode_56f41083-10be-4161-8803= -ee1106bd3f54, (null) null, (Timestamp) 2015-08-17 06:49:01.613, (null) nul= l, (String) A516147113, (null) null]} [code=3D1048, state=3D23000]

at org.apache.openjpa.lib.jdbc.LoggingConnectionDecor= ator.wrap(LoggingConnectionDecorator.java:247)

at org.apache.openjpa.lib.jdbc.LoggingConnectionDecor= ator.wrap(LoggingConnectionDecorator.java:223)

at org.apache.openjpa.lib.jdbc.LoggingConnectionDecor= ator.access$1000(LoggingConnectionDecorator.java:72)

at org.apache.openjpa.lib.jdbc.LoggingConnectionDecor= ator$LoggingConnection$LoggingPreparedStatement.executeUpdate(LoggingConnec= tionDecorator.java:1162)

at org.apache.openjpa.lib.jdbc.DelegatingPreparedStat= ement.executeUpdate(DelegatingPreparedStatement.java:272)

at org.apache.openjpa.jdbc.kernel.JDBCStoreManager$Ca= ncelPreparedStatement.executeUpdate(JDBCStoreManager.java:1741)

at org.apache.openjpa.jdbc.kernel.PreparedStatementMa= nagerImpl.executeUpdate(PreparedStatementManagerImpl.java:267)

at org.apache.openjpa.jdbc.kernel.PreparedStatementMa= nagerImpl.flushAndUpdate(PreparedStatementManagerImpl.java:118)

... 31 more

2015-08-17 06:49:01,691 [pool-8-thread-56] INFO=C2=A0 org.apache.a= iravata.gfac.ssh.provider.impl.SSHProvider=C2=A0 - Saving data for future r= ecovery:=C2=A0

2015-08-17 06:49:01,691 [pool-8-thread-56] INFO=C2=A0 org.apache.a= iravata.gfac.ssh.provider.impl.SSHProvider=C2=A0 -=C2=A0

2015-08-17 06:49:01,721 [pool-8-thread-56] ERROR org.apache.airava= ta.gfac.core.provider.GFacProviderException=C2=A0 - Error persisting job st= atusError while adding JOB_DETAIL

org.apache.airavata.gfac.GFacException: Error persisting job statu= sError while adding JOB_DETAIL

at org.apache.airavata.gfac.core.utils.GFacUtils.save= JobStatus(GFacUtils.java:242)

at org.apache.airavata.gfac.ssh.provider.impl.SSHProv= ider.execute(SSHProvider.java:215)

at org.apache.airavata.gfac.core.cpi.BetterGfacImpl.e= xecuteProvider(BetterGfacImpl.java:842)

at org.apache.airavata.gfac.core.cpi.BetterGfacImpl.i= nvokeProviderExecute(BetterGfacImpl.java:753)

at org.apache.airavata.gfac.core.cpi.BetterGfacImpl.l= aunch(BetterGfacImpl.java:703)

at org.apache.airavata.gfac.core.cpi.BetterGfacImpl.s= ubmitJob(BetterGfacImpl.java:492)

at org.apache.airavata.gfac.core.cpi.BetterGfacImpl.s= ubmitJob(BetterGfacImpl.java:183)

at org.apache.airavata.gfac.core.utils.InputHandlerWo= rker.run(InputHandlerWorker.java:47)

at java.util.concurrent.ThreadPoolExecutor.runWorker(= ThreadPoolExecutor.java:1142)

at java.util.concurrent.ThreadPoolExecutor$Worker.run= (ThreadPoolExecutor.java:617)

at java.lang.Thread.run(Thread.java:745)

Caused by: org.apache.airavata.registry.cpi.RegistryException: Err= or while adding JOB_DETAIL

at org.apache.airavata.persistance.registry.jpa.impl.= RegistryImpl.add(RegistryImpl.java:182)

at org.apache.airavata.gfac.core.utils.GFacUtils.save= JobStatus(GFacUtils.java:232)

... 10 more

Caused by: org.apache.airavata.registry.cpi.RegistryException: org= .apache.airavata.registry.cpi.RegistryException: <openjpa-2.2.0-r422266:= 1244990 fatal store error> org.apache.openjpa.persistence.RollbackExcept= ion: The transaction has been rolled back.=C2=A0 See the nested exceptions for details on the errors that occurred.<= /p>

FailedObject: org.apache.airavata.persistance.registry.jpa.model.J= obDetail@6f1d32ff

at org.apache.airavata.persistance.registry.jpa.impl.= ExperimentRegistry.addJobDetails(ExperimentRegistry.java:1231)

at org.apache.airavata.persistance.registry.jpa.impl.= RegistryImpl.add(RegistryImpl.java:157)

... 11 more

Caused by: org.apache.airavata.registry.cpi.RegistryException: <= ;openjpa-2.2.0-r422266:1244990 fatal store error> org.apache.openjpa.per= sistence.RollbackException: The transaction has been rolled back.=C2=A0 See= the nested exceptions for details on the errors that occurred.

FailedObject: org.apache.airavata.persistance.registry.jpa.model.J= obDetail@6f1d32ff

at org.apache.airavata.persistance.registry.jpa.resou= rces.JobDetailResource.save(JobDetailResource.java:312)

at org.apache.airavata.persistance.registry.jpa.impl.= ExperimentRegistry.addJobDetails(ExperimentRegistry.java:1203)

... 12 more

Caused by: <openjpa-2.2.0-r422266:1244990 fatal store error>= org.apache.openjpa.persistence.RollbackException: The transaction has been= rolled back.=C2=A0 See the nested exceptions for details on the errors tha= t occurred.

FailedObject: org.apache.airavata.persistance.registry.jpa.model.J= obDetail@6f1d32ff

at org.apache.openjpa.persistence.EntityManagerImpl.c= ommit(EntityManagerImpl.java:594)

at org.apache.airavata.persistance.registry.jpa.resou= rces.JobDetailResource.save(JobDetailResource.java:308)

... 13 more

Caused by: <openjpa-2.2.0-r422266:1244990 fatal store error>= org.apache.openjpa.persistence.EntityExistsException: The transaction has = been rolled back.=C2=A0 See the nested exceptions for details on the errors= that occurred.

FailedObject: org.apache.airavata.persistance.registry.jpa.model.J= obDetail@6f1d32ff

at org.apache.openjpa.kernel.BrokerImpl.newFlushExcep= tion(BrokerImpl.java:2337)

at org.apache.openjpa.kernel.BrokerImpl.flush(BrokerI= mpl.java:2174)

at org.apache.openjpa.kernel.BrokerImpl.flushSafe(Bro= kerImpl.java:2072)

at org.apache.openjpa.kernel.BrokerImpl.beforeComplet= ion(BrokerImpl.java:1990)

at org.apache.openjpa.kernel.LocalManagedRuntime.comm= it(LocalManagedRuntime.java:81)

at org.apache.openjpa.kernel.BrokerImpl.commit(Broker= Impl.java:1514)

at org.apache.openjpa.kernel.DelegatingBroker.commit(= DelegatingBroker.java:933)

at org.apache.openjpa.persistence.EntityManagerImpl.c= ommit(EntityManagerImpl.java:570)

... 14 more

Caused by: <openjpa-2.2.0-r422266:1244990 fatal store error>= org.apache.openjpa.persistence.EntityExistsException: Column 'JOB_DESC= RIPTION' cannot be null {prepstmnt 1744538295=C2=A0

INSERT INTO JOB_DETAIL (JOB_ID, TASK_ID, COMPUTE_RESOURCE_CONSUMED= ,=C2=A0

=C2=A0 =C2=A0 =C2=A0 =C2=A0 CREATION_TIME, JOB_DESCRIPTION, JOBNAM= E, WORKING_DIR)=C2=A0

=C2=A0 =C2=A0 VALUES (?, ?, ?, ?, ?, ?, ?)=C2=A0

[params=3D(String) none, (String) tempNode_56f41083-10be-4161-8803= -ee1106bd3f54, (null) null, (Timestamp) 2015-08-17 06:49:01.613, (null) nul= l, (String) A516147113, (null) null]} [code=3D1048, state=3D23000]

FailedObject: org.apache.airavata.persistance.registry.jpa.model.J= obDetail@6f1d32ff

at org.apache.openjpa.jdbc.sql.DBDictionary.narrow(DB= Dictionary.java:4903)

at org.apache.openjpa.jdbc.sql.DBDictionary.newStoreE= xception(DBDictionary.java:4878)

at org.apache.openjpa.jdbc.sql.SQLExceptions.getStore= (SQLExceptions.java:136)

at org.apache.openjpa.jdbc.sql.SQLExceptions.getStore= (SQLExceptions.java:78)

at org.apache.openjpa.jdbc.kernel.PreparedStatementMa= nagerImpl.flushAndUpdate(PreparedStatementManagerImpl.java:143)

at org.apache.openjpa.jdbc.kernel.BatchingPreparedSta= tementManagerImpl.flushAndUpdate(BatchingPreparedStatementManagerImpl.java:= 79)

at org.apache.openjpa.jdbc.kernel.PreparedStatementMa= nagerImpl.flushInternal(PreparedStatementManagerImpl.java:99)

at org.apache.openjpa.jdbc.kernel.PreparedStatementMa= nagerImpl.flush(PreparedStatementManagerImpl.java:87)

at org.apache.openjpa.jdbc.kernel.ConstraintUpdateMan= ager.flush(ConstraintUpdateManager.java:550)

at org.apache.openjpa.jdbc.kernel.ConstraintUpdateMan= ager.flush(ConstraintUpdateManager.java:106)

at org.apache.openjpa.jdbc.kernel.BatchingConstraintU= pdateManager.flush(BatchingConstraintUpdateManager.java:59)

at org.apache.openjpa.jdbc.kernel.AbstractUpdateManag= er.flush(AbstractUpdateManager.java:103)

at org.apache.openjpa.jdbc.kernel.AbstractUpdateManag= er.flush(AbstractUpdateManager.java:76)

at org.apache.openjpa.jdbc.kernel.JDBCStoreManager.fl= ush(JDBCStoreManager.java:713)

at org.apache.openjpa.kernel.DelegatingStoreManager.f= lush(DelegatingStoreManager.java:131)

... 21 more

Caused by: org.apache.openjpa.lib.jdbc.ReportingSQLException: Colu= mn 'JOB_DESCRIPTION' cannot be null {prepstmnt 1744538295=C2=A0

INSERT INTO JOB_DETAIL (JOB_ID, TASK_ID, COMPUTE_RESOURCE_CONSUMED= ,=C2=A0

=C2=A0 =C2=A0 =C2=A0 =C2=A0 CREATION_TIME, JOB_DESCRIPTION, JOBNAM= E, WORKING_DIR)=C2=A0

=C2=A0 =C2=A0 VALUES (?, ?, ?, ?, ?, ?, ?)=C2=A0

[params=3D(String) none, (String) tempNode_56f41083-10be-4161-8803= -ee1106bd3f54, (null) null, (Timestamp) 2015-08-17 06:49:01.613, (null) nul= l, (String) A516147113, (null) null]} [code=3D1048, state=3D23000]

at org.apache.openjpa.lib.jdbc.LoggingConnectionDecor= ator.wrap(LoggingConnectionDecorator.java:247)

at org.apache.openjpa.lib.jdbc.LoggingConnectionDecor= ator.wrap(LoggingConnectionDecorator.java:223)

at org.apache.openjpa.lib.jdbc.LoggingConnectionDecor= ator.access$1000(LoggingConnectionDecorator.java:72)

at org.apache.openjpa.lib.jdbc.LoggingConnectionDecor= ator$LoggingConnection$LoggingPreparedStatement.executeUpdate(LoggingConnec= tionDecorator.java:1162)

at org.apache.openjpa.lib.jdbc.DelegatingPreparedStat= ement.executeUpdate(DelegatingPreparedStatement.java:272)

at org.apache.openjpa.jdbc.kernel.JDBCStoreManager$Ca= ncelPreparedStatement.executeUpdate(JDBCStoreManager.java:1741)

at org.apache.openjpa.jdbc.kernel.PreparedStatementMa= nagerImpl.executeUpdate(PreparedStatementManagerImpl.java:267)

at org.apache.openjpa.jdbc.kernel.PreparedStatementMa= nagerImpl.flushAndUpdate(PreparedStatementManagerImpl.java:118)

... 31 more

2015-08-17 06:49:01,723 [pool-8-thread-56] ERROR org.apache.airava= ta.gfac.GFacException=C2=A0 - Error while executing provider org.apache.air= avata.gfac.ssh.provider.impl.SSHProvider functionality.

org.apache.airavata.gfac.core.provider.GFacProviderException: Erro= r persisting job statusError while adding JOB_DETAIL

at org.apache.airavata.gfac.ssh.provider.impl.SSHProv= ider.execute(SSHProvider.java:231)

at org.apache.airavata.gfac.core.cpi.BetterGfacImpl.e= xecuteProvider(BetterGfacImpl.java:842)

at org.apache.airavata.gfac.core.cpi.BetterGfacImpl.i= nvokeProviderExecute(BetterGfacImpl.java:753)

at org.apache.airavata.gfac.core.cpi.BetterGfacImpl.l= aunch(BetterGfacImpl.java:703)

at org.apache.airavata.gfac.core.cpi.BetterGfacImpl.s= ubmitJob(BetterGfacImpl.java:492)

at org.apache.airavata.gfac.core.cpi.BetterGfacImpl.s= ubmitJob(BetterGfacImpl.java:183)

at org.apache.airavata.gfac.core.utils.InputHandlerWo= rker.run(InputHandlerWorker.java:47)

at java.util.concurrent.ThreadPoolExecutor.runWorker(= ThreadPoolExecutor.java:1142)

at java.util.concurrent.ThreadPoolExecutor$Worker.run= (ThreadPoolExecutor.java:617)

at java.lang.Thread.run(Thread.java:745)

Caused by: org.apache.airavata.gfac.GFacException: Error persistin= g job statusError while adding JOB_DETAIL

at org.apache.airavata.gfac.core.utils.GFacUtils.save= JobStatus(GFacUtils.java:242)

at org.apache.airavata.gfac.ssh.provider.impl.SSHProv= ider.execute(SSHProvider.java:215)

... 9 more

Caused by: org.apache.airavata.registry.cpi.RegistryException: Err= or while adding JOB_DETAIL

at org.apache.airavata.persistance.registry.jpa.impl.= RegistryImpl.add(RegistryImpl.java:182)

at org.apache.airavata.gfac.core.utils.GFacUtils.save= JobStatus(GFacUtils.java:232)

... 10 more

Caused by: org.apache.airavata.registry.cpi.RegistryException: org= .apache.airavata.registry.cpi.RegistryException: <openjpa-2.2.0-r422266:= 1244990 fatal store error> org.apache.openjpa.persistence.RollbackExcept= ion: The transaction has been rolled back.=C2=A0 See the nested exceptions for details on the errors that occurred.<= /p>

FailedObject: org.apache.airavata.persistance.registry.jpa.model.J= obDetail@6f1d32ff

at org.apache.airavata.persistance.registry.jpa.impl.= ExperimentRegistry.addJobDetails(ExperimentRegistry.java:1231)

at org.apache.airavata.persistance.registry.jpa.impl.= RegistryImpl.add(RegistryImpl.java:157)

... 11 more

Caused by: org.apache.airavata.registry.cpi.RegistryException: <= ;openjpa-2.2.0-r422266:1244990 fatal store error> org.apache.openjpa.per= sistence.RollbackException: The transaction has been rolled back.=C2=A0 See= the nested exceptions for details on the errors that occurred.

FailedObject: org.apache.airavata.persistance.registry.jpa.model.J= obDetail@6f1d32ff

at org.apache.airavata.persistance.registry.jpa.resou= rces.JobDetailResource.save(JobDetailResource.java:312)

at org.apache.airavata.persistance.registry.jpa.impl.= ExperimentRegistry.addJobDetails(ExperimentRegistry.java:1203)

... 12 more

Caused by: <openjpa-2.2.0-r422266:1244990 fatal store error>= org.apache.openjpa.persistence.RollbackException: The transaction has been= rolled back.=C2=A0 See the nested exceptions for details on the errors tha= t occurred.

FailedObject: org.apache.airavata.persistance.registry.jpa.model.J= obDetail@6f1d32ff

at org.apache.openjpa.persistence.EntityManagerImpl.c= ommit(EntityManagerImpl.java:594)

at org.apache.airavata.persistance.registry.jpa.resou= rces.JobDetailResource.save(JobDetailResource.java:308)

... 13 more

Caused by: <openjpa-2.2.0-r422266:1244990 fatal store error>= org.apache.openjpa.persistence.EntityExistsException: The transaction has = been rolled back.=C2=A0 See the nested exceptions for details on the errors= that occurred.

FailedObject: org.apache.airavata.persistance.registry.jpa.model.J= obDetail@6f1d32ff

at org.apache.openjpa.kernel.BrokerImpl.newFlushExcep= tion(BrokerImpl.java:2337)

at org.apache.openjpa.kernel.BrokerImpl.flush(BrokerI= mpl.java:2174)

at org.apache.openjpa.kernel.BrokerImpl.flushSafe(Bro= kerImpl.java:2072)

at org.apache.openjpa.kernel.BrokerImpl.beforeComplet= ion(BrokerImpl.java:1990)

at org.apache.openjpa.kernel.LocalManagedRuntime.comm= it(LocalManagedRuntime.java:81)

at org.apache.openjpa.kernel.BrokerImpl.commit(Broker= Impl.java:1514)

at org.apache.openjpa.kernel.DelegatingBroker.commit(= DelegatingBroker.java:933)

at org.apache.openjpa.persistence.EntityManagerImpl.c= ommit(EntityManagerImpl.java:570)

... 14 more

Caused by: <openjpa-2.2.0-r422266:1244990 fatal store error>= org.apache.openjpa.persistence.EntityExistsException: Column 'JOB_DESC= RIPTION' cannot be null {prepstmnt 1744538295=C2=A0

INSERT INTO JOB_DETAIL (JOB_ID, TASK_ID, COMPUTE_RESOURCE_CONSUMED= ,=C2=A0

=C2=A0 =C2=A0 =C2=A0 =C2=A0 CREATION_TIME, JOB_DESCRIPTION, JOBNAM= E, WORKING_DIR)=C2=A0

=C2=A0 =C2=A0 VALUES (?, ?, ?, ?, ?, ?, ?)=C2=A0

[params=3D(String) none, (String) tempNode_56f41083-10be-4161-8803= -ee1106bd3f54, (null) null, (Timestamp) 2015-08-17 06:49:01.613, (null) nul= l, (String) A516147113, (null) null]} [code=3D1048, state=3D23000]

FailedObject: org.apache.airavata.persistance.registry.jpa.model.J= obDetail@6f1d32ff

at org.apache.openjpa.jdbc.sql.DBDictionary.narrow(DB= Dictionary.java:4903)

at org.apache.openjpa.jdbc.sql.DBDictionary.newStoreE= xception(DBDictionary.java:4878)

at org.apache.openjpa.jdbc.sql.SQLExceptions.getStore= (SQLExceptions.java:136)

at org.apache.openjpa.jdbc.sql.SQLExceptions.getStore= (SQLExceptions.java:78)

at org.apache.openjpa.jdbc.kernel.PreparedStatementMa= nagerImpl.flushAndUpdate(PreparedStatementManagerImpl.java:143)

at org.apache.openjpa.jdbc.kernel.BatchingPreparedSta= tementManagerImpl.flushAndUpdate(BatchingPreparedStatementManagerImpl.java:= 79)

at org.apache.openjpa.jdbc.kernel.PreparedStatementMa= nagerImpl.flushInternal(PreparedStatementManagerImpl.java:99)

at org.apache.openjpa.jdbc.kernel.PreparedStatementMa= nagerImpl.flush(PreparedStatementManagerImpl.java:87)

at org.apache.openjpa.jdbc.kernel.ConstraintUpdateMan= ager.flush(ConstraintUpdateManager.java:550)

at org.apache.openjpa.jdbc.kernel.ConstraintUpdateMan= ager.flush(ConstraintUpdateManager.java:106)

at org.apache.openjpa.jdbc.kernel.BatchingConstraintU= pdateManager.flush(BatchingConstraintUpdateManager.java:59)

at org.apache.openjpa.jdbc.kernel.AbstractUpdateManag= er.flush(AbstractUpdateManager.java:103)

at org.apache.openjpa.jdbc.kernel.AbstractUpdateManag= er.flush(AbstractUpdateManager.java:76)

at org.apache.openjpa.jdbc.kernel.JDBCStoreManager.fl= ush(JDBCStoreManager.java:713)

at org.apache.openjpa.kernel.DelegatingStoreManager.f= lush(DelegatingStoreManager.java:131)

... 21 more

Caused by: org.apache.openjpa.lib.jdbc.ReportingSQLException: Colu= mn 'JOB_DESCRIPTION' cannot be null {prepstmnt 1744538295=C2=A0

INSERT INTO JOB_DETAIL (JOB_ID, TASK_ID, COMPUTE_RESOURCE_CONSUMED= ,=C2=A0

=C2=A0 =C2=A0 =C2=A0 =C2=A0 CREATION_TIME, JOB_DESCRIPTION, JOBNAM= E, WORKING_DIR)=C2=A0

=C2=A0 =C2=A0 VALUES (?, ?, ?, ?, ?, ?, ?)=C2=A0

[params=3D(String) none, (String) tempNode_56f41083-10be-4161-8803= -ee1106bd3f54, (null) null, (Timestamp) 2015-08-17 06:49:01.613, (null) nul= l, (String) A516147113, (null) null]} [code=3D1048, state=3D23000]

at org.apache.openjpa.lib.jdbc.LoggingConnectionDecor= ator.wrap(LoggingConnectionDecorator.java:247)

at org.apache.openjpa.lib.jdbc.LoggingConnectionDecor= ator.wrap(LoggingConnectionDecorator.java:223)

at org.apache.openjpa.lib.jdbc.LoggingConnectionDecor= ator.access$1000(LoggingConnectionDecorator.java:72)

at org.apache.openjpa.lib.jdbc.LoggingConnectionDecor= ator$LoggingConnection$LoggingPreparedStatement.executeUpdate(LoggingConnec= tionDecorator.java:1162)

at org.apache.openjpa.lib.jdbc.DelegatingPreparedStat= ement.executeUpdate(DelegatingPreparedStatement.java:272)

at org.apache.openjpa.jdbc.kernel.JDBCStoreManager$Ca= ncelPreparedStatement.executeUpdate(JDBCStoreManager.java:1741)

at org.apache.openjpa.jdbc.kernel.PreparedStatementMa= nagerImpl.executeUpdate(PreparedStatementManagerImpl.java:267)

at org.apache.openjpa.jdbc.kernel.PreparedStatementMa= nagerImpl.flushAndUpdate(PreparedStatementManagerImpl.java:118)

... 31 more

2015-08-17 06:49:01,724 [pool-8-thread-56] ERROR org.apache.airava= ta.gfac.core.monitor.MonitorID=C2=A0 - There is not job created at this poi= nt

2015-08-17 06:49:01,724 [pool-8-thread-56] INFO=C2=A0 org.apache.a= iravata.gfac.core.monitor.GfacInternalStatusUpdator=C2=A0 - Gfac internal s= tate changed to: FAILED

2015-08-17 06:49:01,724 [pool-8-thread-56] INFO=C2=A0 org.apache.a= iravata.gfac.core.monitor.GfacInternalStatusUpdator=C2=A0 - Experiment Fail= ed, So removing the ZK entry for the experimentdimuthu_experiment_gau_11_6e= 6ffc70-5dd2-4a9f-94b6-8cd6e672ec08

2015-08-17 06:49:01,724 [pool-8-thread-56] INFO=C2=A0 org.apache.a= iravata.gfac.core.monitor.GfacInternalStatusUpdator=C2=A0 - Zookeeper exper= iment Path: /gfac-experiments/gfac-node0/dimuthu_experiment_gau_11_6e6ffc70= -5dd2-4a9f-94b6-8cd6e672ec08

2015-08-17 06:49:01,725 [pool-8-thread-56] ERROR org.apache.airava= ta.gfac.core.cpi.BetterGfacImpl=C2=A0 - Error occured during updating the s= tatuses of Experiments,tasks or Job statuses to failed, NullPointerExceptio= n occurred because at this point there might not have Job Created

org.apache.airavata.gfac.GFacException: Error while executing prov= ider org.apache.airavata.gfac.ssh.provider.impl.SSHProvider functionality.<= /span>

at org.apache.airavata.gfac.core.cpi.BetterGfacImpl.e= xecuteProvider(BetterGfacImpl.java:844)

at org.apache.airavata.gfac.core.cpi.BetterGfacImpl.i= nvokeProviderExecute(BetterGfacImpl.java:753)

at org.apache.airavata.gfac.core.cpi.BetterGfacImpl.l= aunch(BetterGfacImpl.java:703)

at org.apache.airavata.gfac.core.cpi.BetterGfacImpl.s= ubmitJob(BetterGfacImpl.java:492)

at org.apache.airavata.gfac.core.cpi.BetterGfacImpl.s= ubmitJob(BetterGfacImpl.java:183)

at org.apache.airavata.gfac.core.utils.InputHandlerWo= rker.run(InputHandlerWorker.java:47)

at java.util.concurrent.ThreadPoolExecutor.runWorker(= ThreadPoolExecutor.java:1142)

at java.util.concurrent.ThreadPoolExecutor$Worker.run= (ThreadPoolExecutor.java:617)

at java.lang.Thread.run(Thread.java:745)

Caused by: org.apache.airavata.gfac.core.provider.GFacProviderExce= ption: Error persisting job statusError while adding JOB_DETAIL

at org.apache.airavata.gfac.ssh.provider.impl.SSHProv= ider.execute(SSHProvider.java:231)

at org.apache.airavata.gfac.core.cpi.BetterGfacImpl.e= xecuteProvider(BetterGfacImpl.java:842)

... 8 more

Caused by: org.apache.airavata.gfac.GFacException: Error persistin= g job statusError while adding JOB_DETAIL

at org.apache.airavata.gfac.core.utils.GFacUtils.save= JobStatus(GFacUtils.java:242)

at org.apache.airavata.gfac.ssh.provider.impl.SSHProv= ider.execute(SSHProvider.java:215)

... 9 more

Caused by: org.apache.airavata.registry.cpi.RegistryException: Err= or while adding JOB_DETAIL

at org.apache.airavata.persistance.registry.jpa.impl.= RegistryImpl.add(RegistryImpl.java:182)

at org.apache.airavata.gfac.core.utils.GFacUtils.save= JobStatus(GFacUtils.java:232)

... 10 more

Caused by: org.apache.airavata.registry.cpi.RegistryException: org= .apache.airavata.registry.cpi.RegistryException: <openjpa-2.2.0-r422266:= 1244990 fatal store error> org.apache.openjpa.persistence.RollbackExcept= ion: The transaction has been rolled back.=C2=A0 See the nested exceptions for details on the errors that occurred.<= /p>

FailedObject: org.apache.airavata.persistance.registry.jpa.model.J= obDetail@6f1d32ff

at org.apache.airavata.persistance.registry.jpa.impl.= ExperimentRegistry.addJobDetails(ExperimentRegistry.java:1231)

at org.apache.airavata.persistance.registry.jpa.impl.= RegistryImpl.add(RegistryImpl.java:157)

... 11 more

Caused by: org.apache.airavata.registry.cpi.RegistryException: <= ;openjpa-2.2.0-r422266:1244990 fatal store error> org.apache.openjpa.per= sistence.RollbackException: The transaction has been rolled back.=C2=A0 See= the nested exceptions for details on the errors that occurred.

FailedObject: org.apache.airavata.persistance.registry.jpa.model.J= obDetail@6f1d32ff

at org.apache.airavata.persistance.registry.jpa.resou= rces.JobDetailResource.save(JobDetailResource.java:312)

at org.apache.airavata.persistance.registry.jpa.impl.= ExperimentRegistry.addJobDetails(ExperimentRegistry.java:1203)

... 12 more

Caused by: <openjpa-2.2.0-r422266:1244990 fatal store error>= org.apache.openjpa.persistence.RollbackException: The transaction has been= rolled back.=C2=A0 See the nested exceptions for details on the errors tha= t occurred.

FailedObject: org.apache.airavata.persistance.registry.jpa.model.J= obDetail@6f1d32ff

at org.apache.openjpa.persistence.EntityManagerImpl.c= ommit(EntityManagerImpl.java:594)

at org.apache.airavata.persistance.registry.jpa.resou= rces.JobDetailResource.save(JobDetailResource.java:308)

... 13 more

Caused by: <openjpa-2.2.0-r422266:1244990 fatal store error>= org.apache.openjpa.persistence.EntityExistsException: The transaction has = been rolled back.=C2=A0 See the nested exceptions for details on the errors= that occurred.

FailedObject: org.apache.airavata.persistance.registry.jpa.model.J= obDetail@6f1d32ff

at org.apache.openjpa.kernel.BrokerImpl.newFlushExcep= tion(BrokerImpl.java:2337)

at org.apache.openjpa.kernel.BrokerImpl.flush(BrokerI= mpl.java:2174)

at org.apache.openjpa.kernel.BrokerImpl.flushSafe(Bro= kerImpl.java:2072)

at org.apache.openjpa.kernel.BrokerImpl.beforeComplet= ion(BrokerImpl.java:1990)

at org.apache.openjpa.kernel.LocalManagedRuntime.comm= it(LocalManagedRuntime.java:81)

at org.apache.openjpa.kernel.BrokerImpl.commit(Broker= Impl.java:1514)

at org.apache.openjpa.kernel.DelegatingBroker.commit(= DelegatingBroker.java:933)

at org.apache.openjpa.persistence.EntityManagerImpl.c= ommit(EntityManagerImpl.java:570)

... 14 more

Caused by: <openjpa-2.2.0-r422266:1244990 fatal store error>= org.apache.openjpa.persistence.EntityExistsException: Column 'JOB_DESC= RIPTION' cannot be null {prepstmnt 1744538295=C2=A0

INSERT INTO JOB_DETAIL (JOB_ID, TASK_ID, COMPUTE_RESOURCE_CONSUMED= ,=C2=A0

=C2=A0 =C2=A0 =C2=A0 =C2=A0 CREATION_TIME, JOB_DESCRIPTION, JOBNAM= E, WORKING_DIR)=C2=A0

=C2=A0 =C2=A0 VALUES (?, ?, ?, ?, ?, ?, ?)=C2=A0

[params=3D(String) none, (String) tempNode_56f41083-10be-4161-8803= -ee1106bd3f54, (null) null, (Timestamp) 2015-08-17 06:49:01.613, (null) nul= l, (String) A516147113, (null) null]} [code=3D1048, state=3D23000]

FailedObject: org.apache.airavata.persistance.registry.jpa.model.J= obDetail@6f1d32ff

at org.apache.openjpa.jdbc.sql.DBDictionary.narrow(DB= Dictionary.java:4903)

at org.apache.openjpa.jdbc.sql.DBDictionary.newStoreE= xception(DBDictionary.java:4878)

at org.apache.openjpa.jdbc.sql.SQLExceptions.getStore= (SQLExceptions.java:136)

at org.apache.openjpa.jdbc.sql.SQLExceptions.getStore= (SQLExceptions.java:78)

at org.apache.openjpa.jdbc.kernel.PreparedStatementMa= nagerImpl.flushAndUpdate(PreparedStatementManagerImpl.java:143)

at org.apache.openjpa.jdbc.kernel.BatchingPreparedSta= tementManagerImpl.flushAndUpdate(BatchingPreparedStatementManagerImpl.java:= 79)

at org.apache.openjpa.jdbc.kernel.PreparedStatementMa= nagerImpl.flushInternal(PreparedStatementManagerImpl.java:99)

at org.apache.openjpa.jdbc.kernel.PreparedStatementMa= nagerImpl.flush(PreparedStatementManagerImpl.java:87)

at org.apache.openjpa.jdbc.kernel.ConstraintUpdateMan= ager.flush(ConstraintUpdateManager.java:550)

at org.apache.openjpa.jdbc.kernel.ConstraintUpdateMan= ager.flush(ConstraintUpdateManager.java:106)

at org.apache.openjpa.jdbc.kernel.BatchingConstraintU= pdateManager.flush(BatchingConstraintUpdateManager.java:59)

at org.apache.openjpa.jdbc.kernel.AbstractUpdateManag= er.flush(AbstractUpdateManager.java:103)

at org.apache.openjpa.jdbc.kernel.AbstractUpdateManag= er.flush(AbstractUpdateManager.java:76)

at org.apache.openjpa.jdbc.kernel.JDBCStoreManager.fl= ush(JDBCStoreManager.java:713)

at org.apache.openjpa.kernel.DelegatingStoreManager.f= lush(DelegatingStoreManager.java:131)

... 21 more

Caused by: org.apache.openjpa.lib.jdbc.ReportingSQLException: Colu= mn 'JOB_DESCRIPTION' cannot be null {prepstmnt 1744538295=C2=A0

INSERT INTO JOB_DETAIL (JOB_ID, TASK_ID, COMPUTE_RESOURCE_CONSUMED= ,=C2=A0

=C2=A0 =C2=A0 =C2=A0 =C2=A0 CREATION_TIME, JOB_DESCRIPTION, JOBNAM= E, WORKING_DIR)=C2=A0

=C2=A0 =C2=A0 VALUES (?, ?, ?, ?, ?, ?, ?)=C2=A0

[params=3D(String) none, (String) tempNode_56f41083-10be-4161-8803= -ee1106bd3f54, (null) null, (Timestamp) 2015-08-17 06:49:01.613, (null) nul= l, (String) A516147113, (null) null]} [code=3D1048, state=3D23000]

at org.apache.openjpa.lib.jdbc.LoggingConnectionDecor= ator.wrap(LoggingConnectionDecorator.java:247)

at org.apache.openjpa.lib.jdbc.LoggingConnectionDecor= ator.wrap(LoggingConnectionDecorator.java:223)

at org.apache.openjpa.lib.jdbc.LoggingConnectionDecor= ator.access$1000(LoggingConnectionDecorator.java:72)

at org.apache.openjpa.lib.jdbc.LoggingConnectionDecor= ator$LoggingConnection$LoggingPreparedStatement.executeUpdate(LoggingConnec= tionDecorator.java:1162)

at org.apache.openjpa.lib.jdbc.DelegatingPreparedStat= ement.executeUpdate(DelegatingPreparedStatement.java:272)

at org.apache.openjpa.jdbc.kernel.JDBCStoreManager$Ca= ncelPreparedStatement.executeUpdate(JDBCStoreManager.java:1741)

at org.apache.openjpa.jdbc.kernel.PreparedStatementMa= nagerImpl.executeUpdate(PreparedStatementManagerImpl.java:267)

at org.apache.openjpa.jdbc.kernel.PreparedStatementMa= nagerImpl.flushAndUpdate(PreparedStatementManagerImpl.java:118)

... 31 more

2015-08-17 06:49:01,726 [pool-8-thread-56] INFO=C2=A0 org.apache.a= iravata.gfac.core.monitor.AiravataWorkflowNodeStatusUpdator=C2=A0 - expId -= dimuthu_experiment_gau_11_6e6ffc70-5dd2-4a9f-94b6-8cd6e672ec08: Updating w= orkflow node status for tempNode_20c62522-62c4-4342-81d8-9541dbf13862:FAILE= D

2015-08-17 06:49:02,129 [pool-8-thread-56] INFO=C2=A0 org.apache.a= iravata.messaging.core.impl.RabbitMQStatusPublisher=C2=A0 - Publishing stat= us to rabbitmq...

2015-08-17 06:49:03,300 [pool-8-thread-56] INFO=C2=A0 org.apache.a= iravata.api.server.listener.AiravataExperimentStatusUpdator=C2=A0 - Updatin= g the experiment status of experiment: dimuthu_experiment_gau_11_6e6ffc70-5= dd2-4a9f-94b6-8cd6e672ec08 to FAILED

2015-08-17 06:49:03,368 [pool-8-thread-56] INFO=C2=A0 org.apache.a= iravata.messaging.core.impl.RabbitMQStatusPublisher=C2=A0 - Publishing stat= us to rabbitmq...

2015-08-17 06:49:03,376 [pool-8-thread-56] INFO=C2=A0 org.apache.a= iravata.api.server.listener.AiravataExperimentStatusUpdator=C2=A0 - Clean u= p task - Acknowledging delivery Tag - 93

2015-08-17 06:49:03,611 [pool-8-thread-56] ERROR org.apache.airava= ta.gfac.GFacException=C2=A0 - Error while executing provider org.apache.air= avata.gfac.ssh.provider.impl.SSHProvider functionality.

org.apache.airavata.gfac.GFacException: Error while executing prov= ider org.apache.airavata.gfac.ssh.provider.impl.SSHProvider functionality.<= /span>

at org.apache.airavata.gfac.core.cpi.BetterGfacImpl.e= xecuteProvider(BetterGfacImpl.java:844)

at org.apache.airavata.gfac.core.cpi.BetterGfacImpl.i= nvokeProviderExecute(BetterGfacImpl.java:753)

at org.apache.airavata.gfac.core.cpi.BetterGfacImpl.l= aunch(BetterGfacImpl.java:703)

at org.apache.airavata.gfac.core.cpi.BetterGfacImpl.s= ubmitJob(BetterGfacImpl.java:492)

at org.apache.airavata.gfac.core.cpi.BetterGfacImpl.s= ubmitJob(BetterGfacImpl.java:183)

at org.apache.airavata.gfac.core.utils.InputHandlerWo= rker.run(InputHandlerWorker.java:47)

at java.util.concurrent.ThreadPoolExecutor.runWorker(= ThreadPoolExecutor.java:1142)

at java.util.concurrent.ThreadPoolExecutor$Worker.run= (ThreadPoolExecutor.java:617)

at java.lang.Thread.run(Thread.java:745)

Caused by: org.apache.airavata.gfac.core.provider.GFacProviderExce= ption: Error persisting job statusError while adding JOB_DETAIL

at org.apache.airavata.gfac.ssh.provider.impl.SSHProv= ider.execute(SSHProvider.java:231)

at org.apache.airavata.gfac.core.cpi.BetterGfacImpl.e= xecuteProvider(BetterGfacImpl.java:842)

... 8 more

Caused by: org.apache.airavata.gfac.GFacException: Error persistin= g job statusError while adding JOB_DETAIL

at org.apache.airavata.gfac.core.utils.GFacUtils.save= JobStatus(GFacUtils.java:242)

at org.apache.airavata.gfac.ssh.provider.impl.SSHProv= ider.execute(SSHProvider.java:215)

... 9 more

Caused by: org.apache.airavata.registry.cpi.RegistryException: Err= or while adding JOB_DETAIL

at org.apache.airavata.persistance.registry.jpa.impl.= RegistryImpl.add(RegistryImpl.java:182)

at org.apache.airavata.gfac.core.utils.GFacUtils.save= JobStatus(GFacUtils.java:232)

... 10 more

Caused by: org.apache.airavata.registry.cpi.RegistryException: org= .apache.airavata.registry.cpi.RegistryException: <openjpa-2.2.0-r422266:= 1244990 fatal store error> org.apache.openjpa.persistence.RollbackExcept= ion: The transaction has been rolled back.=C2=A0 See the nested exceptions for details on the errors that occurred.<= /p>

FailedObject: org.apache.airavata.persistance.registry.jpa.model.J= obDetail@6f1d32ff

at org.apache.airavata.persistance.registry.jpa.impl.= ExperimentRegistry.addJobDetails(ExperimentRegistry.java:1231)

at org.apache.airavata.persistance.registry.jpa.impl.= RegistryImpl.add(RegistryImpl.java:157)

... 11 more

Caused by: org.apache.airavata.registry.cpi.RegistryException: <= ;openjpa-2.2.0-r422266:1244990 fatal store error> org.apache.openjpa.per= sistence.RollbackException: The transaction has been rolled back.=C2=A0 See= the nested exceptions for details on the errors that occurred.

FailedObject: org.apache.airavata.persistance.registry.jpa.model.J= obDetail@6f1d32ff

at org.apache.airavata.persistance.registry.jpa.resou= rces.JobDetailResource.save(JobDetailResource.java:312)

at org.apache.airavata.persistance.registry.jpa.impl.= ExperimentRegistry.addJobDetails(ExperimentRegistry.java:1203)

... 12 more

Caused by: <openjpa-2.2.0-r422266:1244990 fatal store error>= org.apache.openjpa.persistence.RollbackException: The transaction has been= rolled back.=C2=A0 See the nested exceptions for details on the errors tha= t occurred.

FailedObject: org.apache.airavata.persistance.registry.jpa.model.J= obDetail@6f1d32ff

at org.apache.openjpa.persistence.EntityManagerImpl.c= ommit(EntityManagerImpl.java:594)

at org.apache.airavata.persistance.registry.jpa.resou= rces.JobDetailResource.save(JobDetailResource.java:308)

... 13 more

Caused by: <openjpa-2.2.0-r422266:1244990 fatal store error>= org.apache.openjpa.persistence.EntityExistsException: The transaction has = been rolled back.=C2=A0 See the nested exceptions for details on the errors= that occurred.

FailedObject: org.apache.airavata.persistance.registry.jpa.model.J= obDetail@6f1d32ff

at org.apache.openjpa.kernel.BrokerImpl.newFlushExcep= tion(BrokerImpl.java:2337)

at org.apache.openjpa.kernel.BrokerImpl.flush(BrokerI= mpl.java:2174)

at org.apache.openjpa.kernel.BrokerImpl.flushSafe(Bro= kerImpl.java:2072)

at org.apache.openjpa.kernel.BrokerImpl.beforeComplet= ion(BrokerImpl.java:1990)

at org.apache.openjpa.kernel.LocalManagedRuntime.comm= it(LocalManagedRuntime.java:81)

at org.apache.openjpa.kernel.BrokerImpl.commit(Broker= Impl.java:1514)

at org.apache.openjpa.kernel.DelegatingBroker.commit(= DelegatingBroker.java:933)

at org.apache.openjpa.persistence.EntityManagerImpl.c= ommit(EntityManagerImpl.java:570)

... 14 more

Caused by: <openjpa-2.2.0-r422266:1244990 fatal store error>= org.apache.openjpa.persistence.EntityExistsException: Column 'JOB_DESC= RIPTION' cannot be null {prepstmnt 1744538295=C2=A0

INSERT INTO JOB_DETAIL (JOB_ID, TASK_ID, COMPUTE_RESOURCE_CONSUMED= ,=C2=A0

=C2=A0 =C2=A0 =C2=A0 =C2=A0 CREATION_TIME, JOB_DESCRIPTION, JOBNAM= E, WORKING_DIR)=C2=A0

=C2=A0 =C2=A0 VALUES (?, ?, ?, ?, ?, ?, ?)=C2=A0

[params=3D(String) none, (String) tempNode_56f41083-10be-4161-8803= -ee1106bd3f54, (null) null, (Timestamp) 2015-08-17 06:49:01.613, (null) nul= l, (String) A516147113, (null) null]} [code=3D1048, state=3D23000]

FailedObject: org.apache.airavata.persistance.registry.jpa.model.J= obDetail@6f1d32ff

at org.apache.openjpa.jdbc.sql.DBDictionary.narrow(DB= Dictionary.java:4903)

at org.apache.openjpa.jdbc.sql.DBDictionary.newStoreE= xception(DBDictionary.java:4878)

at org.apache.openjpa.jdbc.sql.SQLExceptions.getStore= (SQLExceptions.java:136)

at org.apache.openjpa.jdbc.sql.SQLExceptions.getStore= (SQLExceptions.java:78)

at org.apache.openjpa.jdbc.kernel.PreparedStatementMa= nagerImpl.flushAndUpdate(PreparedStatementManagerImpl.java:143)

at org.apache.openjpa.jdbc.kernel.BatchingPreparedSta= tementManagerImpl.flushAndUpdate(BatchingPreparedStatementManagerImpl.java:= 79)

at org.apache.openjpa.jdbc.kernel.PreparedStatementMa= nagerImpl.flushInternal(PreparedStatementManagerImpl.java:99)

at org.apache.openjpa.jdbc.kernel.PreparedStatementMa= nagerImpl.flush(PreparedStatementManagerImpl.java:87)

at org.apache.openjpa.jdbc.kernel.ConstraintUpdateMan= ager.flush(ConstraintUpdateManager.java:550)

at org.apache.openjpa.jdbc.kernel.ConstraintUpdateMan= ager.flush(ConstraintUpdateManager.java:106)

at org.apache.openjpa.jdbc.kernel.BatchingConstraintU= pdateManager.flush(BatchingConstraintUpdateManager.java:59)

at org.apache.openjpa.jdbc.kernel.AbstractUpdateManag= er.flush(AbstractUpdateManager.java:103)

at org.apache.openjpa.jdbc.kernel.AbstractUpdateManag= er.flush(AbstractUpdateManager.java:76)

at org.apache.openjpa.jdbc.kernel.JDBCStoreManager.fl= ush(JDBCStoreManager.java:713)

at org.apache.openjpa.kernel.DelegatingStoreManager.f= lush(DelegatingStoreManager.java:131)

... 21 more

Caused by: org.apache.openjpa.lib.jdbc.ReportingSQLException: Colu= mn 'JOB_DESCRIPTION' cannot be null {prepstmnt 1744538295=C2=A0

INSERT INTO JOB_DETAIL (JOB_ID, TASK_ID, COMPUTE_RESOURCE_CONSUMED= ,=C2=A0

=C2=A0 =C2=A0 =C2=A0 =C2=A0 CREATION_TIME, JOB_DESCRIPTION, JOBNAM= E, WORKING_DIR)=C2=A0

=C2=A0 =C2=A0 VALUES (?, ?, ?, ?, ?, ?, ?)=C2=A0

[params=3D(String) none, (String) tempNode_56f41083-10be-4161-8803= -ee1106bd3f54, (null) null, (Timestamp) 2015-08-17 06:49:01.613, (null) nul= l, (String) A516147113, (null) null]} [code=3D1048, state=3D23000]

at org.apache.openjpa.lib.jdbc.LoggingConnectionDecor= ator.wrap(LoggingConnectionDecorator.java:247)

at org.apache.openjpa.lib.jdbc.LoggingConnectionDecor= ator.wrap(LoggingConnectionDecorator.java:223)

at org.apache.openjpa.lib.jdbc.LoggingConnectionDecor= ator.access$1000(LoggingConnectionDecorator.java:72)

at org.apache.openjpa.lib.jdbc.LoggingConnectionDecor= ator$LoggingConnection$LoggingPreparedStatement.executeUpdate(LoggingConnec= tionDecorator.java:1162)

at org.apache.openjpa.lib.jdbc.DelegatingPreparedStat= ement.executeUpdate(DelegatingPreparedStatement.java:272)

at org.apache.openjpa.jdbc.kernel.JDBCStoreManager$Ca= ncelPreparedStatement.executeUpdate(JDBCStoreManager.java:1741)

at org.apache.openjpa.jdbc.kernel.PreparedStatementMa= nagerImpl.executeUpdate(PreparedStatementManagerImpl.java:267)

at org.apache.openjpa.jdbc.kernel.PreparedStatementMa= nagerImpl.flushAndUpdate(PreparedStatementManagerImpl.java:118)

... 31 more

2015-08-17 06:49:03,692 [pool-8-thread-56] ERROR org.apache.airava= ta.gfac.GFacException=C2=A0 - Error launching the Job

org.apache.airavata.gfac.GFacException: Error while executing prov= ider org.apache.airavata.gfac.ssh.provider.impl.SSHProvider functionality.<= /span>

at org.apache.airavata.gfac.core.cpi.BetterGfacImpl.l= aunch(BetterGfacImpl.java:743)

at org.apache.airavata.gfac.core.cpi.BetterGfacImpl.s= ubmitJob(BetterGfacImpl.java:492)

at org.apache.airavata.gfac.core.cpi.BetterGfacImpl.s= ubmitJob(BetterGfacImpl.java:183)

--001a1134bad072383e051d820fb7--