ofbiz-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Giulio Speri - MpStyle Srl <giulio.sp...@mpstyle.it>
Subject Re: Scheduled jobs weird behavior
Date Tue, 03 Apr 2018 08:13:46 GMT
Hi Nicolas,

really interesting your solution to have ftp data persisted into the db; it
improves tracking and debugging of the transfers, for sure.
As first step I think that I'll try to tidy up and improve the code of the
service (specially the ftp part), with your previous suggestions and see if
the problem comes to a solution.
If not, I will head for the "db persistency solution".
I'll keep you informed on this.

Thank you all for your help!

Have a good day,
Giulio

2018-04-03 9:28 GMT+02:00 Nicolas Malin <nicolas.malin@nereide.fr>:

> Hi Giulio,
>
> When we realized ftp transfer, we process as follow :
> 1. create a content linked to the OFBiz element. In you case we would be
> created an OrderContent-Content with all OrderHeader present in the file.
> 2. Initialize a communicationEvent with the content, a type FTP_TRANSFER
> and a contactMechIdTo with the destination ftp location
> 3. run a batch that transfer all ftp communication
> With this process we haven't lost data and when a problem come it's easily
> tracking.
>
> Gil Portenseigne create an issue [1] with the code base improvement with
> own customer site feedback
>
> Nicolas
>
> [1] https://issues.apache.org/jira/browse/OFBIZ-10245 File transfer
> management with communicationEvent and new contactMech FTP_ADDRESS
>
> PS: I send it to dev because my response cover an improvement but it's
> also interesting some user ...
>
> On 02/04/2018 09:54, Rishi Solanki wrote:
>
>> Hi Giulio,
>>
>> +1 for Michael suggestion, you should try to backup the file locally
>> before
>> transferring.
>>
>> Another item I could see in your code (thanks for sharing) that, it just
>> connect and transfer the file to ftp location. I see many methods with
>> FTPClient class. One is to set the buffer size.
>> FTPClient.setBufferSize(int), which set the internal buffer size for
>> buffered data streams.
>>
>> As you mentioned initially mostly you got the successful transfer, and
>> problem occurs very few time. Please consider the size of the file to
>> transfer and increase its buffer size should resolve your problem. Also
>> there are many other methods which allows you to get/set the transfer
>> status and mode etc should help.
>>
>> Reference:
>> https://commons.apache.org/proper/commons-net/apidocs/org/
>> apache/commons/net/ftp/FTPClient.html
>>
>> HTH!
>>
>>
>>
>> Rishi Solanki
>> Sr Manager, Enterprise Software Development
>> HotWax Systems Pvt. Ltd.
>> Direct: +91-9893287847
>> http://www.hotwaxsystems.com
>> www.hotwax.co
>>
>> On Fri, Mar 30, 2018 at 10:42 PM, Michael Brohl <michael.brohl@ecomify.de
>> >
>> wrote:
>>
>> Hi Giulio,
>>>
>>> It seems you directly store the generated file to the ftp server,
>>> correct?
>>>
>>> In such cases, we store the file to be transferred locally in an archive
>>> directory and then transfer it to the ftp server.
>>>
>>> You'll see if the file was written and you have a backup in case there
>>> was
>>> something wrong with the ftp transfer.
>>>
>>> Best regards,
>>>
>>> Michael
>>>
>>>
>>> Am 30.03.18 um 16:16 schrieb Giulio Speri - MpStyle Srl:
>>>
>>> Hi Rishi,
>>>
>>>> thanks for the reply!
>>>>
>>>> The api used for the ftp transfer are the apache FTP api, group
>>>> "commons-net" (version 3.5);
>>>> The code used to connect and store file via fftp is the following:
>>>>
>>>> ftp = new FTPClient()
>>>> ftp.connect "xxx.yyyyyy.ww"
>>>> ftp.enterLocalPassiveMode()
>>>> ftp.login "username", "password"
>>>> ftp.changeWorkingDirectory "INPUT_DIRECTORY"
>>>> //put
>>>> InputStream file_put= new FileInputStream(directoryZip);
>>>> ftp.setFileType(FTP.BINARY_FILE_TYPE);
>>>> ftp.storeFile(zipName, file_put);
>>>> file_put.close()
>>>> ftp.logout()
>>>> ftp.disconnect()
>>>>
>>>> Shouldn't Groovy automatically raise exception in case of errors?
>>>>
>>>>
>>>>
>>>>
>>>> 2018-03-30 15:07 GMT+02:00 Rishi Solanki <rishisolankii@gmail.com>:
>>>>
>>>> Giulio,
>>>>
>>>>> In addition to what Nicolas suggested please share or look into the
>>>>> specific code which logs the files and export the orders at ftp
>>>>> locaion.
>>>>> Probable cause for such error writer create the file at particular
>>>>> location
>>>>> but could not push the data/file due to some reason but send no
>>>>> exception/failure.
>>>>>
>>>>> My guess problem should be related to pushing the exported data and
>>>>> logs
>>>>> to
>>>>> ftp location and is giving success but not pushing. Kind of case when
>>>>> we
>>>>> tell the writer object to write and close. Sometime writer.flush() is
>>>>> also
>>>>> required.
>>>>>
>>>>> Also if possible at your end then share the relevant code and api you
>>>>> are
>>>>> using, to better understand the problem.
>>>>>
>>>>> HTH!
>>>>>
>>>>>
>>>>> Rishi Solanki
>>>>> Sr Manager, Enterprise Software Development
>>>>> HotWax Systems Pvt. Ltd.
>>>>> Direct: +91-9893287847
>>>>> http://www.hotwaxsystems.com
>>>>> www.hotwax.co
>>>>>
>>>>> On Fri, Mar 30, 2018 at 5:48 PM, Nicolas Malin <
>>>>> nicolas.malin@nereide.fr
>>>>> wrote:
>>>>>
>>>>> Hello Giulio,
>>>>>
>>>>>> On 30/03/2018 14:05, Giulio Speri - MpStyle Srl wrote:
>>>>>>
>>>>>> Hi all,
>>>>>>
>>>>>>> [...] Now, here is the issue; sometimes (runtime in which happen
>>>>>>> seems
>>>>>>> random), I
>>>>>>> notice that:
>>>>>>> - the service in the Job List screen, has the state of "Finished"
(no
>>>>>>> errors);
>>>>>>> - No log file nor zip file is written;
>>>>>>> - the flag on the OrderHeader entity has been set to "Y" (by
default
>>>>>>> is
>>>>>>> null this field);
>>>>>>> - no error or exception found in ofbiz/error log files;
>>>>>>>
>>>>>>> The Job Poller on the second instance of ofbiz is disabled
>>>>>>> ("poll-enable=false" in serviceengine.xml), so that all the scheduled
>>>>>>>
>>>>>>> jobs
>>>>>> are run only on the first instance.
>>>>>>
>>>>>>> I am quite confused by this situation and I really don't know
what
>>>>>>> and
>>>>>>> where to look, to understand what happens.
>>>>>>>
>>>>>>> Does anyone ever experienced a similar issue?
>>>>>>>
>>>>>>> I worked several time with service engine and I interpreted your
>>>>>>>
>>>>>> feedback
>>>>>> as follow :
>>>>>> * the service in the Job List screen, has the state of "Finished"
->
>>>>>> Ok
>>>>>> for OFBiz the service return a success
>>>>>> * the flag on the OrderHeader entity has been set to "Y" (by default
>>>>>> is
>>>>>> null this field); -> If your service run under transaction, no
reason
>>>>>>
>>>>>> that
>>>>>
>>>>> your service correctly
>>>>>> * No log file nor zip file is written
>>>>>>     ** I see different case, if you are sure that your code works
>>>>>> properly
>>>>>> check if you didn't manage some silent exception ( as
>>>>>> try{...}catch(Exception){ }) and is it's the case review you exception
>>>>>> management.
>>>>>>     ** Increase your code with more log with good log4j configuration
>>>>>> for
>>>>>> track your service.
>>>>>>     ** Check if you haven't external element that can be delete your
>>>>>> file
>>>>>>
>>>>>> by
>>>>>
>>>>> inadvertence
>>>>>>     ** a doubt, send an email with the generate file to your developer
>>>>>> team
>>>>>> in // to the storage.
>>>>>>     ** no solution -> Use the remote debugger
>>>>>>
>>>>>> I hope you can find a solution to your problem
>>>>>> Nicolas
>>>>>>
>>>>>> Thanks in advance for your help.
>>>>>>
>>>>>> Giulio
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>
>>>
>
>


-- 
Giulio Speri


*Mp Styl**e Srl*
via Antonio Meucci, 37
41019 Limidi di Soliera (MO)
T 059/684916
M 334/3779851

www.mpstyle.it

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message