chemistry-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mike IT Expert <mikeitexp...@gmail.com>
Subject Re: Does cmislib python support multi-threading?
Date Sat, 26 May 2018 19:42:37 GMT
Hi Laurent,

Please let me know if there has been update on this. I have added my
employer just to see there is anything we can do on our side to make the
cmislib thread safe.

Regards,
M

On Fri, May 25, 2018 at 7:12 PM, Mike IT Expert <mikeitexpert@gmail.com>
wrote:

>
> Apparently the exception doesn't occur if I use separate parent folder for
> each thread, but the new documents SOMETIMES are created under incorrect
> folder (created by other threads).
>
> Attached is the source code but I cleaned up the code a little and took
> out unrelated stuff of mine.
>
> Please let me know the outcome ....
>
> mike
>
>
>
>
>
> On Fri, May 25, 2018 at 5:59 PM, Mignon, Laurent <laurent.mignon@acsone.eu
> > wrote:
>
>> Not sure if it's the root cause but new files are added at same time into
>> the same parent folder. That means that the parent folder is updated at
>> same time.
>> Do you have the same problem if each thread create files in its own parent
>> folder?
>>
>> lmignon
>>
>> On Fri, May 25, 2018 at 1:52 PM, Mike IT Expert <mikeitexpert@gmail.com>
>> wrote:
>>
>> > You are right but my script does't update the same resource/document
>> > simultaneously because each thread uploads new files under new unique
>> names
>> > on the server, in that each thread
>> >
>> > *1) creates its own instance of the repository*
>> >
>> > * # 2) upload files and update properties*
>> > * # connect to the server *
>> > * client =  cmislib.CmisClient(SERVER_URL, USER, PASSWD)*
>> >
>> > * # get default repository*
>> > * repo = client.defaultRepository*
>> >
>> > *2) retrieves the target folder's node id*
>> >
>> > * # get the target folder to upload new documents into*
>> > * results = repo.query("select * from cmis:folder where cmis:name =
>> '%s'"%(
>> > TARGET_ALFRESCO_FOLDER, ))*
>> > * targetFolder = results[0]*
>> > * targetFolder = repo.getObject( targetFolder.id )*
>> >
>> > *3) for each file (eg. file_i ) in a loop python statement*
>> >
>> >
>> >
>> > * newDoc = targetFolder.createDocument( file_i['docName'], contentFile =
>> > inFile ) *
>> >
>> > * props = {*
>> > * u'cm:title': file_i[ HDR_TITLE ],*
>> > * u'cm:description': file_i[ HDR_DESCR ],*
>> > * }*
>> >
>> >
>> > * newDoc.updateProperties(props)*
>> >
>> >
>> > So what I am doing is that I access the same folder and upload new
>> > documents using unique names. I am 100% sure the document names (ie.
>> > * file_i['docName']*) are unique under the same folder.
>> >
>> > It just doesn't seem to me to be any concurrency problem with above
>> code.
>> >
>> >
>> > Many thanks anyways,
>> > M
>> >
>> >
>> > On Fri, May 25, 2018 at 3:57 PM, Mignon, Laurent <
>> laurent.mignon@acsone.eu
>> > >
>> > wrote:
>> >
>> > > Hi Mike,
>> > >
>> > > I'm not aware of a design issue that could prevent to use cmislib
>> python
>> > > into multi threads.
>> > > The problem you describe is a normal behaviour when two actors (human
>> or
>> > > prog) update the same resource at same time into alfresco.
>> > >
>> > > Regards,
>> > >
>> > > lmignon
>> > >
>> > > On Fri, May 25, 2018 at 12:19 PM, Mike IT Expert <
>> mikeitexpert@gmail.com
>> > >
>> > > wrote:
>> > >
>> > >> Dear All,
>> > >>
>> > >> Apologies in advance for sending this message to dev mailing list (I
>> > wish
>> > >> there was userlist also I wasn't sure it is bug or not.)
>> > >>
>> > >> I have started working with cmislib since past week using Alfresco
as
>> > the
>> > >> local server.
>> > >>
>> > >> My goal is to upload documents provided by json rcord list in a  for
>> > loop
>> > >> as you see below *which is part of run-overiden method of
>> > >> threading.Thread.run method for python 2.7*.
>> > >>
>> > >> The main code is attached just in case.
>> > >>
>> > >>
>> > >>
>> > >> The above code works find using ONE thread only. If I use more than
>> one
>> > >> thread I get the below exception for some records which is odd.
>> > >>
>> > >>
>> > >> File "module_proj_2_parts_1_and_2.py", line 82, in run
>> > >>     newDoc = targetFolder.createDocument( docName, contentFile =
>> inFile
>> > )
>> > >>   File "/home/mike/Desktop/hassan_proj2/.pyenv2/local/lib/python2.
>> > >> 7/site-packages/cmislib/atompub/binding.py", line 2788, in
>> > createDocument
>> > >>     contentEncoding)
>> > >>   File "/home/mike/Desktop/hassan_proj2/.pyenv2/local/lib/python2.
>> > >> 7/site-packages/cmislib/atompub/binding.py", line 1807, in
>> > createDocument
>> > >>     ATOM_XML_ENTRY_TYPE)
>> > >>   File "/home/mike/Desktop/hassan_proj2/.pyenv2/local/lib/python2.
>> > >> 7/site-packages/cmislib/atompub/binding.py", line 188, in post
>> > >>     self._processCommonErrors(resp, url)
>> > >>   File "/home/mike/Desktop/hassan_proj2/.pyenv2/local/lib/python2.
>> > >> 7/site-packages/cmislib/cmis_services.py", line 63, in
>> > >> _processCommonErrors
>> > >>     raise UpdateConflictException(error[\'status\'],
>> > >> url)\nUpdateConflictException: Error 409 at
>> > >> http://127.0.0.1:8080/alfresco/api/-default-/public/cmis/
>> > >> versions/1.1/atom/children?id=4baddda1-87eb-45eb-8d0b-9958b8942c4f\n
>> > >>
>> > >>
>> > >> I doubt if it is bug ... but if it is please let me know where I can
>> > >> start to fix this.
>> > >>
>> > >> Also, my pip freeze package versions are as below:
>> > >>
>> > >> certifi==2018.4.16
>> > >> chardet==3.0.4
>> > >> cmislib==0.6.0
>> > >> httplib2==0.11.3
>> > >> idna==2.6
>> > >> iso8601==0.1.12
>> > >> requests==2.18.4
>> > >> urllib3==1.22
>> > >>
>> > >> I appreciate any clue/comment.
>> > >>
>> > >> Best Regards,
>> > >> Mike
>> > >>
>> > >>
>> > >>
>> > >>
>> > >>
>> > >
>> > >
>> > > --
>> > > *Laurent Mignon*
>> > > Senior software engineer
>> > >
>> > > Tel: +352 20 21 10 20 32
>> > > Fax: +352 20 21 10 21
>> > > Email: laurent.mignon@acsone.eu
>> > >
>> > > Acsone SA, Succursale de Luxembourg
>> > > 22, Zone Industrielle
>> > > ​ ​
>> > > ​| L-8287 Kehlen, Luxembourg
>> > > TVA LU24733605
>> > > ​ | ​RCS B160400​
>> > >
>> > > ​
>> > > Acsone sa/nv
>> > > Waterloo Atrium, Drève Richelle 167
>> > > <https://maps.google.com/?q=Dr%C3%A8ve+Richelle+167&entry=gm
>> ail&source=g
>> > >
>> > >  | B-1410 Waterloo
>> > > RPM Bruxelles 0835.207.216 RPR Brussel
>> > >
>> > > www.acsone.eu
>> > >
>> > >
>> > >
>> > >
>> >
>>
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message