chemistry-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Mignon, Laurent" <laurent.mig...@acsone.eu>
Subject Re: Does cmislib python support multi-threading?
Date Tue, 29 May 2018 11:25:28 GMT
HI

"from cmislib.browser import BrowserBinding" works in all my projects.


On Mon, May 28, 2018 at 8:25 PM, Mike IT Expert <mikeitexpert@gmail.com>
wrote:

> I tried
>
> client =  cmislib.CmisClient(SERVER_URL, USER, PASSWD,
> binding=cmislib.browser.BrowserBinding())
>
> But apparently it couldn't be found :
>
> Exception in thread Thread-15:
> Traceback (most recent call last):
>   File "/usr/lib/python3.6/threading.py", line 916, in _bootstrap_inner
>     self.run()
>   File "cmislib_multi_threading_test_.py", line 40, in run
>     client =  cmislib.CmisClient(SERVER_URL, USER, PASSWD,
> binding=cmislib.browser.BrowserBinding())
> AttributeError: module 'cmislib' has no attribute 'browser'
>
> Where is BrowserBinding?
>
>
>
>
>
> On Mon, May 28, 2018 at 5:04 PM, Mignon, Laurent <laurent.mignon@acsone.eu
> > wrote:
>
>> By default the cmislib uses the 'atompub' binding. Can you try with the
>> 'browser' binding.
>>
>> from cmislib.browser.binding import BrowserBinding
>> client  = CmisClient(.., .., binding=BrowserBinding())
>>
>>
>>
>>
>> On Mon, May 28, 2018 at 2:26 PM, Mike IT Expert <mikeitexpert@gmail.com>
>> wrote:
>>
>> > Thanks you so much for quick resposne.
>> >
>> > Still the same no difference. (files from one thread goes to folder for
>> > another thread)
>> >
>> > I checked out the code like below:
>> >
>> > git clone
>> > git fetch origin pull/12/head
>> > git checkout 1073eb533fc3478515f4af3a6f779b63c1b651b2
>> >
>> >
>> > I used Python 3.6.
>> >
>> > I don't think it is related to Alfresco, I used the same logic for
>> > multi-threading using REST API it works just fine.
>> >
>> > Thanks,
>> > Mike
>> >
>> >
>> >
>> >
>> >
>> > On Mon, May 28, 2018 at 12:26 PM, Mignon, Laurent <
>> > laurent.mignon@acsone.eu>
>> > wrote:
>> >
>> > > Hi Mike,
>> > >
>> > > I'm sorry but I've no time to investigate if the problem is on the
>> > cmislib
>> > > side or on the alfresco side. Nevertheless, I recently completely
>> > reworked
>> > > the cmislib library to make it py2 py3 compatible and at the same
>> time I
>> > > replaced the dependency on httplib2 by requests. (
>> > > https://github.com/apache/chemistry-cmislib/pull/12)
>> > > It could be interesting to make a try with this new version of the
>> > library
>> > > and see if the problem is also present into this new one. If the
>> problem
>> > > comes from cmislib, personally I would prefer to correct it in this
>> new
>> > > version and speed up thus its release.
>> > >
>> > > Regards,
>> > >
>> > > Laurent Mignon
>> > >
>> > >
>> > > On Sat, May 26, 2018 at 9:42 PM, Mike IT Expert <
>> mikeitexpert@gmail.com>
>> > > wrote:
>> > >
>> > > > Hi Laurent,
>> > > >
>> > > > Please let me know if there has been update on this. I have added
my
>> > > > employer just to see there is anything we can do on our side to make
>> > the
>> > > > cmislib thread safe.
>> > > >
>> > > > Regards,
>> > > > M
>> > > >
>> > > > On Fri, May 25, 2018 at 7:12 PM, Mike IT Expert <
>> > mikeitexpert@gmail.com>
>> > > > wrote:
>> > > >
>> > > > >
>> > > > > Apparently the exception doesn't occur if I use separate parent
>> > folder
>> > > > for
>> > > > > each thread, but the new documents SOMETIMES are created under
>> > > incorrect
>> > > > > folder (created by other threads).
>> > > > >
>> > > > > Attached is the source code but I cleaned up the code a little
and
>> > took
>> > > > > out unrelated stuff of mine.
>> > > > >
>> > > > > Please let me know the outcome ....
>> > > > >
>> > > > > mike
>> > > > >
>> > > > >
>> > > > >
>> > > > >
>> > > > >
>> > > > > On Fri, May 25, 2018 at 5:59 PM, Mignon, Laurent <
>> > > > laurent.mignon@acsone.eu
>> > > > > > wrote:
>> > > > >
>> > > > >> Not sure if it's the root cause but new files are added at
same
>> time
>> > > > into
>> > > > >> the same parent folder. That means that the parent folder
is
>> updated
>> > > at
>> > > > >> same time.
>> > > > >> Do you have the same problem if each thread create files
in its
>> own
>> > > > parent
>> > > > >> folder?
>> > > > >>
>> > > > >> lmignon
>> > > > >>
>> > > > >> On Fri, May 25, 2018 at 1:52 PM, Mike IT Expert <
>> > > mikeitexpert@gmail.com
>> > > > >
>> > > > >> wrote:
>> > > > >>
>> > > > >> > You are right but my script does't update the same
>> > resource/document
>> > > > >> > simultaneously because each thread uploads new files
under new
>> > > unique
>> > > > >> names
>> > > > >> > on the server, in that each thread
>> > > > >> >
>> > > > >> > *1) creates its own instance of the repository*
>> > > > >> >
>> > > > >> > * # 2) upload files and update properties*
>> > > > >> > * # connect to the server *
>> > > > >> > * client =  cmislib.CmisClient(SERVER_URL, USER, PASSWD)*
>> > > > >> >
>> > > > >> > * # get default repository*
>> > > > >> > * repo = client.defaultRepository*
>> > > > >> >
>> > > > >> > *2) retrieves the target folder's node id*
>> > > > >> >
>> > > > >> > * # get the target folder to upload new documents into*
>> > > > >> > * results = repo.query("select * from cmis:folder where
>> cmis:name
>> > =
>> > > > >> '%s'"%(
>> > > > >> > TARGET_ALFRESCO_FOLDER, ))*
>> > > > >> > * targetFolder = results[0]*
>> > > > >> > * targetFolder = repo.getObject( targetFolder.id )*
>> > > > >> >
>> > > > >> > *3) for each file (eg. file_i ) in a loop python statement*
>> > > > >> >
>> > > > >> >
>> > > > >> >
>> > > > >> > * newDoc = targetFolder.createDocument( file_i['docName'],
>> > > > contentFile =
>> > > > >> > inFile ) *
>> > > > >> >
>> > > > >> > * props = {*
>> > > > >> > * u'cm:title': file_i[ HDR_TITLE ],*
>> > > > >> > * u'cm:description': file_i[ HDR_DESCR ],*
>> > > > >> > * }*
>> > > > >> >
>> > > > >> >
>> > > > >> > * newDoc.updateProperties(props)*
>> > > > >> >
>> > > > >> >
>> > > > >> > So what I am doing is that I access the same folder
and upload
>> new
>> > > > >> > documents using unique names. I am 100% sure the document
names
>> > (ie.
>> > > > >> > * file_i['docName']*) are unique under the same folder.
>> > > > >> >
>> > > > >> > It just doesn't seem to me to be any concurrency problem
with
>> > above
>> > > > >> code.
>> > > > >> >
>> > > > >> >
>> > > > >> > Many thanks anyways,
>> > > > >> > M
>> > > > >> >
>> > > > >> >
>> > > > >> > On Fri, May 25, 2018 at 3:57 PM, Mignon, Laurent <
>> > > > >> laurent.mignon@acsone.eu
>> > > > >> > >
>> > > > >> > wrote:
>> > > > >> >
>> > > > >> > > Hi Mike,
>> > > > >> > >
>> > > > >> > > I'm not aware of a design issue that could prevent
to use
>> > cmislib
>> > > > >> python
>> > > > >> > > into multi threads.
>> > > > >> > > The problem you describe is a normal behaviour
when two
>> actors
>> > > > (human
>> > > > >> or
>> > > > >> > > prog) update the same resource at same time into
alfresco.
>> > > > >> > >
>> > > > >> > > Regards,
>> > > > >> > >
>> > > > >> > > lmignon
>> > > > >> > >
>> > > > >> > > On Fri, May 25, 2018 at 12:19 PM, Mike IT Expert
<
>> > > > >> mikeitexpert@gmail.com
>> > > > >> > >
>> > > > >> > > wrote:
>> > > > >> > >
>> > > > >> > >> Dear All,
>> > > > >> > >>
>> > > > >> > >> Apologies in advance for sending this message
to dev mailing
>> > list
>> > > > (I
>> > > > >> > wish
>> > > > >> > >> there was userlist also I wasn't sure it is
bug or not.)
>> > > > >> > >>
>> > > > >> > >> I have started working with cmislib since past
week using
>> > > Alfresco
>> > > > as
>> > > > >> > the
>> > > > >> > >> local server.
>> > > > >> > >>
>> > > > >> > >> My goal is to upload documents provided by
json rcord list
>> in a
>> > > > for
>> > > > >> > loop
>> > > > >> > >> as you see below *which is part of run-overiden
method of
>> > > > >> > >> threading.Thread.run method for python 2.7*.
>> > > > >> > >>
>> > > > >> > >> The main code is attached just in case.
>> > > > >> > >>
>> > > > >> > >>
>> > > > >> > >>
>> > > > >> > >> The above code works find using ONE thread
only. If I use
>> more
>> > > than
>> > > > >> one
>> > > > >> > >> thread I get the below exception for some records
which is
>> odd.
>> > > > >> > >>
>> > > > >> > >>
>> > > > >> > >> File "module_proj_2_parts_1_and_2.py", line
82, in run
>> > > > >> > >>     newDoc = targetFolder.createDocument( docName,
>> contentFile
>> > =
>> > > > >> inFile
>> > > > >> > )
>> > > > >> > >>   File "/home/mike/Desktop/hassan_proj2/.pyenv2/local/lib/
>> > > python2.
>> > > > >> > >> 7/site-packages/cmislib/atompub/binding.py",
line 2788, in
>> > > > >> > createDocument
>> > > > >> > >>     contentEncoding)
>> > > > >> > >>   File "/home/mike/Desktop/hassan_proj2/.pyenv2/local/lib/
>> > > python2.
>> > > > >> > >> 7/site-packages/cmislib/atompub/binding.py",
line 1807, in
>> > > > >> > createDocument
>> > > > >> > >>     ATOM_XML_ENTRY_TYPE)
>> > > > >> > >>   File "/home/mike/Desktop/hassan_proj2/.pyenv2/local/lib/
>> > > python2.
>> > > > >> > >> 7/site-packages/cmislib/atompub/binding.py",
line 188, in
>> post
>> > > > >> > >>     self._processCommonErrors(resp, url)
>> > > > >> > >>   File "/home/mike/Desktop/hassan_proj2/.pyenv2/local/lib/
>> > > python2.
>> > > > >> > >> 7/site-packages/cmislib/cmis_services.py",
line 63, in
>> > > > >> > >> _processCommonErrors
>> > > > >> > >>     raise UpdateConflictException(error[\'status\'],
>> > > > >> > >> url)\nUpdateConflictException: Error 409 at
>> > > > >> > >> http://127.0.0.1:8080/alfresco/api/-default-/public/cmis/
>> > > > >> > >> versions/1.1/atom/children?id=4baddda1-87eb-45eb-8d0b-
>> > > > 9958b8942c4f\n
>> > > > >> > >>
>> > > > >> > >>
>> > > > >> > >> I doubt if it is bug ... but if it is please
let me know
>> where
>> > I
>> > > > can
>> > > > >> > >> start to fix this.
>> > > > >> > >>
>> > > > >> > >> Also, my pip freeze package versions are as
below:
>> > > > >> > >>
>> > > > >> > >> certifi==2018.4.16
>> > > > >> > >> chardet==3.0.4
>> > > > >> > >> cmislib==0.6.0
>> > > > >> > >> httplib2==0.11.3
>> > > > >> > >> idna==2.6
>> > > > >> > >> iso8601==0.1.12
>> > > > >> > >> requests==2.18.4
>> > > > >> > >> urllib3==1.22
>> > > > >> > >>
>> > > > >> > >> I appreciate any clue/comment.
>> > > > >> > >>
>> > > > >> > >> Best Regards,
>> > > > >> > >> Mike
>> > > > >> > >>
>> > > > >> > >>
>> > > > >> > >>
>> > > > >> > >>
>> > > > >> > >>
>> > > > >> > >
>> > > > >> > >
>> > > > >> > > --
>> > > > >> > > *Laurent Mignon*
>> > > > >> > > Senior software engineer
>> > > > >> > >
>> > > > >> > > Tel: +352 20 21 10 20 32
>> > > > >> > > Fax: +352 20 21 10 21
>> > > > >> > > Email: laurent.mignon@acsone.eu
>> > > > >> > >
>> > > > >> > > Acsone SA, Succursale de Luxembourg
>> > > > >> > > 22, Zone Industrielle
>> > > > >> > > ​ ​
>> > > > >> > > ​| L-8287 Kehlen, Luxembourg
>> > > > >> > > TVA LU24733605
>> > > > >> > > ​ | ​RCS B160400​
>> > > > >> > >
>> > > > >> > > ​
>> > > > >> > > Acsone sa/nv
>> > > > >> > > Waterloo Atrium, Drève Richelle 167
>> <https://maps.google.com/?q=Dr%C3%A8ve+Richelle+167&entry=gmail&source=g>
>> > > > >> > > <https://maps.google.com/?q=Dr%C3%A8ve+Richelle+167&entry=gm
>> > > > >> ail&source=g
>> > > > >> > >
>> > > > >> > >  | B-1410 Waterloo
>> > > > >> > > RPM Bruxelles 0835.207.216 RPR Brussel
>> > > > >> > >
>> > > > >> > > www.acsone.eu
>> > > > >> > >
>> > > > >> > >
>> > > > >> > >
>> > > > >> > >
>> > > > >> >
>> > > > >>
>> > > > >
>> > > > >
>> > > >
>> > >
>> > >
>> > >
>> > > --
>> > > *Laurent Mignon*
>> > > Senior software engineer
>> > >
>> > > Tel: +352 20 21 10 20 32
>> > > Fax: +352 20 21 10 21
>> > > Email: laurent.mignon@acsone.eu
>> > >
>> > > Acsone SA, Succursale de Luxembourg
>> > > 22, Zone Industrielle
>> > > ​ ​
>> > > ​| L-8287 Kehlen, Luxembourg
>> > > TVA LU24733605
>> > > ​ | ​RCS B160400​
>> > >
>> > > ​
>> > > Acsone sa/nv
>> > > Waterloo Atrium, Drève Richelle 167
>> <https://maps.google.com/?q=Dr%C3%A8ve+Richelle+167+%0D%0A+%C2%A0+%7C+B-1410+Waterloo&entry=gmail&source=g>
>> > >  | B-1410 Waterloo
>> <https://maps.google.com/?q=Dr%C3%A8ve+Richelle+167+%0D%0A+%C2%A0+%7C+B-1410+Waterloo&entry=gmail&source=g>
>> > > RPM Bruxelles 0835.207.216 RPR Brussel
>> > >
>> > > www.acsone.eu
>> > >
>> >
>>
>>
>>
>> --
>> *Laurent Mignon*
>> Senior software engineer
>>
>> Tel: +352 20 21 10 20 32
>> Fax: +352 20 21 10 21
>> Email: laurent.mignon@acsone.eu
>>
>> Acsone SA, Succursale de Luxembourg
>> 22, Zone Industrielle
>> ​ ​
>> ​| L-8287 Kehlen, Luxembourg
>> TVA LU24733605
>> ​ | ​RCS B160400​
>>
>> ​
>> Acsone sa/nv
>> Waterloo Atrium, Drève Richelle 167
>> <https://maps.google.com/?q=Dr%C3%A8ve+Richelle+167+%0D%0A%C2%A0%7C+B-1410+Waterloo&entry=gmail&source=g>
>>  | B-1410 Waterloo
>> RPM Bruxelles 0835.207.216 RPR Brussel
>>
>> www.acsone.eu
>>
>
>


-- 
*Laurent Mignon*
Senior software engineer

Tel: +352 20 21 10 20 32
Fax: +352 20 21 10 21
Email: laurent.mignon@acsone.eu

Acsone SA, Succursale de Luxembourg
22, Zone Industrielle
​ ​
​| L-8287 Kehlen, Luxembourg
TVA LU24733605
​ | ​RCS B160400​

​
Acsone sa/nv
Waterloo Atrium, Drève Richelle 167
 | B-1410 Waterloo
RPM Bruxelles 0835.207.216 RPR Brussel

www.acsone.eu

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message