ant-ivy-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "James Poli" <James.P...@sas.com>
Subject RE: problem with work files created in cache by ivy:resolve (ivy 1.4)
Date Wed, 10 Oct 2007 20:09:32 GMT
Xavier and Robert,

Sorry I haven't opened an issue yet.  Robert thanks for the information
on filelocks.  I knew they
had problems on various platforms/hardware so I'd tested using them with
several use cases to verify
they worked where we needed them before I got too far into
implementation.  FWIW, I did have to change 
the use of the filelock on Linux from blocking to non-blocking after
noticing a blocked lock took 
several minutes to see a release!  BTW, I'm doing all the filelocking
via an ANT task I wrote before
and after the ivy:resolve task. 

Jim 

-----Original Message-----
From: Xavier Hanin [mailto:xavier.hanin@gmail.com] 
Sent: Wednesday, October 10, 2007 1:46 PM
To: ivy-user@incubator.apache.org
Subject: Re: problem with work files created in cache by ivy:resolve
(ivy 1.4)

On 10/10/07, Buck, Robert <rbuck@verisign.com> wrote:
>
> Hi,
>
> I am new to this thread. I don't mean to butt in, but I have one
> question regarding it, and this may be slightly unrelated...
>
> I see mention use of the FileLock class. When I saw a reference to
this,
> I was immediately concerned.
>
> IVY itself does NOT use FileLock, correct?  (I hope not)


Correct.

The reason I ask is that at VeriSign we discovered that FileLock does
> not work on any of the NFS devices we have. The class itself throws an
> exception, whose localized message effectively states, "FileLock is
> unsupported on NFS devices.". Additionally, there are many File
oriented
> classes in NIO that either: (a) do not work at all, or (b) are a LOT
> slower than old IO for IO intensive applications.
>
> So this is all to say, please, only use NIO when you have NFS, NetAPP,
> or EMC, hardware to test on, and you can prove with complete certainty
> the individual API works. You may be surprised to find that the more
> crucial parts of NIO is effectively broken.


Thanks for your experience feedback, we will take that into account when
we
will address this issue. Maybe you could add this comment to the issue
too
(not sure if James already opened it).

Xavier

Thanks in advance, no response necessary,
>
> Bob
>
> -----Original Message-----
> From: Xavier Hanin [mailto:xavier.hanin@gmail.com]
> Sent: Wednesday, October 10, 2007 1:38 AM
> To: ivy-user@incubator.apache.org
> Subject: Re: problem with work files created in cache by ivy:resolve
> (ivy 1.4)
>
> On 10/9/07, James Poli <James.Poli@sas.com> wrote:
> >
> > Xavier,
> >
> > Thanks for the reply.  Yes I can open an issue.  Question: if I lock
> > the entire resolve should I also lock the publish?  In other words
can
>
> > the Ivy resolve task be updating the module resolution results while
> > an Ivy publish (in another process) is trying to use them?
>
>
> Yes, you need to lock the publish od the same module, because Ivy
> doesn't update its resolution data transactionnally.
>
> When you open the issue, you can link it to IVY-605, which points to
> similar problem.
>
> Xavier
>
> Thanks again for any help,
> > Jim
> >
> > -----Original Message-----
> > From: Xavier Hanin [mailto:xavier.hanin@gmail.com]
> > Sent: Tuesday, October 09, 2007 1:20 AM
> > To: ivy-user@incubator.apache.org
> > Subject: Re: problem with work files created in cache by ivy:resolve
> > (ivy 1.4)
> >
> > On 10/9/07, JamesPoli <James.Poli@sas.com> wrote:
> > >
> > >
> > > As Jim Adams had stated we'd moved our cache to a tmp location to
> > > get around this problem.  Another solution I've been working on is
> > > using a lock
> > file
> > > to
> > > protect from having two or more resovles use the  cache at the
same
> > time.
> > > To "guard" the creation/deletion of the lock file I'm using a Java
> > > FileLock
> > > - it acts like a Java object monitor that is visible across JVM's.
> > > It appears to work but for multiple processes (15) using resolve.
> > > Could there be something I'm missing?  Should I also lock the
> > > publish?  I'm
> > thinking
> > > if
> > > I lock too many ivy tasks I'll end up defeating the whole purpose
> > > I'm doing it, performance gains by using a single cache.
> >
> >
> > Indeed, using a lock file can defeat the whole purpose of using a
> > single cache. The main problem is that we handle two different
things
> > in the
> > cache:
> > cache of downloaded data (ivy files and artifacts), and cache of a
> > module resolution results. The former is what is interesting to
share
> > and less a problem to lock, the latter is not really interesting to
> > share and more a problem to lock. Once again we see that the cache
> > management in Ivy needs improvement, it's one other vote for
IVY-399.
> > But this could even deserve a new issue: make Ivy able to use a
> > repository cache shared by multiple processes. Could you create such
> > an issue?
> >
> > Xavier
> >
> > Thanks in advance, Jim :super:
> > >
> > >
> > > Jim Adams-6 wrote:
> > > >
> > > > We have something similar but our only recourse was to put the
ivy
> > cache
> > > > in a tmp location and clear it after every build. Ivy can't
handle
>
> > > > multiple processes accessing the cache at the same time.
> > > >
> > > >> -----Original Message-----
> > > >> From: Gregoire Henry [mailto:ghynxmail@gmail.com]
> > > >> Sent: Friday, July 27, 2007 11:21 AM
> > > >> To: ivy-user@incubator.apache.org
> > > >> Subject: problem with work files created in cache by
ivy:resolve
> > > >> (ivy 1.4)
> > > >>
> > > >> hello
> > > >>
> > > >> I have a problem with the files created by ivy in the cache
> > > >> during the resolve phase ..
> > > >> Ivy create some files nammed [org]-[module]-[conf].xml in the
> > > >> cache , should'nt those files be created in a temporary
location
> > > >> outside of the
> > > >> cache, and preferably in project specific directory   ?
> > > >>
> > > >> My use case is the following
> > > >>
> > > >> We have a Continuous Build Server who can build at the same
time
> > > >> several different build from the same project ....
> > > >> In my build i create a clean.resolve task for deleting the
> > > >> [org]-[module]-[conf].xml files at the beginnig of each build
...
>
> > > >> as we can have multiple build at the same time for the same
> > > >> project I sometime have some build failled because of the
> > > >> deletion of this files by another starting build ...
> > > >>
> > > >> I still have created different cache location for my different
> > > >> project , but this use case should drive me to create a
different
>
> > > >> cache for each build !!!
> > > >>
> > > >> Have you a solution for that ?
> > > >> would it be possible to have an ivy.temp.dir properties ?
> > > >> Is there a different implementation in 2.0.0 ?
> > > >>
> > > >>
> > > >> regards
> > > >>
> > > >> Greg
> > > >>
> > > >>
> > > >>
> > > >>
> > > >
> > > >
> > > %-|%-|:confused::super::super::super::super::super::super::super:
> > > --
> > > View this message in context:
> > >
> >
http://www.nabble.com/problem-with-work-files-created-in-cache-by-ivy%
> > 3A
> > resolve-%28ivy-1.4%29-tf4158540.html#a13107058
> > > Sent from the ivy-user mailing list archive at Nabble.com.
> > >
> > >
> >
> >
> > --
> > Xavier Hanin - Independent Java Consultant http://xhab.blogspot.com/
> > http://incubator.apache.org/ivy/ http://www.xoocode.org/
> >
>
>
>
> --
> Xavier Hanin - Independent Java Consultant http://xhab.blogspot.com/
> http://incubator.apache.org/ivy/ http://www.xoocode.org/
>



-- 
Xavier Hanin - Independent Java Consultant
http://xhab.blogspot.com/
http://incubator.apache.org/ivy/
http://www.xoocode.org/

Mime
View raw message