ace-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Bram Pouwelse <b...@pouwelse.com>
Subject Re: How to keep ace performance consistent
Date Sat, 11 Jan 2014 17:12:37 GMT
Hi Marcel,


I've created a Jira issue [1] containing steps to reproduce the
deploymentversionlimit.

I've also worked on a deployment script [2] to replace my current
deployment process that uses the ContinousDeployer that comes with Ace.
I've used one of the scripts you've shared earlier so thanks for sharing
that made my life a lot easier ;). I've added some code to create the
sourcerepo and features based on a bndrun file from a (bnd)tools project.

I've tested this on my dev machine by running the script from the gogo
shell in the ace client how can I start a gosh script without user
interaction from my CI build environment?

Greetings,

Bram

[1] https://issues.apache.org/jira/browse/ACE-449

[2]
dist="test"
feature="myproject-$dist"

bndWorkspace="/path/to/myproject"
bndProject="myproject"
bndFile=$dist".bndrun"

obr=getproperty org.apache.ace.obr
aceObr="http://$obr/obr/repository.xml"

tmprepo=bnd:dist $bndWorkspace $bndProject $bndFile

echo "Define repositories"
sourceindex = (repo:index $tmprepo)
sourcerepo = (repo:repo R5 $sourceindex)
targetrepo = (repo:repo OBR "http://localhost:8084/obr/repository.xml")
releaserepo = (repo:repo OBR
"file:///path/to/myproject/cnf/releaserepo/index.xml,\
http://repository.amdatu.org/dependencies/index.xml.gz,\
http://repository.amdatu.org/release/index.xml.gz,\
http://repository.amdatu.org/snapshot/index.xml.gz")

echo "Deploying bundles"
deployed = repo:cd $releaserepo $sourcerepo $targetrepo

echo "Opening client workspace"
workspace = (ace:cw)

echo "Creating new artifacts"
each $deployed {
  identity = $it getIdentity
  version = $it getVersion
  name = "$identity - $version"
  url = $it getUrl
  mimetype = $it getMimetype

  if { (coll:first ($workspace la "(artifactName=$name)")) } { } { \
  if { $mimetype equals "application/xml:osgi-autoconf" } \
  { $workspace ca [ artifactName="$name" url="$url" mimetype="$mimetype"
filename="$name" processorPid="org.osgi.deployment.rp.autoconf" ] } \
  { $workspace ca [ artifactName="$name" url="$url" mimetype="$mimetype"
Bundle-SymbolicName="$identity" Bundle-Version="$version" ]; echo "Create
$name" }\
  }
}

echo "Ensuring feature exists"
if { (coll:first ($workspace lf "(name=$feature)")) } { echo " -> exists" }
{ $workspace cf "$feature"; echo " -> created" }

echo "Ensuring distribution exists"
if { (coll:first ($workspace ld "(name=$dist)")) } { echo " -> exists" } {
$workspace cd "$dist"; echo " -> created" }

echo "Ensuring f2d association"
if { (coll:first ($workspace lf2d "(leftEndpoint=*name=$feature*)")) } {
echo " -> exists" } { $workspace cf2d "(name=$feature)" "(name=$dist)" "1"
"1"; echo " -> created" }

echo "Removing a2f associations for feature: $feature"
a2f = ($workspace la2f "(rightEndpoint=*name=$feature*)")
each $a2f {
$workspace da2f $it
}

echo "Ensuring a2f associations for feature: $feature"
b = bnd:lb $aceObr $bndWorkspace $bndProject $bndFile
each $b {
  identity = $it getBsn
  version = $it getVersion

  leftEndpoint="(&(Bundle-SymbolicName=$identity)(Bundle-Version=$version))"
  rightEndpoint="(name=$feature)"
  $workspace ca2f $leftEndpoint $rightEndpoint "10000" "1"
}

echo "Committing workspace"
$workspace commit

# Let events settle
misc:sleep 2000
ace:rw $workspace





2014/1/10 Bram Pouwelse <bram@pouwelse.com>

> Hi Marcel,
>
> 2014/1/10 Marcel Offermans <marcel.offermans@luminis.eu>
>
>> Hello Bram,
>>
>> On 09 Jan 2014, at 11:55 am, Bram Pouwelse <bram@pouwelse.com> wrote:
>>
>> > The issue with the deploymentversionlimit is when I re-configure the
>> > deployment version limit after using Ace for a while.
>>
>> From re-reviewing the code I still do not see a problem, even when
>> reconfiguring the limit, it should be applied every time a new version gets
>> added to that repository. Can you describe the steps to reproduce this
>> issue?
>>
>>
> I'll try to reproduce the issue on a fresh instance and provide you with
> the steps to reproduce
>
>
>>  > I noticed that the
>> > repository file was growing over time [1] and wanted to know if that was
>> > causing the performance degradation and if there is a way to keep it
>> from
>> > growing that fast ;).
>>
>> You can. This 'deploymentversionlimit' is one. Roughly, the deployment
>> repository keeps track of all versions of deployments for each target,
>> where a deployment lists all artifacts. So its size is rougly: targets *
>> versions * artifacts. That is also why it can grow quite quickly. From this
>> formula it's easy to see how to limit its growth. Having less targets or
>> artifacts makes little sense, those are givens, but having less versions
>> does make some sense, which is why we introduced that limit.
>>
>> Another parameter to tweak is (and now I risk confusing you) to limit the
>> number of old versions of the deployment repository to keep around. As you
>> might have seen, every time you make a change, a new version of the
>> deployment repository is calculated and saved on disk. All old versions
>> remain on disk as well. There is a configuration option that you can set
>> for each repository, in the org.apache.ace.server.repository.factory
>> configurations called "limit" which allows you to set the number of old
>> versions of the file it should keep for that repository. Now this only
>> saves you disk space, since ACE does not do anything with these old
>> versions anyway, except keep them around as a record of what has changed
>> over time. You don't always need that record, which is why we introduced
>> this option.
>>
>>
> I've seen that and already limited the amount of copies.
>
>
>>  > If it's (currently) not possible to configure the
>> deploymentversionlimit
>> > after you start using Ace I'll start over with a clean server and see
>> how
>> > that works out.
>>
>> It should work. Let's work together to find and fix the issue.
>
>
> Ok :)
>
>
>>  > I've watched the presentation Jago and jou did after you mentioned it
>> in an
>> > earlier thread on this list and I'm already experimenting with it to see
>> > how this works out for us.
>>
>> Let me know if we can help. The gogo shell is not always the easiest to
>> understand. I would be happy to share what we have if that helps you.
>>
>>
> I've found some gogo shell sample code in an earlier thread which helped
> me a lot already. My current deployment implementation uses a bndrun as
> input and creates features and distributions based on the name of the
> bndrun file and project (deploying test.bndrun from project-x makes sure
> all bundles listed in the bndrun are copied to Ace, creates a feature
> "project-x-test" and a distribution test). This is in Java and using the
> rest API.
>
> Now I'm trying out the ContinuousDeployer that is "shipped" with Ace but
> still have to work on creating features based on the bndrun files. If you
> have anything that you can share that would help me with this that would be
> great.
>
>
> During the deployment using ContinuousDeployer I somtimes get an error
> like:
> Uploading released resource:
>  org.amdatu.web.resourcehandler/1.0.3/osgi.bundle -
> file:/Users/brampouwelse/Development/workspaces/fabuland-repository/dependencies/org.amdatu.web.resourcehandler/org.amdatu.web.resourcehandler-1.0.3.jar
> java.lang.IllegalStateException: Exepected 1 result ofter copy
>
> I've exported all bundles from my project and listed my project release
> repository and the amdatu repo's as release repository locations, running
> the deployment again  "solves" the issue for now.
>
>
>
>> Greetiings, Marcel
>>
>>
> Greetings,
> Bram
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message