river-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Bryan Thompson <br...@systap.com>
Subject Re: Questions about the Reference Collection dependencies. Was: Re: Release 3.0
Date Wed, 12 Aug 2015 09:05:19 GMT
We use very similar collections in Blazegraph and they are quite important
(we mainly use them as a cache for index objects)  I do avoid SoftReference
based collections due to the inability to have predictable behavior of the
garbage collector.

It sounds like importing the relevant classes would address people's
concerns?

I assume the license is compatible that that the source code could be
simply imported.

Bryan

On Wednesday, August 12, 2015, Peter <jini@zeus.net.au> wrote:

> In ServiceDiscoveryManager, it's used to to cache ServiceDiscardTimerTasks
> in a ConcurrentHashMap with key's that are weakly referenced by ServiceID.
>
> PreferredClassProvider, uses it to cache weakly referenced Classloaders
> (values) in a ConcurrentHashMap using a LoaderKey that contains URI []
> representing a codebase annotation, with a weakly referenced parent
> ClassLoader.  It's also used to provide a weakly referenced set, the set is
> read and modified by concurrent threads.
>
> In CombinerSecurityManager it's used to cache time referenced
> NavigableSet's of Permission's, these sets are then stored as values in a
> ConcurrentMap with security contexts as keys.
>
> Custard apple adds weak, soft and time (since last access) based reference
> control to existing Java collection implementations.  References can be
> identity or equality based.
>
> Further information is available in the javadoc.
>
> I'm a little busy right now to consider moving custard apple.  If you
> wan't you can always copy only the code in use into org.apache.river.impl
>
> Regards,
>
> Peter.
>
> On 10/08/2015 11:28 PM, Greg Trasuk wrote:
>
>> Peter:
>>
>> All other things equal, I’d like to keep the core of River free of
>> external dependencies, so I’m trying to understand what custard-apple is
>> for.
>>
>> First, I wonder, since you are the author, is it used solely in River?
>> and if so, if you wouldn’t like to just contribute the library to River so
>> we can have an integrated source tree.  That would save us having to figure
>> out how to handle the dependency and save you from having to maintain the
>> library externally (unless of course there are other users that need it).
>> Alternately, perhaps commons-collections would be a good home, that would
>> see a larger audience.
>>
>> Second, I’m trying to understand what the driver is for usage.  For
>> instance, I had a look in ServiceDiscoveryManager, and it looks like you’re
>> using a custard-apple collection, but I don’t see why it’s needed over a
>> ConcurrentHashMap.  Could you enlighten me?
>>
>> Thanks,
>>
>> Greg
>>
>> On Aug 10, 2015, at 7:53 AM, Peter<jini@zeus.net.au>  wrote:
>>>
>>> One more to add to the list to answer Greg's question,
>>> ReggieTestNameServiceProvider was removed from the river qa test sources to
>>> allow tests to run on platforms other than Sun's Jdk, such as J9, to
>>> prevent build compile errors.
>>>
>>> Again this library is only used for testing.
>>>
>>> asm isn't required for downstream projects using Maven.  asm is used for
>>> River's build for historical reasons to support classdep.
>>>
>>> If users don't need the performance improvement that
>>> CombinerSecurityManager provides, they won't require high-scale-lib.
>>> CombinerSecurityManager avoids duplicated security checks (from identical
>>> context) by weakly caching (by caller context's) the result of security
>>> checks, this reduces unnecessary network traffic caused by duplicated
>>> security checks.
>>>
>>> Only custard-apple is required.
>>>
>>> Peter.
>>>
>>> On 10/08/2015 6:04 PM, Peter wrote:
>>>
>>>> Pat,
>>>>
>>>> I don't have much time, but I'll assist you where I can.
>>>>
>>>> First things first, you'll need a Unix environment.
>>>>
>>>> I'd copy Dennis newly created branch to a 3.0 release branch, then run
>>>> the qa test suite and jtreg test suite.
>>>>
>>>> ant all.build
>>>> ant qa.run
>>>> cd ./qa
>>>> ant jtreg
>>>> cd ../
>>>> ant release
>>>>
>>>> To answer Greg's question:
>>>>
>>>>    The custard-apple library is available on Sourceforge, it's a
>>>>    Collections wrapper library that enables weak, soft and strong
>>>>    reference combinations in any java Collection implementation.  This
>>>>    library is required.  high-scale-lib is also required.
>>>>
>>>>    Test code in the jtreg test suite depends on the bouncy castle
>>>>    library to provide Certificate Authority functionality to sign
>>>>    certificates.  All certificates in other branches have expired, I'm
>>>>    sure there are other people on the dev list would have experienced
>>>>    and be aware of these test failures, these tests pass on this
>>>>    suite.  This library isn't required by down stream developers.
>>>>
>>>>    dnsjava is used in the qa test suite, it isn't required by down
>>>>    stream developers.
>>>>
>>>>    animal sniffer was again used for testing api changes and isn't
>>>>    required for down stream development.
>>>>
>>>>    Velocity was used by Sim and at least one other developer for
>>>>    configuration purposes, this library is optional.
>>>>
>>>>    Libraries required by test suites should stay for testing purposes
>>>>    but aren't required by release artifacts.
>>>>
>>>>
>>>> Regaring the current trunk, there are contributions made by Sim
>>>> (network and other improvements), after the stable branch point (chosen for
>>>> qa-refactor) that need to be integrated back at some point.  The test
>>>> failures after the trunk =>  qa-refactor branch point are latent
>>>> concurrency bugs, they are not directly related to Sim's code.
>>>>
>>>> This should help you get started...
>>>>
>>>> Regards,
>>>>
>>>> Peter.
>>>>
>>>> On 10/08/2015 2:00 PM, Patricia Shanahan wrote:
>>>>
>>>>> In that case, I'll take on the actual release manager role, and get
>>>>> going on dealing with the dependency issue.
>>>>>
>>>>> On 8/9/2015 8:38 PM, Greg Trasuk wrote:
>>>>>
>>>>>> Pat:
>>>>>>
>>>>>> I can provide support and information for you.  But I do think we
>>>>>> need to first sort out the dependencies question I pointed out earlier.
>>>>>>
>>>>>> Cheers,
>>>>>>
>>>>>> Greg Trasuk
>>>>>>
>>>>>> On Aug 9, 2015, at 9:58 PM, Patricia Shanahan<pats@acm.org>
 wrote:
>>>>>>>
>>>>>>> I am going to include the lack of a release manager for 3.0 in
the
>>>>>>> board report, and assign myself an action item to fix it.
>>>>>>>
>>>>>>> At this point I think my best bet is to appeal on the
>>>>>>> dev@community.apache.org mailing list for a mentor who is familiar
>>>>>>> with the release process to guide me through the steps. However,
I am very
>>>>>>> open to alternative suggestions.
>>>>>>>
>>>>>>> On 8/8/2015 8:22 AM, Patricia Shanahan wrote:
>>>>>>>
>>>>>>>> Peter? Anyone?
>>>>>>>>
>>>>>>>> I have time, but not the knowledge. I would be willing to
be release
>>>>>>>> manager provided at least one person who knows how it is
done will
>>>>>>>> provide a lot of step-by-step guidance.
>>>>>>>>
>>>>>>>> On 8/8/2015 7:21 AM, Bryan Thompson wrote:
>>>>>>>>
>>>>>>>>> Not really I am afraid.  I am quite heavily committed
on several
>>>>>>>>> projects
>>>>>>>>> right now.  I have done applications development with
river, but I
>>>>>>>>> am not
>>>>>>>>> familiar with the historical river project structure
in any depth
>>>>>>>>> and
>>>>>>>>> have
>>>>>>>>> never done development on river itself. And I have never
>>>>>>>>> participated
>>>>>>>>> in an
>>>>>>>>> Apache project release.  I am just not a good candidate
for this.
>>>>>>>>>
>>>>>>>>> Thanks,
>>>>>>>>> Bryan
>>>>>>>>>
>>>>>>>>> On Saturday, August 8, 2015, Patricia Shanahan<pats@acm.org>
>>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>> Bryan,
>>>>>>>>>>
>>>>>>>>>> Are you able and willing to act as release manager
for 3.0?
>>>>>>>>>>
>>>>>>>>>> On 8/6/2015 11:56 AM, Bryan Thompson wrote:
>>>>>>>>>>
>>>>>>>>>> Just release to encourage people to use it. +1 on
release.for me.
>>>>>>>>>>> On Aug 6, 2015 2:55 PM, "Patricia Shanahan"<pats@acm.org>
>>>>>>>>>>> wrote:
>>>>>>>>>>>
>>>>>>>>>>> Would it be useful to tag it as a 3.0 beta release
initially, or
>>>>>>>>>>> just go
>>>>>>>>>>>
>>>>>>>>>>>> to 3.0 and add point releases as needed?
>>>>>>>>>>>>
>>>>>>>>>>>> I will vote in favor of releasing it either
way.
>>>>>>>>>>>>
>>>>>>>>>>>> On 8/6/2015 9:55 AM, Bryan Thompson wrote:
>>>>>>>>>>>>
>>>>>>>>>>>> Or just release it.  If problems emerge,
people can report them
>>>>>>>>>>>> and
>>>>>>>>>>>> they
>>>>>>>>>>>>
>>>>>>>>>>>>> can get fixed.  There is a known reliable
release now.  Let the
>>>>>>>>>>>>> community
>>>>>>>>>>>>> vote by migrating their code.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Bryan
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Thu, Aug 6, 2015 at 12:41 PM, Patricia
Shanahan<
>>>>>>>>>>>>> pats@acm.org>
>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>> As far as I can tell, the main remaining
release blocker it
>>>>>>>>>>>>> getting more
>>>>>>>>>>>>>
>>>>>>>>>>>>> people to test in more environments.
Perhaps we should put out
>>>>>>>>>>>>>> an
>>>>>>>>>>>>>> appeal
>>>>>>>>>>>>>> on
>>>>>>>>>>>>>> user@river.apache.org?
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> On 8/6/2015 9:33 AM, Bryan Thompson
wrote:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Maybe it would be good to say a few
words about the release
>>>>>>>>>>>>>> goals
>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> 3.0?
>>>>>>>>>>>>>>> I.e., 3.0 provides a deep refactoring
that address
>>>>>>>>>>>>>>> performance,
>>>>>>>>>>>>>>> concurrency
>>>>>>>>>>>>>>> and scaling issues.  Significant
progress has been made
>>>>>>>>>>>>>>> towards the
>>>>>>>>>>>>>>> 3.0
>>>>>>>>>>>>>>> release target.  X new tests
have been developed for this
>>>>>>>>>>>>>>> release.
>>>>>>>>>>>>>>> The
>>>>>>>>>>>>>>> remaining blockers for a release
are ....
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>
>>
>

-- 
----
Bryan Thompson
Chief Scientist & Founder
SYSTAP, LLC
4501 Tower Road
Greensboro, NC 27410
bryan@systap.com
http://blazegraph.com
http://blog.bigdata.com <http://bigdata.com>
http://mapgraph.io

Blazegraph™ <http://www.blazegraph.com/> is our ultra high-performance
graph database that supports both RDF/SPARQL and Tinkerpop/Blueprints
APIs.  MapGraph™ <http://www.systap.com/mapgraph> is our disruptive new
technology to use GPUs to accelerate data-parallel graph analytics.

<http://smartdata2015.dataversity.net/>

CONFIDENTIALITY NOTICE:  This email and its contents and attachments are
for the sole use of the intended recipient(s) and are confidential or
proprietary to SYSTAP. Any unauthorized review, use, disclosure,
dissemination or copying of this email or its contents or attachments is
prohibited. If you have received this communication in error, please notify
the sender by reply email and permanently delete all copies of the email
and its contents and attachments.

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message