camel-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Raul Kripalani <r...@fusesource.com>
Subject Re: [DISCUSS] Camel and MongoDB
Date Fri, 17 Feb 2012 18:32:43 GMT
Hi Glen,

I requested a large account just to be on the safe side, which is in fact
5gb + 30gb soft.

I will clarify with MongoHQ what would happen if we exceed the soft limit.
Obviously we wouldn't want to be charged!

If we can go with the larger option, I would prefer that. At this point we
don't yet know what direction the component will take. Traction and
adoption may shoot up (that's what I hope for!) and we may need to code
more complex load tests, or support large binary data, etc. We still don't
know. So taking the larger plan now might save us future negotiations.

Additionally, due to MongoDB internals, it preallocates large memory
segments, starting off with e.g. 256mb and then multiplying x2 every time
it needs to expand. But I'm sure that MongoHQ refer to *actual* usage and
not *preallocated* usage on their pricing tables.

Regards,
Raúl.

On 17 February 2012 13:51, Glen Mazza <gmazza@talend.com> wrote:

> (Non-binding $0.02).  The Camel PMC might wish to consider creating a
> Special Thanks page, like CXF does (http://cxf.apache.org/**
> special-thanks.html <http://cxf.apache.org/special-thanks.html>) instead
> of scattering the thank-yous throughout the documentation.  (Or, do both
> possibly.)  I could imagine multiple companies providing support for other
> components and it might be cleaner to group them all in one place (possibly
> attracting more sponsors in the process.)
>
> Although generous of MongoHQ to offer their large-sized account,
> integration tests hopefully wouldn't require that much space--if the 2gb
> small account (https://mongohq.com/pricing) is all that Camel would
> conceivably need the team might wish to go for that instead.  Possible
> advantages of the small account is that the 2gb maximum limit is hard so no
> risk of going past 16gb and potentially incurring bills in the process and
> it would be easier for the integration tests to report back errors if any
> tests are continually adding data while forgetting to delete them
> afterwards, as we'll more quickly get out-of-space errors if that happens.
>
> Glen
>
>
>
> On 02/16/2012 10:09 AM, Raul Kripalani wrote:
>
>> Hi everybody,
>>
>> Spoke to MongoHQ, a company that hosts MongoDB databases on the cloud:
>> https://mongohq.com/home.
>>
>> They are happy to donate to Apache Camel a free MongoDB instance on the
>> cloud on a permanent basis (large account - 16gb storage), so that we can
>> run the component's automated integration tests against it without having
>> to worry about setting up a local instance.
>>
>> Is the PMC happy to go forth with this idea? If so, we'd write a thank you
>> note to MongoHQ with a link to their site on the component page if that's
>> okay?
>>
>> Regards,
>> Raúl.
>>
>> On 15 February 2012 18:28, Raul Kripalani<raul@fusesource.com>  wrote:
>>
>>  Hi everyone,
>>>
>>> Attached the source code of camel-mongodb to
>>> https://issues.apache.org/**jira/browse/CAMEL-4878<https://issues.apache.org/jira/browse/CAMEL-4878>for
inclusion in the
>>> Apache Camel codebase.
>>>
>>> Regards,
>>> Raúl Kripalani.
>>> FuseSource
>>>
>>> On 14 February 2012 12:45, Raul Kripalani<raul@fusesource.com>  wrote:
>>>
>>>  If there are to be run on a CI server, the pre-execution and
>>>> post-execution scripts of the build job would simply run and stop
>>>> MongoDB.
>>>> To run them in user machines, I have included the instructions in the
>>>> README.txt in the component's source.
>>>>
>>>> The Unit Test classes detect whether MongoDB is running and if not, they
>>>> skip the testing. So for users not running an instance of MongoDB
>>>> locally
>>>> or elsewhere (to which the test config file points to), their builds
>>>> would
>>>> just skip the unit tests.
>>>>
>>>> Regards,
>>>> Raúl.
>>>>
>>>> On 14 February 2012 12:24, Bilgin Ibryam<bibryam@gmail.com>  wrote:
>>>>
>>>>  On 14 February 2012 12:13, Raul Kripalani<raul@fusesource.com>  wrote:
>>>>>
>>>>>  Hi Bilgin,
>>>>>>
>>>>>> Apologies for the delay in my reply.
>>>>>>
>>>>>> Regarding the unit test depending on an installation of MongoDB,
there
>>>>>>
>>>>> is
>>>>>
>>>>>> really no way out. There is nothing out there to support an embedded
>>>>>> instance of MongoDB (have spent hours searching). Moreover, MongoDB
is
>>>>>> really a C application, so even if we were to automate fetching and
>>>>>> installing MongoDB on a temp directory as part of the test process,
it
>>>>>> would be some hairy logic because you first need to detect what
>>>>>>
>>>>> platform,
>>>>>
>>>>>> environment and bitness the test is running. Overall, the test would
>>>>>>
>>>>> have a
>>>>>
>>>>>> higher chance of failing than the code itself :P
>>>>>>
>>>>>>  hmmm. I wonder how these test would run on CI server or on user
>>>>> machienes.
>>>>> May be there is way that Iam not aware of.
>>>>>
>>>>>  Mocking it out wasn't really an option here as it provides no value
in
>>>>>>
>>>>> this
>>>>>
>>>>>> scenario IMHO.
>>>>>>
>>>>>> Concerning the constant interface pattern, I don't think it's the
way
>>>>>>
>>>>> to
>>>>>
>>>>>> go. It has been named an anti-pattern in a Sun book:
>>>>>>
>>>>>>
>>>>>>  http://stackoverflow.com/**questions/2659593/what-is-the-**
>>>>> use-of-interface-constants<http://stackoverflow.com/questions/2659593/what-is-the-use-of-interface-constants>
>>>>>
>>>>>> .
>>>>>>
>>>>>>  ok, I just notice the repetition of "public static final" on each
>>>>> line,
>>>>> also the new components that are added to Camel seems to be using
>>>>> interfaces... I'm fine with both versions really.
>>>>>
>>>>>
>>>>>  Thanks for the feedback!
>>>>>>
>>>>>> Regards,
>>>>>> Raúl.
>>>>>>
>>>>>> On 3 February 2012 14:19, Bilgin Ibryam<bibryam@gmail.com>
 wrote:
>>>>>>
>>>>>>  Hi Raul,
>>>>>>>
>>>>>>> I had a look and the mongodb component, it looks ready, great
job.
>>>>>>> Few notes:
>>>>>>>
>>>>>>> I think the unit test should not depend on existing mongodb
>>>>>>> installation, it would be great if there is an inmemory or mock
>>>>>>> implementation and the tests start it up for example.
>>>>>>>
>>>>>>> Minor, you can make MongoDbConstants an interface with constants
>>>>>>> And this fields seems not to be used INSERT_RECORDS_AFFECTED
>>>>>>>
>>>>>>> Cheers,
>>>>>>> Bilgin
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> On 27 January 2012 14:19, Raul Kripalani<raul@fusesource.com>
>>>>>>>
>>>>>> wrote:
>>>>>
>>>>>> Hi everybody,
>>>>>>>>
>>>>>>>> The source for the camel-mongodb component is available here:
Camel
>>>>>>>> MongoDB<https://github.com/**raulk/camel-mongodb<https://github.com/raulk/camel-mongodb>
>>>>>>>> >(
>>>>>>>> https://github.com/raulk/**camel-mongodb<https://github.com/raulk/camel-mongodb>
>>>>>>>> ).
>>>>>>>>
>>>>>>>> A few items are still pending on my task list, namely: Javadoc
and
>>>>>>>> reordering of methods to match the (1) getters/setters and
(2)
>>>>>>>> implementation methods pattern. But at this point I wanted
to give
>>>>>>>>
>>>>>>> you
>>>>>
>>>>>> a
>>>>>>
>>>>>>> sneak peek into it. Please feel free to comment on the code.
>>>>>>>>
>>>>>>>> What's the timeline for the 2.10 release? I'd be great if
this
>>>>>>>>
>>>>>>> component
>>>>>>
>>>>>>> makes it into it.
>>>>>>>>
>>>>>>>> Regards,
>>>>>>>> Raúl Kripalani (@raulvk<https://twitter.com/**raulvk<https://twitter.com/raulvk>
>>>>>>>> >)
>>>>>>>> FuseSource
>>>>>>>>
>>>>>>>> On 10 January 2012 02:48, Raul Kripalani<raul@fusesource.com>
>>>>>>>>
>>>>>>> wrote:
>>>>>
>>>>>>  Ticket created: https://issues.apache.org/**jira/browse/CAMEL-4878<https://issues.apache.org/jira/browse/CAMEL-4878>
>>>>>>>>> .
>>>>>>>>>
>>>>>>>>> Bilgin, I'm already working on the code. Will let you
know in
>>>>>>>>>
>>>>>>>> case I
>>>>>
>>>>>> need
>>>>>>>
>>>>>>>> help. Thanks!
>>>>>>>>>
>>>>>>>>> -- Raúl.
>>>>>>>>>
>>>>>>>>> On 9 January 2012 10:38, Bilgin Ibryam<bibryam@gmail.com>
 wrote:
>>>>>>>>>
>>>>>>>>>  Hi Raul,
>>>>>>>>>>
>>>>>>>>>>  For a first embracement of MongoDB, it seems more
natural to
>>>>>>>>>>>
>>>>>>>>>> implement a
>>>>>>>
>>>>>>>>  raw component which can handle at least the following MongoDB
>>>>>>>>>>> operations: insert, findOne, count, findAll,
query, mapReduce,
>>>>>>>>>>>
>>>>>>>>>> getStats,
>>>>>>>
>>>>>>>>  group, save, update, remove, updateMulti.
>>>>>>>>>>>
>>>>>>>>>>>  If you start the development using camel-jira
I can also help
>>>>>>>>>> for
>>>>>>>>>> testing and coding...
>>>>>>>>>>
>>>>>>>>>>  Then we can start thinking about Morphia.
>>>>>>>>>>>
>>>>>>>>>>> Do you agree?
>>>>>>>>>>>
>>>>>>>>>> Makes sense.
>>>>>>>>>>
>>>>>>>>>> Cheers
>>>>>>>>>> Bilgin
>>>>>>>>>>
>>>>>>>>>>  Regards,
>>>>>>>>>>> Raúl.
>>>>>>>>>>>
>>>>>>>>>>> On 4 January 2012 09:35, Ioannis Canellos<iocanel@gmail.com>
>>>>>>>>>>>
>>>>>>>>>> wrote:
>>>>>>
>>>>>>>  I've used morphia a lot last year and I was very happy with
>>>>>>>>>>>>
>>>>>>>>>>> it.
>>>>>
>>>>>>  However, I
>>>>>>>>>>
>>>>>>>>>>> think that we should not base a mongodb component
on that.
>>>>>>>>>>>>
>>>>>>>>>>>> I think that we need a pure camel-mongodb
component and we
>>>>>>>>>>>>
>>>>>>>>>>> could
>>>>>
>>>>>> have a
>>>>>>>
>>>>>>>>  morphia dataformat or a separate morphia component.
>>>>>>>>>>>>
>>>>>>>>>>>> --
>>>>>>>>>>>> *Ioannis Canellos*
>>>>>>>>>>>> *
>>>>>>>>>>>> FuseSource<http://fusesource.**com <http://fusesource.com>>
>>>>>>>>>>>>
>>>>>>>>>>>> **
>>>>>>>>>>>> Blog: http://iocanel.blogspot.com
>>>>>>>>>>>> **
>>>>>>>>>>>> Apache Karaf<http://karaf.apache.org/**>
 Committer&  PMC
>>>>>>>>>>>>
>>>>>>>>>>>> Apache Camel<http://camel.apache.org/**>
 Committer
>>>>>>>>>>>> Apache ServiceMix<http://servicemix.**apache.org/<http://servicemix.apache.org/>>
>>>>>>>>>>>>   Committer
>>>>>>>>>>>> Apache Gora<http://incubator.apache.**org/gora/<http://incubator.apache.org/gora/>>
>>>>>>>>>>>>  Committer
>>>>>>>>>>>> Apache DirectMemory<
>>>>>>>>>>>>
>>>>>>>>>>> http://incubator.apache.org/**directmemory/<http://incubator.apache.org/directmemory/>
>>>>> >
>>>>>
>>>>>>  Committer
>>>>>>>>>>
>>>>>>>>>>> *
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>
>>>>
>
> --
> Glen Mazza
> Talend Community Coders - coders.talend.com
> blog: www.jroller.com/gmazza
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message