camel-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Claus Ibsen <claus.ib...@gmail.com>
Subject Re: Camel Azure component
Date Tue, 14 Feb 2017 08:08:26 GMT
Hi Sergey

Good to see the commit and glad to hear you get time to work on Apache
Camel as well.

This a heads up that there has been a community user doing some Azure
work on a PR
https://github.com/apache/camel/pull/1213

However there has not been any JIRA or information about what it was
doing etc. But maybe you could take a look at the PR and see if it
does something that our code dont do or something.

For the azure component does it work in Karaf? Eg is there a karaf
feature for it, and can the JARs be installed as bundles. If not we
may consider having the SMX team create OSGi bundles of those JARs.

Also the component need to be in the apache-camel kit, which you do in
the parent and apache-camel folder. Just do a git grep for another
component name and see how its done for that guy.



On Mon, Feb 13, 2017 at 10:52 PM, Sergey Beryozkin <sberyozkin@gmail.com> wrote:
> Yes, thanks, keep getting confused about the repos.
>
> So I merged now, and now I can feel less worried about boasting being a
> Camel committer without having done a single commit :-).
>
> I've opened two more related issues: investigate what to do about the
> integration tests and add the docs. My free account expires soon so I'll
> play a bit more with Azure Blob Service or may be Queue before doing the
> docs for the blob service and then once I'm done I suppose I will close
> CAMEL-10786 and the Azure docs issue with more JIRAs to follow as needed
> (for the queue, etc).
>
> Thanks, any concerns about my commit - let me know please
>
> Sergey
>
>
> On 13/02/17 17:18, Colm O hEigeartaigh wrote:
>>
>> You could try cloning this repo instead:
>> https://git-wip-us.apache.org/repos/asf/camel.git
>>
>> Colm.
>>
>> On Mon, Feb 13, 2017 at 4:43 PM, Sergey Beryozkin <sberyozkin@gmail.com>
>> wrote:
>>
>>> Hi Claus, All
>>>
>>> Am I actually allowed to commit yet ?
>>>
>>> I tried to commit but got an
>>> "ERROR: Permission to apache/camel.git denied to sberyozkin."
>>>
>>> 'git remote show origin' shows I cloned from:
>>>
>>> git@github.com:apache/camel.git
>>>
>>> Can you please double check the permissions ?
>>>
>>> Thanks, Sergey
>>>
>>>
>>> On 10/02/17 14:40, Sergey Beryozkin wrote:
>>>
>>>> Hi Claus
>>>>
>>>> Thanks, I've opened another issue for investigating on how to enable the
>>>> Azure (integration) tests
>>>> but I've also realized I do not have a simpler component/endpoint only
>>>> test, so I'll work on it first and then I will commit
>>>>
>>>> Cheers, Sergey
>>>> On 10/02/17 12:14, Claus Ibsen wrote:
>>>>
>>>>> Hi Sergey
>>>>>
>>>>> Yeah sure its good to get this out in the hands of the users. If
>>>>> something is not yet implemented or something we can always state that
>>>>> in the docs.
>>>>>
>>>>> And if there are things you think still must be done its a good idea
>>>>> to log a JIRA with that.
>>>>>
>>>>> For tests we sometimes have a maven profile that runs real integration
>>>>> tests against an online account or something, and then regular unit
>>>>> tests for just simpler tests.
>>>>>
>>>>> As we have more and more of these it may be worthwhile to look into a
>>>>> special jenkins job that only runs these integration tests with those
>>>>> maven profiles enabled.
>>>>>
>>>>> But sure feel free to go ahead and get the code into the master branch.
>>>>>
>>>>> On Thu, Feb 9, 2017 at 6:42 PM, Sergey Beryozkin
>>>>> <sberyozkin@gmail.com> wrote:
>>>>>
>>>>>> Hi
>>>>>>
>>>>>> I've finally attached a patch for the experts to have a glance, I
>>>>>> know I can
>>>>>> commit and look forward to doing my first Camel commit, but it is
my
>>>>>> first
>>>>>> Camel component, so if something obvious is missing - let me know
>>>>>> please.
>>>>>>
>>>>>> It is hard to create a complete functional component from a start
but
>>>>>> I hope
>>>>>> it will be incrementally enhanced and improved.
>>>>>> At the moment it covers an Azure Blob Service (with some tweaks likely
>>>>>> needed later on once we experiment more with it), I'll have a look
at
>>>>>> adding
>>>>>> a Queue service support later on.
>>>>>>
>>>>>> The most obvious thing which proved hard to solve is to have the
>>>>>> tests not
>>>>>> ignored - I run all the tests which are there against the live 30-day
>>>>>> free
>>>>>> account called 'camelazure' but it is a private account nonetheless.
>>>>>>
>>>>>> I spent a lot of time on trying to mock the clients but the Azure
SDK
>>>>>> creators made all of the clients final, so after messing with
>>>>>> PowerMock I
>>>>>> thought it was not worth it as it looks it can add some instability
>>>>>> in the
>>>>>> running tests. The idea of running the integration tests is also
on
>>>>>> the hold
>>>>>> given that they do not have a Maven plugin for starting the emulator
>>>>>> (and no
>>>>>> emulator for Linux yet).
>>>>>>
>>>>>> So I'd like to go ahead with having these tests disabled and let
>>>>>> those who
>>>>>> are interested start playing with this component and provide the
>>>>>> feedback
>>>>>> until we figure out how to test it in Jenkins...
>>>>>>
>>>>>> Claus, what do you think, would you be OK for the commit to go to
>>>>>> 2.19.0-SNAPSHOT in the current form ?
>>>>>>
>>>>>> Cheers, Sergey
>>>>>>
>>>>>>
>>>>>>
>>>>>> On 05/02/17 17:06, Sergey Beryozkin wrote:
>>>>>>
>>>>>>>
>>>>>>> Hi Guys,
>>>>>>>
>>>>>>> I've looked a bit more carefully at the Azure Emulator docs:
>>>>>>>
>>>>>>>
>>>>>>> https://docs.microsoft.com/en-us/azure/storage/storage-use-e
>>>>>>> mulator#start-and-initialize-the-storage-emulator
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> and I do not see now how it can be dynamically prepared to get
the
>>>>>>> integration tests running, my apologies I did not check it initially,
>>>>>>> though if my earlier email would contribute to a Camel Windows
build
>>>>>>> be
>>>>>>> set up then it would be good :-). I believe Francesco managed
to do
>>>>>>> it
>>>>>>> quite easily for Syncope.
>>>>>>>
>>>>>>> I guess as far as this component is concerned I'd need to start
with
>>>>>>> the
>>>>>>> basic mock client to get some coverage done.
>>>>>>>
>>>>>>> Cheers, Sergey
>>>>>>>
>>>>>>> On 05/02/17 12:39, Jean-Baptiste Onofré wrote:
>>>>>>>
>>>>>>>>
>>>>>>>> +1 for Windows build.
>>>>>>>>
>>>>>>>> Regards
>>>>>>>> JB
>>>>>>>>
>>>>>>>> On Feb 3, 2017, 21:27, at 21:27, Willem Jiang
>>>>>>>> <willem.jiang@gmail.com
>>>>>>>>>
>>>>>>>>>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>>
>>>>>>>>> +1 for setting up a Camel Windows build.
>>>>>>>>> As some of our users are still using windows box.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Willem Jiang
>>>>>>>>>
>>>>>>>>> Blog: http://willemjiang.blogspot.com (English)
>>>>>>>>>          http://jnn.iteye.com  (Chinese)
>>>>>>>>> Twitter: willemjiang
>>>>>>>>> Weibo: 姜宁willem
>>>>>>>>>
>>>>>>>>> On Fri, Feb 3, 2017 at 11:22 PM, Sergey Beryozkin
>>>>>>>>> <sberyozkin@gmail.com>
>>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>> Hi
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> I've started prototyping a Camel Azure component
[1].
>>>>>>>>>>
>>>>>>>>>> Azure has a number of services, Blob, Queue, Table
and File, a
>>>>>>>>>> lot of
>>>>>>>>>> commands, etc.
>>>>>>>>>> I'm starting with supporting a Blob Service. Queue
service will
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>> follow,
>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> and I guess the component will keep evolving to support
other
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>> services too.
>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> At the moment I have most of the Blob Service commands
covered at
>>>>>>>>>> the
>>>>>>>>>> Producer side. I'm not sure Consumer will need to
be there given
>>>>>>>>>> that
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Azure
>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Java API only offers an option to download the blobs
to an output
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>> stream
>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> (file most likely). May be only for getting the attributes
if
>>>>>>>>>> really
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>> needed.
>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> I guess Queue Service will though require a Consumer
but that is a
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>> stage 2.
>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> I'm using camel-aws component as a source of ideas.
>>>>>>>>>>
>>>>>>>>>> The biggest issue it how to test it. I have several
basic 'live'
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>> in-only
>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> tests based on the copy from camel-aws/s3 which depend
on a live
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>> security
>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> key and the connection. Microsoft Azure offers an
emulator which
>>>>>>>>>> can
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>> only
>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> be run on Windows. And I'm not keen using a mock
client which can
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>> become
>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> stale and won't really prove the blob has been uploaded.
>>>>>>>>>>
>>>>>>>>>> I'd like to start with having these tests disabled
on the master
>>>>>>>>>> for
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>> those
>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> who are interested to experiment with them using
their own live
>>>>>>>>>> keys
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>> and in
>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> meantime I reckon we can either try to encourage
the Microsoft
>>>>>>>>>> Azure
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>> team
>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> to release their service emulator for the Linux platform
asap or
>>>>>>>>>> set
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>> up a
>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Camel Windows build and run the emulator integration
test on
>>>>>>>>>> Windows
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>> only.
>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Thanks, Sergey
>>>>>>>>>>
>>>>>>>>>> [1] https://issues.apache.org/jira/browse/CAMEL-10786
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>>
>>>>
>>>
>>
>>
>



-- 
Claus Ibsen
-----------------
http://davsclaus.com @davsclaus
Camel in Action 2: https://www.manning.com/ibsen2

Mime
View raw message