camel-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Sergey Beryozkin <sberyoz...@gmail.com>
Subject Re: Camel Azure component
Date Tue, 14 Feb 2017 11:26:54 GMT
Hi Claus

Thanks for the feedback,

On 14/02/17 08:08, Claus Ibsen wrote:
> Hi Sergey
>
> Good to see the commit and glad to hear you get time to work on Apache
> Camel as well.
>
> This a heads up that there has been a community user doing some Azure
> work on a PR
> https://github.com/apache/camel/pull/1213
>
> However there has not been any JIRA or information about what it was
> doing etc. But maybe you could take a look at the PR and see if it
> does something that our code dont do or something.

Right, I was not aware of it, I looked, at the moment there's no 
conflict, as I've started with Azure Blob Service and this PR 
concentrates on the Queue Service.

I don't know what to do in this case, I guess I will comment in that PR 
and invite the PR authors to initiate a discussion on how to add their 
code to the camel-azure, looks like it can go in without  the conflict, 
may be some coordination will be needed
Example, I started with a

asure-blob://camelazure/container1/blob?someOption

where the account name, the container name and the blob name form the 
URI path as opposed to

asure-blob://blob?account=camelazure&container=container1&someOption

as I thought the 1st option was more compact and better conveys the 
'must' requirement on the account & container names at least, but it is 
a minor style issue...

So I will comment there and lets see what happens
>
> For the azure component does it work in Karaf? Eg is there a karaf
> feature for it, and can the JARs be installed as bundles. If not we
> may consider having the SMX team create OSGi bundles of those JARs.
>
Will investigate yes
> Also the component need to be in the apache-camel kit, which you do in
> the parent and apache-camel folder. Just do a git grep for another
> component name and see how its done for that guy.
>
Sure

Cheers. Sergey
>
>
> On Mon, Feb 13, 2017 at 10:52 PM, Sergey Beryozkin <sberyozkin@gmail.com> wrote:
>> Yes, thanks, keep getting confused about the repos.
>>
>> So I merged now, and now I can feel less worried about boasting being a
>> Camel committer without having done a single commit :-).
>>
>> I've opened two more related issues: investigate what to do about the
>> integration tests and add the docs. My free account expires soon so I'll
>> play a bit more with Azure Blob Service or may be Queue before doing the
>> docs for the blob service and then once I'm done I suppose I will close
>> CAMEL-10786 and the Azure docs issue with more JIRAs to follow as needed
>> (for the queue, etc).
>>
>> Thanks, any concerns about my commit - let me know please
>>
>> Sergey
>>
>>
>> On 13/02/17 17:18, Colm O hEigeartaigh wrote:
>>>
>>> You could try cloning this repo instead:
>>> https://git-wip-us.apache.org/repos/asf/camel.git
>>>
>>> Colm.
>>>
>>> On Mon, Feb 13, 2017 at 4:43 PM, Sergey Beryozkin <sberyozkin@gmail.com>
>>> wrote:
>>>
>>>> Hi Claus, All
>>>>
>>>> Am I actually allowed to commit yet ?
>>>>
>>>> I tried to commit but got an
>>>> "ERROR: Permission to apache/camel.git denied to sberyozkin."
>>>>
>>>> 'git remote show origin' shows I cloned from:
>>>>
>>>> git@github.com:apache/camel.git
>>>>
>>>> Can you please double check the permissions ?
>>>>
>>>> Thanks, Sergey
>>>>
>>>>
>>>> On 10/02/17 14:40, Sergey Beryozkin wrote:
>>>>
>>>>> Hi Claus
>>>>>
>>>>> Thanks, I've opened another issue for investigating on how to enable
the
>>>>> Azure (integration) tests
>>>>> but I've also realized I do not have a simpler component/endpoint only
>>>>> test, so I'll work on it first and then I will commit
>>>>>
>>>>> Cheers, Sergey
>>>>> On 10/02/17 12:14, Claus Ibsen wrote:
>>>>>
>>>>>> Hi Sergey
>>>>>>
>>>>>> Yeah sure its good to get this out in the hands of the users. If
>>>>>> something is not yet implemented or something we can always state
that
>>>>>> in the docs.
>>>>>>
>>>>>> And if there are things you think still must be done its a good idea
>>>>>> to log a JIRA with that.
>>>>>>
>>>>>> For tests we sometimes have a maven profile that runs real integration
>>>>>> tests against an online account or something, and then regular unit
>>>>>> tests for just simpler tests.
>>>>>>
>>>>>> As we have more and more of these it may be worthwhile to look into
a
>>>>>> special jenkins job that only runs these integration tests with those
>>>>>> maven profiles enabled.
>>>>>>
>>>>>> But sure feel free to go ahead and get the code into the master branch.
>>>>>>
>>>>>> On Thu, Feb 9, 2017 at 6:42 PM, Sergey Beryozkin
>>>>>> <sberyozkin@gmail.com> wrote:
>>>>>>
>>>>>>> Hi
>>>>>>>
>>>>>>> I've finally attached a patch for the experts to have a glance,
I
>>>>>>> know I can
>>>>>>> commit and look forward to doing my first Camel commit, but it
is my
>>>>>>> first
>>>>>>> Camel component, so if something obvious is missing - let me
know
>>>>>>> please.
>>>>>>>
>>>>>>> It is hard to create a complete functional component from a start
but
>>>>>>> I hope
>>>>>>> it will be incrementally enhanced and improved.
>>>>>>> At the moment it covers an Azure Blob Service (with some tweaks
likely
>>>>>>> needed later on once we experiment more with it), I'll have a
look at
>>>>>>> adding
>>>>>>> a Queue service support later on.
>>>>>>>
>>>>>>> The most obvious thing which proved hard to solve is to have
the
>>>>>>> tests not
>>>>>>> ignored - I run all the tests which are there against the live
30-day
>>>>>>> free
>>>>>>> account called 'camelazure' but it is a private account nonetheless.
>>>>>>>
>>>>>>> I spent a lot of time on trying to mock the clients but the Azure
SDK
>>>>>>> creators made all of the clients final, so after messing with
>>>>>>> PowerMock I
>>>>>>> thought it was not worth it as it looks it can add some instability
>>>>>>> in the
>>>>>>> running tests. The idea of running the integration tests is also
on
>>>>>>> the hold
>>>>>>> given that they do not have a Maven plugin for starting the emulator
>>>>>>> (and no
>>>>>>> emulator for Linux yet).
>>>>>>>
>>>>>>> So I'd like to go ahead with having these tests disabled and
let
>>>>>>> those who
>>>>>>> are interested start playing with this component and provide
the
>>>>>>> feedback
>>>>>>> until we figure out how to test it in Jenkins...
>>>>>>>
>>>>>>> Claus, what do you think, would you be OK for the commit to go
to
>>>>>>> 2.19.0-SNAPSHOT in the current form ?
>>>>>>>
>>>>>>> Cheers, Sergey
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> On 05/02/17 17:06, Sergey Beryozkin wrote:
>>>>>>>
>>>>>>>>
>>>>>>>> Hi Guys,
>>>>>>>>
>>>>>>>> I've looked a bit more carefully at the Azure Emulator docs:
>>>>>>>>
>>>>>>>>
>>>>>>>> https://docs.microsoft.com/en-us/azure/storage/storage-use-e
>>>>>>>> mulator#start-and-initialize-the-storage-emulator
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> and I do not see now how it can be dynamically prepared to
get the
>>>>>>>> integration tests running, my apologies I did not check it
initially,
>>>>>>>> though if my earlier email would contribute to a Camel Windows
build
>>>>>>>> be
>>>>>>>> set up then it would be good :-). I believe Francesco managed
to do
>>>>>>>> it
>>>>>>>> quite easily for Syncope.
>>>>>>>>
>>>>>>>> I guess as far as this component is concerned I'd need to
start with
>>>>>>>> the
>>>>>>>> basic mock client to get some coverage done.
>>>>>>>>
>>>>>>>> Cheers, Sergey
>>>>>>>>
>>>>>>>> On 05/02/17 12:39, Jean-Baptiste Onofré wrote:
>>>>>>>>
>>>>>>>>>
>>>>>>>>> +1 for Windows build.
>>>>>>>>>
>>>>>>>>> Regards
>>>>>>>>> JB
>>>>>>>>>
>>>>>>>>> On Feb 3, 2017, 21:27, at 21:27, Willem Jiang
>>>>>>>>> <willem.jiang@gmail.com
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> +1 for setting up a Camel Windows build.
>>>>>>>>>> As some of our users are still using windows box.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Willem Jiang
>>>>>>>>>>
>>>>>>>>>> Blog: http://willemjiang.blogspot.com (English)
>>>>>>>>>>          http://jnn.iteye.com  (Chinese)
>>>>>>>>>> Twitter: willemjiang
>>>>>>>>>> Weibo: 姜宁willem
>>>>>>>>>>
>>>>>>>>>> On Fri, Feb 3, 2017 at 11:22 PM, Sergey Beryozkin
>>>>>>>>>> <sberyozkin@gmail.com>
>>>>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>> Hi
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> I've started prototyping a Camel Azure component
[1].
>>>>>>>>>>>
>>>>>>>>>>> Azure has a number of services, Blob, Queue,
Table and File, a
>>>>>>>>>>> lot of
>>>>>>>>>>> commands, etc.
>>>>>>>>>>> I'm starting with supporting a Blob Service.
Queue service will
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> follow,
>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> and I guess the component will keep evolving
to support other
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> services too.
>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> At the moment I have most of the Blob Service
commands covered at
>>>>>>>>>>> the
>>>>>>>>>>> Producer side. I'm not sure Consumer will need
to be there given
>>>>>>>>>>> that
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Azure
>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> Java API only offers an option to download the
blobs to an output
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> stream
>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> (file most likely). May be only for getting the
attributes if
>>>>>>>>>>> really
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> needed.
>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> I guess Queue Service will though require a Consumer
but that is a
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> stage 2.
>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> I'm using camel-aws component as a source of
ideas.
>>>>>>>>>>>
>>>>>>>>>>> The biggest issue it how to test it. I have several
basic 'live'
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> in-only
>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> tests based on the copy from camel-aws/s3 which
depend on a live
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> security
>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> key and the connection. Microsoft Azure offers
an emulator which
>>>>>>>>>>> can
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> only
>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> be run on Windows. And I'm not keen using a mock
client which can
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> become
>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> stale and won't really prove the blob has been
uploaded.
>>>>>>>>>>>
>>>>>>>>>>> I'd like to start with having these tests disabled
on the master
>>>>>>>>>>> for
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> those
>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> who are interested to experiment with them using
their own live
>>>>>>>>>>> keys
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> and in
>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> meantime I reckon we can either try to encourage
the Microsoft
>>>>>>>>>>> Azure
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> team
>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> to release their service emulator for the Linux
platform asap or
>>>>>>>>>>> set
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> up a
>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> Camel Windows build and run the emulator integration
test on
>>>>>>>>>>> Windows
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> only.
>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> Thanks, Sergey
>>>>>>>>>>>
>>>>>>>>>>> [1] https://issues.apache.org/jira/browse/CAMEL-10786
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>>
>>
>
>
>



Mime
View raw message