stratos-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Chamila De Alwis <chami...@wso2.com>
Subject Re: [Discuss] Fixing Python Cartridge Agent (PCA) Issues
Date Fri, 13 Feb 2015 04:31:05 GMT
Hi Imesh,

I have almost completed the work to switch to gitpython. However there is
an issue with the type of the auth url, that is given to GitPython. Other
than that, public repo cloning now works with the library.

When a url with credentials is passed, I'm getting the following error.
Investigating further.

[2015-02-13 09:59:13,200] ERROR {eventsubscriber.py:run} - Error processing
'ArtifactUpdatedEvent' event
Traceback (most recent call last):
  File
"/home/chamilad/dev/stratos/components/org.apache.stratos.python.cartridge.agent/target/10c657f1-a3a4-4aee-bfe3-949fda7346a9/cartridge.agent/modules/subscriber/eventsubscriber.py",
line 103, in run
    handler(event_msg)
  File
"/home/chamilad/dev/stratos/components/org.apache.stratos.python.cartridge.agent/target/test-classes/../10c657f1-a3a4-4aee-bfe3-949fda7346a9/cartridge.agent/agent.py",
line 185, in on_artifact_updated
    CartridgeAgent.extension_handler.on_artifact_updated_event(event_obj)
  File
"/home/chamilad/dev/stratos/components/org.apache.stratos.python.cartridge.agent/target/10c657f1-a3a4-4aee-bfe3-949fda7346a9/cartridge.agent/modules/extensions/extensionhandler.py",
line 122, in on_artifact_updated_event
    subscribe_run, updated =
agentgithandler.AgentGitHandler.checkout(repo_info)
  File
"/home/chamilad/dev/stratos/components/org.apache.stratos.python.cartridge.agent/target/10c657f1-a3a4-4aee-bfe3-949fda7346a9/cartridge.agent/modules/artifactmgt/git/agentgithandler.py",
line 89, in checkout
    git_repo = AgentGitHandler.clone(git_repo)
  File
"/home/chamilad/dev/stratos/components/org.apache.stratos.python.cartridge.agent/target/10c657f1-a3a4-4aee-bfe3-949fda7346a9/cartridge.agent/modules/artifactmgt/git/agentgithandler.py",
line 192, in clone
    Repo.clone_from(unicode(git_repo.repo_url, "utf-8"),
unicode(git_repo.local_repo_path, "utf-8"))
  File "/usr/local/lib/python2.7/dist-packages/git/repo/base.py", line 844,
in clone_from
    return cls._clone(Git(os.getcwd()), url, to_path, GitCmdObjectDB,
progress, **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/git/repo/base.py", line 791,
in _clone
    v=True, **add_progress(kwargs, git, progress))
  File "/usr/local/lib/python2.7/dist-packages/git/cmd.py", line 431, in
<lambda>
    return lambda *args, **kwargs: self._call_process(name, *args, **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/git/cmd.py", line 802, in
_call_process
    return self.execute(make_call(), **_kwargs)
  File "/usr/local/lib/python2.7/dist-packages/git/cmd.py", line 556, in
execute
    **subprocess_kwargs
  File "/usr/lib/python2.7/subprocess.py", line 710, in __init__
    errread, errwrite)
  File "/usr/lib/python2.7/subprocess.py", line 1327, in _execute_child
    raise child_exception
TypeError: execve() arg 2 must contain only strings


Regards,
Chamila de Alwis
Software Engineer | WSO2 | +94772207163
Blog: code.chamiladealwis.com



On Fri, Feb 13, 2015 at 9:30 AM, Imesh Gunaratne <imesh@apache.org> wrote:

> I analyzed the issue the NoneType error has occurred due to the EOF
> exception which has been raised while executing git clone shell command:
>
> The problem is in the following line:
>
> git_repo = AgentGitHandler.clone(git_repo)
> AgentGitHandler.log.debug("Git clone executed: [tenant-id] %s [repo-url] %s",
>                           git_repo.tenant_id, git_repo.repo_url)
>
> Here clone() method returns NoneType if clone fails. Anyway the root cause of the problem
is with the Git client implementation. Will switch to GitPython to avoid this intermittent
issue of EOF.
>
> Thanks
>
>
> On Fri, Feb 13, 2015 at 9:20 AM, Imesh Gunaratne <imesh@apache.org> wrote:
>
>> I can see the tenant id attribute in received message, may be it was not
>> properly set:
>>
>> [2015-02-13 03:44:06,394] DEBUG {eventsubscriber.py:on_message} - Message
>> received: instance/notifier/ArtifactUpdatedEvent:
>>
>> {"clusterId":"php1.php.domain","repoUserName":"","repoPassword":"","repoURL":"
>> https://github.com/imesh/stratos-php-applications.git
>> ","tenantId":"-1234","commitEnabled":false}
>>
>> On Fri, Feb 13, 2015 at 9:19 AM, Imesh Gunaratne <imesh@apache.org>
>> wrote:
>>
>>> Also the following error was there in the log:
>>>
>>> [2015-02-13 03:44:28,498] ERROR {agentgithandler.py:clone} - Exception
>>> while executing git clone command on git binary : EOF("End Of File (EOF).
>>> Exception style platform.\n<pexpect.spawn object at 0x25c17d0>\nversion:
>>> 3.3\ncommand: /usr/bin/git\nargs: ['/usr/bin/git', 'clone', '
>>> https://github.com/imesh/stratos-php-applications.git',
>>> '/var/www']\nsearcher: <pexpect.searcher_re object at 0x25c18d0>\nbuffer
>>> (last 100 chars): ''\nbefore (last 100 chars): 'ng deltas:  98% (71/72)
>>> \\rResolving deltas: 100% (72/72)   \\rResolving deltas: 100% (72/72),
>>> done.\\r\\n'\nafter: <class 'pexpect.EOF'>\nmatch: None\nmatch_index:
>>> None\nexitstatus: None\nflag_eof: True\npid: 63\nchild_fd: 11\nclosed:
>>> False\ntimeout: 120\ndelimiter: <class 'pexpect.EOF'>\nlogfile:
>>> None\nlogfile_read: None\nlogfile_send: None\nmaxread: 2000\nignorecase:
>>> False\nsearchwindowsize: None\ndelaybeforesend: 0.05\ndelayafterclose:
>>> 0.1\ndelayafterterminate: 0.1",)
>>> Traceback (most recent call last):
>>>   File
>>> "/mnt/apache-stratos-python-cartridge-agent-4.1.0-SNAPSHOT/modules/artifactmgt/git/agentgithandler.py",
>>> line 200, in clone
>>>     result = clone_op.expect(["Username for .*", "Password for .*",
>>> "Checking connectivity... done."])
>>>   File "/usr/local/lib/python2.7/dist-packages/pexpect/__init__.py",
>>> line 1451, in expect
>>>     timeout, searchwindowsize)
>>>   File "/usr/local/lib/python2.7/dist-packages/pexpect/__init__.py",
>>> line 1466, in expect_list
>>>     timeout, searchwindowsize)
>>>   File "/usr/local/lib/python2.7/dist-packages/pexpect/__init__.py",
>>> line 1554, in expect_loop
>>>     raise EOF(str(err) + '\n' + str(self))
>>> EOF: End Of File (EOF). Exception style platform.
>>> <pexpect.spawn object at 0x25c17d0>
>>>
>>> On Fri, Feb 13, 2015 at 9:17 AM, Imesh Gunaratne <imesh@apache.org>
>>> wrote:
>>>
>>>> The problem was in populate-user-data.sh file [1], APPLICATION_PATH has
>>>> been removed from it. I just fixed it, pushed the new PHP docker image to
>>>> docker hub and did another flow test and found the below error:
>>>>
>>>> [2015-02-13 03:44:28,603] ERROR {eventsubscriber.py:run} - Error
>>>> processing 'ArtifactUpdatedEvent' event
>>>> Traceback (most recent call last):
>>>>   File
>>>> "/mnt/apache-stratos-python-cartridge-agent-4.1.0-SNAPSHOT/modules/subscriber/eventsubscriber.py",
>>>> line 103, in run
>>>>     handler(event_msg)
>>>>   File "agent.py", line 185, in on_artifact_updated
>>>>
>>>> CartridgeAgent.extension_handler.on_artifact_updated_event(event_obj)
>>>>   File
>>>> "/mnt/apache-stratos-python-cartridge-agent-4.1.0-SNAPSHOT/modules/extensions/extensionhandler.py",
>>>> line 122, in on_artifact_updated_event
>>>>     subscribe_run, updated =
>>>> agentgithandler.AgentGitHandler.checkout(repo_info)
>>>>   File
>>>> "/mnt/apache-stratos-python-cartridge-agent-4.1.0-SNAPSHOT/modules/artifactmgt/git/agentgithandler.py",
>>>> line 91, in checkout
>>>>     git_repo.tenant_id, git_repo.repo_url)
>>>> AttributeError: 'NoneType' object has no attribute 'tenant_id'
>>>>
>>>> [1]
>>>> https://github.com/apache/stratos/blob/master/tools/docker-images/cartridge-docker-images/base-image/files/populate-user-data.sh
>>>>
>>>> On Fri, Feb 13, 2015 at 7:43 AM, Imesh Gunaratne <imesh@apache.org>
>>>> wrote:
>>>>
>>>>> Thanks Lakmal, yes as I found basic features we need works with Git
>>>>> python. Will do that.
>>>>> Regarding the above error, seems like it has occurred due to the
>>>>> APPLICATION_PATH not being sent in payload, there was no None check or
a
>>>>> log saying its not found:
>>>>>
>>>>> def clone(git_repo):
>>>>>     if os.path.isdir(git_repo.local_repo_path):
>>>>>         # delete and recreate local repo path if exists
>>>>>         GitUtils.delete_folder_tree(git_repo.local_repo_path)
>>>>>
>>>>>
>>>>> On Fri, Feb 13, 2015 at 7:37 AM, Lakmal Warusawithana <lakmal@wso2.com
>>>>> > wrote:
>>>>>
>>>>>>
>>>>>>
>>>>>> On Fri, Feb 13, 2015 at 7:34 AM, Imesh Gunaratne <imesh@apache.org>
>>>>>> wrote:
>>>>>>
>>>>>>> As it looks like this error has been caused by the agent git
handler
>>>>>>> while executing git clone, IMO its better if we can switch to
GitPython.
>>>>>>>
>>>>>>>
>>>>>> +1, If GitPython has full functionality what we expect, lets move
to
>>>>>> it. I was under impression that it has some limitation. If so please
not
>>>>>> waist time to fix current one.
>>>>>>
>>>>>>
>>>>>>
>>>>>>> Thanks
>>>>>>>
>>>>>>> On Fri, Feb 13, 2015 at 7:31 AM, Imesh Gunaratne <imesh@apache.org>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> Hi Devs,
>>>>>>>>
>>>>>>>> Yesterday we fixed several issues in PCA, now I'm seeing
the below
>>>>>>>> error when trying to receive artifact updated event:
>>>>>>>>
>>>>>>>> [2015-02-13 01:56:07,311] ERROR {eventsubscriber.py:run}
- Error
>>>>>>>> processing 'ArtifactUpdatedEvent' event
>>>>>>>> Traceback (most recent call last):
>>>>>>>>   File
>>>>>>>> "/mnt/apache-stratos-python-cartridge-agent-4.1.0-SNAPSHOT/modules/subscriber/eventsubscriber.py",
>>>>>>>> line 103, in run
>>>>>>>>     handler(event_msg)
>>>>>>>>   File "agent.py", line 185, in on_artifact_updated
>>>>>>>>
>>>>>>>> CartridgeAgent.extension_handler.on_artifact_updated_event(event_obj)
>>>>>>>>   File
>>>>>>>> "/mnt/apache-stratos-python-cartridge-agent-4.1.0-SNAPSHOT/modules/extensions/extensionhandler.py",
>>>>>>>> line 122, in on_artifact_updated_event
>>>>>>>>     subscribe_run, updated =
>>>>>>>> agentgithandler.AgentGitHandler.checkout(repo_info)
>>>>>>>>   File
>>>>>>>> "/mnt/apache-stratos-python-cartridge-agent-4.1.0-SNAPSHOT/modules/artifactmgt/git/agentgithandler.py",
>>>>>>>> line 89, in checkout
>>>>>>>>     git_repo = AgentGitHandler.clone(git_repo)
>>>>>>>>   File
>>>>>>>> "/mnt/apache-stratos-python-cartridge-agent-4.1.0-SNAPSHOT/modules/artifactmgt/git/agentgithandler.py",
>>>>>>>> line 190, in clone
>>>>>>>>     if os.path.isdir(git_repo.local_repo_path):
>>>>>>>>   File "/usr/lib/python2.7/genericpath.py", line 41, in isdir
>>>>>>>>     st = os.stat(s)
>>>>>>>> TypeError: coercing to Unicode: need string or buffer, NoneType
foun
>>>>>>>>
>>>>>>>> Thanks
>>>>>>>>
>>>>>>>>
>>>>>>>> --
>>>>>>>> Imesh Gunaratne
>>>>>>>>
>>>>>>>> Technical Lead, WSO2
>>>>>>>> Committer & PMC Member, Apache Stratos
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>> Imesh Gunaratne
>>>>>>>
>>>>>>> Technical Lead, WSO2
>>>>>>> Committer & PMC Member, Apache Stratos
>>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> Lakmal Warusawithana
>>>>>> Vice President, Apache Stratos
>>>>>> Director - Cloud Architecture; WSO2 Inc.
>>>>>> Mobile : +94714289692
>>>>>> Blog : http://lakmalsview.blogspot.com/
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Imesh Gunaratne
>>>>>
>>>>> Technical Lead, WSO2
>>>>> Committer & PMC Member, Apache Stratos
>>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> Imesh Gunaratne
>>>>
>>>> Technical Lead, WSO2
>>>> Committer & PMC Member, Apache Stratos
>>>>
>>>
>>>
>>>
>>> --
>>> Imesh Gunaratne
>>>
>>> Technical Lead, WSO2
>>> Committer & PMC Member, Apache Stratos
>>>
>>
>>
>>
>> --
>> Imesh Gunaratne
>>
>> Technical Lead, WSO2
>> Committer & PMC Member, Apache Stratos
>>
>
>
>
> --
> Imesh Gunaratne
>
> Technical Lead, WSO2
> Committer & PMC Member, Apache Stratos
>

Mime
View raw message