ignite-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Vyacheslav Daradur <daradu...@gmail.com>
Subject Re: Service grid redesign
Date Wed, 21 Nov 2018 08:39:32 GMT
During each deployment (exchange) process nodes send to the
coordinator (p2p) deployments maps, caused:
- users requests (deploy/undeploy)
- affinity change (if affinity services present)
- topology change (if services present)
- activation (if services descriptors present)

Also, single node deployment results contain deployment errors which
aggregate by coordinator and propagates across the cluster. And this
is important for introduced service deployment failures policy.

Please, see document for details:
https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=95654584
On Wed, Nov 21, 2018 at 11:16 AM Vladimir Ozerov <vozerov@gridgain.com> wrote:
>
> Hi Vyacheslav,
>
> Could you please explain in what situation coordinator needs to collect
> service deployments info from all nodes and share it with the cluster? I
> cannot remember from our design discussion when it is needed. Global state
> normally shared through discovery and only on node join, In this case we
> use "DiscoveryDataBags", not separate messages.
>
> On Wed, Nov 21, 2018 at 11:11 AM Vyacheslav Daradur <daradurvs@gmail.com>
> wrote:
>
> > I think request-response is not suitable terms.
> >
> > Nodes send to coordinator maps of actual service deployments which
> > contains what count of instances of each service node hosts locally.
> >
> > Coordinator sends to the cluster the full map of deployments across the
> > cluster.
> >
> > On Wed, Nov 21, 2018 at 11:04 AM Vladimir Ozerov <vozerov@gridgain.com>
> > wrote:
> > >
> > > I do not know what is correct term :-) What I said is that "exchange" is
> > > counter intuitive here. There is no "exchange", instead nodes send
> > > information to coordinator that they finished some operation. E.g. we do
> > > the same for schema changes (index creation), and as Denis suggested,
> > > Request-Response is correct suffixes here. Message name should explain
> > what
> > > really happened, instead of describing things which are somewhat similar
> > in
> > > internal flow.
> > >
> > > On Wed, Nov 21, 2018 at 10:49 AM Nikolay Izhikov <nizhikov@apache.org>
> > > wrote:
> > >
> > > > Hello, Vladimir.
> > > >
> > > > What is correct term?
> > > >
> > > >
> > > > ср, 21 нояб. 2018 г., 10:29 Vladimir Ozerov vozerov@gridgain.com:
> > > >
> > > > > Agree. Service deployment has nothing to do with PME. We should not
> > use
> > > > the
> > > > > same term for different things.
> > > > >
> > > > > вт, 20 нояб. 2018 г. в 17:19, Denis Mekhanikov <
> > dmekhanikov@gmail.com>:
> > > > >
> > > > > > Vyacheslav,
> > > > > >
> > > > > > I'm in process of reviewing your changes. Sorry for taking so
long.
> > > > > > I posted the first portion of review comments yesterday.
> > > > > > I'd like to finish looking through the code. I'll post more
> > comments
> > > > > later.
> > > > > >
> > > > > > I see, that you called things analogously to partition map
> > exchange.
> > > > > > I realize, that there is an analogy in used procedures, but
I don't
> > > > > really
> > > > > > like the idea to use the same names for everything.
> > > > > > The partition map exchange is called this way because it involves
> > an
> > > > > actual
> > > > > > exchange of information.
> > > > > > All nodes need to tell each other, which partitions they have,
and
> > what
> > > > > > their states are.
> > > > > >
> > > > > > There is no exchange in case of service deployment, so I would
> > skip the
> > > > > > "exchange" part.
> > > > > > And *single message ->* *full message* look more like *request
->
> > > > > response*
> > > > > > in case of services.
> > > > > >
> > > > > > Suppose we abandon the PME procedure and move to something else.
> > > > > > Then *ServiceDeploymentExchange* name won't make sense.
> > > > > > And I don't want to be in a situation, when I say to my colleague
a
> > > > word
> > > > > > "exchange",
> > > > > > and get "which one?" in return.
> > > > > > So, I'm for the meaningful names rather than analogous to something
> > > > else.
> > > > > >
> > > > > > What do you think?
> > > > > >
> > > > > > Denis
> > > > > >
> > > > > > вт, 20 нояб. 2018 г. в 12:09, Vyacheslav Daradur <
> > daradurvs@gmail.com
> > > > >:
> > > > > >
> > > > > > > Denis, Yakov have you had a chance to review the solution?
> > > > > > >
> > > > > > > Igniters, we need to define a list of reviewers, otherwise
no
> > end in
> > > > > > sign.
> > > > > > >
> > > > > > > I'm ready to continue work on the Service Grid, including
new
> > > > features
> > > > > > > like hot-redeployment and versioning, also, I have ideas
about
> > new
> > > > > > > tools for monitoring and management which will be useful
for our
> > > > > > > end-users.
> > > > > > >
> > > > > > > But for continuing work we need to overcome this first
phase.
> > > > > > >
> > > > > > > On Tue, Nov 13, 2018 at 1:09 PM Vyacheslav Daradur <
> > > > > daradurvs@gmail.com>
> > > > > > > wrote:
> > > > > > > >
> > > > > > > > Denis, Yakov, feel free to contact me directly in
case of
> > > > questions.
> > > > > > > Thanks!
> > > > > > > >
> > > > > > > > On Sun, Nov 11, 2018 at 10:09 PM Denis Mekhanikov
<
> > > > > > dmekhanikov@gmail.com>
> > > > > > > wrote:
> > > > > > > > >
> > > > > > > > > Guys,
> > > > > > > > >
> > > > > > > > > I'd like to take a look at the changes before
they are
> > merged.
> > > > > > > > > I'll do my best to finish the review before the
end of the
> > > > upcoming
> > > > > > > week.
> > > > > > > > >
> > > > > > > > > Thanks!
> > > > > > > > > Denis
> > > > > > > > >
> > > > > > > > > сб, 10 нояб. 2018 г. в 14:25, Nikolay
Izhikov <
> > > > nizhikov@apache.org
> > > > > >:
> > > > > > > > >
> > > > > > > > > > Hello, Vladimir.
> > > > > > > > > >
> > > > > > > > > > I'm agree with you.
> > > > > > > > > >
> > > > > > > > > > Can we write the list of reviewers for this
feature?
> > > > > > > > > > Without a date or similar.
> > > > > > > > > > Just a list of experts who should review
this feature.
> > > > > > > > > >
> > > > > > > > > > В Сб, 10/11/2018 в 14:01 +0300, Vladimir
Ozerov пишет:
> > > > > > > > > > > Igniters,
> > > > > > > > > > >
> > > > > > > > > > > This is very huge thing with complex
algorithms behind.
> > We
> > > > > should
> > > > > > > not
> > > > > > > > > > merge
> > > > > > > > > > > it to the product unless several additional
thorough
> > reviews
> > > > > are
> > > > > > > ready,
> > > > > > > > > > > irrespectively of how long will it
take. We are about
> > > > quality,
> > > > > > not
> > > > > > > speed.
> > > > > > > > > > >
> > > > > > > > > > > сб, 10 нояб. 2018 г. в 1:30,
Denis Magda <
> > dmagda@apache.org
> > > > >:
> > > > > > > > > > >
> > > > > > > > > > > > Vyacheslav,
> > > > > > > > > > > >
> > > > > > > > > > > > What are the cases when the service
can be redeployed?
> > > > > > Affinity,
> > > > > > > > > > failure,
> > > > > > > > > > > > etc., right. It would be good
to list all the cases on
> > the
> > > > > wiki
> > > > > > > and
> > > > > > > > > > then
> > > > > > > > > > > > our tech writers will get everything
documented.
> > > > > > > > > > > >
> > > > > > > > > > > > --
> > > > > > > > > > > > Denis
> > > > > > > > > > > >
> > > > > > > > > > > > On Thu, Nov 8, 2018 at 11:06 PM
Vyacheslav Daradur <
> > > > > > > > > > daradurvs@gmail.com>
> > > > > > > > > > > > wrote:
> > > > > > > > > > > >
> > > > > > > > > > > > > Denis,
> > > > > > > > > > > > >
> > > > > > > > > > > > > Services reassignment process
takes into account
> > previous
> > > > > > > assignments
> > > > > > > > > > > > > to avoid redundant redeployments.
> > > > > > > > > > > > > So, in the described case,
ServiceA won't be moved
> > from
> > > > > node1
> > > > > > > to
> > > > > > > > > > node2.
> > > > > > > > > > > > > On Fri, Nov 9, 2018 at 4:41
AM Denis Magda <
> > > > > > dmagda@apache.org>
> > > > > > > > > > wrote:
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > Vyacheslav,
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > First of all, thanks
for archiving this milestone
> > and
> > > > > > > rolling out
> > > > > > > > > > these
> > > > > > > > > > > > >
> > > > > > > > > > > > > new
> > > > > > > > > > > > > > capabilities.
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > Speaking of the topology
change events [1], does
> > the
> > > > new
> > > > > > > > > > architecture
> > > > > > > > > > > > >
> > > > > > > > > > > > > avoid
> > > > > > > > > > > > > > a running service redeployment
when a new node
> > joins?
> > > > For
> > > > > > > instance,
> > > > > > > > > > > >
> > > > > > > > > > > > let's
> > > > > > > > > > > > > > say I have ServiceA
running node1, then node2 joins
> > > > and I
> > > > > > > don't
> > > > > > > > > > want
> > > > > > > > > > > >
> > > > > > > > > > > > the
> > > > > > > > > > > > > > service to be redeployed
to any other node.
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > [1]
> > > > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=95654584#ServiceGridredesign.Phase1.Implementationdetails.-Topologychange
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > --
> > > > > > > > > > > > > > Denis
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > On Wed, Nov 7, 2018
at 7:04 AM Vyacheslav Daradur <
> > > > > > > > > > daradurvs@gmail.com
> > > > > > > > > > > > > > wrote:
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > > Dmitriy, I published
documentation in wiki:
> > > > > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=95654584
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > Thank you!
> > > > > > > > > > > > > > > On Wed, Nov 7,
2018 at 5:10 PM Dmitriy Pavlov <
> > > > > > > > > > dpavlov.spb@gmail.com
> > > > > > > > > > > > > > > wrote:
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > Hi I think
wiki is better than any attached
> > docs.
> > > > > Could
> > > > > > > you
> > > > > > > > > > please
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > create a
> > > > > > > > > > > > > > > > page?
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > ср, 7 нояб.
2018 г., 14:39 Vyacheslav Daradur <
> > > > > > > > > > daradurvs@gmail.com
> > > > > > > > > > > > >
> > > > > > > > > > > > > :
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > I prepared
a description of the implemented
> > > > > solution
> > > > > > > and
> > > > > > > > > > attached
> > > > > > > > > > > > >
> > > > > > > > > > > > > it
> > > > > > > > > > > > > > > > > to the
issue [1].
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > This
should help during a review. Should I
> > post
> > > > the
> > > > > > > document
> > > > > > > > > > into
> > > > > > > > > > > > >
> > > > > > > > > > > > > wiki
> > > > > > > > > > > > > > > or
> > > > > > > > > > > > > > > > > IEP?
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > I'd like
to ask Ignite's experts review the
> > > > > solution
> > > > > > > [1] [2],
> > > > > > > > > > > > >
> > > > > > > > > > > > > please?
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > [1]
> > > > > > https://issues.apache.org/jira/browse/IGNITE-9607
> > > > > > > > > > > > > > > > > [2]
> > https://github.com/apache/ignite/pull/4434
> > > > > > > > > > > > > > > > > On Wed,
Oct 31, 2018 at 5:04 PM Vyacheslav
> > > > Daradur
> > > > > <
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > daradurvs@gmail.com>
> > > > > > > > > > > > > > > > > wrote:
> > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > >
Hi, Igniters! Good news!
> > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > >
Service Grid Redesign Phase 1 - is in Patch
> > > > > > > Available now.
> > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > >
Nikolay Izhikov has reviewed
> > implementation.
> > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > >
However, we need additional review from
> > other
> > > > > > Ignite
> > > > > > > > > > experts.
> > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > >
Here is an umbrella ticket [1] and PR [2].
> > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > >
Could someone step in and do the review?
> > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > >
[1]
> > > > > > > https://issues.apache.org/jira/browse/IGNITE-9607
> > > > > > > > > > > > > > > > > >
[2]
> > https://github.com/apache/ignite/pull/4434
> > > > > > > > > > > > > > > > > >
On Sat, Aug 18, 2018 at 11:44 AM Denis
> > > > > Mekhanikov <
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > dmekhanikov@gmail.com>
> > > > > > > > > > > > > > > > > wrote:
> > > > > > > > > > > > > > > > > >
>
> > > > > > > > > > > > > > > > > >
> Pavel, could you assist?
> > > > > > > > > > > > > > > > > >
>
> > > > > > > > > > > > > > > > > >
> Does it make sense for .Net to specify
> > > > service
> > > > > > > class name
> > > > > > > > > > > > >
> > > > > > > > > > > > > instead
> > > > > > > > > > > > > > > of
> > > > > > > > > > > > > > > > > its
> > > > > > > > > > > > > > > > > >
> implementation?
> > > > > > > > > > > > > > > > > >
>
> > > > > > > > > > > > > > > > > >
> I think, it shouldn't be a problem.
> > > > > > > > > > > > > > > > > >
>
> > > > > > > > > > > > > > > > > >
> Denis
> > > > > > > > > > > > > > > > > >
>
> > > > > > > > > > > > > > > > > >
> On Sat, Aug 18, 2018, 11:33 Vyacheslav
> > > > Daradur
> > > > > <
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > daradurvs@gmail.com>
> > > > > > > > > > > > > > > > > wrote:
> > > > > > > > > > > > > > > > > >
>
> > > > > > > > > > > > > > > > > >
> > I think that the replacement of
> > serialized
> > > > > > > instance
> > > > > > > > > > makes
> > > > > > > > > > > > >
> > > > > > > > > > > > > sense
> > > > > > > > > > > > > > > to me
> > > > > > > > > > > > > > > > > >
> > for Java part.
> > > > > > > > > > > > > > > > > >
> >
> > > > > > > > > > > > > > > > > >
> > But how it should work for .NET client?
> > > > > > > > > > > > > > > > > >
> >
> > > > > > > > > > > > > > > > > >
> > On Tue, Aug 14, 2018 at 4:07 PM Dmitriy
> > > > > > > Setrakyan <
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > dsetrakyan@apache.org>
> > > > > > > > > > > > > > > > > >
> > wrote:
> > > > > > > > > > > > > > > > > >
> > >
> > > > > > > > > > > > > > > > > >
> > > On Tue, Aug 14, 2018 at 6:10 AM,
> > Nikita
> > > > > > > Amelchev <
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > nsamelchev@gmail.com>
> > > > > > > > > > > > > > > > > >
> > > wrote:
> > > > > > > > > > > > > > > > > >
> > >
> > > > > > > > > > > > > > > > > >
> > > > Hello, Igniters.
> > > > > > > > > > > > > > > > > >
> > > >
> > > > > > > > > > > > > > > > > >
> > > > I am working on task [1] that would
> > > > > replace
> > > > > > > > > > serialized
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > service's
> > > > > > > > > > > > > > > > > >
> > instance
> > > > > > > > > > > > > > > > > >
> > > > by service's class name and
> > properties
> > > > > map
> > > > > > in
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > {ServiceConfiguration}.
> > > > > > > > > > > > > > > > > >
> > > >
> > > > > > > > > > > > > > > > > >
> > > > The task describes that we should
> > use
> > > > > > > > > > > > > > > > > >
> > > > {String className} + {Map<String,
> > > > Object>
> > > > > > > > > > properties}
> > > > > > > > > > > > >
> > > > > > > > > > > > > instead
> > > > > > > > > > > > > > > > > {Service
> > > > > > > > > > > > > > > > > >
> > > > srvc}.
> > > > > > > > > > > > > > > > > >
> > > >
> > > > > > > > > > > > > > > > > >
> > > > I'd like to clarify the following
> > > > > > questions:
> > > > > > > > > > > > > > > > > >
> > > >
> > > > > > > > > > > > > > > > > >
> > > > 1. What about public methods?
> > > > > > > > > > > > > > > > > >
> > > > I suggest to mark them as
> > deprecated
> > > > and
> > > > > > use
> > > > > > > class
> > > > > > > > > > name
> > > > > > > > > > > > >
> > > > > > > > > > > > > of
> > > > > > > > > > > > > > > > > provided
> > > > > > > > > > > > > > > > > >
> > > > instance.
> > > > > > > > > > > > > > > > > >
> > > > Also to add deploying methods with
> > new
> > > > > > > parameters:
> > > > > > > > > > > > > > > > > >
> > > >
> > > > > > > > > > > > > > > > > >
> > > > @Deprecated
> > > > > > > > > > > > > > > > > >
> > > > public IgniteInternalFuture<?>
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > deployNodeSingleton(ClusterGroup
> > > > > > > > > > > > > > > > > prj,
> > > > > > > > > > > > > > > > > >
> > > > String
> > > > > > > > > > > > > > > > > >
> > > > name, Service svc)
> > > > > > > > > > > > > > > > > >
> > > >
> > > > > > > > > > > > > > > > > >
> > > > public IgniteInternalFuture<?>
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > deployNodeSingleton(ClusterGroup
> > > > > > > > > > > > > > > > > prj,
> > > > > > > > > > > > > > > > > >
> > > > String
> > > > > > > > > > > > > > > > > >
> > > > name, String srvcClsName,
> > Map<String,
> > > > > > > Object> prop)
> > > > > > > > > > > > > > > > > >
> > > >
> > > > > > > > > > > > > > > > > >
> > >
> > > > > > > > > > > > > > > > > >
> > > I think this makes sense, but I would
> > > > like
> > > > > > > other
> > > > > > > > > > > > >
> > > > > > > > > > > > > committers to
> > > > > > > > > > > > > > > > > confirm.
> > > > > > > > > > > > > > > > > >
> > > Perhaps Vladimir Ozerov should
> > comment
> > > > > here.
> > > > > > > > > > > > > > > > > >
> > >
> > > > > > > > > > > > > > > > > >
> > >
> > > > > > > > > > > > > > > > > >
> > > > 2. Is {Map<String, Object>
> > properties}
> > > > > > > parameter
> > > > > > > > > > > > >
> > > > > > > > > > > > > mandatory
> > > > > > > > > > > > > > > when
> > > > > > > > > > > > > > > > > >
> > deploying a
> > > > > > > > > > > > > > > > > >
> > > > service?
> > > > > > > > > > > > > > > > > >
> > > > Is it make sense to add deploying
> > > > methods
> > > > > > > without
> > > > > > > > > > it?
> > > > > > > > > > > >
> > > > > > > > > > > > For
> > > > > > > > > > > > > > > > > example:
> > > > > > > > > > > > > > > > > >
> > > >
> > > > > > > > > > > > > > > > > >
> > > > public IgniteInternalFuture<?>
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > deployNodeSingleton(ClusterGroup
> > > > > > > > > > > > > > > > > prj,
> > > > > > > > > > > > > > > > > >
> > > > String
> > > > > > > > > > > > > > > > > >
> > > > name, String srvcClsName)
> > > > > > > > > > > > > > > > > >
> > > >
> > > > > > > > > > > > > > > > > >
> > > > public IgniteInternalFuture<?>
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > deployNodeSingleton(ClusterGroup
> > > > > > > > > > > > > > > > > prj,
> > > > > > > > > > > > > > > > > >
> > > > String
> > > > > > > > > > > > > > > > > >
> > > > name, String srvcClsName,
> > Map<String,
> > > > > > > Object> prop)
> > > > > > > > > > > > > > > > > >
> > > >
> > > > > > > > > > > > > > > > > >
> > >
> > > > > > > > > > > > > > > > > >
> > > I would always ask the user to pass
> > the
> > > > > > > property
> > > > > > > > > > map, but
> > > > > > > > > > > > >
> > > > > > > > > > > > > would
> > > > > > > > > > > > > > > > > allow
it
> > > > > > > > > > > > > > > > > >
> > to
> > > > > > > > > > > > > > > > > >
> > > be null.
> > > > > > > > > > > > > > > > > >
> > >
> > > > > > > > > > > > > > > > > >
> > > D.
> > > > > > > > > > > > > > > > > >
> >
> > > > > > > > > > > > > > > > > >
> >
> > > > > > > > > > > > > > > > > >
> >
> > > > > > > > > > > > > > > > > >
> > --
> > > > > > > > > > > > > > > > > >
> > Best Regards, Vyacheslav D.
> > > > > > > > > > > > > > > > > >
> >
> > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > >
--
> > > > > > > > > > > > > > > > > >
Best Regards, Vyacheslav D.
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > --
> > > > > > > > > > > > > > > > > Best
Regards, Vyacheslav D.
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > --
> > > > > > > > > > > > > > > Best Regards, Vyacheslav
D.
> > > > > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > > > --
> > > > > > > > > > > > > Best Regards, Vyacheslav
D.
> > > > > > > > > > > > >
> > > > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > > --
> > > > > > > > Best Regards, Vyacheslav D.
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > --
> > > > > > > Best Regards, Vyacheslav D.
> > > > > > >
> > > > > >
> > > > >
> > > >
> >
> >
> >
> > --
> > Best Regards, Vyacheslav D.
> >



-- 
Best Regards, Vyacheslav D.

Mime
View raw message