phoenix-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Josh Elser <josh.el...@gmail.com>
Subject Re: [DISCUSS] Include python-phoenixdb into Phoenix
Date Thu, 01 Mar 2018 21:20:37 GMT
He appears! Thanks for weighing in. Comments inline..

On Thu, Mar 1, 2018 at 3:55 PM, Lukáš Lalinský <lalinsky@gmail.com> wrote:
> I'm fine with including the code. The license already matches Phoenix, is
> there anything else that you would need regarding that?

There's an official IP clearance process we need to follow, but your
comment here should be sufficient going off of memory (the mailing
list memorializing it is good).

> Releasing Python code involves building the source distribution package and
> uploading it to PyPI. If you end up using the python-phoenixdb project, I
> can give you full access to https://pypi.python.org/pypi/phoenixdb.
>
> Wheels are mostly useful when you have non-Python code in the package, which
> is not the case here.

Ok, thanks. I'm obviously not an expert here.

> This is the (simple) process I use:
>
> https://github.com/lalinsky/python-phoenixdb/blob/master/RELEASING.rst
>
> You could just publish the tar.gz on Apache servers, but uploading to PyPI
> is preferable.

A source distribution is easiest for us at the moment (and fits ASF
policy). I'd need to investigate use publishing to PyPI. It might just
be something we can do when the release is approved (in spirit, it's
no different than Maven central...)

> One thing that needs to be taken care of is CI. Unlike the Phoenix itself,
> the test suite is unable to run an embedded version of Phoenix/HBase.

Got it. We can try to figure out some automation here to make that
happen. IIRC you have something already with Docker. I'm less worried
about this part :)

> Lukas
>
>
>
> On Thu, Mar 1, 2018 at 8:38 PM, Josh Elser <josh.elser@gmail.com> wrote:
>>
>> Obviously, I'm in favor of this. Thanks for bringing it up again, Ankit.
>>
>> I remember Lukas giving his "blessing" to the project being pulled up
>> into Apache Phoenix in the past. It seems like there are multiple
>> authors on the pyPhoenix fork -- we'll need to make sure that we have
>> all authors on board with this decision.
>>
>> It seems like making a Python "Wheel" would be the lowest-barrier to
>> entry on getting something usable for people from the ASF. I don't
>> know what, if any, infrastructure exists to distribute Python modules.
>> https://packaging.python.org/glossary/#term-built-distribution
>>
>> I feel like a sub-directory in the phoenix repository would be the
>> easiest to make this work (for now). A separate repo would also work.
>> I know phoenixdb has some tests -- I assume the same tests (plus more)
>> exist for pyPhoenix but I have not validated this.
>>
>> On Thu, Mar 1, 2018 at 10:34 AM, Ankit Singhal <ankit@apache.org> wrote:
>> > Re-opening the thread as I can extend help towards the IP clearance
>> > process[1] to include python-phoenixdb[2] if Lukas Lalinsky and
>> > community
>> > agrees to it.
>> >
>> > I found both the repositories[2][3] on par but we should let first Lukáš
>> > Lalinský (the original owner) to confirm if he agrees to contribute his
>> > work[2].
>> >
>> > @dev, We can discuss any concerns/suggestions towards it before I'll
>> > start a
>> > formal vote for inclusion.
>> >
>> > Details on packaging and progress will be tracked on JIRA[4].
>> >
>> > [1] http://incubator.apache.org/ip-clearance/ip-clearance-template.html
>> > [2] https://github.com/lalinsky/python-phoenixdb
>> > [3] https://github.com/Pirionfr/pyPhoenix
>> > [4] https://issues.apache.org/jira/browse/PHOENIX-4636
>> >
>> > Regards,
>> > Ankit Singhal
>> >
>> >
>> > On Tue, Apr 11, 2017 at 1:30 AM, James Taylor <jamestaylor@apache.org>
>> > wrote:
>> >>
>> >> Thanks, Dimitri. A pull request would be great. Looks like the
>> >> outstanding
>> >> items are:
>> >> - Fixing licensing
>> >> - Deciding on package structure. FWIW, the phoenix-spark module has
>> >> scala
>> >> code, so this is a little bit of precedence for non Java code.
>> >> - Tweaking our release process/scripts to handle Python appropriately.
>> >> Is
>> >> that an ASF way of releasing Python libraries?
>> >>
>> >> I haven't looked yet, but is there good test coverage for all this?
>> >> What
>> >> about backward compatibility - any tests in place to validate that
>> >> nothing
>> >> gets broken from release to release?
>> >>
>> >> Thanks,
>> >> James
>> >>
>> >> On Mon, Apr 10, 2017 at 8:08 AM, Dimitri <grytes29@gmail.com> wrote:
>> >>>
>> >>> Hi,
>> >>>
>> >>> DB API 2.0 and SQLAlchemy are in pyPhoenix.
>> >>>
>> >>> I can fix the licensing.
>> >>>
>> >>> tell us what you decide.
>> >>>
>> >>>
>> >>> Dimitri.
>> >>>
>> >>>
>> >>> 2017-03-27 0:48 GMT+02:00 Josh Elser <elserj@apache.org>:
>> >>>
>> >>> > First off, I think we need to figure out which version of
>> >>> > "phoenixdb"
>> >>> > we
>> >>> > want to include. Technically, I don't know which would be preferred:
>> >>> > DB
>> >>> > API
>> >>> > 2.0 or SQLAlchemy. Do we have to choose just one? I can only say
>> >>> > that I
>> >>> > know of users have been using Lukas' variant but I don't know if
>> >>> > that's
>> >>> > because it was the one that existed.
>> >>> >
>> >>> > I think Lukas is being very calm/reserved in the fact that his
code
>> >>> > was
>> >>> > re-licensed, his copyright statement removed, and a new copyright
>> >>> > statement
>> >>> > added. Shame on you, Dimitri -- you should fix this.
>> >>> >
>> >>> > I'm not sure how best to integrate this into our Maven project
>> >>> > either.
>> >>> > I
>> >>> > know that there are ways to integrate Python code/tests into our
>> >>> > Maven
>> >>> > project, but I'm not sure how best to make this work. I also don't
>> >>> > know
>> >>> > what kind of centralized distribution platforms exist here at the
>> >>> > ASF
>> >>> > for
>> >>> > us to use (is there something similar to Maven's repo mirroring
for
>> >>> > Python
>> >>> > libs?). I see Thrift[1] has some code deployed, but I'm not sure
how
>> >>> > to
>> >>> > do
>> >>> > that.
>> >>> >
>> >>> > Finally, while I would (selfishly) like this to exist in Avatica,
>> >>> > I'm
>> >>> > not
>> >>> > too selfish to say that it shouldn't exist in Phoenix. The
>> >>> > type-specific
>> >>> > details that Lukas pointed out are a good reason to keep this in
>> >>> > Phoenix
>> >>> > for now. Perhaps we can figure out someway later on to pull it
in
>> >>> > Avatica
>> >>> > and just leave something down here in Phoenix that has the necessary
>> >>> > pieces
>> >>> > overridden.
>> >>> >
>> >>> > - Josh
>> >>> >
>> >>> > [1] https://pypi.python.org/pypi/thrift/0.9.1
>> >>> >
>> >>> > Lukáš Lalinský wrote:
>> >>> >
>> >>> >> Hi James,
>> >>> >>
>> >>> >> How would you prefer the integration into the Phoenix project
to be
>> >>> >> handled? I'm more than happy to do it, but I don't see a clear
>> >>> >> path.
>> >>> >>
>> >>> >> Licensing is not an issue, since it's using the same license
as
>> >>> >> Phoenix itself. I guess I'd just need to sign the CLA?
>> >>> >>
>> >>> >> Adding it to the Phoenix source repo is just a matter of deciding
>> >>> >> on
>> >>> >> the structure, since there is currently no non-Java code as
far as
>> >>> >> I
>> >>> >> know. Any preferences there?
>> >>> >>
>> >>> >> There is a question on whether it should be added to Phoenix
or
>> >>> >> Avatica. I think Phoenix is a better home for the library,
since
>> >>> >> everything in there is only tested on Phoenix and not other
Avatica
>> >>> >> backends and the type handling is Phoenix-specific as well.
>> >>> >>
>> >>> >> Releasing is a different matter. I'm not sure how Phoenix releases
>> >>> >> are
>> >>> >> currently handled, but for a Python library, it makes sense
to
>> >>> >> upload
>> >>> >> the package to PyPI since that's from where most Python developers
>> >>> >> are
>> >>> >> getting packages from. Who would do that? If me, I wouldn't
want to
>> >>> >> block Phoenix releases due to me unavailability, but on the
other
>> >>> >> hand
>> >>> >> it would be bad for the releases to get out of sync.
>> >>> >>
>> >>> >> I'm also happy to abandon the project if you integrate Dimitri's
>> >>> >> version.
>> >>> >>
>> >>> >> Regards,
>> >>> >>
>> >>> >> Lukas
>> >>> >>
>> >>> >>
>> >>> >>
>> >>> >> On Fri, Mar 24, 2017 at 1:24 AM, James
>> >>> >> Taylor<jamestaylor@apache.org>
>> >>> >> wrote:
>> >>> >>
>> >>> >>> Thanks, Dimitri! If you're interested, I think the community
would
>> >>> >>> welcome
>> >>> >>> this as a contribution to Apache Phoenix so it can appear
in our
>> >>> >>> regular
>> >>> >>> distributions.
>> >>> >>>
>> >>> >>> Regards,
>> >>> >>> James
>> >>> >>>
>> >>> >>> On Thu, Mar 23, 2017 at 5:21 PM, Dimitri<grytes29@gmail.com>
>> >>> >>> wrote:
>> >>> >>>
>> >>> >>>> Hi,
>> >>> >>>>
>> >>> >>>> I make some update on my fork of Lukáš Lalinský
phoenix python
>> >>> >>>> lib.
>> >>> >>>>
>> >>> >>>> I integrate sqlalchemy and correct some bug.
>> >>> >>>>
>> >>> >>>> you can find the code source on
>> >>> >>>> https://github.com/Pirionfr/pyPhoenix
>> >>> >>>> and it's available in https://pypi.python.org/pypi/pyPhoenix
>> >>> >>>>
>> >>> >>>> regards
>> >>> >>>> Dimitri Capitaine
>> >>> >>>>
>> >>> >>>>
>> >>> >>>>
>> >>> >>>>
>> >>> >>>>
>> >>> >>>>
>> >>> >>>>
>> >>> >>>> 2016-11-07 18:12 GMT+01:00 Josh Elser<elserj@apache.org>:
>> >>> >>>>
>> >>> >>>> +1
>> >>> >>>>>
>> >>> >>>>> I was poking around with it this weekend. I had
some issues
>> >>> >>>>> (trying
>> >>> >>>>> to
>> >>> >>>>> use
>> >>> >>>>> it from the Avatica side, instead of PQS, specifically),
but for
>> >>> >>>>> the
>> >>> >>>>> most
>> >>> >>>>> part it worked. Definitely feel free to report
any issues you
>> >>> >>>>> run
>> >>> >>>>> into:
>> >>> >>>>> https://bitbucket.org/lalinsky/python-phoenixdb.
It would be
>> >>> >>>>> nice
>> >>> >>>>> to
>> >>> >>>>> make
>> >>> >>>>> this a "first-class citizen".
>> >>> >>>>>
>> >>> >>>>> James Taylor wrote:
>> >>> >>>>>
>> >>> >>>>> Thanks, Dimitri. I created PHOENIX-3459 to update
the website
>> >>> >>>>> with
>> >>> >>>>> the
>> >>> >>>>>> new
>> >>> >>>>>> link. It'd be great to see this make it into
Phoenix proper (or
>> >>> >>>>>> Avatica
>> >>> >>>>>> or
>> >>> >>>>>> a combination of the two).
>> >>> >>>>>>
>> >>> >>>>>>       James
>> >>> >>>>>>
>> >>> >>>>>> On Sun, Nov 6, 2016 at 12:35 PM, Dimitri<grytes29@gmail.com>
>> >>> >>>>>> wrote:
>> >>> >>>>>>
>> >>> >>>>>> erratum
>> >>> >>>>>>
>> >>> >>>>>>> Someone else did the port to protobuf,
Lukáš Lalinský merged
>> >>> >>>>>>> the
>> >>> >>>>>>> pull
>> >>> >>>>>>> request and it's now released as version
0.5.
>> >>> >>>>>>>
>> >>> >>>>>>> Dimitri
>> >>> >>>>>>>
>> >>> >>>>>>> 2016-11-03 22:14 GMT+01:00 Dimitri<grytes29@gmail.com>:
>> >>> >>>>>>>
>> >>> >>>>>>> Hi,
>> >>> >>>>>>>
>> >>> >>>>>>>> Lukáš Lalinský is no longer use
Phoenix, so he is not
>> >>> >>>>>>>> actively
>> >>> >>>>>>>> working
>> >>> >>>>>>>> on
>> >>> >>>>>>>> his lib anymore.
>> >>> >>>>>>>>
>> >>> >>>>>>>> One major problem with his lib is that
it uses the JSON
>> >>> >>>>>>>> interface,
>> >>> >>>>>>>> which
>> >>> >>>>>>>> is now deprecated.
>> >>> >>>>>>>>
>> >>> >>>>>>>> I fork it for my use (phoenix 4.8 and
sqlalchemy.), it's now
>> >>> >>>>>>>> use
>> >>> >>>>>>>> protobuf,I push my code on my github.
>> >>> >>>>>>>> (https://github.com/Pirionfr/
>> >>> >>>>>>>> python-phoenixdb)
>> >>> >>>>>>>>
>> >>> >>>>>>>> For some who want to use it can you
put the link on your site
>> >>> >>>>>>>> (
>> >>> >>>>>>>> https://phoenix.apache.org/phoenix_python.html)
?
>> >>> >>>>>>>>
>> >>> >>>>>>>>
>> >>> >>>>>>>>
>> >>> >>>>>>>>
>> >>> >>>>>>>>
>> >>> >>>
>> >>
>> >>
>> >
>
>

Mime
View raw message