kudu-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Tim Robertson <timrobertson...@gmail.com>
Subject Re: Growing the Kudu community
Date Wed, 18 Jul 2018 15:52:06 GMT
On the mini cluster as a maven artifact:
The Beam KuduIO is in progress here [1], with Integration test (currently I
use Docker) and I just refactored the code so I could mock a KuduService
for unit tests.
It is an ideal time to try your current work Mike/Grant, as I'd use a
minicluster instead of mocking. Perhaps we should continue this on the dev@
list discussion I started a few weeks back [2]?
Do you have a build for OS X by any chance?

> I'm not really sure there is a lot of overlap between creating a Docker
> image and the kind of relocatable artifacts I'm trying to build, aside
from
> the actual compiling part.

I've had to manipulate hosts files with fake entries to get the Docker
images to work while the minicluster seemingly has a FakeDNS thing (?).
That might just be me doing things wrong though.




[1]
https://github.com/timrobertson100/beam/tree/BEAM-2661-KuduIO/sdks/java/io/kudu

[2]
https://lists.apache.org/thread.html/ee697a022b72bbca2761b1af0581773d8fb708f701fc969bc259fc2d@%3Cdev.kudu.apache.org%3E

On Wed, Jul 18, 2018 at 4:37 AM, Mike Percy <mpercy@apache.org> wrote:

> On Tue, Jul 17, 2018 at 12:22 PM Grant Henke <ghenke@cloudera.com.invalid>
> wrote:
>
> > I have started a document for blog post ideas/topics here:
> >
> > https://docs.google.com/document/d/12QFRIhNDMoOI1kOQBgch64xYJ9t6U
> byVt1D3NaTl7lI/edit?usp=sharing
> >
>
> Nice list, Grant. Actually I think that quarterly email would probably make
> for a better blog post instead and I've added it as a suggestion on that
> doc.
>
> On Tue, Jul 17, 2018 at 12:04 PM Mauricio Aristizabal <mauricio@impact.com
> >
> wrote:
>
> > I was disappointed that Strata SJ 2018 didn't have a single session on
> > Kudu, there were no committers in attendance that I could tell, and it
> > wasn't being highlighted at all in the Cloudera booth.  Between Strata
> and
> > ScalaDays I must have enthusiastically mentioned the product to 15 people
> > and none had heard of it.
> >
>
> Hmm, that is disappointing, and a bit surprising. Perhaps everybody thought
> everybody else was going to submit... actually I had intended to submit a
> talk proposal to Strata this year but got busy and missed the deadline. :(
>
> I wonder if folks using Kudu would like to present on their use case? I'm
> sure conference-goers would like to hear from more people using Kudu "in
> anger" (hopefully not angrily).
>
> On Tue, Jul 17, 2018 at 2:59 PM Sailesh Mukil <sailesh@cloudera.com.invalid
> >
> wrote:
>
> > A suggestion to add on to the easily downloadable pre-built packages, is
> to
> > have easily accessible/downloadable example test-data that's fairly
> > representative of real world datasets (but it doesn't have to be too
> > large). Additionally, we can write tutorials in kudu/examples/ that use
> > this test data, to give new users a better feel for the system.
>
>
> That sounds useful. Any ideas on where we could find such a data set?
>
> On Tue, Jul 17, 2018 at 11:59 AM Tim Robertson <timrobertson100@gmail.com>
> wrote:
>
> > ++1 on the mini cluster
> > Perhaps include a docker image build at the same time which presumably
> > wouldn't be much effort?
> >
>
> I'm not really sure there is a lot of overlap between creating a Docker
> image and the kind of relocatable artifacts I'm trying to build, aside from
> the actual compiling part. But I think it would be valuable for Docker
> users to be able to easily pull down a Kudu image.
>
>
> > l'll be happy to contribute on the Java / maven related parts to that. I
> > will use this for the testing framework for the Apache Beam KuduIO and
> will
> > certainly help test / write a blog.
> >
>
> I don't really know how to handle the Maven part where we unpack the
> tarball and set it up somewhere so we can invoke it from the
> KuduMiniCluster. Maybe it that would require writing a custom Maven plugin?
>
> I'd love to see a blog post about how to use Kudu with Beam!
>
> Mike
>

Mime
View raw message