kudu-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Todd Lipcon <t...@cloudera.com>
Subject Re: where is kudu's dump core located?
Date Wed, 06 Apr 2016 01:33:33 GMT
On Tue, Apr 5, 2016 at 6:27 PM, Darren Hoo <darren.hoo@gmail.com> wrote:

> Thanks Todd,
>
> let me try giving a little more details here.
>
> When I first created the table and loaded about 100k records, kudu tablet
>  server started to crash and very often.
>
> So I suspect that maybe the data file is corrupted and I dump the table as
> parquet file ,
> drop the table, recreate the table, and import the parquet file again.
>
> But after I did that, the tablet server still crashes often utill I
> increase the memory limit to 16GB,
> then the tablet server crashes less often, one time for serveral days.
>
> There's one big STRING column in my table, but the column should not be
> bigger than 4k in size
> as kudu document recommends.
>

OK, that's definitely an interesting part of the story. Although we think
that 4k strings should be OK, the testing in this kind of workload has not
been as extensive.

If you are able to share the Parquet file and "create table" command for
the dataset off-list, that would be great. I'll keep it only within our
datacenter and delete it when done debugging.


>
> I will try to create a minmal dataset to reproduce the issue, but I am not
> sure I can create one.
>

Thanks, that would be great if the larger dataset can't be shared.


>
> here's the core dump compressed,
>
> http://188.166.175.200/core.90197.bz2
>
> the exact kudu version is : 0.7.1-1.kudu0.7.1.p0.36   (installed from
> parcel)
>
>
OK, thank you. I"m downloading it now and will take a look tonight or
tomorrow.

-Todd


> On Wed, Apr 6, 2016 at 8:59 AM, Todd Lipcon <todd@cloudera.com> wrote:
>
>> Hi Darren,
>>
>> This is interesting. I haven't seen a crash that looks like this, and not
>> sure why it would cause data to disappear either.
>>
>> By any chance do you have some workload that can reproduce the issue?
>> e.g. a particular data set that you are loading that seems to be causing
>> problems?
>>
>> Maybe you can gzip the core file and send it to me off-list if it isn't
>> too large?
>>
>>
>>


-- 
Todd Lipcon
Software Engineer, Cloudera

Mime
View raw message