asterixdb-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ian Maxon <ima...@uci.edu>
Subject Re: Internal error [NegativeArraySizeException]
Date Wed, 07 Oct 2015 02:36:00 GMT
Hi Namrata,
First, I think the behavior you are experiencing is a bug, so we'll look
into that. The load fails because each row is really large, about 3MB, and
somehow the sort operator doesn't deal with this well.
However it may be good that we ran into this, because, while huge objects
like this should eventually be handled more gracefully in AsterixDB,
they're viewed as being exceptional rather than the norm. Hence the
performance will not be as good when these types of big objects/fields are
accessed while mixed in with comparatively tiny data.
The field I see taking up almost all of the space in the object is the
"mask" field. Is this something that is actually needed? Or can it be
filtered/projected out?

I've attached a version of the sample data where I cut out the "mask"
field, this one seems to load in just fine using the provided DDL.

​
 new_nomask.adm
<https://drive.google.com/a/uci.edu/file/d/0B9fobkjZFASia2xaZ054T25nUFU/view?usp=drive_web>
​

Thanks,
-Ian

On Tue, Oct 6, 2015 at 10:37 AM, Ian Maxon <imaxon@uci.edu> wrote:
> Awesome, Thanks Namrata. I'll give this a close look later today.
>
> -Ian
>
> On Tue, Oct 6, 2015 at 10:24 AM, Malarout, Namrata (398M-Affiliate)
> <Namrata.Malarout@jpl.nasa.gov> wrote:
>> Hi Ian,
>> I just realized I didn¹t provide the DDL. Sorry about that. I¹ve kept it
>> really simple:
>>
>> drop dataverse TestL4 if exists;
>> create dataverse TestL4;
>> use dataverse TestL4;
>>
>>
>> create type GlobL4Type as open {
>> fid: string,
>> }
>>
>>
>> create dataset GlobL4(GlobL4Type)
>> primary key fid;
>>
>> Please let me know if you have any questions.
>> Thanks,
>> Namrata
>>
>>
>>
>> On 10/1/15, 5:33 PM, "Ian Maxon" <imaxon@uci.edu> wrote:
>>
>>>P.S., if you have the data/DDL/so on that caused this error to happen,
>>>I can try to reproduce here locally if the exception/logs may have
>>>gotten lost somewhere.
>>>
>>>-Ian
>>>
>>>On Thu, Oct 1, 2015 at 5:19 PM, Ian Maxon <imaxon@uci.edu> wrote:
>>>> Hey Namrata,
>>>> Those logs are not logs in the diagnostic sense, but rather
>>>> write-ahead logs, so a log of the transactions that are occuring in
>>>> the instance. If you were using the single-machine package I gave you,
>>>> the error's stack trace should actually be on the console.
>>>>
>>>> Thanks,
>>>> -Ian
>>>>
>>>> On Thu, Oct 1, 2015 at 5:13 PM, Malarout, Namrata (398M-Affiliate)
>>>> <Namrata.Malarout@jpl.nasa.gov> wrote:
>>>>> Hi,
>>>>> I got an error while trying to ingest data.
>>>>> Internal error. Please check instance logs for further details.
>>>>> [NegativeArraySizeException]
>>>>>
>>>>> I¹ve attached the logs. When I open them it¹s unreadable. The logs
in
>>>>> ClusterControllerService are empty (screenshot attached).
>>>>>
>>>>> I have had errors when I was using version 0.8.6 ingesting data due to
>>>>>the
>>>>> size of the data. Has anyone encountered this error before?
>>>>> Thanks in advance for the help.
>>>>>
>>>>> Regards,
>>>>> Namrata
>>

Mime
View raw message