asterixdb-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ildar Absalyamov <ildar.absalya...@gmail.com>
Subject Re: Error loading data
Date Tue, 19 Apr 2016 17:11:26 GMT
Magnus,

Since you are using autogenerated key the inserted record should not have that field. To do
that you need to change the record format in the return clause:

insert into dataset posdata(
  for $x in dataset posdata_temp return {
"campus": $x.campus,
"building": $x.building,
"floor": $x.floor,
"timestamp": $x.timestamp,
"dayOfWeek": $x.day,
"hourOfDay": $x.hour,
"latitude": $x.latitude,
"salt_timestamp": $x.salt,
"longitude": $x.longitude,
"id": $x.id,
"accuracy": $x.accuracy
}
)

> On Apr 19, 2016, at 04:54, Magnus Kongshem <kongshem@stud.ntnu.no> wrote:
> 
> Your suggestion does not work because you get duplicate fields. 
> 
> Exception: Duplicate field &quot;uuid&quot; encountered [AlgebricksException]
> 
> Any other suggestions? This is a major issue in my view, and as Mike Carey said: It should
be easy and seamless to add more data to the dataset.
> 
> BG,
> Magnus
> 
> On Thu, Apr 14, 2016 at 6:34 PM, Ildar Absalyamov <ildar.absalyamov@gmail.com <mailto:ildar.absalyamov@gmail.com>>
wrote:
> Magnus,
> 
> You could still add data to non-empty dataset via inserts.
> The easiest way to do that, granted you have the data you want to insert in a files,
is to bulkhead data to new temp dataset and insert it to the desired dataset:
> 
> create dataset posdata_temp(table) primary key uid auto generated;
> load dataset posdata_temp using localfs "path"="localhost:///data/path/to/file/file.adm,localhost:///data/path/to/file/file2.adm,localhost:///data/path/to/file/file3.adm
<>"),("format"="adm"));
> insert into dataset posdata(
>   for $x in dataset posdata_temp return $x
> )
> 
>> On Apr 14, 2016, at 07:41, Magnus Kongshem <kongshem@stud.ntnu.no <mailto:kongshem@stud.ntnu.no>>
wrote:
>> 
>> Does this mean that adding additional data to an instance and dataverse is not supported?
>> 
>> Magnus
>> 
>> On Wed, Mar 30, 2016 at 8:11 PM, Ian Maxon <imaxon@uci.edu <mailto:imaxon@uci.edu>>
wrote:
>> It should just be a quoted string with commas inside separating the URL-ish paths,
so like:
>> 
>> load dataset foo using localfs "path"="localhost:///data/path/to/file/file.adm,localhost:///data/path/to/file/file2.adm,localhost:///data/path/to/file/file3.adm
<>"),("format"="adm"));
>> 
>> On Wed, Mar 30, 2016 at 6:24 AM, Magnus Kongshem <kongshem@stud.ntnu.no <mailto:kongshem@stud.ntnu.no>>
wrote:
>> Yes I am.
>> 
>> So, combinding each file will and doing the command once will solve it, or do I have
to input the AQL for each file like below?
>> 
>> use dataverse bigd;
>> load dataset posdata using localfs
>>     (("path"="localhost:///data/path/to/file/file.adm <>"),("format"="adm"));
>> (("path"="localhost:///data/path/to/file/file2.adm <>"),("format"="adm"));
>> (("path"="localhost:///data/path/to/file/file3.adm <>"),("format"="adm"));
>> 
>> 
>> BG
>> Magnus
>> 
>> 
>> On Wed, Mar 30, 2016 at 3:21 PM, Wail Alkowaileet <wael.y.k@gmail.com <mailto:wael.y.k@gmail.com>>
wrote:
>> Are you trying to load each file separately?
>> That AFAK is not supported.
>> On Mar 30, 2016 16:13, "Magnus Kongshem" <kongshem@stud.ntnu.no <mailto:kongshem@stud.ntnu.no>>
wrote:
>> I will be loading 12 files.
>> 
>> AQL below:
>> 
>> use dataverse bigd;
>> load dataset posdata using localfs
>>     (("path"="localhost:///data/path/to/file/file.adm <>"),("format"="adm"));
>> 
>> Will it be solved if I concatinate the files and do the dataset loading only once?
>> 
>> Magnus
>> 
>> On Wed, Mar 30, 2016 at 3:06 PM, Wail Alkowaileet <wael.y.k@gmail.com <mailto:wael.y.k@gmail.com>>
wrote:
>> How many files you're loading?
>> Can you send the loading AQL?
>> 
>> On Mar 30, 2016 16:01, "Magnus Kongshem" <kongshem@stud.ntnu.no <mailto:kongshem@stud.ntnu.no>>
wrote:
>> Using asterixdb v0.8.8.
>> 
>> I am loading data into my asterixDB instance.
>> 
>> Loading the first file is successful. But when I try to load another file, I get
a "Internal error. Please check instance logs for further details. [NullPointerException]"
>> 
>> The files are of the type adm and as good as equal in size (3 gb).
>> 
>> My instance was initialized with these commands:
>> 
>> drop dataverse bigd if exists;
>>     create dataverse bigd;
>>     use dataverse bigd;
>> 
>>     create type table as open {
>> 	uid: uuid,
>>         campus: string,
>> 	building: string,
>> 	floor: string,
>>         timestamp: int32,
>> 	dayOfWeek: int32,
>> 	hourOfDay: int32,
>>         latitude: double,
>>         salt_timestamp: int32,
>>         longitude: double,
>>         id: string,
>> 	accuracy: double
>>     }
>> create dataset posdata(table)
>>     primary key uid autogenerated;
>> create index stamp on posdata(timestamp);
>> create index hour on posdata(hourOfDay);
>> create index day on posdata(dayOfWeek);
>> 
>> My log file is attached.
>> 
>> Any help?
>> 
>> -- 
>> Mvh
>> 
>> Magnus Kongshem
>> 
>> NTNU
>> +47 415 65 906 <tel:%2B47%20415%2065%20906>
>> 
>> 
>> -- 
>> Mvh
>> 
>> Magnus Alderslyst Kongshem
>> Leder av seniorkomiteen
>> Online, linjeforeningen for informatikk
>> +47 415 65 906 <tel:%2B47%20415%2065%20906>
>> 
>> 
>> -- 
>> Mvh
>> 
>> Magnus Alderslyst Kongshem
>> Leder av seniorkomiteen
>> Online, linjeforeningen for informatikk
>> +47 415 65 906 <tel:%2B47%20415%2065%20906>
>> 
>> 
>> 
>> -- 
>> Mvh
>> 
>> Magnus Alderslyst Kongshem
>> Leder av seniorkomiteen
>> Online, linjeforeningen for informatikk
>> +47 415 65 906 <tel:%2B47%20415%2065%20906>
> Best regards,
> Ildar
> 
> 
> 
> 
> -- 
> Mvh
> 
> Magnus Alderslyst Kongshem
> Leder av seniorkomiteen
> Online, linjeforeningen for informatikk
> +47 415 65 906 <tel:%2B47%20415%2065%20906>
Best regards,
Ildar


Mime
View raw message