accumulo-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From dlmar...@comcast.net
Subject Re: Ingest speed
Date Tue, 05 May 2015 15:57:36 GMT
Fair enough. Point taken. 

----- Original Message -----

From: "Josh Elser" <josh.elser@gmail.com> 
To: dev@accumulo.apache.org 
Sent: Tuesday, May 5, 2015 11:51:35 AM 
Subject: Re: Ingest speed 

Well, let this serve as a big "please" to write _something_. JIRA is not 
a form of documentation for users. 

It's a neat feature, but if there's no documentation about it, it's 
going to die. 

dlmarion@comcast.net wrote: 
> Only what is in the ticket. Not sure there is enough information to fill a chapter. 
> 
> ----- Original Message ----- 
> 
> From: "Josh Elser"<josh.elser@gmail.com> 
> To: dev@accumulo.apache.org 
> Sent: Tuesday, May 5, 2015 11:38:47 AM 
> Subject: Re: Ingest speed 
> 
> Dave -- is there any documentation on this? Seems like it would be very 
> well served by a user-manual chapter. 
> 
> dlmarion@comcast.net wrote: 
>> Revan, 
>> 
>> You could use the scripting[1] feature in the shell for this purpose. The feature
allows you to use JSR-223 compatible languages for your scripts. You will see javascript and
python as examples in the comments on the JIRA ticket. 
>> 
>> [1] https://issues.apache.org/jira/browse/ACCUMULO-1399 
>> 
>> ----- Original Message ----- 
>> 
>> From: "Revan1988"<andrealeoni88@gmail.com> 
>> To: dev@accumulo.apache.org 
>> Sent: Tuesday, May 5, 2015 5:35:53 AM 
>> Subject: Ingest speed 
>> 
>> Hi, 
>> I'm an Italian student and i'm going to be graduated having thesis about 
>> using accumulo. 
>> I've developed a little java application that reads LOGS in a .json file and 
>> insert them into accumulo. 
>> In my virtual machine my app's insert/sec score is about 6'000. 
>> I see that there are some bench test that scores about 50'000 in my VM. 
>> (It's in $ACCUMULO_HOME/test/system/* folder.) 
>> Those test use the shell call to insert a large amount of data. 
>> So I think that i could write all my insert using an accumulo-shell call in 
>> my app. 
>> I've seen that i can execute that command: 
>> 
>> ./bin/accumulo shell -u username -p passw -e "insert row fam qual val [vis] 
>> [timestamp]" 
>> 
>> but there is a problem: i need to set up the table with the command table 
>> (infact insert command has not -t 
>> option). 
>> 
>> So there is any way to execute two commands in a row into accumulo shell? i 
>> tried with&& ; and other separator but with no success. 
>> 
>> The other option that i have is to write a big command file with all 
>> commands that i need and send it to accumulo shell using the command 
>> 
>> ./bin/accumulo shell -f<file> 
>> 
>> it may be a better solution cause i'll connect to accumulo just one time 
>> (and not for every insert)... 
>> 
>> Any suggest? 
>> 
>> Thank you everybody and sorry for my bad english. 
>> 
>> Revan 
>> 
>> 
>> 
> 
> 


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message