asterixdb-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Xikui Wang <xik...@uci.edu>
Subject Re: Build UDF project (Maven) with large model to deploy to AsterixDB [Error assembling jar]
Date Sat, 24 Nov 2018 17:55:27 GMT
Hi Sandra,

How big is the model file that you are using? I guess you are trying to
pack this model file into the UDF jar? I personally haven't seen this error
before. It feels like a Maven building with big files issue. I found this
thread on StackOverflow which describes the similar situation. Could you
try the resolutions there?

As a side note, if you need to use a big model file in UDF, I wouldn't
suggest you pack that into your UDF jar file. It's because this will
significantly slow down your UDF installation, and you will spend a lot of
time redeploying the resource file to the cluster if you only need to
update the UDF code. Alternatively, you could make the resource file path
as a UDF parameter, and let the UDF load that file when it initializes.
This could make the installation much faster and avoid deploying the
resource file multiple times, and the packing issue should be gone as well.
:)

PS If it's ok, could you tell us which use case that you are working on? We
would like to know how our customers use AsterixDB in different scenarios,
so we can help them (you) better!

Best,
Xikui



On Sat, Nov 24, 2018 at 6:05 AM sandraskarshaug@gmail.com <
sandraskarshaug@gmail.com> wrote:

> Hi!
>
> My master thesis partner and I have added a model for word embeddings
> (word2vec) in our project which is quite large. This is supposed to be
> loaded in the initialize phase of the UDF and be used for evaluating the
> incoming records.
>
> However, when trying to build the Maven project before deploying it to
> AsterixDB, we get the error "Error assembling JAR, invalid entry size". Is
> this a problem anyone else have faced when for instance using machine
> learning models in AsterixDB?
>
> If so, we appreciate any help!
>
> Best regards,
> Sandra
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message