samza-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Tommy Becker <tobec...@tivo.com>
Subject Re: How to set up RocksDB
Date Mon, 09 Nov 2015 17:05:29 GMT
The native RocksDB libs are included in the jar file, but they are extracted at runtime before
they can be loaded. Make sure the user running the job has permission to write to...wherever
it is that it tries to put it (sorry don't know offhand). But you shouldn't have to set any
environment variables or anything for it to work.

On 11/09/2015 08:50 AM, Lukáš Havrlant wrote:

Hello,
we have a problem with configuring RocksDB as a key-value store with our
Samza 0.9.0 project. We added this line to our config:

stores.my-store.factory=org.apache.samza.storage.kv.RocksDbKeyValueStorageEngineFactory

But then our Samza job couldn't locate the
RocksDbKeyValueStorageEngineFactory class. So we added compile
"org.apache.samza:samza-kv-rocksdb_2.10:$samzaVersion" to our gradle build
script. It helped but now Samza failed to start because it couldn't
find librocksdbjni-linux64.so library. We added export
ROCKSDB_SHAREDLIB_DIR=/opt/samza/lib to our init script and it didn't help.
Then we manually copied the librocksdbjni-linux64.so from
the rocksdbjni-3.5.1.jar to the /opt/samza/lib folder and Samza still
couldn't start, now because of

2015-11-09T14:35:06,209 INFO [main]
org.apache.samza.container.SamzaContainer$ - Got store consumers:
Map(tracking-info ->
org.apache.samza.system.kafka.KafkaSystemConsumer@78461bc4)
# A fatal error has been detected by the Java Runtime Environment:
#
#  SIGSEGV (0xb) at pc=0x0000003c4de092ab, pid=4410, tid=139728971949824

And we're out of ideas at the moment. Can you please help me set up RocksDB
correctly? I suppose we are doing something completely wrong.

Is it possibble to use another key-value store at least?

Lukáš Havrlant



--
Tommy Becker
Senior Software Engineer

Digitalsmiths
A TiVo Company

www.digitalsmiths.com<http://www.digitalsmiths.com>
tobecker@tivo.com<mailto:tobecker@tivo.com>

________________________________

This email and any attachments may contain confidential and privileged material for the sole
use of the intended recipient. Any review, copying, or distribution of this email (or any
attachments) by others is prohibited. If you are not the intended recipient, please contact
the sender immediately and permanently delete this email and any attachments. No employee
or agent of TiVo Inc. is authorized to conclude any binding agreement on behalf of TiVo Inc.
by email. Binding agreements with TiVo Inc. may only be made by a signed written agreement.

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message