accumulo-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Marc P." <marc.par...@gmail.com>
Subject Re: c++ thrift client crash
Date Mon, 25 Apr 2016 15:27:17 GMT
I ran into this a lot when performing operations like bulk load, writing,
and scanning accumulo from c++. I wasn't using the proxy, but I ran into
this a lot
1) My first likely cause was issues of endianness. I assume the proxy takes
care of this....so perhaps this is moot.
2) The next most likely culprit was pointer mismanagement or scope
problems. Does d_client's update call occur asynchronously or will it
complete the write before that function returns?
3) Invalid thrift operation. Did you do a startUpdate somewhere? I don't
see it. Doesn't mean it didn't occur...but can I see the code where it
occurred?



Hi,
> Anyone here has experience writing C++ client code using Accumulo Thrift
> proxy.
> My C++ code crash when trying to add mutation, and the proxy spits out this
> callstack:
>
> 2016-04-25 14:28:33,952 [server.AbstractNonblockingServer$FrameBuffer] WARN
> : Got an IOException in internalRead!
> java.io.IOException: Connection reset by peer
>         at sun.nio.ch.FileDispatcherImpl.read0(Native Method)
>         at sun.nio.ch.SocketDispatcher.read(SocketDispatcher.java:39)
>         at sun.nio.ch.IOUtil.readIntoNativeBuffer(IOUtil.java:223)
>         at sun.nio.ch.IOUtil.read(IOUtil.java:197)
>         at sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:384)
>         at
>
> org.apache.thrift.transport.TNonblockingSocket.read(TNonblockingSocket.java:141)
>         at
>
> org.apache.thrift.server.AbstractNonblockingServer$FrameBuffer.internalRead(AbstractNonblockingServer.java:537)
>         at
>
> org.apache.thrift.server.AbstractNonblockingServer$FrameBuffer.read(AbstractNonblockingServer.java:338)
>         at
>
> org.apache.thrift.server.AbstractNonblockingServer$AbstractSelectThread.handleRead(AbstractNonblockingServer.java:203)
>         at
>
> org.apache.thrift.server.TNonblockingServer$SelectAcceptThread.select(TNonblockingServer.java:202)
>         at
>
> org.apache.thrift.server.TNonblockingServer$SelectAcceptThread.run(TNonblockingServer.java:158)
>
> I can't figure out what's wrong with the function 'addMutation' in our c++
> code below. Any idea or pointer?
>
> BatchWriter::BatchWriter(std::shared_ptr<::accumulo::AccumuloProxyClient>
> proxyClient,
>                           const std::string& login,
>                           const std::string& tableName,
>                           const int64_t maxMemory,
>                           const int64_t latencyMs,
>                           const int64_t timeoutMs,
>                           const int32_t numThreads)
> : d_client(proxyClient)
> , d_login(login)
> , d_tableName(tableName)
> {
>      ::accumulo::WriterOptions writerOptions;
>      writerOptions.__set_maxMemory(maxMemory);
>      writerOptions.__set_latencyMs(latencyMs);
>      writerOptions.__set_timeoutMs(timeoutMs);
>      writerOptions.__set_threads(numThreads);
>      BOOST_ASSERT_MSG(d_client, "Client must be valid pointer");
>      d_client->createWriter(d_writerToken, login, tableName,
> writerOptions);
> }
>
> void BatchWriter::addMutation(Mutation& mutation)
> {
>      std::map<std::string, std::vector&lt;::accumulo::ColumnUpdate> >
> cells;
>
>      const std::vector<::accumulo::ColumnUpdate> updates =
> mutation.getUpdates();
>      const std::string rowId = mutation.getRowId();
>
>      cells.insert(std::make_pair(rowId, updates));
>
>      d_client->update(d_writerToken, cells);
>
>      mutation.clear();
> }
>
>
> Thanks,
> Z
>
>
>
>
> --
> View this message in context:
>
> http://apache-accumulo.1065345.n5.nabble.com/c-thrift-client-crash-tp16741.html
> Sent from the Developers mailing list archive at Nabble.com.
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message