Repository navigation
Potential race condition or concurrency issue with env.txnRead() or dbi.get() under heavy multithreaded access #253
Description
Summary
When using lmdbjava in a multithreaded environment (~32 threads), I experience sporadic native crashes (SIGSEGV) across various JVMs and OSes during my CI matrix builds. The crashes consistently point to the native method mdb_txn_renew0, invoked during env.txnRead() or or dbi.get().
Environment
- Java versions: Various (e.g., Corretto 21.0.6.7.1, Temurin 21.0.6+7)
- OS: Linux, Windows, macOS (GitHub Actions runners)
- LMDBJava version: Latest via Maven (0.9.1)
- Concurrency: ~32 threads executing
txnRead()andget()in tight loops - Frequency: Millions of calls across all threads
- Crash frequency: Sporadic (impossible to reproduce locally, but more frequent in CI)
Crash Snippet example
# A fatal error has been detected by the Java Runtime Environment:
# SIGSEGV (0xb) at pc=0x00007f62e442b95e, pid=2053, tid=2149
# Problematic frame:
# C [lmdbjava-native-library-8897859339362759578.so+0x595e] mdb_txn_renew0+0x13e
...
Stack: [0x00007f62adddf000,0x00007f62adedf000], sp=0x00007f62adedd290, free space=1016k
Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code)
C [lmdbjava-native-library-8897859339362759578.so+0x595e] mdb_txn_renew0+0x13e
C [lmdbjava-native-library-8897859339362759578.so+0x5eec] mdb_txn_begin+0x26c
C 0x00007f62e44257ec
Java frames: (J=compiled Java code, j=interpreted, Vv=VM code)
J 2884 org.lmdbjava.Library$Lmdb$jnr$ffi$0.mdb_txn_begin$jni$29(JJIJ)I (0 bytes) @ 0x00007f62cc409b02 [0x00007f62cc409aa0+0x0000000000000062]
J 3902 c2 net.ladenthin.bitcoinaddressfinder.ConsumerJava.containsAddress(Ljava/nio/ByteBuffer;)Z (213 bytes) @ 0x00007f62cc516f70 [0x00007f62cc5159c0+0x00000000000015b0]
J 3954% c1 net.ladenthin.bitcoinaddressfinder.ConsumerJava.consumeKeys(Ljava/nio/ByteBuffer;)V (1373 bytes) @ 0x00007f62c4f17b4c [0x00007f62c4f172a0+0x00000000000008ac]
Usage Pattern
I use the following method in high-throughput read scenarios:
@Override
public boolean containsAddress(ByteBuffer hash160) {
try (Txn<ByteBuffer> txn = env.txnRead()) {
ByteBuffer byteBuffer = lmdb_h160ToAmount.get(txn, hash160);
return byteBuffer != null;
}
}hash160is not shared across threads (it’s either thread-local, duplicated, or allocated per call).- The
Dbi<ByteBuffer>(lmdb_h160ToAmount) andEnv<ByteBuffer>are shared and initialized once.
I’ve tried:
hash160.duplicate().rewind()- Allocating a fresh
ByteBuffer.allocateDirect()and copying the contents
None of this resolves the crash.
What I'm Doing
In my code, I'm calling env.txnRead() to create a read-only transaction and then immediately passing it to lmdb_h160ToAmount.get(txn, hash160) to check for the existence of a key. This is part of a high-throughput loop running in about 32 threads.
I'm currently trying to determine whether the native crash originates from the env.txnRead() call itself or from the subsequent get(txn, hash160) operation. Both calls are used extensively under heavy concurrent load, and isolating the failing component has been difficult due to the sporadic nature of the issue.
Try 1 - Fails
I tried synchronizing only the get(txn, hash160) call (while keeping env.txnRead() outside the synchronized block). Even with this partial synchronization, I experienced a native crash during CI testing.
@Override
public boolean containsAddress(ByteBuffer hash160) {
try (Txn<ByteBuffer> txn = env.txnRead()) {
synchronized (this) {
ByteBuffer byteBuffer = lmdb_h160ToAmount.get(txn, hash160);
return byteBuffer != null;
}
}
}Error: Crashed tests:
Error: net.ladenthin.bitcoinaddressfinder.LMDBPersistencePerformanceTest
Error: org.apache.maven.surefire.booter.SurefireBooterForkException: ExecutionException The forked VM terminated without properly saying goodbye. VM crash or System.exit called?
Error: Command was cmd.exe /X /C "C:\hostedtoolcache\windows\Java_Microsoft_jdk\21.0.2\x64\bin\java -javaagent:C:\\Users\\runneradmin\\.m2\\repository\\org\\jacoco\\org.jacoco.agent\\0.8.13\\org.jacoco.agent-0.8.13-runtime.jar=destfile=D:\\a\\BitcoinAddressFinder\\BitcoinAddressFinder\\target\\jacoco.exec -Xmx2g -Xms1g --add-opens=java.base/java.lang=ALL-UNNAMED --add-opens=java.base/java.io=ALL-UNNAMED --add-opens=java.base/java.nio=ALL-UNNAMED --add-opens=java.base/jdk.internal.ref=ALL-UNNAMED --add-opens=java.base/jdk.internal.misc=ALL-UNNAMED --add-opens=java.base/sun.nio.ch=ALL-UNNAMED --add-opens=jdk.management/com.sun.management.internal=ALL-UNNAMED -jar C:\Users\runneradmin\AppData\Local\Temp\surefire10771212932124905688\surefirebooter-20250409125554748_110.jar C:\Users\runneradmin\AppData\Local\Temp\surefire10771212932124905688 2025-04-09T12-55-54_486-jvmRun1 surefire-20250409125554748_108tmp surefire_14-20250409125554748_109tmp"
Error: Error occurred in starting fork, check output in log
Error: Process Exit Code: -1073741819
Error: Crashed tests:
Error: net.ladenthin.bitcoinaddressfinder.LMDBPersistencePerformanceTest
Try 2 - Fails
This code is crashing, which shows me more or less that env.txnRead() has a problem with concurrent execution.
@Override
public boolean containsAddress(ByteBuffer hash160) {
try (Txn<ByteBuffer> txn = env.txnRead()) {
synchronized (this) {
ByteBuffer byteBuffer = lmdb_h160ToAmount.get(txn, hash160);
return byteBuffer != null;
}
}
}#
# A fatal error has been detected by the Java Runtime Environment:
#
# SIGSEGV (0xb) at pc=0x00007fb36801295e, pid=2040, tid=2168
#
# JRE version: OpenJDK Runtime Environment (Alibaba Dragonwell Extended Edition)-21.0.6.0.6+7-GA (21.0.6) (build 21.0.6.0.6)
# Java VM: OpenJDK 64-Bit Server VM (Alibaba Dragonwell Extended Edition)-21.0.6.0.6+7-GA (21.0.6.0.6, mixed mode, sharing, tiered, compressed oops, compressed class ptrs, g1 gc, linux-amd64)
# Problematic frame:
# C [lmdbjava-native-library-6392604367366445385.so+0x595e] mdb_txn_renew0+0x13e
#
# Core dump will be written. Default location: Core dumps may be processed with "/usr/lib/systemd/systemd-coredump %P %u %g %s %t 9223372036854775808 %h" (or dumping to /home/runner/work/BitcoinAddressFinder/BitcoinAddressFinder/core.2040)
#
# If you would like to submit a bug report, please visit:
# mailto:[email protected]
# The crash happened outside the Java Virtual Machine in native code.
# See problematic frame for where to report the bug.
#
--------------- S U M M A R Y ------------
Command Line: -javaagent:/home/runner/.m2/repository/org/jacoco/org.jacoco.agent/0.8.13/org.jacoco.agent-0.8.13-runtime.jar=destfile=/home/runner/work/BitcoinAddressFinder/BitcoinAddressFinder/target/jacoco.exec -Xmx2g -Xms1g --add-opens=java.base/java.lang=ALL-UNNAMED --add-opens=java.base/java.io=ALL-UNNAMED --add-opens=java.base/java.nio=ALL-UNNAMED --add-opens=java.base/jdk.internal.ref=ALL-UNNAMED --add-opens=java.base/jdk.internal.misc=ALL-UNNAMED --add-opens=java.base/sun.nio.ch=ALL-UNNAMED --add-opens=jdk.management/com.sun.management.internal=ALL-UNNAMED /home/runner/work/BitcoinAddressFinder/BitcoinAddressFinder/target/surefire/surefirebooter-20250409134947304_28.jar /home/runner/work/BitcoinAddressFinder/BitcoinAddressFinder/target/surefire 2025-04-09T13-49-47_148-jvmRun1 surefire-20250409134947304_26tmp surefire_3-20250409134947304_27tmp
Host: AMD EPYC 7763 64-Core Processor, 4 cores, 15G, Ubuntu 24.04.2 LTS
Time: Wed Apr 9 13:51:19 2025 UTC elapsed time: 65.874255 seconds (0d 0h 1m 5s)
--------------- T H R E A D ---------------
Current thread (0x00007fb27820c2d0): JavaThread "pool-3-thread-31" [_thread_in_native, id=2168, stack(0x00007fb2ed5e7000,0x00007fb2ed6e7000) (1024K)]
Stack: [0x00007fb2ed5e7000,0x00007fb2ed6e7000], sp=0x00007fb2ed6e5250, free space=1016k
Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code)
C [lmdbjava-native-library-6392604367366445385.so+0x595e] mdb_txn_renew0+0x13e
C [lmdbjava-native-library-6392604367366445385.so+0x5eec] mdb_txn_begin+0x26c
C 0x00007fb36ca137ec
Java frames: (J=compiled Java code, j=interpreted, Vv=VM code)
J 2752 org.lmdbjava.Library$Lmdb$jnr$ffi$0.mdb_txn_begin$jni$29(JJIJ)I (0 bytes) @ 0x00007fb3543c0942 [0x00007fb3543c08e0+0x0000000000000062]
J 3664 c2 net.ladenthin.bitcoinaddressfinder.ConsumerJava.containsAddress(Ljava/nio/ByteBuffer;)Z (213 bytes) @ 0x00007fb35446f560 [0x00007fb35446d260+0x0000000000002300]
J 3891% c1 net.ladenthin.bitcoinaddressfinder.ConsumerJava.consumeKeys(Ljava/nio/ByteBuffer;)V (1373 bytes) @ 0x00007fb34cd57f84 [0x00007fb34cd572a0+0x0000000000000ce4]
Try 3 - Fails
@Override
public boolean containsAddress(ByteBuffer hash160) {
Txn<ByteBuffer> txn;
synchronized (env) {
txn = env.txnRead();
}
try (txn) {
ByteBuffer byteBuffer = lmdb_h160ToAmount.get(txn, hash160);
return byteBuffer != null;
}
}crashes also
#
# A fatal error has been detected by the Java Runtime Environment:
#
# SIGSEGV (0xb) at pc=0x00007ff3940e695e, pid=2026, tid=2134
#
# JRE version: OpenJDK Runtime Environment Zulu21.40+17-CRaC-CA (21.0.6+7) (build 21.0.6+7-LTS)
# Java VM: OpenJDK 64-Bit Server VM Zulu21.40+17-CRaC-CA (21.0.6+7-LTS, mixed mode, sharing, tiered, compressed oops, compressed class ptrs, g1 gc, linux-amd64)
# Problematic frame:
# C [lmdbjava-native-library-1362637522283092738.so+0x595e] mdb_txn_renew0+0x13e
#
# Core dump will be written. Default location: Core dumps may be processed with "/usr/lib/systemd/systemd-coredump %P %u %g %s %t 9223372036854775808 %h" (or dumping to /home/runner/work/BitcoinAddressFinder/BitcoinAddressFinder/core.2026)
#
# If you would like to submit a bug report, please visit:
# http://www.azul.com/support/
# The crash happened outside the Java Virtual Machine in native code.
# See problematic frame for where to report the bug.
#
--------------- S U M M A R Y ------------
Command Line: -javaagent:/home/runner/.m2/repository/org/jacoco/org.jacoco.agent/0.8.13/org.jacoco.agent-0.8.13-runtime.jar=destfile=/home/runner/work/BitcoinAddressFinder/BitcoinAddressFinder/target/jacoco.exec -Xmx2g -Xms1g --add-opens=java.base/java.lang=ALL-UNNAMED --add-opens=java.base/java.io=ALL-UNNAMED --add-opens=java.base/java.nio=ALL-UNNAMED --add-opens=java.base/jdk.internal.ref=ALL-UNNAMED --add-opens=java.base/jdk.internal.misc=ALL-UNNAMED --add-opens=java.base/sun.nio.ch=ALL-UNNAMED --add-opens=jdk.management/com.sun.management.internal=ALL-UNNAMED /home/runner/work/BitcoinAddressFinder/BitcoinAddressFinder/target/surefire/surefirebooter-20250409144845698_28.jar /home/runner/work/BitcoinAddressFinder/BitcoinAddressFinder/target/surefire 2025-04-09T14-48-45_583-jvmRun1 surefire-20250409144845698_26tmp surefire_3-20250409144845698_27tmp
Host: AMD EPYC 7763 64-Core Processor, 4 cores, 15G, Ubuntu 24.04.2 LTS
Time: Wed Apr 9 14:50:17 2025 UTC elapsed time: 65.782065 seconds (0d 0h 1m 5s)
--------------- T H R E A D ---------------
Current thread (0x00007ff2a0244340): JavaThread "pool-3-thread-12" [_thread_in_native, id=2134, stack(0x00007ff362ef0000,0x00007ff362ff0000) (1024K)]
Stack: [0x00007ff362ef0000,0x00007ff362ff0000], sp=0x00007ff362fee3a0, free space=1016k
Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code)
C [lmdbjava-native-library-1362637522283092738.so+0x595e] mdb_txn_renew0+0x13e
C [lmdbjava-native-library-1362637522283092738.so+0x5eec] mdb_txn_begin+0x26c
C 0x00007ff3982137ec
Java frames: (J=compiled Java code, j=interpreted, Vv=VM code)
J 2855 org.lmdbjava.Library$Lmdb$jnr$ffi$0.mdb_txn_begin$jni$29(JJIJ)I (0 bytes) @ 0x00007ff380422d82 [0x00007ff380422d20+0x0000000000000062]
J 3842 c2 net.ladenthin.bitcoinaddressfinder.ConsumerJava.containsAddress(Ljava/nio/ByteBuffer;)Z (213 bytes) @ 0x00007ff38053d2b8 [0x00007ff38053c060+0x0000000000001258]
J 4064% c2 net.ladenthin.bitcoinaddressfinder.ConsumerJava.consumeKeys(Ljava/nio/ByteBuffer;)V (1373 bytes) @ 0x00007ff38057c3f0 [0x00007ff38057c1a0+0x0000000000000250]
j net.ladenthin.bitcoinaddressfinder.ConsumerJava.consumeKeysRunner()V+97
j net.ladenthin.bitcoinaddressfinder.ConsumerJava.lambda$startConsumer$1()Ljava/lang/Void;+8
j net.ladenthin.bitcoinaddressfinder.ConsumerJava$$Lambda+0x00007ff3181a7318.call()Ljava/lang/Object;+4
Usually works, but crashes in about 1 out of 100 runs
A crash has been observed with this version:
@Override
public synchronized boolean containsAddress(ByteBuffer hash160) {
try (Txn<ByteBuffer> txn = env.txnRead()) {
ByteBuffer byteBuffer = lmdb_h160ToAmount.get(txn, hash160);
return byteBuffer != null;
}
}surefire-reports-ubuntu-latest-corretto-java21_synchronized_method.zip
And with this version also:
Synchronize both env.txnRead() and get()
@Override
public boolean containsAddress(ByteBuffer hash160) {
Txn<ByteBuffer> txn;
synchronized (env) {
txn = env.txnRead();
try (txn) {
ByteBuffer byteBuffer = lmdb_h160ToAmount.get(txn, hash160);
return byteBuffer != null;
}
}
}surefire-reports-ubuntu-latest-microsoft-java21_synchronized_env.zip
This strongly suggests the issue is not purely a concurrency problem.
It may indicate a deeper issue in the implementation of one or more of the following:
env.txnRead()Dbi.get(txn, key)- Shared JNI-backed structures within lmdbjava
Questions
Before I try building a minimal reproducer, I’d appreciate clarification on:
- ❓ Is using a shared
Dbi<ByteBuffer>across threads supported? - ❓ Is
env.txnRead()thread-safe? - ❓ Are there known issues when mixing many parallel
Txninstances with a sharedDbi? - ❓ Is my opening for read only correct? Thread safe? https://github.com/bernardladenthin/BitcoinAddressFinder/blob/main/src/main/java/net/ladenthin/bitcoinaddressfinder/persistence/lmdb/LMDBPersistence.java#L114 It is documented here https://github.com/lmdbjava/lmdbjava/wiki/Concurrency
Notes
- I've ruled out key buffer corruption, shared buffer issues, or GC-related problems.
- I’ve seen this crash in different JVMs and platforms (Linux, macOS, Windows) via GitHub Actions.
- I can provide logs, dump files, or a stripped-down reproducer if needed.
- Related issue in my project with logs attached:
BitcoinAddressFinder Issue #50
Activity
cdprete commentedon Jul 17, 2025
@bernardladenthin I don't know the details of your project, but I would suggest you to create shard for each thread and let each thread operate on its own shard.
In a project I deal with at my company, we also use 32 threads to constantly write and time-polling (every 40s) read and I've define 32 shards/Envs for this so that each thread operates in isolation on its own set of data.
Moreover, each shard has its own ByteBuffer for the key and the value directly allocated.
I hope this may help you out. ;)
at055612 commentedon Oct 24, 2025
From the LMDB docs
If you want to share read txns across threads you would need to set the MDB_NOTLS env flag. It is not something we do in our use of LmdbJava, and I think is probably best avoided. Write txns should not be shared, you may want to have a single writer thread that holds the write txn and does all the writes, with other threads passing work to it.
We share Env and Dbi instances between threads. We also added our own concurrency control around the opening of txns as an added layer of java-side protection. This ensures there is only a single thread with a write txn. It also allows us to make writes block reads in cases where we have a lot of writes and want the writes to be able to reuse empty pages (which they can't do if a read txn is open). You do need to make sure the maxReaders is set to higher than the number of threads you expect to be concurrently reading, I think the default is 126.
I can't see anything obviously wrong with your code. Are you still seeing this issue?
bp-alex commentedon Nov 7, 2025
I'll close this issue. If it remains a problem just comment and we can re-open.
bernardladenthin commentedon Jul 24, 2026
Follow-up / resolution, for anyone who lands here from a
mdb_txn_renew0crash: in my case (theConsumerJava.containsAddressstack above) this turned out to be an application-side lifecycle bug, not an lmdbjava defect — and importantly not the read-txn-sharing / MDB_NOTLS concern discussed here.EnvandDbiwere shared correctly and each read txn was thread-local.The real bug: my shutdown path closed the
Envwhile reader threads were still mid-txnRead(). On timeout my consumer proceeded toEnv.close()even though a worker was still insidecontainsAddress → mdb_txn_begin;Env.close()unmapped the mmap out from under it andmdb_txn_renew0then dereferenced freed memory →SIGSEGV (SEGV_MAPERR)(Linux) /EXCEPTION_ACCESS_VIOLATION 0xC0000005(Windows). It looked like a sporadic concurrency race purely because it depended on shutdown timing (which is also why JaCoCo/fork settings changed the frequency).It reproduces deterministically once you stop treating it as a steady-state race and instead close the env with reads in flight: a "close-during-read" harness crashed 5/5, whereas steady-state read churn with the env kept open was 0/48.
Fix on my side: a
ReentrantReadWriteLockin the persistence layer — reads hold the read lock for the whole duration of their txn,close()holds the write lock (so it waits for in-flight native reads and blocks new ones before unmapping). 0/5 after the fix.So @at055612's suggestion to add "our own concurrency control around the opening of txns as an added layer of java-side protection" was exactly right — the missing piece was extending that guard to cover
Env.close()vs. in-flight reads. Thanks all; nothing to change in lmdbjava. Details: bernardladenthin/BitcoinAddressFinder#50at055612 commentedon Aug 3, 2026
@bernardladenthin thanks for the update. Glad you found the issue. I will have a look at the PRs.