Repository navigation
Conversation
HostsFileParser reloaded a changed file by clearing its maps in place and refilling them, while lookups read the maps without the lock. A lookup in that window found nothing, even for an entry present in both versions of the file. The first read had the same window: the maps were published before the file was parsed. Each read of the file now builds its own cache, published with a single volatile write and never cleared, and each lookup works on one snapshot. This also resets the "entire file parsed" flag per read, so a file that grows beyond dnsjava.hostsfile.max_size_bytes is searched again instead of reporting every uncached name as absent. Closes dnsjava#429
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## master #430 +/- ##
============================================
+ Coverage 71.42% 71.44% +0.01%
Complexity 3401 3401
============================================
Files 198 198
Lines 14031 14031
Branches 2245 2245
============================================
+ Hits 10022 10024 +2
+ Misses 3378 3376 -2
Partials 631 631 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
…lling thread state The test paused the first read and slept in a loop until the concurrent lookup showed as BLOCKED. It now holds the read on a latch, checks that no cache has been published and that the concurrent lookup has not answered, then releases the read and expects both lookups to find the entry. A package-private hasCache(), next to cacheSize(), exposes the published state.
|
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.



Closes #429.
Problem
HostsFileParserreloads a changed hosts file by clearing its maps in place and refilling them, whilegetAddressForHost/getNameForReverseLookupread those maps without the lock. A lookup between the clear and the refill returnsOptional.empty()for a name that is in both versions of the file. The first read has the same window, because the empty maps are published before parsing. Separately,isEntireFileParsedis never reset, so a file that grows pastdnsjava.hostsfile.max_size_bytesstops answering uncached names. Line-by-line root cause in #429.How it was found
DNSControl routes JVM-wide DNS through
Lookupand usesHostsFileParserforjdk.net.hosts.file. Its concurrency test runs 32 threads ofLookupwhile the hosts file is atomically replaced 3000 times. Every answer must be the old or the new address. Against 3.7.0's parser, 8 of 10 runs had lookups with no answer. DNSControl currently works around it by swapping in fully read, non-reloading parsers (0 of 20 runs failed), and would like to drop that.Fix
The maps, the file size and the "entire file parsed" flag move into one
HostsCacheobject per read of the file:readHostsFile()builds the nextHostsCacheoff to the side and publishes it with a single volatile write. A published cache is never cleared;createOrClearCache()is gone.validateCache()returns the current cache, and each lookup works only on that snapshot, so it sees either the old or the new file.ConcurrentHashMaps, as before.isEntireFileParsed/size behind.No public API change and no new locking. The fast path added in #369 stays.
Tests
Three new tests in
HostsFileParserTest. The first two pause a read inside the file read with a spiedFileSystemProvider(ashandleNoModificationTimealready does) and coordinate through latches only, with no sleeps or polling:testLookupDuringReloadFindsUnchangedEntrylooks up while a reload is paused and must get the entry from the previous read.testLookupDuringFirstReadWaitsForItholds the first read, checks that no cache has been published and that a concurrent lookup has not answered, then releases the read and expects both lookups to find the entry. A package-privatehasCache(), next to the existingcacheSize(), exposes the published state.masterparsertestLookupDuringReloadFindsUnchangedEntrytestLookupDuringFirstReadWaitsForIttestFileGrownBeyondFullCacheSizeIsSearchedHostsFileParserTesttestsRun with no retries on Corretto 11.0.15, 21.0.6 and 25 (macOS);
HostsFileParserTestalso passed 10 consecutive runs. Raw output withmaster's parser (plus the equivalenthasCache(), i.e.hostsCache != null):mvn verifylocally: checkstyle (0 violations), spotless and forbiddenapis are clean on 11 and 25. The only failures in the full suite areNioUdpClientTest(2) andResolverTest(1), which fail the same way on unmodifiedmasteron this machine. It's a macOS UDP loopback issue ("Can't assign requested address").