Original commit: 0b6ba66238
Co-authored-by: João Barbosa <joao.paulo.barbosa@gmail.com>
Co-authored-by: Jon Atack <jon@atack.com>
Github-Pull: #21422
Rebased-From: c5e53d0d21
It is UB to exceed the bounds of the buffer when doing pointer
arithemetic. That means the following is not a valid bounds check:
if (start + 4 <= limit)
Because if we were at the end of the buffer, we wouldn't be
allowed to add 4 anyway. Instead, this must be written as:
if (limit - start >= 4)
Basic forms of this issue are flagged by UBSan. If building with
-fsanitize=undefined, the following test trips an error:
[ RUN ] HASH.SignedUnsignedIssue
.../leveldb/util/hash.cc:30:15: runtime error: applying non-zero offset 4 to null pointer
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior /usr/local/google/home/davidben/leveldb/util/hash.cc:30:15 in
[ OK ] HASH.SignedUnsignedIssue (1 ms)
(cherry picked from commit 578eeb702ec0fbb6b9780f3d4147b1076630d633)
Github-Pull: #31671
Rebased-From: a8844b23ab02a48a05940987f70bf23d4fa76f58
exFAT is known to cause corruption on macOS. See #28552.
Therefore we should warn when using this fs format for either the blocks
or data directories.
Github-Pull: #31453
Rebased-From: df1ba101419729aafa382d970ee605e2a6273e26
`p2p_dns_seeds.py` would try to connect to the DNS server configured on
the machine and resolve `dummySeed.invalid`.
To block that configure an unavailable proxy which will be used also to
connect to the name server. The test needs 2 successful connections to
other peers (two Python `P2PInterface`s) and they work in spite of the
unavailable proxy because they are on `127.0.0.1` (`NET_UNROUTABLE`) and
the proxy is not used for that.
Github-Pull: #31646
Rebased-From: 2ed161c5ce
`feature_config_args.py` uses a proxy address of `1.2.3.4`. This results
in actually trying to open TCP connections over the internet to
`1.2.3.4:9050`.
The test does not need those to succeed so use `127.0.0.1:1` instead.
Also avoid `-noconnect=0` because that is interpreted as `-connect=1`
which is interpreted as `-connect=0.0.0.1` and a connection to
`0.0.0.1:18444` is attempted.
Github-Pull: #31646
Rebased-From: a5746dc559
Create a separate status for transactions that are confirmed in
a block that is assumed valid pending background validation.
Use the same icon as for transactions with a single confirmation.
Github-Pull: #28616
Rebased-From: 3e281590c7dfa6d97191c41792d3b9604632d1a7
Determine if a given transaction belongs to a block that is assumed to be valid pending background validation.
Add this information to WalletTxStatus.
Github-Pull: #28616
Rebased-From: a6c53bb24436ecc1ae717b032c0cccdd704044eb
This guards against 2 processes running with separate datadirs but the same
blocksdir.
It's not likely to happen currently, but may be more relevant in the future
with applications using the kernel.
Note that the kernel does not currently do any dir locking, but it should.
Github-Pull: #31674
Rebased-From: bdc0a68e67
A subsequent commit will add a .lock file to this dir at startup, meaning that
the blocksdir is never empty by the time the xor key is being read/written.
Ignore all hidden files when determining if this is the first run.
Github-Pull: #31674
Rebased-From: 1db331ba76
The `libevent` package defaults to the "Release" build type, which
overrides our per-build-type optimization flags with `-O3`.
To prevent this behavior, set `CMAKE_BUILD_TYPE` to "None", consistent
with how other packages are handled.
Github-Pull: #31661
Rebased-From: d44626a9c2
The UTXO set has grown significantly, and flushing it from memory to LevelDB often takes over 20 minutes after a successful IBD with large dbcache values.
The final UTXO set is written to disk in batches, which LevelDB sorts into SST files.
By increasing the default batch size, we can reduce overhead from repeated compaction cycles, minimize constant overhead per batch, and achieve more sequential writes.
Experiments with different batch sizes (loaded via assumeutxo at block 840k, then measuring final flush time) show that 64 MiB batches significantly reduce flush time without notably increasing memory usage:
| dbbatchsize | flush_sum (ms) |
|-------------|----------------|
| 8 MiB | ~240,000 |
| 16 MiB | ~220,000 |
| 32 MiB | ~200,000 |
| *64 MiB* | *~150,000* |
| 128 MiB | ~156,000 |
| 256 MiB | ~166,000 |
| 512 MiB | ~186,000 |
| 1 GiB | ~186,000 |
Checking the impact of a `-reindex-chainstate` with `-stopatheight=878000` and `-dbcache=30000` gives:
16 << 20
```
2025-01-12T07:31:05Z Flushed fee estimates to fee_estimates.dat.
2025-01-12T07:31:05Z [warning] Flushing large (26 GiB) UTXO set to disk, it may take several minutes
2025-01-12T07:53:51Z Shutdown: done
```
Flush time: 22 minutes and 46 seconds
64 >> 20
```
2025-01-12T18:30:00Z Flushed fee estimates to fee_estimates.dat.
2025-01-12T18:30:00Z [warning] Flushing large (26 GiB) UTXO set to disk, it may take several minutes
2025-01-12T18:44:43Z Shutdown: done
```
Flush time: ~14 minutes 43 seconds.
Github-Pull: #31645
Rebased-From: d249a353be58868d41d2a7c57357038ffd779eba
It is UB to apply a distance to a pointer or iterator further than the
end itself, even if the distance is (partially) revoked later on.
Fix the issue by advancing the data pointer at most to the end.
Github-Pull: #31655
Rebased-From: fabeca3458
