00618e8745 assumeutxo: Drop block height from metadata (Fabian Jahr)
Pull request description:
Fixes https://github.com/bitcoin/bitcoin/issues/30514 which has more context and shows how the issue can be reproduced. Since the value in question is removed, there is no test to add to reproduce anything.
This is an alternative approach to #30516 with much of the [code being suggested there](https://github.com/bitcoin/bitcoin/pull/30516#discussion_r1689146902).
ACKs for top commit:
maflcko:
re-ACK 00618e8745🎌
achow101:
ACK 00618e8745
theStack:
Code-review ACK 00618e8745
ismaelsadeeq:
Re-ACK 00618e8745
mzumsande:
ACK 00618e8745
Tree-SHA512: db9575247bae838ad7742a27a216faaf55bb11e022f9afdd05752bb09bbf9614717d0ad64304ff5722a16bf41d8dea888af544e4ae26dcaa528c1add0269a4a8
Although it is not explicitly possible to create a default wallet with
descriptors, it is possible to migrate a default wallet and have it end
up being a default wallet with descriptors. These wallets should be
listed by ListDatabases so that it appears in wallet directory listings
to avoid user confusion.
Migration creates backup files in the wallet directory with .bak as the
extension. This pollutes the output of listwalletdir with backup files
that most users should not need to care about.
92c1d7d1f8 validation: Use MAX_TIMEWARP constant as testnet4 timewarp defense delta (Fabian Jahr)
4b2fad502e doc: Add release notes for 29775 (Fabian Jahr)
f7cc97313b doc: Align deprecation warnings (Fabian Jahr)
1163b08378 chainparams: Add initial minimum chain work for Testnet4 (Fabian Jahr)
Pull request description:
This completes follow-ups left open in #29775.
- Adds release notes
- Addresses the [misalignment](https://github.com/bitcoin/bitcoin/pull/29775#discussion_r1706982102) in deprecation warnings and hints at the intention to remove support for Testnet3.
- Adds initial minimum chainwork for Testnet4.
- Use the `MAX_TIMEWARP` constant as the timewarp defense delta, equal to `MAX_FUTURE_BLOCK_TIME`.
ACKs for top commit:
Sjors:
ACK 92c1d7d1f8
achow101:
ACK 92c1d7d1f8
tdb3:
re ACK 92c1d7d1f8
Tree-SHA512: 7ebdac7809f96231f75ca62706af59cd1ed27f713a4c7be5e2ad69fae95832b146b3ea23c712fb03b412da1deda7e8a5dae55bb2bbd2dcfd9f926e85c2a72666
2925bd537c refactor: use c++20 std::views::reverse instead of reverse_iterator.h (stickies-v)
Pull request description:
C++20 introduces [`std::ranges::views::reverse`](https://en.cppreference.com/w/cpp/ranges/reverse_view), which allows us to drop our own `reverse_iterator.h` implementation and also makes it easier to chain views (even though I think we currently don't use this).
ACKs for top commit:
achow101:
ACK 2925bd537c
maflcko:
ACK 2925bd537c🎷
Tree-SHA512: 567666ec44af5d1beb7a271836bcc89c4c577abc77f522fcc18bc6d4de516ae9b0df766d0bfa6dd217569e6878331c2aee1d9815620860375e3510dad7fed476
Enhanced efficiency and readability of CCoinsViewCache::FetchCoin by replacing separate find() and emplace() calls with a single try_emplace(), reducing map lookups and potential insertions.
Multipath descriptors will be imported as multiple separate descriptors.
When there are 2 multipath items, the first descriptor will be for receiving
addresses and the second for change. This mirrors importmulti.
Multipath descriptors will be imported as multiple separate descriptors.
When there are exactly 2 multipath items, the first descriptor will be
for receiving addreses, and the second for change
addresses. When importing a multipath descriptor, 'internal' cannot be
specified.
Instead of applying internal-ness to all keys being imported at the same
time, apply it on a per key basis. So each key that is imported will
carry with it whether it is for the change keypool.
When given a multipath descriptor, derive all of the descriptors.
The derived addresses will be returned in an object
consisting of multiple arrays. For compatibility, when given a single path
descriptor, the addresses are provided in a single array as before.
Multipath specifiers are derivation path indexes of the form `<i;j;k;...>`
used for specifying multiple derivation paths for a descriptor.
Only one multipath specifier is allowed per PubkeyProvider.
This is syntactic sugar which is parsed into multiple distinct descriptors.
One descriptor will have all of the `i` paths, the second all of the `j` paths,
the third all of the `k` paths, and so on.
ParseKeypath will always return a vector of keypaths with the same size
as the multipath specifier. The callers of this function are updated to deal
with this case and return multiple PubkeyProviders. Their callers have
also been updated to handle vectors of PubkeyProviders.
Co-Authored-By: furszy <matiasfurszyfer@protonmail.com>
To prepare for returning multipath descriptors which will be a shorthand
for specifying multiple descriptors, change ParseScript's signature to return a
vector.
589db872e1 validation: don't erase coins cache on prune flushes (Andrew Toth)
0e8918755f Add linked-list test to CCoinsViewCache::SanityCheck (Pieter Wuille)
05cf4e1875 coins: move Sync logic to CoinsViewCacheCursor (Andrew Toth)
7825b8b9ae coins: pass linked list of flagged entries to BatchWrite (Andrew Toth)
a14edada8a test: add cache entry linked list tests (Andrew Toth)
24ce37cb86 coins: track flagged cache entries in linked list (Andrew Toth)
58b7ed156d coins: call ClearFlags in CCoinsCacheEntry destructor (Andrew Toth)
8bd3959fea refactor: require self and sentinel parameters for AddFlags (Andrew Toth)
75f36d241d refactor: add CoinsCachePair alias (Andrew Toth)
f08faeade2 refactor: move flags to private uint8_t and rename to m_flags (Andrew Toth)
4e4fb4cbab refactor: disallow setting flags in CCoinsCacheEntry constructors (Andrew Toth)
8737c0cefa refactor: encapsulate flags setting with AddFlags and ClearFlags (Andrew Toth)
9715d3bf1e refactor: encapsulate flags get access for all other checks (Andrew Toth)
df34a94e57 refactor: encapsulate flags access for dirty and fresh checks (Andrew Toth)
Pull request description:
Since https://github.com/bitcoin/bitcoin/pull/17487 we no longer need to clear the coins cache when syncing to disk. A warm coins cache significantly speeds up block connection, and only needs to be fully flushed when nearing the `dbcache` limit.
For frequent pruning flushes there's no need to empty the cache and kill connect block speed. However, simply using `Sync` in place of `Flush` actually slows down a pruned full IBD with a high `dbcache` value. This is because as the cache grows, sync takes longer since every coin in the cache is scanned to check if it's dirty. For frequent prune flushes and a large cache this constant scanning starts to really slow IBD down, and just emptying the cache on every prune becomes faster.
To fix this, we can add two pointers to each cache entry and construct a doubly linked list of dirty entries. We can then only iterate through all dirty entries on each `Sync`, and simply clear the pointers after.
With this approach a full IBD with `dbcache=16384` and `prune=550` was 32% faster than master. For default `dbcache=450` speedup was ~9%. All benchmarks were run with `stopatheight=800000`.
| | prune | dbcache | time | max RSS | speedup |
|-----------:|----------:|------------:|--------:|-------------:|--------------:|
| master | 550 | 16384 | 8:52:57 | 2,417,464k | - |
| branch | 550 | 16384 | 6:01:00 | 16,216,736k | 32% |
| branch | 550 | 450 | 8:05:08 | 2,818,072k | 8.8% |
| master | 10000 | 5000 | 8:19:59 | 2,962,752k | - |
| branch | 10000 | 5000| 5:56:39 | 6,179,764k | 28.8% |
| master | 0 | 16384 | 4:51:53 | 14,726,408k | - |
| branch | 0 | 16384 | 4:43:11 | 16,526,348k | 2.7% |
| master | 0 | 450 | 7:08:07 | 3,005,892k | - |
| branch | 0 | 450 | 6:57:24 | 3,013,556k |2.6%|
While the 2 pointers add memory to each cache entry, it did not slow down IBD. For non-pruned IBD results were similar for this branch and master. When I performed the initial IBD, the full UTXO set could be held in memory when using the max `dbcache` value. For non-pruned IBD with max `dbcache` to tip ended up using 12% more memory, but it was also 2.7% faster somehow. For smaller `dbcache` values the `dbcache` limit is respected so does not consume more memory, and the potentially more frequent flushes were not significant enough to cause any slowdown.
For reviewers, the commits in order do the following:
First 4 commits encapsulate all accesses to `flags` on cache entries, and then the 5th makes `flags` private.
Commits `refactor: add CoinsCachePair alias` to `coins: call ClearFlags in CCoinsCacheEntry destructor` create the linked list head nodes and cache entry self references and pass them into `AddFlags`.
Commit `coins: track flagged cache entries in linked list` actually adds the entries into a linked list when they are flagged DIRTY or FRESH and removes them from the linked list when they are destroyed or the flags are cleared manually. However, the linked list is not yet used anywhere.
Commit `test: add cache entry linked list tests` adds unit tests for the linked list.
Commit `coins: pass linked list of flagged entries to BatchWrite` uses the linked list to iterate through DIRTY entries instead of using the entire coins cache.
Commit `validation: don't erase coins cache on prune flushes` uses `Sync` instead of `Flush` for pruning flushes, so the cache is no longer cleared.
Inspired by [this comment](https://github.com/bitcoin/bitcoin/pull/15265#issuecomment-457720636).
Fixes https://github.com/bitcoin/bitcoin/issues/11315.
ACKs for top commit:
paplorinc:
ACK 589db872e1
sipa:
reACK 589db872e1
achow101:
ACK 589db872e1
mzumsande:
re-ACK 589db872e1
Tree-SHA512: 23b2bc01c83edacb5b39aa60bb0b766de9a74ce17f0c59bf13b97b4328a7b758ad9aff6581c3ca88e2973f7658380651530d497444f48d6e22ea0bfc51cc921d
6bfa26048d testnet: Add timewarp attack prevention for Testnet4 (Fabian Jahr)
0100907ca1 testnet: Add Testnet4 difficulty adjustment rules fix (Fabian Jahr)
74a04f9e7a testnet: Introduce Testnet4 (Fabian Jahr)
Pull request description:
To supplement the [ongoing conceptual discussion about a testnet reset](https://groups.google.com/g/bitcoindev/c/9bL00vRj7OU/m/9yCPo3uUBwAJ) I have drafted a move to v4 including a fix to the difficulty adjustment mechanism, which was part of the motivation that started the discussion.
Conceptual considerations:
- The conceptual discussion about doing a testnet4 or softforking the fix into testnet3 is outside of the scope of this PR and I would ask reviewers to contribute their opinions on this on the ML instead. However, I am happy to adapt this PR to a softfork change on testnet3 if there is consensus for that instead.
- The difficulty adjustment fix suggested here touches the `CalculateNextWorkRequired` function and uses the same logic used in `GetNextWorkRequired` to find the last previous block that was not mined with difficulty 1 under the exceptionf. An alternative fix briefly mentioned on the mailing list by Jameson Lopp would be to "restrict the special testnet minimum difficulty rule so that it can't be triggered on the block right before a difficulty retarget". That would also fix the issue but I find my suggestion here a bit more elegant.
ACKs for top commit:
jsarenik:
tACK 6bfa26048d
achow101:
ACK 6bfa26048d
murchandamus:
tACK 6bfa26048d
Tree-SHA512: 0b8b69a621406a944da5be551b863d065358ba94d85dd3b80d83c412660e230ee93b27316081fbee9b4851cc4ff8585db64c7dfa26cb5148ac835663f2712c3d
1f93e3c360 add deprecation warning for mempoolfullrbf (glozow)
4400c979a3 [doc] update documentation for new mempoolfullrbf default (glozow)
Pull request description:
Followup to #30493. Update bips.md and policy/*.md to reflect new default rules around signaling requirements in RBF.
Also, log a warning when `-mempoolfullrbf=0` that this config option is deprecated and will be removed in a future release.
ACKs for top commit:
petertodd:
ACK 1f93e3c360
instagibbs:
ACK 1f93e3c360
tdb3:
ACK 1f93e3c360
Tree-SHA512: f60a9524f15cfaa4c10c40b6f62b787d3f9865aac48ca883def30efac4f8a118f1359532f1b209ea34e201f0b1c92398abc8bc1e439e6b60910cc7f75c51e9ae
ec973dd197 refactor: remove un-tested early returns (josibake)
72a5822d43 tests: add tests for KeyPair (josibake)
cebb08b121 refactor: move SignSchnorr to KeyPair (josibake)
c39fd39ba8 crypto: add KeyPair wrapper class (josibake)
5d507a0091 tests: add key tweak smoke test (josibake)
f14900b6e4 bench: add benchmark for signing with a taptweak (josibake)
Pull request description:
Broken out from #28201
---
The wallet returns an untweaked internal key for taproot outputs. If the output commits to a tree of scripts, this key needs to be tweaked with the merkle root. Even if the output does not commit to a tree of scripts, BIP341/342 recommend commiting to a hash of the public key.
Previously, this logic for applying the taptweak was implemented in the `CKey::SignSchnorr` method.
This PR moves introduces a KeyPair class which wraps a `secp256k1_keypair` type and refactors SignSchnorr to use this new KeyPair. The KeyPair class is created with an optional merkle_root argument and the logic from BIP341 is applied depending on the state of the merkle_root argument.
The motivation for this refactor is to be able to use the tap tweak logic outside of signing, e.g. in silent payments when retrieving the private key (see #28201).
Outside of silent payments, since we almost always convert a `CKey` to a `secp256k1_keypair` when doing anything with taproot keys, it seems generally useful to have a way to model this type in our code base.
ACKs for top commit:
paplorinc:
ACK ec973dd197 - will happily reack if you decide to apply @ismaelsadeeq's suggestions
ismaelsadeeq:
Code review ACK ec973dd197
itornaza:
trACK ec973dd197
theStack:
Code-review ACK ec973dd197
Tree-SHA512: 34947e3eac39bd959807fa21b6045191fc80113bd650f6f08606e4bcd89aa17d6afd48dd034f6741ac4ff304b104fa8c1c1898e297467edcf262d5f97425da7b
6714276d72 miniscript: Use `ToIntegral` instead of `ParseInt64` (brunoerg)
Pull request description:
Currently, miniscript code uses `ParseInt64` function for `after`, `older`, `multi` and `thresh` fragments. It means that a leading `+` or whitespace, among other things, are accepted into the fragments. However, these cases are not useful and cause Bitcoin Core to behave differently compared to other miniscript implementations (see https://github.com/brunoerg/bitcoinfuzz/issues/34). This PR fixes it.
ACKs for top commit:
achow101:
ACK 6714276d72
tdb3:
cr ACK 6714276d72
danielabrozzoni:
tACK 6714276d72
darosior:
utACK 6714276d72
Tree-SHA512: d9eeb93f380f346d636513eeaf26865285e7b0907b8ed258fe1e02153a9eb69d484c82180eb1c78b0ed77ad5f0e5b244be6672c2f890b1d9fddc9e844bee6dde
After loading a snapshot, pindexLastCommonBlock is usually already set
to some block for existing peers. That means we'd continue syncing the
background chain from those peers instead of prioritising the snapshot
chain, which defeats the purpose of doing assumeutxo in the first place.
Only existing peers are affected by this bug.
If the best chain of the peer doesn't include the snapshot
block, it is futile to download blocks from this chain,
because we couldn't reorg to it. We'd also crash
trying to reorg because this scenario is not handled.
fa18fc7050 log: Remove NOLINT(bitcoin-unterminated-logprintf) (MarcoFalke)
Pull request description:
`NOLINT(bitcoin-unterminated-logprintf)` is used to document a missing trailing `\n` char in the format string. This has many issues:
* It is just documentation, assuming that a trailing `\n` ends up in the formatted string. It is not enforced at compile-time, so it is brittle.
* If the newline was truly missing and `NOLINT(bitcoin-unterminated-logprintf)` were used to document a "continued" line, the log stream would be racy/corrupt, because any other thread may inject a log message in the meantime.
* If the newline was accidentally missing, nothing is there to correct the mistake.
* The intention of all code is to always end a log line with a new line. However, historic code exists to deal with the case where the new line was missing (`m_started_new_line`). This is problematic, because the presumed dead code has to be maintained (https://github.com/bitcoin/bitcoin/pull/30386#discussion_r1682963306).
Fix almost all issues by removing the `NOLINT(bitcoin-unterminated-logprintf)`, ensuring that a new line is always present.
A follow-up will remove the dead logging code.
ACKs for top commit:
TheCharlatan:
ACK fa18fc7050
ryanofsky:
Code review ACK fa18fc7050
Tree-SHA512: bf8a83723cca84e21187658edc19612da79c34f7ef2e1f6e9353e7ba70e4ecc0a878a2ae32290045fb90cba9a44451e35341a36ef2ec1169d13592393aa4a8ca
e9de0a76b9 doc: release note for 30212 (willcl-ark)
87b1880525 rpc: clarify ALREADY_IN_CHAIN rpc errors (willcl-ark)
Pull request description:
Closes: #19363
Renaming this error improves clarity around the returned error both internally and externally when a transactions' outputs are already found in the utxo set (`TransactionError::ALREADY_IN_CHAIN -> TransactionError::ALREADY_IN_UTXO_SET`)
ACKs for top commit:
tdb3:
ACK e9de0a76b9
ismaelsadeeq:
ACK e9de0a76b9
ryanofsky:
Code review ACK e9de0a76b9.
Tree-SHA512: 7d2617200909790340951fe56a241448f9ce511900777cb2a712e8b9c0778a27d1f912b460f82335844224f1abb4322bc898ca076440959edade55c082a09237
59c0ece0a7 fuzz: replace hardcoded numbers for bech32 limits (josibake)
Pull request description:
Follow-up to #30047 to replace a hardcoded value that was missed in the original PR
ACKs for top commit:
paplorinc:
ACK 59c0ece0a7
dergoegge:
utACK 59c0ece0a7
marcofleon:
ACK 59c0ece0a7. Ran the test a bit to be sure, lgtm.
brunoerg:
utACK 59c0ece0a7
Tree-SHA512: 89799928feb6752a533259117340b087ff7299f9bf204b165dd87708e15b99a338521f2ac9f9e1fd91dc48b93be839059768d9e68b172e36328232174d1dfa3f
Erase spent cache entries and clear flags of unspent
entries inside the BatchWrite loop, instead of an
additional loop after BatchWrite.
Co-Authored-By: Pieter Wuille <pieter@wuille.net>
No visible behavior change. This commit tracks the flagged
entries internally but the list is not iterated by anything.
Co-Authored-By: Pieter Wuille <pieter@wuille.net>
Co-Authored-By: l0rinc <pap.lorinc@gmail.com>
fa895c7283 mingw: Document mode wbx workaround (MarcoFalke)
fa359255fe Add -blocksxor boolean option (MarcoFalke)
fa7f7ac040 Return XOR AutoFile from BlockManager::Open*File() (MarcoFalke)
Pull request description:
Currently the *.dat files in the blocksdir store the data received from remote peers as-is. This may be problematic when a program other than Bitcoin Core tries to interpret them by accident. For example, an anti-virus program or other program may scan them and move them into quarantine, or delete them, or corrupt them. This may cause Bitcoin Core to fail a reorg, or fail to reply to block requests (via P2P, RPC, REST, ...).
Fix this, similar to https://github.com/bitcoin/bitcoin/pull/6650, by rolling a random XOR pattern over the dat files when writing or reading them.
Obviously this can only protect against programs that accidentally and unintentionally are trying to mess with the dat files. Any program that intentionally wants to mess with the dat files can still trivially do so.
The XOR pattern is only applied when the blocksdir is freshly created, and there is an option to disable it (on creation), so that people can disable it, if needed.
ACKs for top commit:
achow101:
ACK fa895c7283
TheCharlatan:
Re-ACK fa895c7283
hodlinator:
ACK fa895c7283
Tree-SHA512: c92a6a717da83bc33a9b8671a779eeefde2c63b192362ba1d71e6535ee31d08e2802b74acc908345197de9daac6930e4771595ee25b09acd5a67f7ea34854720
f3cfbd65f5 net: log connections failures via SOCKS5 with less severity (Vasil Dimov)
Pull request description:
It is expected to have some Bitcoin nodes unreachable some of the time. A failure to connect to an IPv4 or IPv6 node is already properly logged under category=net/severity=debug. Do the same when a connection fails when using a SOCKS5 proxy. This could be either to an .onion address or to an IPv4 or IPv6 address (via a Tor exit node).
Related: https://github.com/bitcoin/bitcoin/issues/29759
ACKs for top commit:
achow101:
ACK f3cfbd65f5
mzumsande:
Code Review ACK f3cfbd65f5
tdb3:
Code Review ACK f3cfbd65f5
Tree-SHA512: c6e83568783cb5233edac7840a00f708d27be9af87480fc73093ad99fe4bd8670d3f2c97fd6b6e2c54b8d9337746eacb9a5db6eefecc1486951996bfbb0a37f7
172c1ad026 test: expand LimitOrphan and EraseForPeer coverage (Greg Sanders)
28dbe218fe refactor: move orphanage constants to header file (Greg Sanders)
Pull request description:
Inspired by refactorings in #30000 as the coverage appeared a bit sparse.
Added some minimal border value testing, timeouts, and tightened existing assertions.
ACKs for top commit:
achow101:
ACK 172c1ad026
rkrux:
reACK [172c1ad](172c1ad026)
glozow:
reACK 172c1ad026
Tree-SHA512: e8fa9b1de6a8617612bbe9b132c9c0c9b5a651ec94fd8c91042a34a8c91c5f9fa7ec4175b47e2b97d1320d452c23775be671a9970613533e68e81937539a7d70
fa530ec543 rpc: Return precise loadtxoutset error messages (MarcoFalke)
faa5c86dbf refactor: Use untranslated error message in ActivateSnapshot (MarcoFalke)
Pull request description:
The error messages should never happen in normal operation. However, if
they do, they are helpful to return to the user to debug the issue. For
example, to notice a truncated file.
This fixes https://github.com/bitcoin/bitcoin/issues/28621
Also includes a minor refactor commit.
ACKs for top commit:
fjahr:
Code review ACK fa530ec543
ryanofsky:
Code review ACK fa530ec543, just adjusting error messages a little since last review. (Thanks!)
Tree-SHA512: 224968c9b13d082ca2ed1f6a8fcc5f51ff16d6c96bd38c3679699505b54337b99cccaf7a8474391f6b11f9ccb101977b4e626898c1217eae95802e290cf105f1
2d9d752e4f scripted-diff: Replace uint256S("str") -> uint256{"str"} (Hodlinator)
c06f2368e2 refactor: Hand-replace some uint256S -> uint256 (Hodlinator)
b74d8d58fa refactor: Add consteval uint256(hex_str) (Hodlinator)
Pull request description:
Motivation:
* Validates and converts the hex string at compile time instead of at runtime into the resulting bytes.
* Makes it possible to derive other compile time constants from `uint256`.
* Potentially eliminates runtime dependencies (`SetHexDeprecated()` is called in less places).
* Has stricter requirements than the deprecated `uint256S()` (requiring 64 chars exactly, disallows garbage at the end) and replaces it in a bunch of places.
* Makes the binary smaller (tested Guix-built x86_64-linux-gnu bitcoind binary).
* Minor: should shave off a few cycles of start-up time.
Extracted from #30377 which diverged into exploring `consteval` `ParseHex()` solutions.
ACKs for top commit:
maflcko:
rebase re-cr-ACK 2d9d752e4f🎐
stickies-v:
re-ACK 2d9d752e4f
paplorinc:
ACK 2d9d752e4f
Tree-SHA512: 39bd9320db0ed81950b5d71495eaa1d06508cc008466f2308874d70ac9ff32bc69798d2e3ef6a784868c1633fb519f60cc2111a9d0718c2663b28e78b67f7cde
When using `sendrawtransaction` the ALREADY_IN_CHAIN error help string
may be confusing.
Rename TransactionError::ALREADY_IN_CHAIN to
TransactionError::ALREADY_IN_UTXO_SET and update the rpc help string.
Remove backwards compatibility alias as no longer required.
fa3ea3b83c test: Fix intermittent issue in p2p_v2_misbehaving.py (MarcoFalke)
55555574d1 net: Log accepted connection after m_nodes.push_back (MarcoFalke)
Pull request description:
Fix the two issues reported in https://github.com/bitcoin/bitcoin/pull/30468/files#r1688444784:
* Delay a debug log line for consistency.
* Fix an intermittent test issue.
They are completely separate fixes, but both `net` related.
ACKs for top commit:
0xB10C:
Code Review ACK fa3ea3b83c
stratospher:
tested ACK fa3ea3b.
Tree-SHA512: cd6b6e164b317058a305a5c3e38c56c9a814a7469039e1143f1d7addfbc91b0a28506873356b373d97448b46cb6fbe94a1309df82e34c855540b241a09489e8b
bfd3c29e4f fuzz: fix timeout in crypter target (brunoerg)
Pull request description:
Fixes#30503
- Move SetKeyFromPassphrase to out of LIMITED_WHILE
- Remove `SetKey` calls since it is already called internally by other functions.
- Reduce number of iterations (100 is enough, no need for 10,000).
ACKs for top commit:
maflcko:
review ACK bfd3c29e4f📆
dergoegge:
utACK bfd3c29e4f
Tree-SHA512: 275ab7d07a20bfd07279a23613678993c10c166f40cdc900213b9f4d5afb107462d5f88518a0f4ce2a52f3b7950ff2c01cf74292042f16996909fcb96f827d3e
chainparams.cpp - workaround for MSVC bug triggering C7595 - Calling consteval constructors in initializer lists fails, but works on GCC (13.2.0) & Clang (17.0.6).
Complements uint256::FromHex() nicely in that it naturally does all error checking at compile time and so doesn't need to return an std::optional.
Will be used in the following 2 commits to replace many calls to uint256S(). uint256S() calls taking C-string literals are littered throughout the codebase and executed at runtime to perform parsing unless a given optimizer was surprisingly efficient. While this may not be a hot spot, it's better hygiene in C++20 to store the parsed data blob directly in the binary, without any parsing at runtime.
bf0efb4fc7 scripted-diff: Modernize naming of nChainTx and nTxCount (Fabian Jahr)
72e5d1be1f test: Add basic check for nChainTx type (Fabian Jahr)
dc2938e979 chainparams: Change nChainTx to uint64_t (Fabian Jahr)
Pull request description:
This picks up the work from #29331 and closes#29258.
This simply changes the type and addresses the comments from #29331 by changing the type in all relevant places and removing unnecessary casts. This also adds an extremely simple unit test.
Additionally this modernizes the name of `nChainTx` which helps reviewers check all use of the symbol and can make silent merge conflicts.
ACKs for top commit:
maflcko:
only rebase in scripted-diff, re-ACK bf0efb4fc7🔈
glozow:
reACK bf0efb4fc7 via range-diff
Tree-SHA512: ee4020926d0800236fe655d0c7b127215ab36b553b04d5f91494f4b7fac6e1cfe7ee298b07c0983db5a3f4786932acaa54f5fd2ccd45f2fcdcfa13427358dc3b
bbcee5a0d6 clusterlin: improve rechunking in LinearizationChunking (optimization) (Pieter Wuille)
04d7a04ea4 clusterlin: add MergeLinearizations function + fuzz test + benchmark (Pieter Wuille)
4f8958d756 clusterlin: add PostLinearize + benchmarks + fuzz tests (Pieter Wuille)
0e2812d293 clusterlin: add algorithms for connectedness/connected components (Pieter Wuille)
0e52728a2d clusterlin: rename Intersect -> IntersectPrefixes (Pieter Wuille)
Pull request description:
Part of cluster mempool: #30289
Depends on #30126, and was split off from it. #28676 depends on this.
This adds the algorithms for merging & postprocessing linearizations.
The `PostLinearize(depgraph, linearization)` function performs an in-place improvement of `linearization`, using two iterations of the [Linearization post-processing](https://delvingbitcoin.org/t/linearization-post-processing-o-n-2-fancy-chunking/201/8) algorithm. The first running from back to front, the second from front to back.
The `MergeLinearizations(depgraph, linearization1, linearization2)` function computes a new linearization for the provided cluster, given two existing linearizations for that cluster, which is at least as good as both inputs. The algorithm is described at a high level in [merging incomparable linearizations](https://delvingbitcoin.org/t/merging-incomparable-linearizations/209).
For background and references, see [Introduction to cluster linearization](https://delvingbitcoin.org/t/introduction-to-cluster-linearization/1032).
ACKs for top commit:
sdaftuar:
ACK bbcee5a0d6
glozow:
code review ACK bbcee5a0d6
instagibbs:
ACK bbcee5a0d6
Tree-SHA512: d2b5a3f132d1ef22ddf9c56421ab8b397efe45b3c4c705548dda56f5b39fe4b8f57a0d2a4c65b338462d80bb5b9b84a9a39efa1b4f390420a8005ce31817774e
73e3fa10b4 doc + test: Correct uint256 hex string endianness (Hodlinator)
Pull request description:
This PR is a follow-up to #30436.
Only changes test-code and modifies/adds comments.
Byte order of hex string representation was wrongfully documented as little-endian, but are in fact closer to "big-endian" (endianness is a memory-order concept rather than a numeric concept). `[arith_]uint256` both store their data in arrays with little-endian byte order (`arith_uint256` has host byte order within each `uint32_t` element).
**uint256_tests.cpp** - Avoid using variable from the left side of the condition in the right side. Credits to @maflcko: https://github.com/bitcoin/bitcoin/pull/30436#discussion_r1688273553
**setup_common.cpp** - Skip needless ArithToUint256-conversion. Credits to @stickies-v: https://github.com/bitcoin/bitcoin/pull/30436#discussion_r1688621638
---
<details>
<summary>
## Logical reasoning for endianness
</summary>
1. Comparing an `arith_uint256` (`base_uint<256>`) to a `uint64_t` compares the beginning of the array, and verifies the remaining elements are zero.
```C++
template <unsigned int BITS>
bool base_uint<BITS>::EqualTo(uint64_t b) const
{
for (int i = WIDTH - 1; i >= 2; i--) {
if (pn[i])
return false;
}
if (pn[1] != (b >> 32))
return false;
if (pn[0] != (b & 0xfffffffful))
return false;
return true;
}
```
...that is consistent with little endian ordering of the array.
2. They have the same endianness (but `arith_*` has host-ordering of each `uint32_t` element):
```C++
arith_uint256 UintToArith256(const uint256 &a)
{
arith_uint256 b;
for(int x=0; x<b.WIDTH; ++x)
b.pn[x] = ReadLE32(a.begin() + x*4);
return b;
}
```
### String conversions
The reversal of order which happens when converting hex-strings <=> uint256 means strings are actually closer to big-endian, see the end of `base_blob<BITS>::SetHexDeprecated`:
```C++
unsigned char* p1 = m_data.data();
unsigned char* pend = p1 + WIDTH;
while (digits > 0 && p1 < pend) {
*p1 = ::HexDigit(trimmed[--digits]);
if (digits > 0) {
*p1 |= ((unsigned char)::HexDigit(trimmed[--digits]) << 4);
p1++;
}
}
```
Same reversal here:
```C++
template <unsigned int BITS>
std::string base_blob<BITS>::GetHex() const
{
uint8_t m_data_rev[WIDTH];
for (int i = 0; i < WIDTH; ++i) {
m_data_rev[i] = m_data[WIDTH - 1 - i];
}
return HexStr(m_data_rev);
}
```
It now makes sense to me that `SetHexDeprecated`, upon receiving a shorter hex string that requires zero-padding, would pad as if the missing hex chars where towards the end of the little-endian byte array, as they are the most significant bytes. "Big-endian" string representation is also consistent with the case where `SetHexDeprecated` receives too many hex digits and discards the leftmost ones, as a form of integer narrowing takes place.
### How I got it wrong in #30436
Previously I used the less than (`<`) comparison to prove endianness, but for `uint256` it uses `memcmp` and thereby gives priority to the *lower* bytes at the beginning of the array.
```C++
constexpr int Compare(const base_blob& other) const { return std::memcmp(m_data.data(), other.m_data.data(), WIDTH); }
```
`arith_uint256` is different in that it begins by comparing the bytes from the end, as it is using little endian representation, where the bytes toward the end are more significant.
```C++
template <unsigned int BITS>
int base_uint<BITS>::CompareTo(const base_uint<BITS>& b) const
{
for (int i = WIDTH - 1; i >= 0; i--) {
if (pn[i] < b.pn[i])
return -1;
if (pn[i] > b.pn[i])
return 1;
}
return 0;
}
```
(The commit documents that `base_blob::Compare()` is doing lexicographic ordering unlike the `arith_*`-variant which is doing numeric ordering).
</details>
ACKs for top commit:
paplorinc:
ACK 73e3fa10b4
ryanofsky:
Code review ACK 73e3fa10b4
Tree-SHA512: 121630c37ab01aa7f7097f10322ab37da3cbc0696a6bbdbf2bbd6db180dc5938c7ed91003aaa2df7cf4a4106f973f5118ba541b5e077cf3588aa641bbd528f4e
a7432dd6ed logging: clarify -debug and -debugexclude descriptions (Anthony Towns)
74dd33cb0a rpc: make logging method reject "0" category and correct the help text (Vasil Dimov)
8c6f3bf163 logging, refactor: minor encapsulation improvement and use BCLog::NONE instead of 0 (Vasil Dimov)
160706aa38 logging, refactor: make category special cases explicit (Ryan Ofsky)
Pull request description:
* Move special cases from `LOG_CATEGORIES_BY_STR` to `GetLogCategory()` (suggested [here](https://github.com/bitcoin/bitcoin/pull/29419#discussion_r1547990373)).
* Remove `"none"` and `"0"` from RPC `logging` help because that help text was wrong. `"none"` resulted in an error and `"0"` was ignored itself (contrary to what the help text suggested).
* Remove unused `LOG_CATEGORIES_BY_STR[""]` (suggested [here](https://github.com/bitcoin/bitcoin/pull/29419#discussion_r1548018694)).
This is a followup to https://github.com/bitcoin/bitcoin/pull/29419, addressing leftover suggestions + more.
ACKs for top commit:
LarryRuane:
ACK a7432dd6ed
ryanofsky:
Code review ACK a7432dd6ed. Only changes since last review are removing dead if statement and adding AJ's suggested -debug and -debugexclude help improvements, which look accurate and much more clear.
Tree-SHA512: 41b997b06fccdb4c1d31f57d4752c83caa744cb3280276a337ef4a9b7012a04eb945071db6b8fad24c6a6cf8761f2f800fe6d8f3d8836f5b39c25e4f11c85bf0
Replace early returns in KeyPair::KeyPair() with asserts.
The if statements imply there is an error we are handling, but keypair_xonly_pub
and xonly_pubkey_serialize can only fail if the keypair object is malformed, i.e.,
it was created with a bad secret key. Since we check that the keypair was created
successfully before attempting to extract the public key, using asserts more
accurately documents what we expect here and removes untested branches from the code.
Move `SignSchnorr` to `KeyPair`. This makes `CKey::SignSchnorr` now
compute a `KeyPair` object and then call `KeyPair::SignSchorr`. The
notable changes are:
* Move the merkle_root tweaking out of the sign function and into
the KeyPair constructor
* Remove the temporary secp256k1_keypair object and have the
functions access m_keypair->data() directly
Current logging RPC method documentation claims to accept "0" and "none"
categories, but the "none" argument is actually rejected and the "0"
argument is ignored. Update the implementation to refuse both
categories, and remove the help text claiming to support them.
* Make the standalone function `LogCategoryToStr()` private inside
`logging.cpp` (aka `static`) - it is only used in that file.
* Make the method `Logger::GetLogPrefix()` `private` - it is only
used within the class.
* Use `BCLog::NONE` to initialize `m_categories` instead of `0`.
We later check whether it is `BCLog::NONE` (in
`Logger::DefaultShrinkDebugFile()`).
Make special cases explicit in GetLogCategory() and LogCategoryToStr()
functions. Simplify the LOG_CATEGORIES_BY_STR and LOG_CATEGORIES_BY_FLAG
mappings and LogCategoriesList() function.
This makes the maps `LOG_CATEGORIES_BY_STR` and `LOG_CATEGORIES_BY_FLAG`
consistent (one is exactly the opposite of the other).
Follow-up to #30436.
uint256 string representation was wrongfully documented as little-endian due to them being reversed by GetHex() etc, and base_blob::Compare() giving most significance to the beginning of the internal array. They are closer to "big-endian", but this commit tries to be even more precise than that.
uint256_tests.cpp - Avoid using variable from the left side of the condition in the right side.
setup_common.cpp - Skip needless ArithToUint256-conversion.
Add a `KeyPair` class which wraps the `secp256k1_keypair`. This keeps
the secret data in secure memory and enables passing the
`KeyPair` object directly to libsecp256k1 functions expecting a
`secp256k1_keypair`.
Motivation: when passing `CKeys` for taproot outputs to libsecp256k1 functions,
the first step is to create a `secp256k1_keypair` data type and use that
instead. This is so the libsecp256k1 function can determine if the key
needs to be negated, e.g., when signing.
This is a bit clunky in that it creates an extra step when using a `CKey`
for a taproot output and also involves copying the secret data into a
temporary object, which the caller must then take care to cleanse. In
addition, the logic for applying the merkle_root tweak currently
only exists in the `SignSchnorr` function.
In a later commit, we will add the merkle_root tweaking logic to this
function, which will make the merkle_root logic reusable outside of
signing by using the `KeyPair` class directly.
Co-authored-by: Cory Fields <cory-nospam-@coryfields.com>
Sanity check that using CKey/CPubKey directly vs using secp256k1_keypair objects
returns the same results for BIP341 key tweaking.
Co-authored-by: l0rinc <pap.lorinc@gmail.com>
Add benchmarks for signing with null and non-null merkle_root arguments.
Null and non-null merkle_root arguments will apply the taptweaks
H_TapTweak(P) and H_TapTweak(P | merkle_root), respectively, to the
private key during signing.
This benchmark is added to verify there are no significant performance
changes after moving the taptweak signing logic in a later commit.
Co-authored-by: l0rinc <pap.lorinc@gmail.com>
75648cea5a test: add P2A ProduceSignature coverage (Greg Sanders)
7998ce6b20 Add release note for P2A output feature (Greg Sanders)
71c9b02a04 test: add P2A coverage for decodescript (Greg Sanders)
1349e9ec15 test: Add anchor mempool acceptance test (Greg Sanders)
9d89209937 policy: stop 3rd party wtxid malleability of anchor spend (Greg Sanders)
b60aaf8b23 policy: make anchor spend standard (Greg Sanders)
455fca86cf policy: Add OP_1 <0x4e73> as a standard output type (Greg Sanders)
Pull request description:
This is a sub-feature taken out of the original proposal for ephemeral anchors #30239
This PR makes *spending* of `OP_1 <0x4e73>` (i.e. `bc1pfeessrawgf`) standard. Creation of this output type is already standard.
Any future witness output types are considered relay-standard to create, but not to spend. This preserves upgrade hooks, such as a completely new output type for a softfork such as BIP341. It also gives us a bit of room to use a new output type for policy uses.
This particular sized witness program has no other known use-cases (https://bitcoin.stackexchange.com/a/110664/17078), s it affords insufficient cryptographic security for a secure commitment to data, such as a script or a public key. This makes this type of output "keyless", or unauthenticated.
As a witness program, the `scriptSig` of the input MUST be blank, by BIP141. This helps ensure txid-stability of the spending transaction, which may be required for smart contracting wallets. If we do not use segwit, a miner can simply insert an `OP_NOP` in the `scriptSig` without effecting the result of program execution.
An additional relay restriction is to disallow non-empty witness data, which an adversary may use to penalize the "honest" transactor when RBF'ing the transaction due to the incremental fee requirement of RBF rules.
The intended use-case for this output type is to "anchor" the transaction with a spending child to bring exogenous CPFP fees into the transaction package, encouraging the inclusion of the package in a block. The minimal size of creation and spending of this output makes it an attractive contrast to outputs like `p2sh(OP_TRUE)` and `p2wsh(OP_TRUE)` which
are significantly larger in vbyte terms.
Combined with TRUC transactions which limits the size of child transactions significantly, this is an attractive option for presigned transactions that need to be fee-bumped after the fact.
ACKs for top commit:
sdaftuar:
utACK 75648cea5a
theStack:
re-ACK 75648cea5a
ismaelsadeeq:
re-ACK 75648cea5a via [diff](e7ce6dc070..75648cea5a)
glozow:
ACK 75648cea5a
tdb3:
ACK 75648cea5a
Tree-SHA512: d529de23d20857e6cdb40fa611d0446b49989eaafed06c28280e8fd1897f1ed8d89a4eabbec1bbf8df3d319910066c3dbbba5a70a87ff0b2967d5205db32ad1e
189c987386 Showing local addresses on the Node Window (Jadi)
a5d7aff867 net: Providing an interface for mapLocalHost (Jadi)
Pull request description:
This change adds a new row to the Node Window (debugwindow.ui)
under the Network section which shows the LocalAddresses.
fixes#564
<!--
*** Please remove the following help text before submitting: ***
Pull requests without a rationale and clear improvement may be closed
immediately.
GUI-related pull requests should be opened against
https://github.com/bitcoin-core/gui
first. See CONTRIBUTING.md
-->
<!--
Please provide clear motivation for your patch and explain how it improves
Bitcoin Core user experience or Bitcoin Core developer experience
significantly:
* Any test improvements or new tests that improve coverage are always welcome.
* All other changes should have accompanying unit tests (see `src/test/`) or
functional tests (see `test/`). Contributors should note which tests cover
modified code. If no tests exist for a region of modified code, new tests
should accompany the change.
* Bug fixes are most welcome when they come with steps to reproduce or an
explanation of the potential issue as well as reasoning for the way the bug
was fixed.
* Features are welcome, but might be rejected due to design or scope issues.
If a feature is based on a lot of dependencies, contributors should first
consider building the system outside of Bitcoin Core, if possible.
* Refactoring changes are only accepted if they are required for a feature or
bug fix or otherwise improve developer experience significantly. For example,
most "code style" refactoring changes require a thorough explanation why they
are useful, what downsides they have and why they *significantly* improve
developer experience or avoid serious programming bugs. Note that code style
is often a subjective matter. Unless they are explicitly mentioned to be
preferred in the [developer notes](/doc/developer-notes.md), stylistic code
changes are usually rejected.
-->
<!--
Bitcoin Core has a thorough review process and even the most trivial change
needs to pass a lot of eyes and requires non-zero or even substantial time
effort to review. There is a huge lack of active reviewers on the project, so
patches often sit for a long time.
-->
ACKs for top commit:
pablomartin4btc:
re-ACK 189c987386
furszy:
utACK 189c987
Tree-SHA512: 93f201bc6d21d81b27b87be050a447b841f01e3efb69b9eca2cc7af103023d7cd69eb5e16e2875855573ef51a5bf74a6ee6028636c1b6798cb4bb11567cb4996
Move `SetKeyFromPassphrase` to out of LIMITED_WHILE,
remove `SetKey` calls since it is already called
internally by other functions and reduce the number
of iterations.
642c885b61 Merge bitcoin-core/secp256k1#1575: release: prepare for 0.5.1
cdf08c1a2b Merge bitcoin-core/secp256k1#1576: doc: mention `needs-changelog` github label in release process
40d87b8e45 release: prepare for 0.5.1
5770226176 changelog: clarify CMake option
759bd4bbc8 doc: mention `needs-changelog` github label in release process
fded437c4c Merge bitcoin-core/secp256k1#1574: Fix compilation when extrakeys module isn't enabled
763d938cf0 ci: only enable extrakeys module when schnorrsig is enabled
af551ab9db tests: do not use functions from extrakeys module
0055b86780 Merge bitcoin-core/secp256k1#1551: Add ellswift usage example
ea2d5f0f17 Merge bitcoin-core/secp256k1#1563: doc: Add convention for defaults
ca06e58b2c Merge bitcoin-core/secp256k1#1564: build, ci: Adjust the default size of the precomputed table for signing
e2af491263 ci: Switch to the new default value of the precomputed table for signing
d94a9273f8 build: Adjust the default size of the precomputed table for signing
fcc5d7381b Merge bitcoin-core/secp256k1#1565: cmake: Bump CMake minimum required version up to 3.16
9420eece24 cmake: Bump CMake minimum required version up to 3.16
16685649d2 doc: Add convention for defaults
a5269373fa Merge bitcoin-core/secp256k1#1555: Fixed O3 replacement
b8fe33332b cmake: Fixed O3 replacement
31f84595c4 Add ellswift usage example
fe4fbaa7f3 examples: fix case typos in secret clearing paragraphs (s/, Or/, or/)
git-subtree-dir: src/secp256k1
git-subtree-split: 642c885b6102725e25623738529895a95addc4f4
7231c7630e qt: Replace deprecated LogPrintf with LogInfo in GUIUtil::LogQtInfo() (Hennadii Stepanov)
b3d3ae0680 qt, build: Drop `QT_STATICPLUGIN` macro (Hennadii Stepanov)
Pull request description:
Broken out of https://github.com/bitcoin/bitcoin/pull/30454.
Our `QT_STATICPLUGIN` macro is effectively equivalent to the Qt's `QT_STATIC` macro.
It is easy to see in the `_BITCOIN_QT_IS_STATIC` macro implementation: ebd82fa9fa/build-aux/m4/bitcoin_qt.m4 (L269-L292)
No need to handle both macros.
ACKs for top commit:
maflcko:
re-ACK 7231c7630e
TheCharlatan:
ACK 7231c7630e
Tree-SHA512: abbf21859b7ac2aaf47c5b0e075403e4cc9bc540b1565d23f51650b8932dde314586aca67fd4ed5daadebc89268baf8c18f65348fa2b836078ac24543c14cfd6
2e86f2b201 rpc: fix maybe-uninitialized compile warning in getchaintxstats (Michael Dietz)
Pull request description:
This resolves the compiler warning about potential uninitialized use of window_tx_count introduced in fa2dada.
The warning:
```
CXX rpc/libbitcoin_node_a-blockchain.o
rpc/blockchain.cpp: In function ‘getchaintxstats()::<lambda(const RPCHelpMan&, const JSONRPCRequest&)>’:
rpc/blockchain.cpp:1742:38: warning: ‘*(std::_Optional_payload_base<unsigned int>::_Storage<unsigned int, true>*)((char*)&window_tx_count + offsetof(const std::optional<unsigned int>,std::optional<unsigned int>::<unnamed>.std::_Optional_base<unsigned int, true, true>::<unnamed>)).std::_Optional_payload_base<unsigned int>::_Storage<unsigned int, true>::_M_value’ may be used uninitialized in this function [-Wmaybe-uninitialized]
1742 | ret.pushKV("txrate", double(*window_tx_count) / nTimeDiff);
|
```
ACKs for top commit:
maflcko:
lgtm ACK 2e86f2b201
theStack:
ACK 2e86f2b201
tdb3:
ACK 2e86f2b201
Tree-SHA512: c087e8f1cd68dd8df734a8400d30a95abe57ebd56cd53aef4230e425b33a23aa55b3af42abfd162e3be8c937a4c27e56abb70a4fedb10e2df64d52d577e0f262
Contributes to #564 by providing an interface for mapLocalHost
through net -> node interface -> clientModel. Later this value can be
read by GUI to show the local addresses.
No behavior change because any entries that are added in EmplaceCoinInternalDANGER
have DIRTY assigned to them after, and if they
are not inserted then they will not be
modified as before.
This prepares moving the cache entry
flags field to private access.
Co-Authored-By: Martin Leitner-Ankerl <martin.ankerl@gmail.com>
When the transactions being marked done exactly match the first chunk of
what remains of the linearization, we can just remember to skip that
chunk instead of computing a full rechunking.
Further, chop off prefixes of the input linearization that are already done,
so they don't need to be reconsidered for further rechunkings.
be419674da qt: Update translation source file (Hennadii Stepanov)
e49d858aab qt: Bump Transifex slug for 28.x (Hennadii Stepanov)
31b33019b7 qt: Pull recent translations from Transifex (Hennadii Stepanov)
Pull request description:
This PR follows our [Release Process](4c62f4b535/doc/release-process.md).
Note: (possible) vandalism/damage has been prevented by reverting the deletion of `bitcoin_af`, `bitcoin_es_MX`, and `bitcoin_ru` translations.
Required to open Transifex translations for v28.0 as it's scheduled in https://github.com/bitcoin/bitcoin/issues/29891.
The previous similar PR: https://github.com/bitcoin/bitcoin/pull/29397.
ACKs for top commit:
stickies-v:
ACK be419674da
Tree-SHA512: 76f7947af9c156c2aaf24c7f926f82e4d8e2664beb5ebde5c7cda8dd7a8dbf672b4a886302c8d189e0cb2145c0ed755f45f9cdb545e29d38bb1ec90ca18fa539
f553e6d86f refactor: remove TxidFromString (stickies-v)
285ab50ace test: replace WtxidFromString with Wtxid::FromHex (stickies-v)
9a0b2a69c4 fuzz: increase FromHex() coverage (stickies-v)
526a87ba6b test: add uint256::FromHex unittest coverage (stickies-v)
Pull request description:
Since fab6ddbee6, `TxidFromString()` has been deprecated because it is less robust than the `transaction_identifier::FromHex()` introduced in [the same PR](https://github.com/bitcoin/bitcoin/pull/30482). Specifically, it tries to recover from length-mismatches, recover from untrimmed whitespace, 0x-prefix and garbage at the end, instead of simply requiring exactly 64 hex-only characters.
In this PR, `TxidFromString` is removed completely to clean up the code and prevent further unsafe usage. Unit and fuzz test coverage on `uint256::FromHex()` and functions that wrap it is increased.
Note: `TxidFromSring` allowed users to prefix strings with "0x", this is no longer allowed for `transaction_identifier::FromHex()`, so a helper function for input validation may prove helpful in the future _(this overlaps with the `uint256::uint256S()` vs `uint256::FromHex()` future cleanup)_. It is not relevant to this PR, though, besides the fact that this unused (except for in tests) functionality is removed.
The only users of `TxidFromString` are:
- `test`, where it is straightforward to drop in the new `FromHex()` methods without much further concern
- `qt` coincontrol. There is no need for input validation here, but txids are not guaranteed to be 64 characters. This is already handled by the existing code, so again, using `FromHex()` here seems quite straightforward.
Addresses @maflcko's suggestion: https://github.com/bitcoin/bitcoin/pull/30482#discussion_r1691826934
Also removes `WtxidFromString()`, which is a test-only helper function.
### Testing GUI changes
To test the GUI coincontrol affected lines, `regtest` is probably the easiest way to quickly get some test coins, you can use e.g.
```
alias cli="./src/bitcoin-cli -regtest"
cli createwallet "coincontrol"
# generate 10 spendable outputs on 1 address
cli generatetoaddress 10 $(cli -rpcwallet=coincontrol getnewaddress)
# generate 10 spendable outputs on another address
cli generatetoaddress 10 $(cli -rpcwallet=coincontrol getnewaddress)
# make previous outputs spendable
cli generatetoaddress 100 $(cli -rpcwallet=coincontrol getnewaddress)
```
ACKs for top commit:
maflcko:
re-ACK f553e6d86f🔻
hodlinator:
ACK f553e6d86f
paplorinc:
ACK f553e6d86f
TheCharlatan:
Nice, ACK f553e6d86f
Tree-SHA512: c1c7e6ea4cbf05cf660ba178ffc4f35f0328f7aa6ad81872e2462fb91a6a22e4681ff64b3d0202a5a9abcb650c939561585cd309164a69ab6081c0765ee271ef
afd237bb5d [fuzz] Harness for version handshake (dergoegge)
a90ab4aec9 scripted-diff: Rename lazily initialized bloom filters (dergoegge)
82de1bc478 [net processing] Lazily initialize m_recent_confirmed_transactions (dergoegge)
fa0c87f19c [net processing] Lazily initialize m_recent_rejects_reconsiderable (dergoegge)
662e8db2d3 [net processing] Lazily initialize m_recent_rejects (dergoegge)
Pull request description:
This adds a fuzzing harness dedicated to the version handshake. To avoid determinism issues, the harness creates necessary components each iteration (addrman, peerman, etc). A harness like this would have easily caught https://bitcoincore.org/en/2024/07/03/disclose-timestamp-overflow/.
As a performance optimization, this PR includes a change to `PeerManager` to lazily initialize various filters (to avoid large unnecessary memory allocations each iteration).
ACKs for top commit:
brunoerg:
ACK afd237bb5d
marcofleon:
Tested ACK afd237bb5d. I compared the coverage of `net_processing` from this harness to the `process_message` and `process_messages` harnesses to see the differences. This target hits more specific parts of the version handshake. The stability looks good as well, at about 94%.
glozow:
utACK afd237bb5d lazy blooms look ok
mzumsande:
Code Review ACK afd237bb5d
Tree-SHA512: 62bba20aec0cd220e62368354891f9790b81ad75e8adf7b22a76a6d4663bd26aedc4cae8083658a75ea9043d60aad3f0e58ad36bd7bbbf93ff1d16e317bf15cc
93fb0e7897 kernel: Only setup kernel context globals once (TheCharlatan)
Pull request description:
The globals setup by the function calls when creating a new kernel context only need to be setup once. Calling them multiple times may be wasteful and has no apparent benefit.
Besides kernel users potentially creating multiple contexts, this change may also be useful for tests creating multiple setups.
ACKs for top commit:
stickies-v:
re-ACK 93fb0e7897
maflcko:
ACK 93fb0e7897👝
tdb3:
re ACK 93fb0e7897
Tree-SHA512: c8418c23b34883b9b6af2b93c48760a931c246c9190fae372fb808f573408d332f53ca43b9c783eef561c4a6681e2fb63f215c939b40a87d597c0518dabea22a
These outputs are called anchors, and allow
key-less anchor spends which are vsize-minimized
versus keyed anchors which require larger outputs
when creating and inputs when spending.
41a1a8615d gui: Hide peers details (@RandyMcMillan)
Pull request description:
Add a close (X) button to the Peers Detail panel.
Reuse the same icon used in the Console Tab.
The close button deselects the peer highlighted
in the PeerTableView and hides the detail panel.
fixes#485
Co-authored-by: @w0xlt <w0xlt@users.noreply.github.com>
ACKs for top commit:
pablomartin4btc:
re ACK 41a1a8615d
hebasto:
ACK 41a1a8615d, tested on Ubuntu 23.10.
Tree-SHA512: fc692891eec61bd1e6878f2433b478de3c69bf0b3ce3471f2faafda6f63d371e2cc125ae8290fd2ac3e4d8659031b79d85665318cfc5a9481e967ef99d245f9c
The globals setup by the function calls when creating a new kernel
context only need to be setup once. Calling them multiple times may be
wasteful and has no apparent benefit.
Besides kernel users potentially creating multiple contexts, this change
may also be useful for tests creating multiple setups.
Co-authored-by: stickies-v <stickies-v@protonmail.com>
17845e7f21 rpc: add utxo's blockhash and number of confirmations to scantxoutset output (Luis Schwab)
Pull request description:
This PR resolves#30478 by adding two fields to the `scantxoutset` RPC:
- blockhash: the blockhash that an UTXO was created
- confirmations: the number of confirmations an UTXO has relative to the chaintip.
The rationale for the first field is that a blockhash is a much more reliable identifier than the height:
> When using the scantxoutset RPC, the current behaviour is to show the block height of the UTXO. This is not optimal, as block height is ambiguous, especially in the case of a block reorganization happening at the same instant of the query. In this case, an UTXO that does not exist would be assumed to exist, unless the chain's tip hash is recorded before the scan, and make sure it still exists after, as per https://github.com/bitcoindevkit/bdk/issues/895#issuecomment-1475766797 comment by evanlinjin.
The second one was suggested by maflcko, and I agree it's useful for human users:
> While touching this, another thing to add could be the number of confirmations? I understand that this wouldn't help machine consumers of the interface, but human callers may find it useful?
This will yield an RPC output like so:
```diff
bitcoin-cli scantxoutset start "[\"addr(bc1q5q9344vdyjkcgv79ve3tldz4jmx4lf7knmnx6r)\"]"
{
"success": true,
"txouts": 185259116,
"height": 853622,
"bestblock": "00000000000000000002e97d9be8f0ddf31829cf873061b938c10b0f80f708b2",
"unspents": [
{
"txid": "fae435084345fe26e464994aebc6544875bca0b897bf4ce52a65901ae28ace92",
"vout": 0,
"scriptPubKey": "0014a00b1ad58d24ad8433c56662bfb45596cd5fa7d6",
"desc": "addr(bc1q5q9344vdyjkcgv79ve3tldz4jmx4lf7knmnx6r)#smk4xmt7",
"amount": 0.00091190,
"coinbase": false,
"height": 852741,
+ "blockhash": "00000000000000000002eefe7e7db44d5619c3dace4c65f3fdcd2913d4945c13",
+ "confirmations": 882
}
],
"total_amount": 0.00091190
}
```
ACKs for top commit:
sipa:
utACK 17845e7f21
Eunovo:
ACK 17845e7f21
tdb3:
ACK 17845e7f21
Tree-SHA512: 02366d0004e5d547522115ef0efe6794a35978db53dda12c675cfae38197bf43f0bf89ca99a3d79e3d2cff95186015fe1ab764abb8ab82bda440ae9302ad973b
The error messages should never happen in normal operation. However, if
they do, they are helpful to return to the user to debug the issue. For
example, to notice a truncated file.
b4dd7ab43e logging: use std::string_view (Anthony Towns)
558df5c733 logging: Apply formatting to early log messages (Anthony Towns)
6cf9b34440 logging: Limit early logging buffer (Anthony Towns)
0b1960f1b2 logging: Add DisableLogging() (Anthony Towns)
6bbc2dd6c5 logging: Add thread safety annotations (Anthony Towns)
Pull request description:
In order to cope gracefully with `Log*()` calls that are invoked prior to logging being fully configured (indicated by calling `StartLogging()` we buffer early log messages in `m_msgs_before_open`. This has a couple of minor issues:
* if there are many such log messages the buffer can become arbitrarily large; this can be a problem for users of libkernel that might not wish to worry about logging at all, and as a result never invoke `StartLogging()`
* early log messages are formatted before the formatting options are configured, leading to inconsistent output
Fix those issues by buffering the log info prior to formatting it, and setting a limit on the size of the buffer (dropping the oldest lines, and reporting the number of lines skipped).
Also adds some thread safety annotations, and the ability to invoke `LogInstance().DisableLogging()` if you want to disable logging entirely, for a minor efficiency improvement.
ACKs for top commit:
maflcko:
re-ACK b4dd7ab43e 🕴
ryanofsky:
Code review ACK b4dd7ab43e
TheCharlatan:
Nice, ACK b4dd7ab43e
Tree-SHA512: 966660181276939225a9f776de6ee0665e44577d2ee9cc76b06c8937297217482e6e426bdc5772d1ce533a0ba093a8556b6a50857d4c876ad8923e432a200440
fae0db0360 fuzz: Deglobalize signature cache in sigcache test (TheCharlatan)
Pull request description:
The body of the fuzz test should ideally be a pure function. If data is persisted in the cache over many iterations, and there is a crash, reproducing it from the input might be difficult. Solve this by getting rid of the global state. This is a follow-up from #30425.
ACKs for top commit:
dergoegge:
utACK fae0db0360
ryanofsky:
Code review ACK fae0db0360
Tree-SHA512: 93dcbb9f2497f13856970469042d6870f04de10fe206827a8db1aae7fc8f3ac7fd900bee7945b5fe4c9e33883268dabb15be7e7bc91cf353ffc0d118cd60e97d
647fa37cdb bench: add cluster linearization improvement benchmark (Pieter Wuille)
28549791b3 clusterlin: permit passing in existing linearization to Linearize (Pieter Wuille)
97d98718b0 clusterlin: add LinearizationChunking class (Pieter Wuille)
d5918dc3c6 clusterlin: randomize the SearchCandidateFinder search order (Pieter Wuille)
991ff9a9a4 clusterlin: use bounded BFS exploration (optimization) (Pieter Wuille)
d9b235e7d2 bench: Candidate finding and linearization benchmarks (Pieter Wuille)
46aad9b099 clusterlin: add Linearize function (Pieter Wuille)
ee0ddfe4f6 clusterlin: add chunking algorithm (Pieter Wuille)
2a41f151af clusterlin: add SearchCandidateFinder class (Pieter Wuille)
4828079db3 clusterlin: add AncestorCandidateFinder class (Pieter Wuille)
58f7e01db4 tests: framework for testing DepGraph class (Pieter Wuille)
a6e07e769a clusterlin: introduce cluster_linearize.h with Cluster and DepGraph types (Pieter Wuille)
Pull request description:
Part of cluster mempool: #30289
This introduces low-level cluster linearization code, including tests and some benchmarks. It is currently not hooked up to anything.
Ultimately, what this PR adds is a function `Linearize` which operates on instances of `DepGraph` (instances of which represent pre-processed transaction clusters) to produce and/or improve linearizations for that cluster.
To provide assurance, the code heavily relies on fuzz tests. A novel approach is used here, where the fuzz input is parsed using the serialization.h framework rather than `FuzzedDataProvider`, with a custom serializer/deserializer for `DepGraph` objects. By including serialization, it's possible to ascertain that the format can represent every relevant cluster, as well as potentially permitting the construction of ad-hoc fuzz inputs from clusters (not included in this PR, but used during development).
---
The `Linearize(depgraph, iteration_limit, rng_seed, old_linearization)` function is an implementation of the (single) LIMO algorithm, with the $S$ in every iteration found as the best out of (a) the best remaining ancestor set and (b) randomized computationally-bounded search. It incrementally builds up a linearization by finding good topologically-valid subsets to move to the front, in such a way that the resulting linearization has a diagram that is at least as good as the `old_linearization` passed in (if any).
* Despite using both best ancestor set and search, this is not Double LIMO, as no intersections between these are involved; just the best of the two.
* The `iteration_limit` and `rng_seed` only control the (b) randomized search. Even with 0 iterations, the result will be as good as the old linearization, and the included sets at every point will have a feerate at least as high as the best remaining ancestor set at that point.
The search algorithm used in the (b) step is very basic, and largely matches Section 2.1 of [How to Linearize your Cluster.](https://delvingbitcoin.org/t/how-to-linearize-your-cluster/303#h-21-searching-6). See #30286 for optimizations to make it more efficient.
For background and references, see [Introduction to cluster linearization](https://delvingbitcoin.org/t/introduction-to-cluster-linearization/1032).
ACKs for top commit:
instagibbs:
reACK 647fa37cdb
glozow:
reACK 647fa37cdb, both code and mermaid diagram look correct to me
sdaftuar:
ACK 647fa37cdb
Tree-SHA512: 52c8aa3d1d91190bf1265a947d2712e9d12f745313ffceef6ae7e3ff517d01d8b3b9b4ce6066298d59751c4ba90555a3c0171229868ba50100f588a2aa6a486d
7aa8994c6f refactor: Add FlatFileSeq member variables in BlockManager (TheCharlatan)
Pull request description:
Instead of constructing a new class every time a file operation is done, construct them once for each of the undo and block file when a new BlockManager is created.
In future, this might make it easier to introduce an abstract block store.
Historically, this was not easily possible prior to #27125.
ACKs for top commit:
danielabrozzoni:
ACK 7aa8994c6f
tdb3:
ACK 7aa8994c6f
stickies-v:
ACK 7aa8994c6f
brunoerg:
utACK 7aa8994c6f
Tree-SHA512: 7c181968c270956c90fa0f3687562239912a973b6a35ddbf49fc58733247ea9d986303cbf6f8fc16e8c2d9bf4505e866aed37f030a8c9be72e95bf3752902aa6
c399c80a09 cleanse: Use SecureZeroMemory for mingw-w64 (release) builds (fanquake)
Pull request description:
This PR switches our Windows release builds to use the [`SecureZeroMemory()`](https://learn.microsoft.com/en-us/previous-versions/windows/desktop/legacy/aa366877(v=vs.85)) provided by mingw-w64.
ACKs for top commit:
sipa:
utACK c399c80a09
TheCharlatan:
ACK c399c80a09
Tree-SHA512: dbb20b16c85061d2f9408a3cf69cecc16765f8f61b25a1707146767b664c7ad0caf36975380814ef8e7c49a30199daebac6d5d7a3585354d1adac8e9770199c6
It encapsulates a given linearization in chunked form, permitting arbitrary
subsets of transactions to be removed from the linearization. Its purpose
is adding the Intersect function, which is a crucial operation that will
be used in a further commit to make Linearize improve existing linearizations.
Switch to BFS exploration of the search tree in SearchCandidateFinder
instead of DFS exploration. This appears to behave better for real
world clusters.
As BFS has the downside of needing far larger search queues, switch
back to DFS temporarily when the queue grows too large.
Add benchmarks for known bad graphs for the purpose of search (as
an upper bound on work per search iterations) and ancestor sorting
(as an upper bound on linearization work with no search iterations).
This adds a first version of the overall linearization interface, which given
a DepGraph constructs a good linearization, by incrementally including good
candidate sets (found using AncestorCandidateFinder and SearchCandidateFinder).
This introduces a bespoke fuzzing-focused serialization format for DepGraphs,
and then tests that this format can represent any graph, roundtrips, and then
uses that to test the correctness of DepGraph itself.
This forms the basis for future fuzz tests that need to work with interesting
graphs.
This primarily adds the DepGraph class, which encapsulates precomputed
ancestor/descendant information for a given transaction cluster, with a
number of utility features (inspectors for set feerates, computing
reduced parents/children, adding transactions, adding dependencies), which
will become needed in future commits.
f46b220256 fuzz: Use BasicTestingSetup for coins_view target (TheCharlatan)
9e2a723d5d test: Add arguments for creating a slimmer setup (TheCharlatan)
Pull request description:
This adds arguments to some of the testing setup constructors for creating an environment without networking and a validation interface. This is useful for improving the performance of the utxo snapshot fuzz test, which constructs a new TestingSetup on each iteration.
Using this slimmed down `TestingSetup` in future might also make the tests a bit faster when run in aggregate.
ACKs for top commit:
maflcko:
review ACK f46b220256
dergoegge:
utACK f46b220256
Tree-SHA512: 9dc62512b127b781fc9e2d8ef2b5a9b06ebb927a8294b6d872001c553984a7eb1f348e0257b32435b34b5505b5d0323f73bdd572a673da272d3e1e8538ab49d6
fac0c3d4bf doc: Add release notes for two pull requests (MarcoFalke)
fa7b57e5f5 refactor: Replace ParseHashStr with FromHex (MarcoFalke)
fa90777245 rest: Reject truncated hex txid early in getutxos parsing (MarcoFalke)
fab6ddbee6 refactor: Expose FromHex in transaction_identifier (MarcoFalke)
fad2991ba0 refactor: Implement strict uint256::FromHex() (MarcoFalke)
fa103db2bb scripted-diff: Rename SetHex to SetHexDeprecated (MarcoFalke)
fafe4b8051 test: refactor: Replace SetHex with uint256 constructor directly (MarcoFalke)
Pull request description:
In `rest_getutxos` truncated txids such as `aa` or `ff` are accepted. This is brittle at best.
Fix it by rejecting any truncated (or overlarge) input.
----
Review note: This also starts a major refactor to rework hex parsing in Bitcoin Core, meaning that a few refactor commits are included as well. They are explained individually in the commit message and the work will be continued in the future.
ACKs for top commit:
stickies-v:
re-ACK fac0c3d4bf - only doc and test updates to address review comments, thanks!
hodlinator:
ACK fac0c3d4bf
Tree-SHA512: 473feb3fcf6118443435d1dd321006135b0b54689bfbbcb1697bb5811a449bef51f475c715de6911ff3c4ea3bdb75f601861ff93347bc4414d6b9e5298105dd7
25bf86a225 [test]: ensure `estimatesmartfee` default mode is `economical` (ismaelsadeeq)
41a2545046 [fees]: change `estimatesmartfee` default mode to `economical` (ismaelsadeeq)
Pull request description:
Fixes#30009
This PR changes the `estimatesmartfee` default mode to `economical`.
This was also suggested on IRC https://bitcoin-irc.chaincode.com/bitcoin-core-dev/2024-04-26#1021609
- `conservative` mode: This is the `estimatesmartfee` RPC mode which considers a longer history of blocks. It potentially returns a higher fee rate and is more likely to be sufficient for the desired target, but it is not as responsive to short-term drops in the prevailing fee market.
- `economical` mode: This is the `estimatesmartfee` RPC mode where estimates are potentially lower and more responsive to short-term drops in the prevailing fee market.
Since users are likely to use the default mode, this change will reduce overestimation for many users. The conservative mode remains available for those who wish to opt-in.
For an in-depth analysis of how significantly the `conservative` mode overestimates, see
https://delvingbitcoin.org/t/bitcoind-policy-estimator-modes-analysis/964.
ACKs for top commit:
instagibbs:
reACK 25bf86a225
glozow:
ACK 25bf86a225
willcl-ark:
ACK 25bf86a225
Tree-SHA512: 78ebda667eb9c8f87dcc2f0e6c14968bd1de30358dc77a13611b186fb8427ad97d9f537bad6e32e0a1aa477ccd8c64fee4d41e19308ef3cb184ff1664e6ba8a6
This is a safe replacement of the previous SetHex, which now returns an
optional to indicate success or failure.
The code is similar to the ParseHashStr helper, which will be removed in
a later commit.
These cause compile failures with _LIBCPP_REMOVE_TRANSITIVE_INCLUDES.
i.e:
```bash
In file included from init.cpp:8:
./init.h:46:54: error: no template named 'atomic' in namespace 'std'
46 | bool AppInitBasicSetup(const ArgsManager& args, std::atomic<int>& exit_status);
| ~~~~~^
1 error generated.
```
See: https://libcxx.llvm.org/DesignDocs/HeaderRemovalPolicy.html.
Now that m_txrequest and m_recent_confirmed_transactions are guarded by
the same mutex, there is no benefit to processing them separately.
Instead, just loop through pblock->vtx once.
c85accecaf [refactor] delete EraseTxNoLock, just use EraseTx (glozow)
6ff84069a5 remove obsoleted TxOrphanage::m_mutex (glozow)
61745c7451 lock m_recent_confirmed_transactions using m_tx_download_mutex (glozow)
723ea0f9a5 remove obsoleted hashRecentRejectsChainTip (glozow)
18a4355250 update recent_rejects filters on ActiveTipChange (glozow)
36f170d879 add ValidationInterface::ActiveTipChange (glozow)
3eb1307df0 guard TxRequest and rejection caches with new mutex (glozow)
Pull request description:
See #27463 for full project tracking.
This contains the first few commits of #30110, which require some thinking about thread safety in review.
- Introduce a new `m_tx_download_mutex` which guards the transaction download data structures including `m_txrequest`, the rolling bloom filters, and `m_orphanage`. Later this should become the mutex guarding `TxDownloadManager`.
- `m_txrequest` doesn't need to be guarded using `cs_main` anymore
- `m_recent_confirmed_transactions` doesn't need its own lock anymore
- `m_orphanage` doesn't need its own lock anymore
- Adds a new `ValidationInterface` event, `ActiveTipChanged`, which is a synchronous callback whenever the tip of the active chainstate changes.
- Flush `m_recent_rejects` and `m_recent_rejects_reconsiderable` on `ActiveTipChanged` just once instead of checking the tip every time `AlreadyHaveTx` is called. This should speed up calls to that function (no longer comparing a block hash each time) and removes the need to lock `cs_main` every time it is called.
Motivation:
- These data structures need synchronization. While we are holding `m_tx_download_mutex`, these should hold:
- a tx hash in `m_txrequest` is not also in `m_orphanage`
- a tx hash in `m_txrequest` is not also in `m_recent_rejects` or `m_recent_confirmed_transactions`
- In the future, orphan resolution tracking should also be synchronized. If a tx has an entry in the orphan resolution tracker, it is also in `m_orphanage`, and not in `m_txrequest`, etc.
- Currently, `cs_main` is used to e.g. sync accesses to `m_txrequest`. We should not broaden the scope of things it locks.
- Currently, we need to know the current chainstate every time we call `AlreadyHaveTx` so we can decide whether we should update it. Every call compares the current tip hash with `hashRecentRejectsChainTip`. It is more efficient to have a validation interface callback that updates the rejection filters whenever the chain tip changes.
ACKs for top commit:
instagibbs:
reACK c85accecaf
dergoegge:
Code review ACK c85accecaf
theStack:
Light code-review ACK c85accecaf
hebasto:
ACK c85accecaf, I have reviewed the code and it looks OK.
Tree-SHA512: c3bd524b5de1cafc9a10770dadb484cc479d6d4c687d80dd0f176d339fd95f73b85cb44cb3b6b464d38a52e20feda00aa2a1da5a73339e31831687e4bd0aa0c5
Instead of constructing a new class every time a file operation is done,
construct them once for each of the undo and block file when a new
BlockManager is created.
In future, this might make it easier to introduce an abstract block
store.
SetHex is fragile, because it accepts any non-hex input or any length of
input, without error feedback. This can lead to issues when the input is
truncated or otherwise corrupted.
Document the problem by renaming the method.
In the future, the fragile method should be removed from the public
interface.
-BEGIN VERIFY SCRIPT-
sed -i 's/SetHex/SetHexDeprecated/g' $( git grep -l SetHex ./src )
-END VERIFY SCRIPT-
09ce3501fa fix: Make TxidFromString() respect string_view length (Hodlinator)
01e314ce0a refactor: Change base_blob::SetHex() to take std::string_view (Hodlinator)
2f5577dc2e test: uint256 - Garbage suffixes and zero padding (Hodlinator)
f11f816800 refactor: Make uint256_tests no longer use deprecated BOOST_CHECK() (Hodlinator)
f0eeee2dc1 test: Add test for TxidFromString() behavior (Ryan Ofsky)
Pull request description:
### Problem
Prior to this, `TxidFromString()` was passing `string_view::data()` into `uint256S()` which meant it would only receive the a naked `char*` pointer and potentially scan past the `string_view::length()` until it found a null terminator (or some other non-hex character).
Appears to have been a fully dormant bug as callers were either passing a string literal or `std::string` directly to `TxidFromFromString()`, meaning a null terminator always existed at `pointer[length()]`. Bug existed since original merge of `TxidFromString()`.
### Solution
Make `uint256S()` (and `base_blob::SetHex()`) take and operate on `std::string_view` instead of `const char*` and have `TxidFromString()` pass that in.
(PR was prompted by comment in https://github.com/bitcoin/bitcoin/pull/30377#issuecomment-2208857200 (referring to https://github.com/bitcoin/bitcoin/pull/28922#discussion_r1404437378)).
ACKs for top commit:
maflcko:
re-ACK 09ce3501fa🕓
paplorinc:
ACK 09ce3501fa
ryanofsky:
Code review ACK 09ce3501fa. I think the current code changes are about as small as you could make to fix the bug without introducing a string copy, and the surrounding test improvements are all very nice and welcome.
Tree-SHA512: c2c10551785fb6688d1e2492ba42a8eee4c19abbe8461bb0774d56a70c23cd6b0718d2641632890bee880c06202dee148126447dd2264eaed4f5fee7e1bcb581
29eafd5733 rpc: doc: use "output script" terminology consistently in "asm"/"hex" results (Sebastian Falbesoner)
Pull request description:
The wording "public key script" was likely chosen as a human-readable form of the technical term `scriptPubKey`, but it doesn't seem to be really widespread. Replace it by the more (probably most?) common term "output script" instead. Note that the argument for the `decodescript` RPC is not necessarily an output script (it could e.g. be also a redeem script), so in this case we just stay generic and use "script".
See also the draft BIP "Terminology for Transaction Components" (https://github.com/murchandamus/bips/blob/2022-04-tx-terminology/bip-tx-terminology.mediawiki) from murchandamus which suggests to use "output script" as well.
Affects the help text of the following RPCs:
- decodepsbt
- decoderawtransaction
- decodescript
- getblock (if verbosity=3)
- getrawtransaction (if verbosity=2,3)
- gettxout
ACKs for top commit:
maflcko:
ACK 29eafd5733
achow101:
ACK 29eafd5733
BrandonOdiwuor:
ACK 29eafd5733
tdb3:
ACK 29eafd5733
Tree-SHA512: 62eb92d42bc44e36dc3090df7b248a123868a74af253d2046de02086e688bf6ff98307b927ba2fee3d599f85e073aeb8eca90ed15105ca63b648b6796cfa340b
Otherwise, the debug log could read confusingly, when the getpeerinfo()
RPC (calling GetNodeStats) happens after the "accepted connection" log
line, but returns an empty list.
For example, the following timeline in the debug log could correspond to
a getpeerinfo reply that is empty:
[net] [net.cpp:3764] [CNode] Added connection peer=0
[net] [net.cpp:1814] [CreateNodeFromAcceptedSocket] connection from 127.0.0.1:45154 accepted
[http] [httpserver.cpp:305] [http_request_cb] Received a POST request for / from 127.0.0.1:33320
[httpworker.1] [rpc/request.cpp:232] [parse] ThreadRPCServer method=getpeerinfo user=__cookie__
Fix it by moving the log line.
Prior to this, passing string_view::data() into uint256S() meant the latter would only receive the a naked char* pointer and potentially scan past the string_view::length() until it found a null terminator (or some other non-hex character).
Appears to have been a fully dormant bug as callers were either passing a string literal or std::string directly to TxidFromFromString(), meaning null terminator always existed at pointer[length()]. Bug existed since original merge of TxidFromString(), discussed in https://github.com/bitcoin/bitcoin/pull/28922#discussion_r1404437378.
6a5e9e40e1 doc: use proper doxygen formatting for CTxMemPool::cs (Vasil Dimov)
Pull request description:
Having `@par title` followed by an empty line renders improperly in Doxygen - it results in a paragraph with a title but without a body.
https://www.doxygen.nl/manual/commands.html#cmdpar
This also results in a compiler warning (or error) with Clang 19:
```
./txmempool.h:368:34: error: empty paragraph passed to '@par' command [-Werror,-Wdocumentation]
368 | * @par Consistency guarantees
| ~~~~~~~~~~~~~~~~~~~~~~~~~~^
1 error generated.
```
ACKs for top commit:
maflcko:
review ACK 6a5e9e40e1
tdb3:
ACK 6a5e9e40e1
Tree-SHA512: 2c4c9e5fd4bd44754800a9bcfff74df101afc060b84451c45aa098e4ceb05a47f28a36f8473b31222552fad6339b752a148e6b1c7d41c2003f515b3eb4060902
Having `@par title` followed by an empty line renders improperly in
Doxygen - it results in a paragraph with a title but without a body.
https://www.doxygen.nl/manual/commands.html#cmdpar
This also results in a compiler warning (or error) with Clang 19:
```
./txmempool.h:368:34: error: empty paragraph passed to '@par' command [-Werror,-Wdocumentation]
368 | * @par Consistency guarantees
| ~~~~~~~~~~~~~~~~~~~~~~~~~~^
1 error generated.
```
When using CMake, the user can select the MSVC runtime library to be:
1) Statically-linked (with the corresponding `x64-windows-static` vcpkg
triplet) or
2) Dynamically-linked (with the corresponding `x64-windows` vcpkg
triplet)
In the latter case, the compiler emits the C4273 warning.
As the "Necessary on some platforms" comment does not apply to MSVC,
skip the declaration for MSVC.
The body of the fuzz test should ideally be a pure function. If data is
persisted in the cache over many iterations, and there is a crash,
reproducing it from the input might be difficult.
Adds more testing options for creating an environment without networking
and a validation interface. This is useful for improving the performance
of the utxo snapshot fuzz test, which constructs a new TestingSetup on
each iteration.
The formatting of log messages isn't defined until StartLogging() is
called; so can't be correctly applied to early log messages from prior
to that call. Instead of saving the output log message, save the inputs
to the logging invocation (including time, mocktime and thread name),
and format those inputs into a log message when StartLogging() is called.
23333b7ed2 net: Allow DNS lookups on nodes with IPV6 lo only (Max Edwards)
Pull request description:
This is similar to (but does not fix) https://github.com/bitcoin/bitcoin/issues/13155 which I believe is the same issue but in libevent.
The issue is on a host that has IPV6 enabled but only a loopback IP address `-proxy=[::1]` will fail as `[::1]` is not considered valid by `getaddrinfo` with `AI_ADDRCONFIG` flag. I think the loopback interface should be considered valid and we have a functional test that will try to test this: `feature_proxy.py`.
To replicate the issue, run `feature_proxy.py` inside a docker container that has IPV6 loopback ::1 address without specifically giving that container an external IPV6 address. This should be the default with recent versions of docker. IPV6 on loopback interface was enabled in docker engine 26 and later ([https://docs.docker.com/engine/release-notes/26.0/#bug-fixes-and-enhancements-2](https://docs.docker.com/engine/release-notes/26.0/#bug-fixes-and-enhancements-2)).
`AI_ADDRCONFIG` was introduced to prevent slow DNS lookups on systems that were IPV4 only.
References:
Man section on `AI_ADDRCONFIG`:
```
If hints.ai_flags includes the AI_ADDRCONFIG flag, then IPv4 addresses are returned in the list pointed to by res only if the local system has at least one IPv4 address configured, and IPv6 addresses
are returned only if the local system has at least one IPv6 address configured. The loopback address is not considered for this case as valid as a configured address. This flag is useful on, for ex‐
ample, IPv4-only systems, to ensure that getaddrinfo() does not return IPv6 socket addresses that would always fail in connect(2) or bind(2).
```
[AI_ADDRCONFIG considered harmful Wiki entry by Fedora](https://fedoraproject.org/wiki/QA/Networking/NameResolution/ADDRCONFIG)
[Mozilla discussing slow DNS without AI_ADDRCONFIG and also localhost issues with it](https://bugzilla.mozilla.org/show_bug.cgi?id=467497)
ACKs for top commit:
achow101:
ACK 23333b7ed2
tdb3:
ACK 23333b7ed2
pinheadmz:
ACK 23333b7ed2
Tree-SHA512: 5ecd8c72d1e1c28e3ebff07346381d74eaddef98dca830f6d3dbf098380562fa68847d053c0d84cc8ed19a45148ceb5fb244e4820cf63dccb10ab3db53175020
55b6d7be68 validation: Don't load a snapshot if it's not in the best header chain. (Martin Zumsande)
Pull request description:
This was suggested by me in the discussion of #30288, which has more context.
If the snapshot is not an ancestor of the most-work header (`m_best_header`), syncing from that alternative chain leading to `m_best_header` should be prioritised. Therefore it's not useful loading the snapshot in this situation.
If the other chain turns out to be invalid or the chain with the snapshot retrieves additional headers so that it's the most-work one again (see functional test), `m_best_header` will change and loading the snapshot will be possible again.
Because of the work required to generate a conflicting headers chain, a situation with two conflicting chains should only be possible under extreme circumstances, such as major forks.
ACKs for top commit:
fjahr:
re-ACK 55b6d7be68
achow101:
ACK 55b6d7be68
alfonsoromanz:
Re ACK 55b6d7be68
Tree-SHA512: 4fbea5ab1038ae353fc949a186041cf9b397e7ce4ac59ff36f881c9437b4f22ada922490ead5b2661389eb1ca0f3d1e7e7e6a4261057678643e71594a691ac36
