6ed424f2db wallet: fix, detect blank legacy wallets in IsLegacy (furszy)
Pull request description:
Blank legacy wallets do not have active SPKM. They can only be
detected by checking the descriptors' flag or the db format.
This enables the migration of blank legacy wallets in the GUI.
To test this:
1) Create a blank legacy wallet.
2) Try to migrate it using the GUI's toolbar "Migrate Wallet" button.
-> In master: The button will be disabled because `CWallet::IsLegacy()` returns false for blank legacy wallet.
-> In this PR: the button will be enabled, allowing the migration of legacy wallets.
ACKs for top commit:
achow101:
ACK 6ed424f2db
tdb3:
ACK 6ed424f2db
glozow:
ACK 6ed424f2db
Tree-SHA512: c06c4c4c2e546ccb033287b9aa3aee4ca36b47aeb2fac6fbed5de774b65caef9c818fc8dfdaac6ce78839b2d5d642a5632a5b44c5e889ea169ced80ed50501a7
The re-init is expensive, so skip it if there is no need.
Also, add an even faster fuzz target utxo_snapshot_invalid, which does
not need any re-init at all.
f550a8e035 Rename ReleaseWallet to FlushAndDeleteWallet (furszy)
64e736d79e wallet: WaitForDeleteWallet, do not expect thread safety (Ryan Ofsky)
8872b4a6ca wallet: rename UnloadWallet to WaitForDeleteWallet (furszy)
5d15485aaf wallet: unload, notify GUI as soon as possible (furszy)
Pull request description:
Coming from #29073.
Applied ryanofsky suggested changes on https://github.com/bitcoin/bitcoin/issues/29073#issuecomment-2274237242 with few modifications coming from https://github.com/bitcoin/bitcoin/pull/18338#issuecomment-605060348.
The only point I did not tackle from https://github.com/bitcoin/bitcoin/pull/18338#issuecomment-605060348 is:
> * Move log print and flush out of ReleaseWallet into CWallet destructor
Because it would mean every `CWallet` object would flush data to disk during destruction. Which is not necessary for wallet tool utilities and unit tests.
ACKs for top commit:
achow101:
ACK f550a8e035
ryanofsky:
Code review ACK f550a8e035. Just a simple rename since last review
ismaelsadeeq:
Re-ACK f550a8e035
Tree-SHA512: e2eb69bf36883c514f601f4838ae6a41113996b9559abf8dc2b46e16bbcdad401195ac0f2b9d1fb55a10e78bb8ea9953788a168c80474e3f101350d208cb3bd2
1610643c8b chainparams: add mainnet assumeutxo param at height 840_000 (Sjors Provoost)
Pull request description:
This adds snapshot parameters for mainnet block 840,000.
You can generate the snapshot yourself using `./contrib/devtools/utxo_snapshot.sh` or download my torrent:
* torrent: `magnet:?xt=urn:btih:596c26cc709e213fdfec997183ff67067241440c&dn=utxo-840000.dat&tr=udp%3A%2F%2Ftracker.bitcoin.sprovoost.nl%3A6969`
It would be a good idea to test:
1. That you can produce the same snapshot file, sha256 sum:
```
dc4bb43d58d6a25e91eae93eb052d72e3318bd98ec62a5d0c11817cefbba177b utxo-840000.dat
```
2. That the snapshot works
ACKs for top commit:
fjahr:
re-ACK 1610643c8b
achow101:
ACK 1610643c8b
theStack:
Tested ACK 1610643c8b
mzumsande:
tested ACK 1610643c8b
willcl-ark:
tACK 1610643c8b
Tree-SHA512: 581d8e86379bb044324f04f8559dd0a8946b6e2b145d5f25b38727b30b8cf13d6ac3c8777ff06554d3cf1a072809f7b5fbd693239868578f25dceafe5ba5f57c
9b29755520 Deduplicate list of chain strings in RPC help texts (Martin Saposnic)
Pull request description:
As mentioned in issue https://github.com/bitcoin/bitcoin/issues/30645:
Many command line parameter and RPC help texts currently contain the list of chain/network names hardcoded ("main, test, testnet4, signet, regtest"), which is error-prone as it can easily happen to miss an instance if the list ever changes again.
This PR deduplicates the list of possible chain/network strings in RPC/parameter help texts, and it creates a macro `LIST_CHAIN_NAMES` in src/chainparamsbase.h. In the future, there is only 1 place where that list of possible values lives, so maintainability is improved and errors are avoided.
All three places where this change impacts:
```
./bitcoin-cli --help
./bitcoin-cli help getblockchaininfo
./bitcoin-cli help getmininginfo
```
They all return the correct string `"main, test, testnet4, signet, regtest"`
See https://github.com/bitcoin/bitcoin/pull/30642#discussion_r1714711575
ACKs for top commit:
maflcko:
lgtm ACK 9b29755520
achow101:
ACK 9b29755520
MarnixCroes:
ACK 9b29755520
theStack:
ACK 9b29755520
danielabrozzoni:
ACK 9b29755520
Tree-SHA512: 1e961bcbe40b0f17a87a2437eb4ba1bb89468fd1b5a39599d72a00ef75cb4009e7d2f05d0a621bb904fecf681c55b8a219fcfe4d44d5d27f27cdda20882b1323
8f2522d242 gui: Use menu for wallet migration (Ava Chow)
d56a450bf5 gui: Use wallet name for wallet migration rather than WalletModel (Ava Chow)
c3918583dd gui: don't remove wallet manually before migration (furszy)
bfba63880f gui: Consolidate wallet display name to GUIUtil function (Ava Chow)
28fc562f26 wallet, interfaces: Include database format in listWalletDir (Ava Chow)
Pull request description:
Currently the Migrate Wallet menu item can only be used to migrate the currently loaded wallet. This is not suitable for the future when legacy wallets can no longer be loaded at all, but should still be able to be migrated. This PR changes that menu item into a menu list like Open Wallet and lets users migrate any legacy wallet in their wallet directory regardless of the wallets loaded.
One issue I ran into was dealing with encrypted wallets. Ideally, we would detect whether a wallet is encrypted, and prompt the user for their passphrase at that time. However, that's actually difficult to do in the GUI since migration will unload the wallet if it was already loaded, and reload it without connecting it to any signals or interfaces. Only then can it detect whether a wallet is encrypted, but then there is no `WalletModel` or even an `interfaces::Wallet` that the GUI could use to unlock it via a callback.
To deal with this, I've opted to just add a button to the migration dialog box that has the user enter their passphrase first, along with instructional text to use that button if their wallet was encrypted. If the user enters the wrong passphrase or clicked the other button that does not prompt for the passphrase, migration will fail with a message indicating that the passphrase was incorrect.
ACKs for top commit:
hebasto:
ACK 8f2522d242.
furszy:
ACK 8f2522d
Tree-SHA512: a0e3b70dbfcacb89617956510ebcea94cad8617a987c68fe39fa16ac1721190b7cf7afc156c39b9032920cfb67b5d4ca28791681f5021d92d16acc691387afa1
Once legacy wallets can no longer be loaded, we need to be able to
migrate them without loading. Thus we should use a menu that lists the
wallets in the wallet directory instead of an action which migrates the
currently loaded wallet.
In 6bfa26048d the testnet4 timewarp
attack fix block time variation was increased from the Great
Consensus Cleanup value of 600s to 7200s on the thesis that this
allows miners to always create blocks with the current time. Sadly,
doing so does allow for some nonzero inflation, even if not a huge
amount.
While it could be that some hardware ignores the timestamp provided
to it over Stratum and forces the block header timestamp to the
current time, I'm not aware of any such hardware, and it would also
likely suffer from random invalid blocks due to relying on NTP
anyway, making its existence highly unlikely.
This leaves the only concern being pools, but most of those rely on
work generated by Bitcoin Core (in one way or another, though when
spy mining possibly not), and it seems likely that they will also
not suffer any lost work. While its possible that a pool does
generate invalid work due to spy mining or otherwise custom logic,
it seems unlikely that a substantial portion of hashrate would do
so, making the difference somewhat academic (any pool that screws
this up will only do so once and the network would come out just
fine).
Further, while we may end up deciding these assumptions were
invalid and we should instead use 7200s, it seems prudent to try
with the value we "want" on testnet4, giving us the ability to
learn if the compatibility concerns are an issue before we go to
mainnet.
To prepare for migrating wallets that are not loaded, when migration
occurs in the GUI, it should not rely on a WalletModel existing.
Co-authored-by: furszy <matiasfurszyfer@protonmail.com>
fa6fe43207 net: Clarify that m_addr_local is only set once (MarcoFalke)
Pull request description:
The function is supposed to be only called once when the version msg arrives (a single time). Calling it twice would be an internal logic bug. However, the `LogError` in this function has many issues:
* If the error happens in tests, as is the case for the buggy fuzz test, it will go unnoticed
* It is dead code, unless a bug is introduced to execute it
Fix all issues by using `Assume(!m_addr_local.IsValid())` instead. Idea taken from https://github.com/bitcoin/bitcoin/pull/30364#discussion_r1680530382
ACKs for top commit:
achow101:
ACK fa6fe43207
mzumsande:
utACK fa6fe43207
glozow:
ACK fa6fe43207
Tree-SHA512: 8c1e8c524768f4f36cc50110ae54ee423e057a963ff78f736f3bf92df1ce5af28e3e0149153780897944e1d5c22ddbca9dac9865d9f4d44afffa152bc8559405
It is expensive to construct, and only one test uses it.
Fix both issues by disallowing the construction and moving it to the
single test that uses it.
Blank legacy wallets do not have active SPKM. They can
only be detected by checking the descriptors' flag or
the db format.
This enables the migration of blank legacy wallets in
the GUI.
The following bitcoind parameters / RPC calls missed the "testnet4"
network string:
- `-chain=` parameter
- `getblockchaininfo` RPC, "chain" result
- `getmininginfo` RPC, "chain" result
86b38529d5 qa: a fuzz target for the block index database (Antoine Poinsot)
Pull request description:
This introduces a small fuzz target for `CBlockTreeDB` which asserts a few invariants by using an in-memory LevelDb.
ACKs for top commit:
achow101:
ACK 86b38529d5
TheCharlatan:
Re-ACK 86b38529d5
maflcko:
review ACK 86b38529d5🥒
brunoerg:
utACK 86b38529d5
Tree-SHA512: ab75b4ae1c7e0a4b15f8a6ceffdf509fbc79833e6ea073ecef68558d53b83663d1b30362aaa2d77c22b8890a572f5b1d4b1c5abbca483c8c8f9b1fb5b276a59a
401cc4ec70 fuzz: improve scriptpubkeyman target (brunoerg)
Pull request description:
Fixes#30541
This PR aims to improve `scriptpubkeyman` target to avoid timeouts. The input provided in #30541 takes too much time to run because it basically calls only `MarkUnusedAddresses` (300 times * number of spks). The following changes were made to improve it:
- Reduce keypool size.
- When calling `MarkUnusedAddresses`, do it with one of the spks per iteration.
- Remove the specific `AddDescriptorKey` call since it is already covered with `AddWalletDescriptor`.
- Limit number of iterations to a reasonable value.
ACKs for top commit:
maflcko:
lgtm ACK 401cc4ec70
achow101:
ACK 401cc4ec70
Tree-SHA512: 941812bc6d991dd03675a2974ce1b839494ca7f6e6d8a22c689d4bf4fed2dac5491246998f19cb15dbff516fdd8eeda27e7628c3206d45f57dc292bc05624a5c
15aa7d0236 gui, qt: brintToFront workaround for Wayland (pablomartin4btc)
Pull request description:
There are known issues around handling windows focus in `Wayland` ([this one specific](https://bugs.kde.org/show_bug.cgi?id=462574) in KDE but also in [gnome](https://gitlab.gnome.org/GNOME/gnome-shell/-/issues/730)).
The idea is that the workaround will be executed if `bitcoin-qt` is running using `Wayland` platform (e.g.: `QT_QPA_PLATFORM=wayland ./src/qt/bitcoin-qt -regtest`), since the workaround behaviour looks like re-opening the window again (which I tried to fix by moving the window to the original position and/ or re-setting the original geometry without success) while in `X11` (not sure in Mac) the current `GUIUtil::brintToFront` actually sets the focus to the desired window, keeping its original position as expected, and I didn't want to change that (`X11` behaviour).
The solution was [initially discussed](https://github.com/bitcoin-core/gui/pull/817#issuecomment-2256158902) with hebasto in #817.
ACKs for top commit:
hebasto:
ACK 15aa7d0236.
Tree-SHA512: 141d6cc4a618026e551627b9f4cc284285980db02a54a7b19c7de91e8c5adccf0c1d67380625146b5413e58c59f39c9e944ed5ba68cb8644f67647518918b6f7
6b2dcba076 wallet: List sqlite wallets with empty string name (Ava Chow)
3ddbdd1815 wallet: Ignore .bak files when listing wallet files (Ava Chow)
Pull request description:
When the default wallet is migrated, we do not rename the wallet so we end up having a descriptor wallet with the empty string as its name and the wallet.dat file in the root of the walletdir. This is supposed to be an unsupported configuration and there is no other way to achieve this (other than file copying), but the wallet loading code does not disallow loading such wallets. However `listwalletdir` does not currently list the default wallet if it is sqlite. This is confusing to users, so change `listwalletdir` to include these wallets.
Additionally, the migration of the default wallet, and of any plain wallet files in the walletdir, produces a backup file in the walletdir itself. Since these backups are a BDB file, `listwalletdir` will detect them as being another wallet that we could open, but this is erroneous and could lead to confusion and potentially funds loss if both the backup and the migrated wallet are in use simultaneously. To reduce the likelihood of this issue, don't list these wallets in `listwalletdir`.
***
Possibly we could have more stringent checks on loading to resolve these issues, but I'm concerned that that will just confuse users and gratuitously break things that already worked.
Since the original intent was to disallow default wallets for sqlite/descriptors, a possible alternative would be to prevent people from loading such wallets and change migration to rename those wallets. However, given that this behavior with migrating default wallets has existed since default wallet migration was fixed, I think that making such a change would be confusing and break things for no good reason. Although perhaps we should still do the renaming.
For the backups, we could also change loading to refuse to load any wallet named with `.bak` (or `.legacy.bak`) as such wallets can still be loaded by giving the path to them directly, which some users may do to "restore" the backup. However restricting what can be loaded based on filename seems a little heavyhanded. It wouldn't be funds loss though since the correct way to restore the backup is with `restorewallet`.
ACKs for top commit:
fjahr:
Code review ACK 6b2dcba076
furszy:
Code ACK 6b2dcba076
glozow:
ACK 6b2dcba076
Tree-SHA512: 0b033f6ed55830f8a054afea3fb2cf1fa82a94040053ebfaf123bda36c99f45d3f01a2aec4ed02fed9c61bb3d320b047ed892d7f6644b5a356a7bc5974b10cff
055bc05792 policy/feerate.h: avoid constraint self-dependency (Matt Whitlock)
138f867156 add missing #include <cstdint> for GCC 15 (Matt Whitlock)
Pull request description:
#30612 with changes made.
GCC 15 introduces three build failures:
* Two are related to missing includes. You can't use `uint16_t` et al. without including `<cstdint>`.
* The third is harder to understand but easy to fix. GCC changed something about the way templates are instantiated when checking type constraints, and now there is a dependency loop while checking `std::optional<CFeeRate>`. This manifests as the following compile-time mess:
```
In file included from /usr/lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/format:48,
from /usr/lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/bits/chrono_io.h:39,
from /usr/lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/chrono:3362,
from ./util/time.h:9,
from ./primitives/block.h:12,
from ./blockencodings.h:8,
from blockencodings.cpp:5:
/usr/lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/type_traits: In substitution of 'template<class _Up> requires !(is_same_v<std::optional<_Tp>, typename std::remove_cvref<_It2>::type>) && (is_constructible_v<_Tp, const _Up&>) && (__construct_from_contained_value<_Up, typename std::remove_cv< <template-parameter-1-1> >::type>) constexpr std::optional<CFeeRate>::optional(const std::optional<_Tp>&) [with _Up = CFeeRate]':
/usr/lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/type_traits:1140:25: required by substitution of 'template<class _Tp, class ... _Args> using std::__is_constructible_impl = std::__bool_constant<__is_constructible(_Tp, _Args ...)> [with _Tp = CFeeRate; _Args = {std::optional<CFeeRate>&}]'
1140 | = __bool_constant<__is_constructible(_Tp, _Args...)>;
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/usr/lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/type_traits:1145:12: required from 'struct std::is_constructible<CFeeRate, std::optional<CFeeRate>&>'
1145 | struct is_constructible
| ^~~~~~~~~~~~~~~~
/usr/lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/type_traits:178:35: required by substitution of 'template<class ... _Bn> std::__detail::__first_t<std::integral_constant<bool, false>, typename std::enable_if<(!(bool)(_Bn::value)), void>::type ...> std::__detail::__or_fn(int) [with _Bn = {std::is_constructible<CFeeRate, std::optional<CFeeRate>&>, std::is_convertible<std::optional<CFeeRate>&, CFeeRate>, std::is_constructible<CFeeRate, std::optional<CFeeRate> >, std::is_convertible<std::optional<CFeeRate>, CFeeRate>, std::is_constructible<CFeeRate, const std::optional<CFeeRate>&>, std::is_convertible<const std::optional<CFeeRate>&, CFeeRate>, std::is_constructible<CFeeRate, const std::optional<CFeeRate> >, std::is_convertible<const std::optional<CFeeRate>, CFeeRate>}]'
178 | __enable_if_t<!bool(_Bn::value)>...>;
| ^~~~~
/usr/lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/type_traits:196:41: required from 'struct std::__or_<std::is_constructible<CFeeRate, std::optional<CFeeRate>&>, std::is_convertible<std::optional<CFeeRate>&, CFeeRate>, std::is_constructible<CFeeRate, std::optional<CFeeRate> >, std::is_convertible<std::optional<CFeeRate>, CFeeRate>, std::is_constructible<CFeeRate, const std::optional<CFeeRate>&>, std::is_convertible<const std::optional<CFeeRate>&, CFeeRate>, std::is_constructible<CFeeRate, const std::optional<CFeeRate> >, std::is_convertible<const std::optional<CFeeRate>, CFeeRate> >'
196 | : decltype(__detail::__or_fn<_Bn...>(0))
| ~~~~~~~~~~~~~~~~~~~~~~~~~^~~
/usr/lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/optional:824:45: required from 'constexpr const bool std::optional<CFeeRate>::__construct_from_contained_value<CFeeRate, CFeeRate>'
824 | = !__converts_from_optional<_Tp, _From>::value;
| ^~~~~
/usr/lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/optional:884:7: required by substitution of 'template<class _Up> requires !(is_same_v<std::optional<_Tp>, typename std::remove_cvref<_It2>::type>) && (is_constructible_v<_Tp, const _Up&>) && (__construct_from_contained_value<_Up, typename std::remove_cv< <template-parameter-1-1> >::type>) constexpr std::optional<CFeeRate>::optional(const std::optional<_Tp>&) [with _Up = CFeeRate]'
884 | && __construct_from_contained_value<_Up>
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
./validation.h:164:41: required from here
164 | return MempoolAcceptResult(state);
| ^
/usr/lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/optional:886:2: required by the constraints of 'template<class _Tp> template<class _Up> requires !(is_same_v<std::optional<_Tp>, typename std::remove_cvref<_It2>::type>) && (is_constructible_v<_Tp, const _Up&>) && (__construct_from_contained_value<_Up, typename std::remove_cv< <template-parameter-1-1> >::type>) constexpr std::optional<_Tp>::optional(const std::optional<_From>&)'
/usr/lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/optional:884:14: error: satisfaction of atomic constraint '__construct_from_contained_value<_Up, typename std::remove_cv< <template-parameter-1-1> >::type> [with _Tp = _Tp; _Up = _Up]' depends on itself
884 | && __construct_from_contained_value<_Up>
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
```
It is easiest to solve this by changing the `static_assert` in the explicit `CFeeRate` constructor to a SFINAE by using a type constraint on the function template parameter.
We already [downstreamed](https://github.com/gentoo/gentoo/pull/38015) these fixes in Gentoo.
ACKs for top commit:
stickies-v:
ACK 055bc05792
Tree-SHA512: ce9cb27bcd9b0f4bbc80951e45cf7127112dcb7f9937bcb0167b362026d35beecb1255354746de0aac82e03c41eaccbe26acbfe0ddff2ee1e5a8634673f4f4ba
fa5755b0a8 doc: rpc: Use "output script" consistently (2/2) (MarcoFalke)
Pull request description:
Small follow-up to https://github.com/bitcoin/bitcoin/pull/30408 to fixup the RPCs that were forgotten.
ACKs for top commit:
theStack:
lgtm ACK fa5755b0a8
Tree-SHA512: f1fc0aabb59017da216d6fe0f08a2274336d04db332ad6ce3d9608cd6f03667be1c76423f24a489ac8e7d536011a129dca752ab64b4621b7bc1d4d53f68602e4
49d569cb1f p2p: For assumeutxo, download snapshot chain before background chain (Martin Zumsande)
7a885518d5 p2p: Restrict downloading of blocks for snapshot chain (Martin Zumsande)
Pull request description:
After loading a snapshot, `pindexLastCommonBlock` is usually already set to some block for existing peers. That means we'd continue syncing the background chain from those peers instead of prioritising the snapshot chain, which defeats the purpose of doing assumeutxo in the first place. Only existing peers are affected by this bug.
ACKs for top commit:
fjahr:
re-ACK 49d569cb1f
achow101:
ACK 49d569cb1f
Sjors:
tACK 49d569cb1f
Tree-SHA512: 0eaebe1c29a8510d5ced57e14c09b128ccb34b491692815291df68bf12e2a15b52b1e7bf8d9f34808904e7f7bc20f70b0ad0f7e14df93bbdf456bd12cc02a5d2
00618e8745 assumeutxo: Drop block height from metadata (Fabian Jahr)
Pull request description:
Fixes https://github.com/bitcoin/bitcoin/issues/30514 which has more context and shows how the issue can be reproduced. Since the value in question is removed, there is no test to add to reproduce anything.
This is an alternative approach to #30516 with much of the [code being suggested there](https://github.com/bitcoin/bitcoin/pull/30516#discussion_r1689146902).
ACKs for top commit:
maflcko:
re-ACK 00618e8745🎌
achow101:
ACK 00618e8745
theStack:
Code-review ACK 00618e8745
ismaelsadeeq:
Re-ACK 00618e8745
mzumsande:
ACK 00618e8745
Tree-SHA512: db9575247bae838ad7742a27a216faaf55bb11e022f9afdd05752bb09bbf9614717d0ad64304ff5722a16bf41d8dea888af544e4ae26dcaa528c1add0269a4a8
Although it is not explicitly possible to create a default wallet with
descriptors, it is possible to migrate a default wallet and have it end
up being a default wallet with descriptors. These wallets should be
listed by ListDatabases so that it appears in wallet directory listings
to avoid user confusion.
Migration creates backup files in the wallet directory with .bak as the
extension. This pollutes the output of listwalletdir with backup files
that most users should not need to care about.
92c1d7d1f8 validation: Use MAX_TIMEWARP constant as testnet4 timewarp defense delta (Fabian Jahr)
4b2fad502e doc: Add release notes for 29775 (Fabian Jahr)
f7cc97313b doc: Align deprecation warnings (Fabian Jahr)
1163b08378 chainparams: Add initial minimum chain work for Testnet4 (Fabian Jahr)
Pull request description:
This completes follow-ups left open in #29775.
- Adds release notes
- Addresses the [misalignment](https://github.com/bitcoin/bitcoin/pull/29775#discussion_r1706982102) in deprecation warnings and hints at the intention to remove support for Testnet3.
- Adds initial minimum chainwork for Testnet4.
- Use the `MAX_TIMEWARP` constant as the timewarp defense delta, equal to `MAX_FUTURE_BLOCK_TIME`.
ACKs for top commit:
Sjors:
ACK 92c1d7d1f8
achow101:
ACK 92c1d7d1f8
tdb3:
re ACK 92c1d7d1f8
Tree-SHA512: 7ebdac7809f96231f75ca62706af59cd1ed27f713a4c7be5e2ad69fae95832b146b3ea23c712fb03b412da1deda7e8a5dae55bb2bbd2dcfd9f926e85c2a72666
2925bd537c refactor: use c++20 std::views::reverse instead of reverse_iterator.h (stickies-v)
Pull request description:
C++20 introduces [`std::ranges::views::reverse`](https://en.cppreference.com/w/cpp/ranges/reverse_view), which allows us to drop our own `reverse_iterator.h` implementation and also makes it easier to chain views (even though I think we currently don't use this).
ACKs for top commit:
achow101:
ACK 2925bd537c
maflcko:
ACK 2925bd537c🎷
Tree-SHA512: 567666ec44af5d1beb7a271836bcc89c4c577abc77f522fcc18bc6d4de516ae9b0df766d0bfa6dd217569e6878331c2aee1d9815620860375e3510dad7fed476
Enhanced efficiency and readability of CCoinsViewCache::FetchCoin by replacing separate find() and emplace() calls with a single try_emplace(), reducing map lookups and potential insertions.
Multipath descriptors will be imported as multiple separate descriptors.
When there are 2 multipath items, the first descriptor will be for receiving
addresses and the second for change. This mirrors importmulti.
Multipath descriptors will be imported as multiple separate descriptors.
When there are exactly 2 multipath items, the first descriptor will be
for receiving addreses, and the second for change
addresses. When importing a multipath descriptor, 'internal' cannot be
specified.
Instead of applying internal-ness to all keys being imported at the same
time, apply it on a per key basis. So each key that is imported will
carry with it whether it is for the change keypool.
When given a multipath descriptor, derive all of the descriptors.
The derived addresses will be returned in an object
consisting of multiple arrays. For compatibility, when given a single path
descriptor, the addresses are provided in a single array as before.
Multipath specifiers are derivation path indexes of the form `<i;j;k;...>`
used for specifying multiple derivation paths for a descriptor.
Only one multipath specifier is allowed per PubkeyProvider.
This is syntactic sugar which is parsed into multiple distinct descriptors.
One descriptor will have all of the `i` paths, the second all of the `j` paths,
the third all of the `k` paths, and so on.
ParseKeypath will always return a vector of keypaths with the same size
as the multipath specifier. The callers of this function are updated to deal
with this case and return multiple PubkeyProviders. Their callers have
also been updated to handle vectors of PubkeyProviders.
Co-Authored-By: furszy <matiasfurszyfer@protonmail.com>
To prepare for returning multipath descriptors which will be a shorthand
for specifying multiple descriptors, change ParseScript's signature to return a
vector.
589db872e1 validation: don't erase coins cache on prune flushes (Andrew Toth)
0e8918755f Add linked-list test to CCoinsViewCache::SanityCheck (Pieter Wuille)
05cf4e1875 coins: move Sync logic to CoinsViewCacheCursor (Andrew Toth)
7825b8b9ae coins: pass linked list of flagged entries to BatchWrite (Andrew Toth)
a14edada8a test: add cache entry linked list tests (Andrew Toth)
24ce37cb86 coins: track flagged cache entries in linked list (Andrew Toth)
58b7ed156d coins: call ClearFlags in CCoinsCacheEntry destructor (Andrew Toth)
8bd3959fea refactor: require self and sentinel parameters for AddFlags (Andrew Toth)
75f36d241d refactor: add CoinsCachePair alias (Andrew Toth)
f08faeade2 refactor: move flags to private uint8_t and rename to m_flags (Andrew Toth)
4e4fb4cbab refactor: disallow setting flags in CCoinsCacheEntry constructors (Andrew Toth)
8737c0cefa refactor: encapsulate flags setting with AddFlags and ClearFlags (Andrew Toth)
9715d3bf1e refactor: encapsulate flags get access for all other checks (Andrew Toth)
df34a94e57 refactor: encapsulate flags access for dirty and fresh checks (Andrew Toth)
Pull request description:
Since https://github.com/bitcoin/bitcoin/pull/17487 we no longer need to clear the coins cache when syncing to disk. A warm coins cache significantly speeds up block connection, and only needs to be fully flushed when nearing the `dbcache` limit.
For frequent pruning flushes there's no need to empty the cache and kill connect block speed. However, simply using `Sync` in place of `Flush` actually slows down a pruned full IBD with a high `dbcache` value. This is because as the cache grows, sync takes longer since every coin in the cache is scanned to check if it's dirty. For frequent prune flushes and a large cache this constant scanning starts to really slow IBD down, and just emptying the cache on every prune becomes faster.
To fix this, we can add two pointers to each cache entry and construct a doubly linked list of dirty entries. We can then only iterate through all dirty entries on each `Sync`, and simply clear the pointers after.
With this approach a full IBD with `dbcache=16384` and `prune=550` was 32% faster than master. For default `dbcache=450` speedup was ~9%. All benchmarks were run with `stopatheight=800000`.
| | prune | dbcache | time | max RSS | speedup |
|-----------:|----------:|------------:|--------:|-------------:|--------------:|
| master | 550 | 16384 | 8:52:57 | 2,417,464k | - |
| branch | 550 | 16384 | 6:01:00 | 16,216,736k | 32% |
| branch | 550 | 450 | 8:05:08 | 2,818,072k | 8.8% |
| master | 10000 | 5000 | 8:19:59 | 2,962,752k | - |
| branch | 10000 | 5000| 5:56:39 | 6,179,764k | 28.8% |
| master | 0 | 16384 | 4:51:53 | 14,726,408k | - |
| branch | 0 | 16384 | 4:43:11 | 16,526,348k | 2.7% |
| master | 0 | 450 | 7:08:07 | 3,005,892k | - |
| branch | 0 | 450 | 6:57:24 | 3,013,556k |2.6%|
While the 2 pointers add memory to each cache entry, it did not slow down IBD. For non-pruned IBD results were similar for this branch and master. When I performed the initial IBD, the full UTXO set could be held in memory when using the max `dbcache` value. For non-pruned IBD with max `dbcache` to tip ended up using 12% more memory, but it was also 2.7% faster somehow. For smaller `dbcache` values the `dbcache` limit is respected so does not consume more memory, and the potentially more frequent flushes were not significant enough to cause any slowdown.
For reviewers, the commits in order do the following:
First 4 commits encapsulate all accesses to `flags` on cache entries, and then the 5th makes `flags` private.
Commits `refactor: add CoinsCachePair alias` to `coins: call ClearFlags in CCoinsCacheEntry destructor` create the linked list head nodes and cache entry self references and pass them into `AddFlags`.
Commit `coins: track flagged cache entries in linked list` actually adds the entries into a linked list when they are flagged DIRTY or FRESH and removes them from the linked list when they are destroyed or the flags are cleared manually. However, the linked list is not yet used anywhere.
Commit `test: add cache entry linked list tests` adds unit tests for the linked list.
Commit `coins: pass linked list of flagged entries to BatchWrite` uses the linked list to iterate through DIRTY entries instead of using the entire coins cache.
Commit `validation: don't erase coins cache on prune flushes` uses `Sync` instead of `Flush` for pruning flushes, so the cache is no longer cleared.
Inspired by [this comment](https://github.com/bitcoin/bitcoin/pull/15265#issuecomment-457720636).
Fixes https://github.com/bitcoin/bitcoin/issues/11315.
ACKs for top commit:
paplorinc:
ACK 589db872e1
sipa:
reACK 589db872e1
achow101:
ACK 589db872e1
mzumsande:
re-ACK 589db872e1
Tree-SHA512: 23b2bc01c83edacb5b39aa60bb0b766de9a74ce17f0c59bf13b97b4328a7b758ad9aff6581c3ca88e2973f7658380651530d497444f48d6e22ea0bfc51cc921d
6bfa26048d testnet: Add timewarp attack prevention for Testnet4 (Fabian Jahr)
0100907ca1 testnet: Add Testnet4 difficulty adjustment rules fix (Fabian Jahr)
74a04f9e7a testnet: Introduce Testnet4 (Fabian Jahr)
Pull request description:
To supplement the [ongoing conceptual discussion about a testnet reset](https://groups.google.com/g/bitcoindev/c/9bL00vRj7OU/m/9yCPo3uUBwAJ) I have drafted a move to v4 including a fix to the difficulty adjustment mechanism, which was part of the motivation that started the discussion.
Conceptual considerations:
- The conceptual discussion about doing a testnet4 or softforking the fix into testnet3 is outside of the scope of this PR and I would ask reviewers to contribute their opinions on this on the ML instead. However, I am happy to adapt this PR to a softfork change on testnet3 if there is consensus for that instead.
- The difficulty adjustment fix suggested here touches the `CalculateNextWorkRequired` function and uses the same logic used in `GetNextWorkRequired` to find the last previous block that was not mined with difficulty 1 under the exceptionf. An alternative fix briefly mentioned on the mailing list by Jameson Lopp would be to "restrict the special testnet minimum difficulty rule so that it can't be triggered on the block right before a difficulty retarget". That would also fix the issue but I find my suggestion here a bit more elegant.
ACKs for top commit:
jsarenik:
tACK 6bfa26048d
achow101:
ACK 6bfa26048d
murchandamus:
tACK 6bfa26048d
Tree-SHA512: 0b8b69a621406a944da5be551b863d065358ba94d85dd3b80d83c412660e230ee93b27316081fbee9b4851cc4ff8585db64c7dfa26cb5148ac835663f2712c3d
1f93e3c360 add deprecation warning for mempoolfullrbf (glozow)
4400c979a3 [doc] update documentation for new mempoolfullrbf default (glozow)
Pull request description:
Followup to #30493. Update bips.md and policy/*.md to reflect new default rules around signaling requirements in RBF.
Also, log a warning when `-mempoolfullrbf=0` that this config option is deprecated and will be removed in a future release.
ACKs for top commit:
petertodd:
ACK 1f93e3c360
instagibbs:
ACK 1f93e3c360
tdb3:
ACK 1f93e3c360
Tree-SHA512: f60a9524f15cfaa4c10c40b6f62b787d3f9865aac48ca883def30efac4f8a118f1359532f1b209ea34e201f0b1c92398abc8bc1e439e6b60910cc7f75c51e9ae
ec973dd197 refactor: remove un-tested early returns (josibake)
72a5822d43 tests: add tests for KeyPair (josibake)
cebb08b121 refactor: move SignSchnorr to KeyPair (josibake)
c39fd39ba8 crypto: add KeyPair wrapper class (josibake)
5d507a0091 tests: add key tweak smoke test (josibake)
f14900b6e4 bench: add benchmark for signing with a taptweak (josibake)
Pull request description:
Broken out from #28201
---
The wallet returns an untweaked internal key for taproot outputs. If the output commits to a tree of scripts, this key needs to be tweaked with the merkle root. Even if the output does not commit to a tree of scripts, BIP341/342 recommend commiting to a hash of the public key.
Previously, this logic for applying the taptweak was implemented in the `CKey::SignSchnorr` method.
This PR moves introduces a KeyPair class which wraps a `secp256k1_keypair` type and refactors SignSchnorr to use this new KeyPair. The KeyPair class is created with an optional merkle_root argument and the logic from BIP341 is applied depending on the state of the merkle_root argument.
The motivation for this refactor is to be able to use the tap tweak logic outside of signing, e.g. in silent payments when retrieving the private key (see #28201).
Outside of silent payments, since we almost always convert a `CKey` to a `secp256k1_keypair` when doing anything with taproot keys, it seems generally useful to have a way to model this type in our code base.
ACKs for top commit:
paplorinc:
ACK ec973dd197 - will happily reack if you decide to apply @ismaelsadeeq's suggestions
ismaelsadeeq:
Code review ACK ec973dd197
itornaza:
trACK ec973dd197
theStack:
Code-review ACK ec973dd197
Tree-SHA512: 34947e3eac39bd959807fa21b6045191fc80113bd650f6f08606e4bcd89aa17d6afd48dd034f6741ac4ff304b104fa8c1c1898e297467edcf262d5f97425da7b
6714276d72 miniscript: Use `ToIntegral` instead of `ParseInt64` (brunoerg)
Pull request description:
Currently, miniscript code uses `ParseInt64` function for `after`, `older`, `multi` and `thresh` fragments. It means that a leading `+` or whitespace, among other things, are accepted into the fragments. However, these cases are not useful and cause Bitcoin Core to behave differently compared to other miniscript implementations (see https://github.com/brunoerg/bitcoinfuzz/issues/34). This PR fixes it.
ACKs for top commit:
achow101:
ACK 6714276d72
tdb3:
cr ACK 6714276d72
danielabrozzoni:
tACK 6714276d72
darosior:
utACK 6714276d72
Tree-SHA512: d9eeb93f380f346d636513eeaf26865285e7b0907b8ed258fe1e02153a9eb69d484c82180eb1c78b0ed77ad5f0e5b244be6672c2f890b1d9fddc9e844bee6dde
After loading a snapshot, pindexLastCommonBlock is usually already set
to some block for existing peers. That means we'd continue syncing the
background chain from those peers instead of prioritising the snapshot
chain, which defeats the purpose of doing assumeutxo in the first place.
Only existing peers are affected by this bug.
If the best chain of the peer doesn't include the snapshot
block, it is futile to download blocks from this chain,
because we couldn't reorg to it. We'd also crash
trying to reorg because this scenario is not handled.
fa18fc7050 log: Remove NOLINT(bitcoin-unterminated-logprintf) (MarcoFalke)
Pull request description:
`NOLINT(bitcoin-unterminated-logprintf)` is used to document a missing trailing `\n` char in the format string. This has many issues:
* It is just documentation, assuming that a trailing `\n` ends up in the formatted string. It is not enforced at compile-time, so it is brittle.
* If the newline was truly missing and `NOLINT(bitcoin-unterminated-logprintf)` were used to document a "continued" line, the log stream would be racy/corrupt, because any other thread may inject a log message in the meantime.
* If the newline was accidentally missing, nothing is there to correct the mistake.
* The intention of all code is to always end a log line with a new line. However, historic code exists to deal with the case where the new line was missing (`m_started_new_line`). This is problematic, because the presumed dead code has to be maintained (https://github.com/bitcoin/bitcoin/pull/30386#discussion_r1682963306).
Fix almost all issues by removing the `NOLINT(bitcoin-unterminated-logprintf)`, ensuring that a new line is always present.
A follow-up will remove the dead logging code.
ACKs for top commit:
TheCharlatan:
ACK fa18fc7050
ryanofsky:
Code review ACK fa18fc7050
Tree-SHA512: bf8a83723cca84e21187658edc19612da79c34f7ef2e1f6e9353e7ba70e4ecc0a878a2ae32290045fb90cba9a44451e35341a36ef2ec1169d13592393aa4a8ca
e9de0a76b9 doc: release note for 30212 (willcl-ark)
87b1880525 rpc: clarify ALREADY_IN_CHAIN rpc errors (willcl-ark)
Pull request description:
Closes: #19363
Renaming this error improves clarity around the returned error both internally and externally when a transactions' outputs are already found in the utxo set (`TransactionError::ALREADY_IN_CHAIN -> TransactionError::ALREADY_IN_UTXO_SET`)
ACKs for top commit:
tdb3:
ACK e9de0a76b9
ismaelsadeeq:
ACK e9de0a76b9
ryanofsky:
Code review ACK e9de0a76b9.
Tree-SHA512: 7d2617200909790340951fe56a241448f9ce511900777cb2a712e8b9c0778a27d1f912b460f82335844224f1abb4322bc898ca076440959edade55c082a09237
59c0ece0a7 fuzz: replace hardcoded numbers for bech32 limits (josibake)
Pull request description:
Follow-up to #30047 to replace a hardcoded value that was missed in the original PR
ACKs for top commit:
paplorinc:
ACK 59c0ece0a7
dergoegge:
utACK 59c0ece0a7
marcofleon:
ACK 59c0ece0a7. Ran the test a bit to be sure, lgtm.
brunoerg:
utACK 59c0ece0a7
Tree-SHA512: 89799928feb6752a533259117340b087ff7299f9bf204b165dd87708e15b99a338521f2ac9f9e1fd91dc48b93be839059768d9e68b172e36328232174d1dfa3f
Erase spent cache entries and clear flags of unspent
entries inside the BatchWrite loop, instead of an
additional loop after BatchWrite.
Co-Authored-By: Pieter Wuille <pieter@wuille.net>
No visible behavior change. This commit tracks the flagged
entries internally but the list is not iterated by anything.
Co-Authored-By: Pieter Wuille <pieter@wuille.net>
Co-Authored-By: l0rinc <pap.lorinc@gmail.com>
fa895c7283 mingw: Document mode wbx workaround (MarcoFalke)
fa359255fe Add -blocksxor boolean option (MarcoFalke)
fa7f7ac040 Return XOR AutoFile from BlockManager::Open*File() (MarcoFalke)
Pull request description:
Currently the *.dat files in the blocksdir store the data received from remote peers as-is. This may be problematic when a program other than Bitcoin Core tries to interpret them by accident. For example, an anti-virus program or other program may scan them and move them into quarantine, or delete them, or corrupt them. This may cause Bitcoin Core to fail a reorg, or fail to reply to block requests (via P2P, RPC, REST, ...).
Fix this, similar to https://github.com/bitcoin/bitcoin/pull/6650, by rolling a random XOR pattern over the dat files when writing or reading them.
Obviously this can only protect against programs that accidentally and unintentionally are trying to mess with the dat files. Any program that intentionally wants to mess with the dat files can still trivially do so.
The XOR pattern is only applied when the blocksdir is freshly created, and there is an option to disable it (on creation), so that people can disable it, if needed.
ACKs for top commit:
achow101:
ACK fa895c7283
TheCharlatan:
Re-ACK fa895c7283
hodlinator:
ACK fa895c7283
Tree-SHA512: c92a6a717da83bc33a9b8671a779eeefde2c63b192362ba1d71e6535ee31d08e2802b74acc908345197de9daac6930e4771595ee25b09acd5a67f7ea34854720
f3cfbd65f5 net: log connections failures via SOCKS5 with less severity (Vasil Dimov)
Pull request description:
It is expected to have some Bitcoin nodes unreachable some of the time. A failure to connect to an IPv4 or IPv6 node is already properly logged under category=net/severity=debug. Do the same when a connection fails when using a SOCKS5 proxy. This could be either to an .onion address or to an IPv4 or IPv6 address (via a Tor exit node).
Related: https://github.com/bitcoin/bitcoin/issues/29759
ACKs for top commit:
achow101:
ACK f3cfbd65f5
mzumsande:
Code Review ACK f3cfbd65f5
tdb3:
Code Review ACK f3cfbd65f5
Tree-SHA512: c6e83568783cb5233edac7840a00f708d27be9af87480fc73093ad99fe4bd8670d3f2c97fd6b6e2c54b8d9337746eacb9a5db6eefecc1486951996bfbb0a37f7
172c1ad026 test: expand LimitOrphan and EraseForPeer coverage (Greg Sanders)
28dbe218fe refactor: move orphanage constants to header file (Greg Sanders)
Pull request description:
Inspired by refactorings in #30000 as the coverage appeared a bit sparse.
Added some minimal border value testing, timeouts, and tightened existing assertions.
ACKs for top commit:
achow101:
ACK 172c1ad026
rkrux:
reACK [172c1ad](172c1ad026)
glozow:
reACK 172c1ad026
Tree-SHA512: e8fa9b1de6a8617612bbe9b132c9c0c9b5a651ec94fd8c91042a34a8c91c5f9fa7ec4175b47e2b97d1320d452c23775be671a9970613533e68e81937539a7d70
fa530ec543 rpc: Return precise loadtxoutset error messages (MarcoFalke)
faa5c86dbf refactor: Use untranslated error message in ActivateSnapshot (MarcoFalke)
Pull request description:
The error messages should never happen in normal operation. However, if
they do, they are helpful to return to the user to debug the issue. For
example, to notice a truncated file.
This fixes https://github.com/bitcoin/bitcoin/issues/28621
Also includes a minor refactor commit.
ACKs for top commit:
fjahr:
Code review ACK fa530ec543
ryanofsky:
Code review ACK fa530ec543, just adjusting error messages a little since last review. (Thanks!)
Tree-SHA512: 224968c9b13d082ca2ed1f6a8fcc5f51ff16d6c96bd38c3679699505b54337b99cccaf7a8474391f6b11f9ccb101977b4e626898c1217eae95802e290cf105f1
2d9d752e4f scripted-diff: Replace uint256S("str") -> uint256{"str"} (Hodlinator)
c06f2368e2 refactor: Hand-replace some uint256S -> uint256 (Hodlinator)
b74d8d58fa refactor: Add consteval uint256(hex_str) (Hodlinator)
Pull request description:
Motivation:
* Validates and converts the hex string at compile time instead of at runtime into the resulting bytes.
* Makes it possible to derive other compile time constants from `uint256`.
* Potentially eliminates runtime dependencies (`SetHexDeprecated()` is called in less places).
* Has stricter requirements than the deprecated `uint256S()` (requiring 64 chars exactly, disallows garbage at the end) and replaces it in a bunch of places.
* Makes the binary smaller (tested Guix-built x86_64-linux-gnu bitcoind binary).
* Minor: should shave off a few cycles of start-up time.
Extracted from #30377 which diverged into exploring `consteval` `ParseHex()` solutions.
ACKs for top commit:
maflcko:
rebase re-cr-ACK 2d9d752e4f🎐
stickies-v:
re-ACK 2d9d752e4f
paplorinc:
ACK 2d9d752e4f
Tree-SHA512: 39bd9320db0ed81950b5d71495eaa1d06508cc008466f2308874d70ac9ff32bc69798d2e3ef6a784868c1633fb519f60cc2111a9d0718c2663b28e78b67f7cde
When using `sendrawtransaction` the ALREADY_IN_CHAIN error help string
may be confusing.
Rename TransactionError::ALREADY_IN_CHAIN to
TransactionError::ALREADY_IN_UTXO_SET and update the rpc help string.
Remove backwards compatibility alias as no longer required.
fa3ea3b83c test: Fix intermittent issue in p2p_v2_misbehaving.py (MarcoFalke)
55555574d1 net: Log accepted connection after m_nodes.push_back (MarcoFalke)
Pull request description:
Fix the two issues reported in https://github.com/bitcoin/bitcoin/pull/30468/files#r1688444784:
* Delay a debug log line for consistency.
* Fix an intermittent test issue.
They are completely separate fixes, but both `net` related.
ACKs for top commit:
0xB10C:
Code Review ACK fa3ea3b83c
stratospher:
tested ACK fa3ea3b.
Tree-SHA512: cd6b6e164b317058a305a5c3e38c56c9a814a7469039e1143f1d7addfbc91b0a28506873356b373d97448b46cb6fbe94a1309df82e34c855540b241a09489e8b
bfd3c29e4f fuzz: fix timeout in crypter target (brunoerg)
Pull request description:
Fixes#30503
- Move SetKeyFromPassphrase to out of LIMITED_WHILE
- Remove `SetKey` calls since it is already called internally by other functions.
- Reduce number of iterations (100 is enough, no need for 10,000).
ACKs for top commit:
maflcko:
review ACK bfd3c29e4f📆
dergoegge:
utACK bfd3c29e4f
Tree-SHA512: 275ab7d07a20bfd07279a23613678993c10c166f40cdc900213b9f4d5afb107462d5f88518a0f4ce2a52f3b7950ff2c01cf74292042f16996909fcb96f827d3e
chainparams.cpp - workaround for MSVC bug triggering C7595 - Calling consteval constructors in initializer lists fails, but works on GCC (13.2.0) & Clang (17.0.6).
Complements uint256::FromHex() nicely in that it naturally does all error checking at compile time and so doesn't need to return an std::optional.
Will be used in the following 2 commits to replace many calls to uint256S(). uint256S() calls taking C-string literals are littered throughout the codebase and executed at runtime to perform parsing unless a given optimizer was surprisingly efficient. While this may not be a hot spot, it's better hygiene in C++20 to store the parsed data blob directly in the binary, without any parsing at runtime.
bf0efb4fc7 scripted-diff: Modernize naming of nChainTx and nTxCount (Fabian Jahr)
72e5d1be1f test: Add basic check for nChainTx type (Fabian Jahr)
dc2938e979 chainparams: Change nChainTx to uint64_t (Fabian Jahr)
Pull request description:
This picks up the work from #29331 and closes#29258.
This simply changes the type and addresses the comments from #29331 by changing the type in all relevant places and removing unnecessary casts. This also adds an extremely simple unit test.
Additionally this modernizes the name of `nChainTx` which helps reviewers check all use of the symbol and can make silent merge conflicts.
ACKs for top commit:
maflcko:
only rebase in scripted-diff, re-ACK bf0efb4fc7🔈
glozow:
reACK bf0efb4fc7 via range-diff
Tree-SHA512: ee4020926d0800236fe655d0c7b127215ab36b553b04d5f91494f4b7fac6e1cfe7ee298b07c0983db5a3f4786932acaa54f5fd2ccd45f2fcdcfa13427358dc3b
bbcee5a0d6 clusterlin: improve rechunking in LinearizationChunking (optimization) (Pieter Wuille)
04d7a04ea4 clusterlin: add MergeLinearizations function + fuzz test + benchmark (Pieter Wuille)
4f8958d756 clusterlin: add PostLinearize + benchmarks + fuzz tests (Pieter Wuille)
0e2812d293 clusterlin: add algorithms for connectedness/connected components (Pieter Wuille)
0e52728a2d clusterlin: rename Intersect -> IntersectPrefixes (Pieter Wuille)
Pull request description:
Part of cluster mempool: #30289
Depends on #30126, and was split off from it. #28676 depends on this.
This adds the algorithms for merging & postprocessing linearizations.
The `PostLinearize(depgraph, linearization)` function performs an in-place improvement of `linearization`, using two iterations of the [Linearization post-processing](https://delvingbitcoin.org/t/linearization-post-processing-o-n-2-fancy-chunking/201/8) algorithm. The first running from back to front, the second from front to back.
The `MergeLinearizations(depgraph, linearization1, linearization2)` function computes a new linearization for the provided cluster, given two existing linearizations for that cluster, which is at least as good as both inputs. The algorithm is described at a high level in [merging incomparable linearizations](https://delvingbitcoin.org/t/merging-incomparable-linearizations/209).
For background and references, see [Introduction to cluster linearization](https://delvingbitcoin.org/t/introduction-to-cluster-linearization/1032).
ACKs for top commit:
sdaftuar:
ACK bbcee5a0d6
glozow:
code review ACK bbcee5a0d6
instagibbs:
ACK bbcee5a0d6
Tree-SHA512: d2b5a3f132d1ef22ddf9c56421ab8b397efe45b3c4c705548dda56f5b39fe4b8f57a0d2a4c65b338462d80bb5b9b84a9a39efa1b4f390420a8005ce31817774e
73e3fa10b4 doc + test: Correct uint256 hex string endianness (Hodlinator)
Pull request description:
This PR is a follow-up to #30436.
Only changes test-code and modifies/adds comments.
Byte order of hex string representation was wrongfully documented as little-endian, but are in fact closer to "big-endian" (endianness is a memory-order concept rather than a numeric concept). `[arith_]uint256` both store their data in arrays with little-endian byte order (`arith_uint256` has host byte order within each `uint32_t` element).
**uint256_tests.cpp** - Avoid using variable from the left side of the condition in the right side. Credits to @maflcko: https://github.com/bitcoin/bitcoin/pull/30436#discussion_r1688273553
**setup_common.cpp** - Skip needless ArithToUint256-conversion. Credits to @stickies-v: https://github.com/bitcoin/bitcoin/pull/30436#discussion_r1688621638
---
<details>
<summary>
## Logical reasoning for endianness
</summary>
1. Comparing an `arith_uint256` (`base_uint<256>`) to a `uint64_t` compares the beginning of the array, and verifies the remaining elements are zero.
```C++
template <unsigned int BITS>
bool base_uint<BITS>::EqualTo(uint64_t b) const
{
for (int i = WIDTH - 1; i >= 2; i--) {
if (pn[i])
return false;
}
if (pn[1] != (b >> 32))
return false;
if (pn[0] != (b & 0xfffffffful))
return false;
return true;
}
```
...that is consistent with little endian ordering of the array.
2. They have the same endianness (but `arith_*` has host-ordering of each `uint32_t` element):
```C++
arith_uint256 UintToArith256(const uint256 &a)
{
arith_uint256 b;
for(int x=0; x<b.WIDTH; ++x)
b.pn[x] = ReadLE32(a.begin() + x*4);
return b;
}
```
### String conversions
The reversal of order which happens when converting hex-strings <=> uint256 means strings are actually closer to big-endian, see the end of `base_blob<BITS>::SetHexDeprecated`:
```C++
unsigned char* p1 = m_data.data();
unsigned char* pend = p1 + WIDTH;
while (digits > 0 && p1 < pend) {
*p1 = ::HexDigit(trimmed[--digits]);
if (digits > 0) {
*p1 |= ((unsigned char)::HexDigit(trimmed[--digits]) << 4);
p1++;
}
}
```
Same reversal here:
```C++
template <unsigned int BITS>
std::string base_blob<BITS>::GetHex() const
{
uint8_t m_data_rev[WIDTH];
for (int i = 0; i < WIDTH; ++i) {
m_data_rev[i] = m_data[WIDTH - 1 - i];
}
return HexStr(m_data_rev);
}
```
It now makes sense to me that `SetHexDeprecated`, upon receiving a shorter hex string that requires zero-padding, would pad as if the missing hex chars where towards the end of the little-endian byte array, as they are the most significant bytes. "Big-endian" string representation is also consistent with the case where `SetHexDeprecated` receives too many hex digits and discards the leftmost ones, as a form of integer narrowing takes place.
### How I got it wrong in #30436
Previously I used the less than (`<`) comparison to prove endianness, but for `uint256` it uses `memcmp` and thereby gives priority to the *lower* bytes at the beginning of the array.
```C++
constexpr int Compare(const base_blob& other) const { return std::memcmp(m_data.data(), other.m_data.data(), WIDTH); }
```
`arith_uint256` is different in that it begins by comparing the bytes from the end, as it is using little endian representation, where the bytes toward the end are more significant.
```C++
template <unsigned int BITS>
int base_uint<BITS>::CompareTo(const base_uint<BITS>& b) const
{
for (int i = WIDTH - 1; i >= 0; i--) {
if (pn[i] < b.pn[i])
return -1;
if (pn[i] > b.pn[i])
return 1;
}
return 0;
}
```
(The commit documents that `base_blob::Compare()` is doing lexicographic ordering unlike the `arith_*`-variant which is doing numeric ordering).
</details>
ACKs for top commit:
paplorinc:
ACK 73e3fa10b4
ryanofsky:
Code review ACK 73e3fa10b4
Tree-SHA512: 121630c37ab01aa7f7097f10322ab37da3cbc0696a6bbdbf2bbd6db180dc5938c7ed91003aaa2df7cf4a4106f973f5118ba541b5e077cf3588aa641bbd528f4e
a7432dd6ed logging: clarify -debug and -debugexclude descriptions (Anthony Towns)
74dd33cb0a rpc: make logging method reject "0" category and correct the help text (Vasil Dimov)
8c6f3bf163 logging, refactor: minor encapsulation improvement and use BCLog::NONE instead of 0 (Vasil Dimov)
160706aa38 logging, refactor: make category special cases explicit (Ryan Ofsky)
Pull request description:
* Move special cases from `LOG_CATEGORIES_BY_STR` to `GetLogCategory()` (suggested [here](https://github.com/bitcoin/bitcoin/pull/29419#discussion_r1547990373)).
* Remove `"none"` and `"0"` from RPC `logging` help because that help text was wrong. `"none"` resulted in an error and `"0"` was ignored itself (contrary to what the help text suggested).
* Remove unused `LOG_CATEGORIES_BY_STR[""]` (suggested [here](https://github.com/bitcoin/bitcoin/pull/29419#discussion_r1548018694)).
This is a followup to https://github.com/bitcoin/bitcoin/pull/29419, addressing leftover suggestions + more.
ACKs for top commit:
LarryRuane:
ACK a7432dd6ed
ryanofsky:
Code review ACK a7432dd6ed. Only changes since last review are removing dead if statement and adding AJ's suggested -debug and -debugexclude help improvements, which look accurate and much more clear.
Tree-SHA512: 41b997b06fccdb4c1d31f57d4752c83caa744cb3280276a337ef4a9b7012a04eb945071db6b8fad24c6a6cf8761f2f800fe6d8f3d8836f5b39c25e4f11c85bf0
Replace early returns in KeyPair::KeyPair() with asserts.
The if statements imply there is an error we are handling, but keypair_xonly_pub
and xonly_pubkey_serialize can only fail if the keypair object is malformed, i.e.,
it was created with a bad secret key. Since we check that the keypair was created
successfully before attempting to extract the public key, using asserts more
accurately documents what we expect here and removes untested branches from the code.
Move `SignSchnorr` to `KeyPair`. This makes `CKey::SignSchnorr` now
compute a `KeyPair` object and then call `KeyPair::SignSchorr`. The
notable changes are:
* Move the merkle_root tweaking out of the sign function and into
the KeyPair constructor
* Remove the temporary secp256k1_keypair object and have the
functions access m_keypair->data() directly
Current logging RPC method documentation claims to accept "0" and "none"
categories, but the "none" argument is actually rejected and the "0"
argument is ignored. Update the implementation to refuse both
categories, and remove the help text claiming to support them.
* Make the standalone function `LogCategoryToStr()` private inside
`logging.cpp` (aka `static`) - it is only used in that file.
* Make the method `Logger::GetLogPrefix()` `private` - it is only
used within the class.
* Use `BCLog::NONE` to initialize `m_categories` instead of `0`.
We later check whether it is `BCLog::NONE` (in
`Logger::DefaultShrinkDebugFile()`).
Make special cases explicit in GetLogCategory() and LogCategoryToStr()
functions. Simplify the LOG_CATEGORIES_BY_STR and LOG_CATEGORIES_BY_FLAG
mappings and LogCategoriesList() function.
This makes the maps `LOG_CATEGORIES_BY_STR` and `LOG_CATEGORIES_BY_FLAG`
consistent (one is exactly the opposite of the other).
Follow-up to #30436.
uint256 string representation was wrongfully documented as little-endian due to them being reversed by GetHex() etc, and base_blob::Compare() giving most significance to the beginning of the internal array. They are closer to "big-endian", but this commit tries to be even more precise than that.
uint256_tests.cpp - Avoid using variable from the left side of the condition in the right side.
setup_common.cpp - Skip needless ArithToUint256-conversion.
Add a `KeyPair` class which wraps the `secp256k1_keypair`. This keeps
the secret data in secure memory and enables passing the
`KeyPair` object directly to libsecp256k1 functions expecting a
`secp256k1_keypair`.
Motivation: when passing `CKeys` for taproot outputs to libsecp256k1 functions,
the first step is to create a `secp256k1_keypair` data type and use that
instead. This is so the libsecp256k1 function can determine if the key
needs to be negated, e.g., when signing.
This is a bit clunky in that it creates an extra step when using a `CKey`
for a taproot output and also involves copying the secret data into a
temporary object, which the caller must then take care to cleanse. In
addition, the logic for applying the merkle_root tweak currently
only exists in the `SignSchnorr` function.
In a later commit, we will add the merkle_root tweaking logic to this
function, which will make the merkle_root logic reusable outside of
signing by using the `KeyPair` class directly.
Co-authored-by: Cory Fields <cory-nospam-@coryfields.com>
Sanity check that using CKey/CPubKey directly vs using secp256k1_keypair objects
returns the same results for BIP341 key tweaking.
Co-authored-by: l0rinc <pap.lorinc@gmail.com>
Add benchmarks for signing with null and non-null merkle_root arguments.
Null and non-null merkle_root arguments will apply the taptweaks
H_TapTweak(P) and H_TapTweak(P | merkle_root), respectively, to the
private key during signing.
This benchmark is added to verify there are no significant performance
changes after moving the taptweak signing logic in a later commit.
Co-authored-by: l0rinc <pap.lorinc@gmail.com>
75648cea5a test: add P2A ProduceSignature coverage (Greg Sanders)
7998ce6b20 Add release note for P2A output feature (Greg Sanders)
71c9b02a04 test: add P2A coverage for decodescript (Greg Sanders)
1349e9ec15 test: Add anchor mempool acceptance test (Greg Sanders)
9d89209937 policy: stop 3rd party wtxid malleability of anchor spend (Greg Sanders)
b60aaf8b23 policy: make anchor spend standard (Greg Sanders)
455fca86cf policy: Add OP_1 <0x4e73> as a standard output type (Greg Sanders)
Pull request description:
This is a sub-feature taken out of the original proposal for ephemeral anchors #30239
This PR makes *spending* of `OP_1 <0x4e73>` (i.e. `bc1pfeessrawgf`) standard. Creation of this output type is already standard.
Any future witness output types are considered relay-standard to create, but not to spend. This preserves upgrade hooks, such as a completely new output type for a softfork such as BIP341. It also gives us a bit of room to use a new output type for policy uses.
This particular sized witness program has no other known use-cases (https://bitcoin.stackexchange.com/a/110664/17078), s it affords insufficient cryptographic security for a secure commitment to data, such as a script or a public key. This makes this type of output "keyless", or unauthenticated.
As a witness program, the `scriptSig` of the input MUST be blank, by BIP141. This helps ensure txid-stability of the spending transaction, which may be required for smart contracting wallets. If we do not use segwit, a miner can simply insert an `OP_NOP` in the `scriptSig` without effecting the result of program execution.
An additional relay restriction is to disallow non-empty witness data, which an adversary may use to penalize the "honest" transactor when RBF'ing the transaction due to the incremental fee requirement of RBF rules.
The intended use-case for this output type is to "anchor" the transaction with a spending child to bring exogenous CPFP fees into the transaction package, encouraging the inclusion of the package in a block. The minimal size of creation and spending of this output makes it an attractive contrast to outputs like `p2sh(OP_TRUE)` and `p2wsh(OP_TRUE)` which
are significantly larger in vbyte terms.
Combined with TRUC transactions which limits the size of child transactions significantly, this is an attractive option for presigned transactions that need to be fee-bumped after the fact.
ACKs for top commit:
sdaftuar:
utACK 75648cea5a
theStack:
re-ACK 75648cea5a
ismaelsadeeq:
re-ACK 75648cea5a via [diff](e7ce6dc070..75648cea5a)
glozow:
ACK 75648cea5a
tdb3:
ACK 75648cea5a
Tree-SHA512: d529de23d20857e6cdb40fa611d0446b49989eaafed06c28280e8fd1897f1ed8d89a4eabbec1bbf8df3d319910066c3dbbba5a70a87ff0b2967d5205db32ad1e
189c987386 Showing local addresses on the Node Window (Jadi)
a5d7aff867 net: Providing an interface for mapLocalHost (Jadi)
Pull request description:
This change adds a new row to the Node Window (debugwindow.ui)
under the Network section which shows the LocalAddresses.
fixes#564
<!--
*** Please remove the following help text before submitting: ***
Pull requests without a rationale and clear improvement may be closed
immediately.
GUI-related pull requests should be opened against
https://github.com/bitcoin-core/gui
first. See CONTRIBUTING.md
-->
<!--
Please provide clear motivation for your patch and explain how it improves
Bitcoin Core user experience or Bitcoin Core developer experience
significantly:
* Any test improvements or new tests that improve coverage are always welcome.
* All other changes should have accompanying unit tests (see `src/test/`) or
functional tests (see `test/`). Contributors should note which tests cover
modified code. If no tests exist for a region of modified code, new tests
should accompany the change.
* Bug fixes are most welcome when they come with steps to reproduce or an
explanation of the potential issue as well as reasoning for the way the bug
was fixed.
* Features are welcome, but might be rejected due to design or scope issues.
If a feature is based on a lot of dependencies, contributors should first
consider building the system outside of Bitcoin Core, if possible.
* Refactoring changes are only accepted if they are required for a feature or
bug fix or otherwise improve developer experience significantly. For example,
most "code style" refactoring changes require a thorough explanation why they
are useful, what downsides they have and why they *significantly* improve
developer experience or avoid serious programming bugs. Note that code style
is often a subjective matter. Unless they are explicitly mentioned to be
preferred in the [developer notes](/doc/developer-notes.md), stylistic code
changes are usually rejected.
-->
<!--
Bitcoin Core has a thorough review process and even the most trivial change
needs to pass a lot of eyes and requires non-zero or even substantial time
effort to review. There is a huge lack of active reviewers on the project, so
patches often sit for a long time.
-->
ACKs for top commit:
pablomartin4btc:
re-ACK 189c987386
furszy:
utACK 189c987
Tree-SHA512: 93f201bc6d21d81b27b87be050a447b841f01e3efb69b9eca2cc7af103023d7cd69eb5e16e2875855573ef51a5bf74a6ee6028636c1b6798cb4bb11567cb4996
Move `SetKeyFromPassphrase` to out of LIMITED_WHILE,
remove `SetKey` calls since it is already called
internally by other functions and reduce the number
of iterations.
642c885b61 Merge bitcoin-core/secp256k1#1575: release: prepare for 0.5.1
cdf08c1a2b Merge bitcoin-core/secp256k1#1576: doc: mention `needs-changelog` github label in release process
40d87b8e45 release: prepare for 0.5.1
5770226176 changelog: clarify CMake option
759bd4bbc8 doc: mention `needs-changelog` github label in release process
fded437c4c Merge bitcoin-core/secp256k1#1574: Fix compilation when extrakeys module isn't enabled
763d938cf0 ci: only enable extrakeys module when schnorrsig is enabled
af551ab9db tests: do not use functions from extrakeys module
0055b86780 Merge bitcoin-core/secp256k1#1551: Add ellswift usage example
ea2d5f0f17 Merge bitcoin-core/secp256k1#1563: doc: Add convention for defaults
ca06e58b2c Merge bitcoin-core/secp256k1#1564: build, ci: Adjust the default size of the precomputed table for signing
e2af491263 ci: Switch to the new default value of the precomputed table for signing
d94a9273f8 build: Adjust the default size of the precomputed table for signing
fcc5d7381b Merge bitcoin-core/secp256k1#1565: cmake: Bump CMake minimum required version up to 3.16
9420eece24 cmake: Bump CMake minimum required version up to 3.16
16685649d2 doc: Add convention for defaults
a5269373fa Merge bitcoin-core/secp256k1#1555: Fixed O3 replacement
b8fe33332b cmake: Fixed O3 replacement
31f84595c4 Add ellswift usage example
fe4fbaa7f3 examples: fix case typos in secret clearing paragraphs (s/, Or/, or/)
git-subtree-dir: src/secp256k1
git-subtree-split: 642c885b6102725e25623738529895a95addc4f4
7231c7630e qt: Replace deprecated LogPrintf with LogInfo in GUIUtil::LogQtInfo() (Hennadii Stepanov)
b3d3ae0680 qt, build: Drop `QT_STATICPLUGIN` macro (Hennadii Stepanov)
Pull request description:
Broken out of https://github.com/bitcoin/bitcoin/pull/30454.
Our `QT_STATICPLUGIN` macro is effectively equivalent to the Qt's `QT_STATIC` macro.
It is easy to see in the `_BITCOIN_QT_IS_STATIC` macro implementation: ebd82fa9fa/build-aux/m4/bitcoin_qt.m4 (L269-L292)
No need to handle both macros.
ACKs for top commit:
maflcko:
re-ACK 7231c7630e
TheCharlatan:
ACK 7231c7630e
Tree-SHA512: abbf21859b7ac2aaf47c5b0e075403e4cc9bc540b1565d23f51650b8932dde314586aca67fd4ed5daadebc89268baf8c18f65348fa2b836078ac24543c14cfd6
2e86f2b201 rpc: fix maybe-uninitialized compile warning in getchaintxstats (Michael Dietz)
Pull request description:
This resolves the compiler warning about potential uninitialized use of window_tx_count introduced in fa2dada.
The warning:
```
CXX rpc/libbitcoin_node_a-blockchain.o
rpc/blockchain.cpp: In function ‘getchaintxstats()::<lambda(const RPCHelpMan&, const JSONRPCRequest&)>’:
rpc/blockchain.cpp:1742:38: warning: ‘*(std::_Optional_payload_base<unsigned int>::_Storage<unsigned int, true>*)((char*)&window_tx_count + offsetof(const std::optional<unsigned int>,std::optional<unsigned int>::<unnamed>.std::_Optional_base<unsigned int, true, true>::<unnamed>)).std::_Optional_payload_base<unsigned int>::_Storage<unsigned int, true>::_M_value’ may be used uninitialized in this function [-Wmaybe-uninitialized]
1742 | ret.pushKV("txrate", double(*window_tx_count) / nTimeDiff);
|
```
ACKs for top commit:
maflcko:
lgtm ACK 2e86f2b201
theStack:
ACK 2e86f2b201
tdb3:
ACK 2e86f2b201
Tree-SHA512: c087e8f1cd68dd8df734a8400d30a95abe57ebd56cd53aef4230e425b33a23aa55b3af42abfd162e3be8c937a4c27e56abb70a4fedb10e2df64d52d577e0f262
Contributes to #564 by providing an interface for mapLocalHost
through net -> node interface -> clientModel. Later this value can be
read by GUI to show the local addresses.
No behavior change because any entries that are added in EmplaceCoinInternalDANGER
have DIRTY assigned to them after, and if they
are not inserted then they will not be
modified as before.
This prepares moving the cache entry
flags field to private access.
Co-Authored-By: Martin Leitner-Ankerl <martin.ankerl@gmail.com>
When the transactions being marked done exactly match the first chunk of
what remains of the linearization, we can just remember to skip that
chunk instead of computing a full rechunking.
Further, chop off prefixes of the input linearization that are already done,
so they don't need to be reconsidered for further rechunkings.
