This commit is part of the libbitcoinkernel project and seeks to remove
the ChainstateManager's and, more generally, the kernel library's
dependency on interface_ui with options methods in this and the following
few commits. By removing interface_ui from the kernel library, its
dependency on boost is reduced to just boost::multi_index.
This commit is part of the libbitcoinkernel project and seeks to remove
the ChainstateManager's and, more generally, the kernel library's
dependency on interface_ui with options methods in this and the following
few commits. By removing interface_ui from the kernel library, its
dependency on boost is reduced to just boost::multi_index.
Define a new kernel notification class with virtual methods for
notifying about internal kernel events. Create a new file in the node
library for defining a function creating the default set of notification
methods such that these do not need to be re-defined all over the
codebase. As a first step, add a `blockTip` method, wrapping
`uiInterface.NotifyBlockTip`.
6b605b91c1 [fuzz] Add MiniMiner target + diff fuzz against BlockAssembler (glozow)
3f3f2d59ea [unit test] GatherClusters and MiniMiner unit tests (glozow)
59afcc8354 Implement Mini version of BlockAssembler to calculate mining scores (glozow)
56484f0fdc [mempool] find connected mempool entries with GatherClusters(…) (glozow)
Pull request description:
Implement Mini version of BlockAssembler to calculate mining scores
Run the mining algorithm on a subset of the mempool, only disturbing the
mempool to copy out fee information for relevant entries. Intended to be
used by wallet to calculate amounts needed for fee-bumping unconfirmed
transactions.
From comments of sipa and glozow below:
> > In what way does the code added here differ from the real block assembly code?
>
> * Only operates on the relevant transactions rather than full mempool
> * Has the ability to remove transactions that will be replaced so they don't impact their ancestors
> * Does not hold mempool lock outside of the constructor, makes copies of the entries it needs instead (though I'm not sure if this has an effect in practice)
> * Doesn't do the sanity checks like keeping weight within max block weight and `IsFinalTx()`
> * After the block template is built, additionally calculates fees to bump remaining ancestor packages to target feerate
ACKs for top commit:
achow101:
ACK 6b605b91c1
Xekyo:
> ACK [6b605b9](6b605b91c1) modulo `miniminer_overlap` test.
furszy:
ACK 6b605b91 modulo `miniminer_overlap` test.
theStack:
Code-review ACK 6b605b91c1
Tree-SHA512: f86a8b4ae0506858a7b15d90f417ebceea5038b395c05c825e3796123ad3b6cb8a98ebb948521316802a4c6d60ebd7041093356b1e2c2922a06b3b96b3b8acb6
fa953f15bf build: Bump minimum supported GCC to g++-9 (MarcoFalke)
fa69955e74 ci: Bump centos:stream8 to centos:stream9 (MarcoFalke)
fa6a755d9f ci: Document the false positive error for g++-9 (MarcoFalke)
Pull request description:
It is a bit frustrating to write valid C++ code only to realize that g++-8 fails to parse it later on.
The only non-EOL operating system still shipping with g++-8 is CentOS Stream 8. I think it is reasonable for users of affected Linux distributions to:
* Upgrade their operating system, or compiler to a supported version.
* Alternatively, stay with a previous release of Bitcoin Core as long as it is supported.
Fixes https://github.com/bitcoin/bitcoin/issues/27537
ACKs for top commit:
hebasto:
ACK fa953f15bf
fanquake:
ACK fa953f15bf
Tree-SHA512: b9cf7e763d3071e1e008c5010de19601d4773afe46d58cf869d3f59285c53240c739a1cd7235a5525ede1bbdf6b6cb6fb091c8fc314864a28d5b27a400bb7632
69d43905b7 test: add coverage for wallet read write db deadlock (furszy)
12daf6fcdc walletdb: scope bdb::EraseRecords under a single db txn (furszy)
043fcb0b05 wallet: bugfix, GetNewCursor() misses to provide batch ptr to BerkeleyCursor (furszy)
Pull request description:
Decoupled from #26644 so it can closed in favor of #26715.
Basically, with bdb, we can't make a write operation while we are traversing the db with the same db handler. These two operations are performed in different txn contexts and cause a deadlock.
Added coverage by using `EraseRecords()` which is the simplest function that executes this process.
To replicate it, need bdb support and drop the first commit. The test will run forever.
ACKs for top commit:
achow101:
ACK 69d43905b7
hebasto:
re-ACK 69d43905b7
Tree-SHA512: b3773be78925f674e962f4a5c54b398a9d0cfe697148c01c3ec0d68281cc5c1444b38165960d219ef3cf1a57c8ce6427f44a876275958d49bbc0808486e19d7d
This is a change in behavior so that if for some reason we request a block from a peer, we don't allow an unsolicited CMPCT_BLOCK announcement for that same block to cause a request for a full block from the uninvited peer (as some type of request is already outstanding from the original peer)
7014e08015 doc: remove mention of glibc 2.10+ (fanquake)
Pull request description:
We already require glibc 2.27+, so mentioning a much older version here is redundant.
ACKs for top commit:
TheCharlatan:
ACK 7014e08015
Tree-SHA512: 883a566a80cabe34bfb5d902990f3eca08d0e11438e6c128d311e558f373ec232b0934deb85d12d796baacfeae590af8c73aa1b2faef07f27ffa9011270ffd96
This is achieved by letting the index sync thread wait until
reindex-chainstate is finished.
This also disables the pruning check when reindexing the chainstate (which is
incompatible with prune mode) because there would be no chain at this point
in init.
The index sync code has logic to go back the chain to the forking point, while
also updating index-specific state, which is necessary to prevent
possible corruption of the coinstatsindex.
Also add a test for this (a reorg happens while the index is deactivated)
that would not pass before this change.
This incorrectly assumed num_blocks_total would be greater than 0. This is not guaranteed until the ConnectBlock call right below it.
The total and average metric is not very useful because it does not distinguish between blocks read from disk and those loaded from memory. So rather than fixing the divide by zero issue, we just drop the metric.
1b1ffbd014 Build: Log when test -f fails in Makefile (TheCharlatan)
541012e621 Build: Use AM_V_GEN in Makefiles where appropriate (TheCharlatan)
Pull request description:
This PR triages some behavior around Makefile recipe echoing suppression.
When generating new files as part of the Makefile the recipe is sometimes suppressed with $(AM_V_GEN) and sometimes with `@`. We should prefer $(AM_V_GEN), since this also prints the lines in silent mode. This is arguably more in style with the current recipe echoing.
Before:
`Generated test/data/script_tests.json.h`
Now:
` GEN test/data/script_tests.json.h`
A side effect of this change is that the recipe for generating build.h is now echoed on each make run. Arguably this makes its generation more transparent.
Sometimes the error emitted by `test -f` is currently thrown without any logging. This makes it a bit harder to debug. Instead, print a helpful log message to point the developer in the right direction.
Alternatively this could have been implemented by just removing the recipe echo suppression (@), but the subsequent make output became too noisy.
ACKs for top commit:
fanquake:
ACK 1b1ffbd014
Tree-SHA512: e31869fab25e72802b692ce6735f9561912caea903c1577101b64c9cb115c98de01a59300e8ffe7b05b998345c1b64a79226231d7d1453236ac338c62dc9fbb3
so we erase all the records atomically or abort the entire
procedure.
and, at the same time, we can share the same db txn context
for the db cursor and the erase functionality.
extra note from the Db.cursor doc:
"If transaction protection is enabled, cursors must be
opened and closed within the context of a transaction"
thus why added a `CloseCursor` call before calling to
`TxnAbort/TxnCommit`.
If the batch ptr is not passed, the cursor will not use the db active
txn context which could lead to a deadlock if the code tries to modify
the db while it is traversing it.
E.g. the 'EraseRecords()' function.
33e2b82a4f wallet, bench: Remove unused database options from WalletBenchLoading (Andrew Chow)
80ace042d8 tests: Modify records directly in wallet ckey loading test (Andrew Chow)
b3bb17d5d0 tests: Update DuplicateMockDatabase for MockableDatabase (Andrew Chow)
f0eecf5e40 scripted-diff: Replace CreateMockWalletDB with CreateMockableWalletDB (Andrew Chow)
075962bc25 wallet, tests: Include wallet/test/util.h (Andrew Chow)
14aa4cb1e4 wallet: Move DummyDatabase to salvage (Andrew Chow)
f67a385556 wallet, tests: Replace usage of dummy db with mockable db (Andrew Chow)
33c6245ac1 Introduce MockableDatabase for wallet unit tests (Andrew Chow)
Pull request description:
For the wallet's unit tests, we currently use either `DummyDatabase` or memory-only versions of either BDB or SQLite. The tests that use `DummyDatabase` just need a `WalletDatabase` so that the `CWallet` can be constructed, while the tests using the memory-only databases just need a backing data store. There is also a `FailDatabase` that is similar to `DummyDatabase` except it fails be default or can have a configured return value. Having all of these different database types can make it difficult to write tests, particularly tests that work when either BDB or SQLite is disabled.
This PR unifies all of these different unit test database classes into a single `MockableDatabase`. Like `DummyDatabase`, most functions do nothing and just return true. Like `FailDatabase`, the return value of some functions can be configured on the fly to test various failure cases. Like the memory-only databases, records can actually be written to the `MockableDatabase` and be retrieved later, but all of this is still held in memory. Using `MockableDatabase` completely removes the need for having BDB or SQLite backed wallets in the unit tests for the tests that are not actually testing specific database behaviors.
Because `MockableDatabase`s can be created by each unit test, we can also control what records are stored in the database. Records can be added and removed externally from the typical database modification functions. This will give us greater ability to test failure conditions, particularly those involving corrupted records.
Possible alternative to #26644
ACKs for top commit:
furszy:
ACK 33e2b82
TheCharlatan:
ACK 33e2b82a4f
Tree-SHA512: c2b09eff9728d063d2d4aea28a0f0e64e40b76483e75dc53f08667df23bd25834d52656cd4eafb02e552db0b9e619cfdb1b1c65b26b5436ee2c971d804768bcc
In `blockDisconnected`, for each transaction in the block, look
for any wallet transactions spending the same inputs. If any of
these transactions were marked conflicted, they are now marked as
inactive.
Co-authored-by: ariard <antoine.riard@gmail.com>
5b3406094f net_processing: Boost inv trickle rate (Anthony Towns)
228e9201ef txmempool: have CompareDepthAndScore sort missing txs first (Anthony Towns)
Pull request description:
Couple of performance improvements when draining the inventory-to-send queue:
* drop txs that have already been evicted from the mempool (or included in a block) immediately, rather than at the end of processing
* marginally increase outgoing trickle rate during spikes in tx volume
ACKs for top commit:
willcl-ark:
ACK 5b34060
instagibbs:
ACK 5b3406094f
darosior:
utACK 5b3406094f
glozow:
code review ACK 5b3406094f
dergoegge:
utACK 5b3406094f
Tree-SHA512: 155cd3b5d150ba3417c1cd126f2be734497742e85358a19c9d365f4f97c555ff9e846405bbeada13c3575b3713c3a7eb2f780879a828cbbf032ad9a6e5416b30
5ff63a09a9 refactor, blockstorage: Replace stopafterblockimport arg (TheCharlatan)
18e5ba7c80 refactor, blockstorage: Replace blocksdir arg (TheCharlatan)
02a0899527 refactor, BlockManager: Replace fastprune from arg with options (TheCharlatan)
a498d699e3 refactor/iwyu: Complete includes for blockmanager_args (TheCharlatan)
f0bb1021f0 refactor: Move functions to BlockManager methods (TheCharlatan)
cfbb212493 zmq: Pass lambda to zmq's ZMQPublishRawBlockNotifier (TheCharlatan)
8ed4ff8e05 refactor: Declare g_zmq_notification_interface as unique_ptr (TheCharlatan)
Pull request description:
The libbitcoin_kernel library should not rely on the `ArgsManager`, but rather use option structs that can be passed to the various classes it uses. This PR removes reliance on the `ArgsManager` from the `blockstorage.*` files. Like similar prior work, it uses the options struct in the `BlockManager` that can be populated with `ArgsManager` values.
Some related prior work: https://github.com/bitcoin/bitcoin/pull/26889https://github.com/bitcoin/bitcoin/pull/25862https://github.com/bitcoin/bitcoin/pull/25527https://github.com/bitcoin/bitcoin/pull/25487
Related PR removing blockstorage globals: https://github.com/bitcoin/bitcoin/pull/25781
ACKs for top commit:
ryanofsky:
Code review ACK 5ff63a09a9. Since last ACK just added std::move and fixed commit title. Sorry for the noise!
mzumsande:
Code Review ACK 5ff63a09a9
Tree-SHA512: 4bde8fd140a40b97eca923e9016d85dcea6fad6fd199731f158376294af59c3e8b163a0725aa47b4be3519b61828044e0a042deea005e0c28de21d8b6c3e1ea7
72efc26439 util: improve streams.h:FindByte() performance (Larry Ruane)
604df63f6c [bench] add streams findbyte (gzhao408)
Pull request description:
This PR is strictly a performance improvement; there is no functional change. The `CBufferedFile::FindByte()` method searches for the next occurrence of the given byte in the file. Currently, this is done by explicitly inspecting each byte in turn. This PR takes advantage of `std::find()` to do the same more efficiently, improving its CPU runtime by a factor of about 25 in typical use.
ACKs for top commit:
achow101:
re-ACK 72efc26439
stickies-v:
re-ACK 72efc26439
Tree-SHA512: ddf0bff335cc8aa34f911aa4e0558fa77ce35d963d602e4ab1c63090b4a386faf074548daf06ee829c7f2c760d06eed0125cf4c34e981c6129cea1804eb3b719
It's unnecessary to keep the data around, as it doesn't do anything. If
prioritisetransaction is called again, we'll make a new entry in
mapDeltas.
These entries are only deleted when the transaction is mined or conflicted
from a block (i.e. not in replacement or eviction), are persisted in
mempool.dat, and never expire. If node operators use the RPC to
regularly prioritise/de-prioritise transactions, these (meaningless)
map entries may hang around forever and take up valuable mempool memory.
Add a stop_after_block_import field to the BlockManager options. Use
this field instead of the global gArgs.
This should allow users of the BlockManager to not rely on the global
Args.
Add a blocks_dir field to the BlockManager options. Move functions
relying on the global gArgs to get the blocks_dir into the BlockManager
class.
This should eventually allow users of the BlockManager to not rely on
the global Args and instead pass in their own options.
Remove access to the global gArgs for the fastprune argument and
replace it by adding a field to the existing BlockManager Options
struct.
When running `clang-tidy-diff` on this commit, there is a diagnostic
error: `unknown type name 'uint64_t' [clang-diagnostic-error] uint64_t
prune_target{0};`, which is fixed by including cstdint.
This should eventually allow users of the BlockManager to not rely on
the global gArgs and instead pass in their own options.
This is a commit in preparation for the next few commits. The functions
are moved to methods to avoid their re-declaration for the purpose of
passing in BlockManager options.
The functions that were now moved into the BlockManager should no longer
use the params as an argument, but instead use the member variable.
In the moved ReadBlockFromDisk and UndoReadFromDisk, change
the function signature to accept a reference to a CBlockIndex instead of
a raw pointer. The pointer is expected to be non-null, so reflect that
in the type.
To allow for the move of functions to BlockManager methods all call
sites require an instantiated BlockManager, or a callback to one.
The lambda captures a reference to the chainman unique_ptr to retrieve
block data. An assert is added on the chainman to ensure that the lambda
is not used while the chainman is uninitialized.
This is done in preparation for the following commits where blockstorage
functions are made BlockManager methods.
fa266c4bbf Temporarily work around gcc-13 warning bug in interfaces_tests (MarcoFalke)
fa28850562 Fix clang-tidy performance-unnecessary-copy-initialization warnings (MarcoFalke)
faaa60a30e Remove unused find_value global function (MarcoFalke)
fa422aeec2 scripted-diff: Use UniValue::find_value method (MarcoFalke)
fa548ac872 Add UniValue::find_value method (MarcoFalke)
Pull request description:
The global function has issues:
* It causes gcc-13 warnings, see https://github.com/bitcoin/bitcoin/issues/26926
* There is no rationale for it being a global function, when it acts like a member function
* `performance-unnecessary-copy-initialization` clang-tidy isn't run on it
Fix all issues by making it a member function.
ACKs for top commit:
achow101:
ACK fa266c4bbf
hebasto:
re-ACK fa266c4bbf
Tree-SHA512: 6c4e25da3122cd3b91c376bef73ea94fb3beb7bf8ef5cb3853c5128d95bfbacbcbfb16cc843eb7b1a7ebd350c2b6311f8085eeacf9aeeab3366987037d209e44
Ensures better memory safety for this global. This came up during
discussion of the following commit, but is not strictly required for its
implementation.
If transactions are being added to the mempool at a rate faster than 7tx/s
(INVENTORY_BROADCAST_PER_SECOND) then peers' inventory_to_send queue can
become relatively large. If this happens, increase the number of txids
we include in an INV message (normally capped at 35) by 5 for each 1000
txids in the queue.
This will tend to clear a temporary excess out reasonably quickly; an
excess of 4000 invs to send will be cleared down to 1000 in about 30
minutes, while an excess of 20000 invs would be cleared down to 1000 in
about 60 minutes.
We use CompareDepthAndScore to choose an order of txs to inv. Rather
than sorting txs that have been evicted from the mempool at the end
of the list, sort them at the beginning so they are removed from
the queue immediately.
d168458d1f scripted-diff: Remove unused chainparamsbase includes (TheCharlatan)
e9ee8aaf3a Add missing definitions in prep for scripted diff (TheCharlatan)
ba8fc7d788 refactor: Replace string chain name constants with ChainTypes (TheCharlatan)
401453df41 refactor: Introduce ChainType getters for ArgsManager (TheCharlatan)
bfc21c31b2 refactor: Create chaintype files (TheCharlatan)
Pull request description:
This pull request is part of the `libbitcoinkernel` project https://github.com/bitcoin/bitcoin/issues/24303https://github.com/bitcoin/bitcoin/projects/18 and more specifically its "Step 2: Decouple most non-consensus code from libbitcoinkernel". It is also a follow up to #26177.
It replaces pull request https://github.com/bitcoin/bitcoin/pull/27294, which just moved the constants to a new file, but did not re-declare them as enums.
The code move of the chain name constants out of the `chainparamsbase` to their own separate header allows the kernel `chainparams` to no longer include `chainparamsbase`. The `chainparamsbase` contain references to the `ArgsManager` and networking related options that should not belong to the kernel library. Besides this move, the constants are re-declared as enums with helper functions facilitating string conversions.
ACKs for top commit:
ryanofsky:
Code review ACK d168458d1f. Just suggested changes since last review.
Tree-SHA512: ac2fbe5cbbab4f52eae1e30af1f16700b6589eb4764c328a151a712adfc37f326cc94a65c385534c57d4bc92cc1a13bf1777d92bc924a20dbb30440e7380b316
fae1d9cded refactor: Remove unused GetTimeMillis (MarcoFalke)
Pull request description:
The function is unused, not type-safe, and does not denote the underlying clock type. So remove it.
ACKs for top commit:
willcl-ark:
tACK fae1d9cded
Tree-SHA512: 41ea7125d1964192b85a94265be974d02bf1e79b1feb61bff11486dc0ac811745156940ec5cad2ad1f94b653936f8ae563c959c1c4142203a55645fcb83203e8
This is a follow-up to previous commits moving the chain constants out
of chainparamsbase.
The script removes the chainparamsbase header in all files where it is
included, but not used. This is done by filtering against all defined
symbols of the header as well as its respective .cpp file.
The kernel chainparams now no longer relies on chainparamsbase.
-BEGIN VERIFY SCRIPT-
sed -i '/#include <chainparamsbase.h>/d' $( git grep -l 'chainparamsbase.h' | xargs grep -L 'CBaseChainParams\|CreateBaseChainParams\|SetupChainParamsBaseOptions\|BaseParams\|SelectBaseParams\|chainparamsbase.cpp' )
-END VERIFY SCRIPT-
This commit effectively moves the definition of these constants
out of the chainparamsbase to their own file.
Using the ChainType enums provides better type safety compared to
passing around strings.
The commit is part of an ongoing effort to decouple the libbitcoinkernel
library from the ArgsManager and other functionality that should not be
part of the kernel library.
These are introduced for the next commit where the usage of the
ChainType is adopted throughout the code.
Co-authored-by: Russell Yanofsky <russ@yanofsky.org>
Co-authored-by: TheCharlatan <seb.kung@gmail.com>
This is the first of a number of commits with the goal of moving the
chain type definitions out of chainparamsbase to their own file and
implementing them as enums instead of constant strings. The goal is to
allow the kernel chainparams to no longer include chainparamsbase.
The commit is part of an ongoing effort to decouple the libbitcoinkernel
library from the ArgsManager and other functionality that should not be
part of the kernel library.
fe49f06c0e doc: clarify PR 26076 release note (Sjors Provoost)
bd13dc2f46 Switch hardened derivation marker to h in descriptors (Sjors Provoost)
Pull request description:
This makes it easier to handle descriptor strings manually, especially when importing from another Bitcoin Core wallet.
For example the `importdescriptors` RPC call is easiest to use `h` as the marker: `'["desc": ".../0h/..."]'`, avoiding the need for escape characters. With this change `listdescriptors` will use `h`, so you can copy-paste the result, without having to add escape characters or switch `'` to 'h' manually.
Both markers can still be parsed.
The `hdkeypath` field in `getaddressinfo` is also impacted by this change, except for legacy wallets. The latter is to prevent accidentally breaking ancient software that uses our legacy wallet.
See discussion in #15740
ACKs for top commit:
achow101:
ACK fe49f06c0e
darosior:
re-ACK fe49f06c0e
Tree-SHA512: f78bc873b24a6f7a2bf38f5dd58f2b723e35e6b10e4d65c36ec300e2d362d475eeca6e5afa04b3037ab4bee0bf8ebc93ea5fc18102a2111d3d88fc873c08dc89
fa83fb3161 wallet: Use steady clock to calculate number of derive iterations (MarcoFalke)
fa2c099cec wallet: Use steady clock to measure scanning duration (MarcoFalke)
fa97621804 qt: Use steady clock to throttle GUI notifications (MarcoFalke)
fa1d8044ab test: Use steady clock in index tests (MarcoFalke)
fa454dcb20 net: Use steady clock in InterruptibleRecv (MarcoFalke)
Pull request description:
`GetTimeMillis` has multiple issues:
* It doesn't denote the underlying clock type
* It isn't type-safe
* It is used incorrectly in places that should use a steady clock
Fix all issues here.
ACKs for top commit:
willcl-ark:
ACK fa83fb3161
martinus:
Code review ACK fa83fb3161, also ran all tests. All usages of the steady_clock are just for duration measurements, so the change to a different epoch is ok.
Tree-SHA512: 5ec4fede8c7f97e2e08863c011856e8304f16ba30a68fdeb42f96a50a04961092cbe46ccf9ea6ac99ff5203c09f9e0924eb483eb38d7df0759addc85116c8a9f
Avoid use of the expensive mod operator (%) when calculating the
buffer offset. No functional difference.
Co-authored-by: Hennadii Stepanov <32963518+hebasto@users.noreply.github.com>
c4981e7f63 prune, import: fixes#23852 (mruddy)
Pull request description:
Fixes#23852
This allows pruning to work during the `-loadblock` import process.
An example use case is where you have a clean set of block files and you want to create a pruned node from them, but you don't want to alter the input set of block files.
#23852 noted that pruning was not working reliably during the loadblock import process. The reason why the loadblock process was not pruning regularly as it progressed is that the pruning process (`BlockManager::FindFilesToPrune`) checks the tip height of the active chainstate, and `CChainState::ActivateBestChain` was not called (which updates that tip height) in `ThreadImport` until after all the import files were processed.
An example bash command line that makes it easy to import a bunch of block files:
```
./src/qt/bitcoin-qt -debug -logthreadnames -datadir=/tmp/btc -prune=550 -loadblock=/readonly/btc/main/blk{00000..00043}.dat
```
One interesting side note is that `CChainState::ActivateBestChain` can be called while the import process is running (in the `loadblk` thread) by concurrent network message processing activity in the `msghand` thread. For example, one way to reproduce this easily is with the `getblockfrompeer` RPC (requesting a block with height greater than 100000) run from a node connected to an importing node. There are other ways too, but this is an easy way. I only mention this to explain how the `max_prune_height=225719` log message in the original issue came to occur.
ACKs for top commit:
achow101:
re-ACK c4981e7f63
Tree-SHA512: d287c7753952c22f598ba782914c47f45ad44ce60b0fbce9561354e701f1a2a98bafaaaa106c8428690b814e281305ca3622b177ed3cb2eb7559f07c958ab537
Also add flag to allow RPC methods that intendionally accept options and
parameters with the same name bypass the check.
Check and flag were suggested by ajtowns
https://github.com/bitcoin/bitcoin/pull/26485#issuecomment-1507916357
Co-authored-by: Anthony Towns <aj@erisian.com.au>
OBJ_NAMED_PARAMS type works the same as OBJ type except it registers the object
keys to be accepted as top-level named-only RPC parameters. Generated
documentation also lists the object keys seperately in a new "Named arguments"
section of help text.
Named-only RPC parameters have the same semantics as python keyword-only
arguments (https://peps.python.org/pep-3102/). They are always required to be
passed by name, so they don't affect interpretation of positional arguments,
and aren't affected when positional arguments are added or removed.
The new OBJ_NAMED_PARAMS type is used in the next commit to make it easier to
pass options values to various RPC methods.
Co-authored-by: Andrew Chow <github@achow101.com>
daba95700b refactor: Make ListSelected return vector (Sebastian Falbesoner)
94776621ba wallet: Move CoinCointrol definitions to .cpp (Aurèle Oulès)
1db23da6e1 wallet: Use std::optional for GetExternalOutput and fixups (Aurèle Oulès)
becc45b589 scripted-diff: Rename setSelected->m_selected_inputs (Aurèle Oulès)
Pull request description:
- Moves CoinControl function definitions from `coincontrol.h` to `coincontrol.cpp`
- Adds more documentation
- Renames class member for an improved comprehension
- Use `std::optional` for `GetExternalOutput`
ACKs for top commit:
achow101:
ACK daba95700b
Xekyo:
ACK daba95700b
Tree-SHA512: 3bf2dc834a3246c2f53f8c55154258e605fcb169431d3f7b156931f33c7e3b1ae28e03e16b37f9140a827890eb7798be485b2c36bfc23ff29bb01763f289a07c
In the wallet ckey loading test, we modify various ckey records to test
corruption handling. As the database is now a mockable database, we can
modify the records that the database will be initialized with. This
avoids having to use the verbose database reading and writing functions.
Since we have a mockable wallet database, we don't really need to be
using BDB or SQLite's in-memory database capabilities. It doesn't really
help us to be using those as we aren't doing anything that requires one
type of db over the other, and will just prefer SQLite if it's
available.
MockableDatabase is suitable for these uses, so use
CreateMockableWalletDatabase to use that.
-BEGIN VERIFY SCRIPT-
sed -i "s/CreateMockWalletDatabase(options)/CreateMockableWalletDatabase()/" $(git grep -l "CreateMockWalletDatabase(options)" -- ":(exclude)src/wallet/walletdb.*")
sed -i "s/CreateMockWalletDatabase/CreateMockableWalletDatabase/" $(git grep -l "CreateMockWalletDatabase" -- ":(exclude)src/wallet/walletdb.*")
-END VERIFY SCRIPT-
MockableDatabase is a WalletDatabase that allows us to interact with the
records to change them independently from the wallet, as well as
changing the return values from within the tests. This will give us
greater flexibility in testing the wallet.
Under which circumstances we process received 'mempool' P2P messages
caused confusion in #27426. Rather than bikeshedding the formulation
of the IF-statement, this adds a comment clarifing when we process
the message. Also, correcting the comment of `m_send_mempool`.
Co-authored-by: willcl-ark <will8clark@gmail.com>
710b83938a rpc: return block hash & height in getbalances, gettransaction & getwalletinfo JSONs (Harris)
Pull request description:
Reopens#18570 and closes#18567.
I have rebased the original PR.
Not sure why the original got closed as it was about to get merged.
ACKs for top commit:
achow101:
ACK 710b83938a
Tree-SHA512: d4478d990be98b1642e9ffb2930987f4a224e8bd64e2e35a5dda927a54c509ec9d712cd0eac35dc2bb89f00a1678e530ce14d7445f1dd93aa3a4cce2bc9b130d
8f14fc8622 test: cover fastprune with excessive block size (Matthew Zipkin)
271c23e87f blockstorage: Adjust fastprune limit if block exceeds blockfile size (Martin Zumsande)
Pull request description:
The debug-only `-fastprune` option used in several tests is not always safe to use:
If a `-fastprune` node receives a block larger than the maximum blockfile size of `64kb` bad things happen: The while loop in `BlockManager::FindBlockPos` never terminates, and the node runs oom because memory for `m_blockfile_info` is allocated in each iteration of the loop.
The same would happen if a naive user used `-fastprune` on anything other than regtest (so this can be tested by syncing on signet for example, the first block that crashes the node is at height 2232).
Change the approach by raising the blockfile size to the size of the block, if that block otherwise wouldn't fit (idea by TheCharlatan).
ACKs for top commit:
ryanofsky:
Code review ACK 8f14fc8622. Added new assert, test, and comment since last review
TheCharlatan:
ACK 8f14fc8622
pinheadmz:
ACK 8f14fc8622
Tree-SHA512: df2fea30613ef9d40ebbc2416eacb574f6d7d96847db5c33dda22a29a2c61a8db831aa9552734ea4477e097f253dbcb6dcb1395d43d2a090cc0588c9ce66eac3
b922f6b526 rpc: scanblocks, add "completed" flag to the result obj (furszy)
ce50acc54f rpc: scanblocks, do not traverse the whole chain block by block (furszy)
Pull request description:
Coming from https://github.com/bitcoin/bitcoin/pull/23549#pullrequestreview-1105712566
The current `scanblocks` flow walks-through every block in the active chain
until hits the chain tip or processes 10k blocks, then calls `lookupFilterRange`
function to obtain all filters from that particular range.
This is only done to obtain the heights range to look up the block
filters. Which is unneeded.
As `scanblocks` only lookup block filters in the active chain, we can
directly calculate the lookup range heights, by using the chain tip,
without requiring to traverse the chain block by block.
ACKs for top commit:
achow101:
ACK b922f6b526
TheCharlatan:
ACK b922f6b526
Tree-SHA512: 0587e6d9cf87a59184adb2dbc26a4e2bce3a16233594c6c330f69feb49bf7dc63fdacf44fc20308e93441159ebc604c63eb7de19204d3e745a2ff16004892b45
be72663a15 test: bumpfee, add coverage for "send coins back to yourself" (furszy)
7bffec6715 bumpfee: enable send coins back to yourself (furszy)
Pull request description:
Simple example:
1) User_1 sends 0.1 btc to user_2 on a low fee transaction.
2) After few hours, the tx is still in the mempool, user_2
is not interested anymore, so user_1 decides to cancel
it by sending coins back to himself.
3) User_1 has the bright idea of opening the explorer and
copy the change output address of the transaction. Then
call bumpfee providing such output (in the "outputs" arg).
Currently, this is not possible. The wallet fails with
"Unable to create transaction. Transaction must have at least
one recipient" error.
The error reason is because we discard the provided output
from the recipients list and set it inside the coin control
so the process adds it later (when the change is calculated).
But.. there is no later if the tx has no outputs.
ACKs for top commit:
ishaanam:
reACK be72663a15
achow101:
ACK be72663a15
Tree-SHA512: c2c38290a998f9b426a830d9624c7feb730158980ac186f8fb0138d5e200935d6538307bc60a2c3d0b7b6ee2b4ffb77a1e98baf8feb1d20a7d825f6055ac377f
9141e4395a rpc, docs: Add note for commands that supports only legacy wallets (Yusuf Sahin HAMZA)
Pull request description:
Refs #25363, apparently issue is not updated since over a month, so i decided to put the same `importaddress` note in #25368 to other rpc commands that needs this note.
Note is added for following commands:
- `importprivkey`
- `importpubkey`
- `importwallet`
- `dumpprivkey`
- `dumpwallet`
- `importmulti`
- `addmultisigaddress`
- `sethdseed`
ACKs for top commit:
achow101:
ACK 9141e4395a
Tree-SHA512: f3dc05d26577fd8dbe2bd69cb5c14ffccebacd6010402af44427b3d01be8484895dfcf33d55dfa766eadb7f9f3bae5cc4c2add3ac816a2ac60e8beb5a97527f3
a5986e82dd refactor: Remove CAddressBookData::destdata (Ryan Ofsky)
5938ad0bdb wallet: Add DatabaseBatch::ErasePrefix method (Ryan Ofsky)
Pull request description:
This is cleanup that doesn't change external behavior. Benefits of the cleanup are:
- Removes awkward `StringMap` intermediate representation for wallet address metadata.
- Simplifies `CWallet`, deals with used address and received request serialization in `walletdb.cpp` instead of higher level wallet code
- Adds test coverage and documentation
This PR doesn't change externally observable behavior. Internally, the only change in behavior is that `EraseDestData` deletes rows directly from the database because they are no longer stored in memory. This is more direct and efficient because it uses a single lookup and scan instead of multiple lookups.
Motivation for this cleanup is making changes like #18550, #18192, #13756 easier to reason about and less likely to result in unintended behavior and bugs
---
This PR is a rebased copy of #18608. For some reason that PR is locked and couldn't be reopened or commented on.
This PR is an alternative to #27215 with differences described in https://github.com/bitcoin/bitcoin/pull/27215#pullrequestreview-1329028143
ACKs for top commit:
achow101:
ACK a5986e82dd
furszy:
Code ACK a5986e82
Tree-SHA512: 6bd3e402f1f60263fbd433760bdc29d04175ddaf8307207c4a67d59f6cffa732e176ba57886e02926f7a1615dce0ed9cda36c8cbc6430aa8e5b56934c23f3fe7
To tell the user whether the process was aborted or not.
Plus, as the process can be aborted prior to the end range,
have also changed the "to_height" result value to return the
last scanned block instead of the end range block.
The current flow walks-through every block in the active chain until
hits the chain tip or processes 10k blocks, then calls
`lookupFilterRange()` to obtain all the filters from that
particular range.
This is only done to obtain the heights range to look up the block
filters. Which is unneeded.
As `scanblocks` only lookup block filters in the active chain, we can
directly calculate the lookup range heights, by using the chain tip,
without requiring to traverse the chain block by block.
0c520679ab doc: add release notes for `abandoned` field in `gettransaction` and `listtransactions` (brunoerg)
a1aaa7f51f rpc, wallet: add `abandoned` field for all categories of transactions in ListTransactions (brunoerg)
Pull request description:
Fixes#25130
ACKs for top commit:
achow101:
re-ACK 0c520679ab
Tree-SHA512: 1864460d76decab7898737c96517d722055eb8f81ca52248fe1035723258c6cd4a93251e06a86ecbbb0b0a80af1466b2c86fb142ace4ccb74cc40d5dc3967d7f
bf77fc9cb4 [test] mempool full in package accept (glozow)
b51ebccc28 [validation] set PackageValidationState when mempool full (glozow)
563a2ee4f5 [policy] disallow transactions under min relay fee, even in packages (glozow)
c4554fe894 [test] package cpfp bumps parents <mempoolminfee but >=minrelaytxfee (glozow)
ac463e87df [test util] mock mempool minimum feerate (glozow)
Pull request description:
Part of package relay, see #27463.
Note that this still allows packages to bump transactions that are below the dynamic mempool minimum feerate, which means this still solves the "mempool is congested and my presigned 1sat/vB tx is screwed" problem for all transactions.
On master, the package policy (only accessible through regtest-only RPC submitpackage) allows 0-fee (or otherwise below min relay feerate) transactions if they are bumped by a child. However, with default package limits, we don't yet have a DoS-resistant way of ensuring these transactions remain bumped throughout their time in the mempool. Primarily, the fee-bumping child may later be replaced by another transaction that doesn't bump the parent(s). The parent(s) could potentially stay bumped by other transactions, but not enough to ever be selected by the `BlockAssembler` (due to `blockmintxfee`).
For example, (tested [here](https://github.com/glozow/bitcoin/commits/26933-motivation)):
- The mempool accepts 24 below-minrelayfeerate transactions ("0-fee parents"), all bumped by a single high-fee transaction ("the fee-bumping child"). The fee-bumping child also spends a confirmed UTXO.
- Two additional children are added to each 0-fee parent. These children each pay a feerate slightly above the minimum relay feerate (e.g. 1.9sat/vB) such that, for each 0-fee parent, the total fees of its two children divided by the total size of the children and parent is above the minimum relay feerate.
- If a block template is built now, all transactions would be selected.
- A transaction replaces the the fee-bumping child, spending only the confirmed UTXO and not any of the outputs from the 0-fee parents.
- The 0-fee parents now each have 2 children. Their descendant feerates are above minrelayfeerate, which means that they remain in the mempool, even if the mempool evicts all below-minrelayfeerate packages.
- If a block template is built now, none of the 0-fee parents or their children would be selected.
- Even more low-feerate descendants can be added to these below-minrelayfeerate packages and they will not be evicted until they expire or the mempool reaches capacity.
Unless we have a DoS-resistant way of ensuring package CPFP-bumped transactions are always bumped, allowing package CPFP to bump below-minrelayfeerate transactions can result in these problematic situations. See #27018 which proposes a partial solution with some limitations, and contains discussion about potential improvements to eviction strategy. While no adequate solution exists, for now, avoid these situations by requiring all transactions to meet min relay feerate.
ACKs for top commit:
ajtowns:
reACK bf77fc9cb4
instagibbs:
re-ACK bf77fc9cb4
Tree-SHA512: 28940f41493a9e280b010284316fb8caf1ed7b2090ba9a4ef8a3b2eafc5933601074b142f4f7d4e3c6c4cce99d3146f5c8e1393d9406c6f2070dd41c817985c9
10a354f174 test: prevent intermittent failures (Amiti Uttarwar)
Pull request description:
Follow up to #27214 - add an address to the tried table before the new table to make sure a new table collision is not possible.
ACKs for top commit:
mzumsande:
Code review ACK 10a354f174 - the fix is what I suggested [here](https://github.com/bitcoin/bitcoin/pull/27214#discussion_r1169169601) and should make these intermittent failures impossible.
Tree-SHA512: 24099f02e1915395130065af0ef6a2a1893955d222517d156d928765541d9c427da00172a9b5a540163f4d6aae93ca3882e8267eeb35ecc595d42178abc6191c
6e9f8bb050 rpc, tests: in `utxoupdatepsbt` also look for the transaction in the txindex (ishaanam)
a5b4883fb4 rpc: extract psbt updating logic into ProcessPSBT (ishaanam)
Pull request description:
Previously the `utxoupdatepsbt` RPC, added in #13932, only updated the inputs spending segwit outputs with the `witness_utxo`, because the full transaction is needed for legacy inputs. Before this RPC looked for just the utxos in the utxo set and the mempool. This PR makes it so that if the user has txindex enabled, then the full transaction is looked for there for all inputs. If it is not found in the txindex or txindex isn't enabled, then the mempool is checked for the full transaction. If the transaction an input is spending from is still not found at that point, then the utxo set is searched and the inputs spending segwit outputs are updated with just the utxo.
ACKs for top commit:
achow101:
ACK 6e9f8bb050
Xekyo:
ACK 6e9f8bb050
Tree-SHA512: 078db3c37a1ecd5816d80a42e8bd1341e416d661f508fa5fce0f4e1249fefd7b92a0d45f44957781cb69d0953145ef096ecdd4545ada39062be27742402dac6f
Currently debug.log will show the wrong bitcoin.conf config file path when
bitcoind is invoked without -conf or -datadir arguments, and there's a default
bitcoin.conf file which specifies another datadir= location. When this happens,
the debug.log will include an incorrect "Config file:" line referring to a
bitcoin.conf file in the other datadir, instead of the referring to the actual
configuration file in the default datadir which was parsed.
The bad log print was reported and originally fixed in
https://github.com/bitcoin/bitcoin/pull/27303 by
Matthew Zipkin <pinheadmz@gmail.com>
This PR takes a slightly different approach to fixing the bug, trying to avoid
future bugs by not allowing the GetConfigFilePath function to be called before
the the configuration is parsed, and deleting GetConfigFile function which
could be confused with GetConfigFilePath. It also includes a test for the bug
which the original fix did not have.
Co-authored-by: Matthew Zipkin <pinheadmz@gmail.com>
faa7144d3c fuzz: re-enable prioritisetransaction & analyzepsbt RPC (MarcoFalke)
Pull request description:
The linked issue seems fixed, so it should be fine to re-enable
ACKs for top commit:
dergoegge:
utACK faa7144d3c
Tree-SHA512: a681c726fceacc27ab5a03d455c7808d33f3cb11fe7d253d455526568af840b29f0c3c1d97c54785ef9277e7891a3aa742ac73ccd3cf115b7606eba50864aaa9
Show an error on startup if a bitcoin datadir that is being used contains a
`bitcoin.conf` file that is ignored. There are two cases where this could
happen:
- One case reported in
https://github.com/bitcoin/bitcoin/issues/27246#issuecomment-1470006043
happens when a bitcoin.conf file in the default datadir (e.g.
$HOME/.bitcoin/bitcoin.conf) has a "datadir=/path" line that sets different
datadir containing a second bitcoin.conf file. Currently the second
bitcoin.conf file is ignored with no warning.
- Another way this could happen is if a -conf= command line argument points
to a configuration file with a "datadir=/path" line and that specified path
contains a bitcoin.conf file, which is currently ignored.
This change only adds an error message and doesn't change anything about way
settings are applied. It also doesn't trigger errors if there are redundant
-datadir or -conf settings pointing at the same configuration file, only if
they are pointing at different files and one file is being ignored.
be55f545d5 move-only: Extract common/args and common/config.cpp from util/system (TheCharlatan)
Pull request description:
This pull request is part of the `libbitcoinkernel` project https://github.com/bitcoin/bitcoin/issues/24303https://github.com/bitcoin/bitcoin/projects/18 and more specifically its "Step 2: Decouple most non-consensus code from libbitcoinkernel". It is part of a series of patches splitting up the `util/system` files. Its preceding pull request is https://github.com/bitcoin/bitcoin/pull/27254.
The pull request contains an extraction of ArgsManager related functions from util/system into their own common/ file.
The background of this commit is an ongoing effort to decouple the libbitcoinkernel library from the ArgsManager. The ArgsManager belongs into the common library, since the kernel library should not depend on it. See [doc/design/libraries.md](https://github.com/bitcoin/bitcoin/blob/master/doc/design/libraries.md) for more information on this rationale.
ACKs for top commit:
MarcoFalke:
re-ACK be55f545d5🚲
ryanofsky:
Code review ACK be55f545d5. Just small cleanups since the last review.
hebasto:
ACK be55f545d5, I have reviewed the code and it looks OK, I agree it can be merged.
Tree-SHA512: 90eb03334af0155b823030b4f2ecf286d35058d700ee2ddbbaa445be19e31eb0fe982656f35bd14ecee3ad2c3d0db3746855cb8f3777eff7253713e42873e111
0076bed45e logging: log ASN when using `-asmap` (brunoerg)
9836c76ae0 net: add `GetMappedAS` in `CConnman` (brunoerg)
Pull request description:
When using `-asmap`, you can check the ASN assigned to the peers only with the RPC command `getpeerinfo` (check `mapped_as` field), however, it's not possible to check it in logs (e.g. see in logs the ASN of the peers when a new outbound peer has been connected). This PR includes the peers' ASN in debug output when using `-asmap`.
Obs: Open this primarily to chase some Concept ACK, I've been using this on my node to facilitate to track the peers' ASN especially when reading the logs.
ACKs for top commit:
Sjors:
tACK 0076bed45e
jamesob:
ACK 0076bed45e ([`jamesob/ackr/27412.1.brunoerg.logging_net_add_asn_from`](https://github.com/jamesob/bitcoin/tree/ackr/27412.1.brunoerg.logging_net_add_asn_from))
achow101:
ACK 0076bed45e
Tree-SHA512: c19cd11e8ab49962021f390459aadf6d33d221ae9a2c3df331a25d6865a8df470e2c8828f6e5219b8a887d6ab5b3450d34be9e26c00cca4d223b4ca64d51111b
25ab14712b refactor: coinselector_tests, unify wallet creation code (furszy)
ba9431c505 test: coverage for bnb max weight (furszy)
5a2bc45ee0 wallet: clean post coin selection max weight filter (furszy)
2d112584e3 coin selection: BnB, don't return selection if exceeds max allowed tx weight (furszy)
d3a1c098e4 test: coin selection, add coverage for SRD (furszy)
9d9689e5a6 coin selection: heap-ify SRD, don't return selection if exceeds max tx weight (furszy)
6107ec2229 coin selection: knapsack, select closest UTXO above target if result exceeds max tx size (furszy)
1284223691 wallet: refactor coin selection algos to return util::Result (furszy)
Pull request description:
Coming from the following comment https://github.com/bitcoin/bitcoin/pull/25729#discussion_r1029324367.
The reason why we are adding hundreds of UTXO from different sources when the target
amount is covered only by one of them is because only SRD returns a usable result.
Context:
In the test, we create 1515 UTXOs with 0.033 BTC each, and 1 UTXO with 50 BTC. Then
perform Coin Selection to fund 49.5 BTC.
As the selection of the 1515 small UTXOs exceeds the max allowed tx size, the
expectation here is to receive a selection result that only contain the big UTXO.
Which is not happening for the following reason:
Knapsack returns a result that exceeds the max allowed transaction size, when
it should return the closest utxo above the target, so we fallback to SRD who
selects coins randomly up until the target is met. So we end up with a selection
result with lot more coins than what is needed.
ACKs for top commit:
S3RK:
ACK 25ab14712b
achow101:
ACK 25ab14712b
Xekyo:
reACK 25ab14712b
theStack:
Code-review ACK 25ab14712b
Tree-SHA512: 2425de4cc479b4db999b3b2e02eb522a2130a06379cca0418672a51c4076971a1d427191173820db76a0f85a8edfff100114e1c38fb3b5dc51598d07cabe1a60
9f947fc3d4 Use PoolAllocator for CCoinsMap (Martin Leitner-Ankerl)
5e4ac5abf5 Call ReallocateCache() on each Flush() (Martin Leitner-Ankerl)
1afca6b663 Add PoolResource fuzzer (Martin Leitner-Ankerl)
e19943f049 Calculate memory usage correctly for unordered_maps that use PoolAllocator (Martin Leitner-Ankerl)
b8401c3281 Add pool based memory resource & allocator (Martin Leitner-Ankerl)
Pull request description:
A memory resource similar to `std::pmr::unsynchronized_pool_resource`, but optimized for node-based containers. The goal is to be able to cache more coins with the same memory usage, and allocate/deallocate faster.
This is a reimplementation of #22702. The goal was to implement it in a way that is simpler to review & test
* There is now a generic `PoolResource` for allocating/deallocating memory. This has practically the same API as `std::pmr::memory_resource`. (Unfortunately I cannot use std::pmr because libc++ simply doesn't implement that API).
* Thanks to sipa there is now a fuzzer for PoolResource! On a fast machine I ran it for ~770 million executions without finding any issue.
* The estimation of the correct node size is now gone, PoolResource now has multiple pools and just needs to be created large enough to have space for the unordered_map nodes.
I ran benchmarks with #22702, mergebase, and this PR. Frequency locked Intel i7-8700, clang++ 13.0.1 to reindex up to block 690000.
```sh
bitcoind -dbcache=5000 -assumevalid=00000000000000000002a23d6df20eecec15b21d32c75833cce28f113de888b7 -reindex-chainstate -printtoconsole=0 -stopatheight=690000
```
The performance is practically identical with #22702, just 0.4% slower. It's ~21% faster than master:
Note that on cache drops mergebase's memory doesnt go so far down because it does not free the `CCoinsMap` bucket array.
ACKs for top commit:
LarryRuane:
ACK 9f947fc3d4
achow101:
re-ACK 9f947fc3d4
john-moffett:
ACK 9f947fc3d4
jonatack:
re-ACK 9f947fc3d4
Tree-SHA512: 48caf57d1775875a612b54388ef64c53952cd48741cacfe20d89049f2fb35301b5c28e69264b7d659a3ca33d4c714d47bafad6fd547c4075f08b45acc87c0f45
17e705428d doc: clarify new_only param for Select function (Amiti Uttarwar)
b0010c83a1 bench: test select for a new table with only one address (Amiti Uttarwar)
9b91aae085 bench: add coverage for addrman select with network parameter (Amiti Uttarwar)
22a4d1489c test: increase coverage of addrman select (without network) (Amiti Uttarwar)
a98e542e0c test: add addrman test for special case (Amiti Uttarwar)
5c8b4baff2 tests: add addrman_select_by_network test (Amiti Uttarwar)
6b229284fd addrman: add functionality to select by network (Amiti Uttarwar)
26c3bf11e2 scripted-diff: rename local variables to match modern conventions (Amiti Uttarwar)
48806412e2 refactor: consolidate select logic for new and tried tables (Amiti Uttarwar)
ca2a9c5f8f refactor: generalize select logic (Amiti Uttarwar)
052fbcd5a7 addrman: Introduce helper to generalize looking up an addrman entry (Amiti Uttarwar)
9bf078f66c refactor: update Select_ function (Amiti Uttarwar)
Pull request description:
For the full context & motivation of this patch, see #27213
This is joint work with mzumsande.
This PR adds functionality to `AddrMan::Select` to enable callers to specify a network they are interested in.
Along the way, it refactors the function to deduplicate the logic, updates the local variables to match modern conventions, adds test coverage for both the new and existing `Select` logic, and adds bench tests for the worst case performance of both the new and existing `Select` logic.
This functionality is used in the parent PR.
ACKs for top commit:
vasild:
ACK 17e705428d
brunoerg:
re-ACK 17e705428d
ajtowns:
ACK 17e705428d
mzumsande:
Code Review ACK 17e705428d
Tree-SHA512: e99d1ce0c44a15601a3daa37deeadfc9d26208a92969ecffbea358d57ca951102d759734ccf77eacd38db368da0bf5b6fede3cd900d8a77b3061f4adc54e52d8
If the added block exceeds the blockfile size in test-only
-fastprune mode, the node would get stuck in an infinite loop and
run out of memory.
Avoid this by raising the blockfile size to the size of the added block
in this situation.
Co-authored-by: TheCharlatan <seb.kung@gmail.com>
This is an extraction of ArgsManager related functions from util/system
into their own common file.
Config file related functions are moved to common/config.cpp.
The background of this commit is an ongoing effort to decouple the
libbitcoinkernel library from the ArgsManager. The ArgsManager belongs
into the common library, since the kernel library should not depend on
it. See doc/design/libraries.md for more information on this rationale.
0d6383fda0 Don't return OutputType::UNKNOWN in ParseOutputType (Pttn)
Pull request description:
Fixes https://github.com/bitcoin/bitcoin/issues/27472 by also handling at the relevant places the case where ParseOutputType returns `OutputType::UNKNOWN`, and not just when it returns `std::nullopt`.
ACKs for top commit:
achow101:
ACK 0d6383fda0
MarcoFalke:
lgtm ACK 0d6383fda0
furszy:
ACK 0d6383fda0
Tree-SHA512: 776793027b926283d3162e69fb9c8883c814b19bcce4574ccdf8e3140a1ec4ebc4aa8ccd1abae7ef3571f942d2e6c35305fd1244259540d90605106e01afc34c
In PR 27374, the semantics of the `setConnected` data structure in
CConnman::ThreadOpenConnections changed from the set of outbound peer
netgroups to those of outbound IPv4/6 peers only.
This commit updates a code comment in this regard about feeler connections and
updates the naming of `setConnected` to `outbound_ipv46_peer_netgroups` to
reflect its new role.
`evhttp_uri_parse` can return a nullptr, for example when the URI
contains invalid characters (e.g. "%").
`GetQueryParameterFromUri` passes the output of `evhttp_uri_parse`
straight into `evhttp_uri_get_query`, which means that anyone calling
a REST endpoint in which query parameters are used (e.g. `rest_headers`)
can cause a segfault.
This bugfix is designed to be minimal and without additional behaviour change.
Follow-up work should be done to resolve this in a more general and robust way,
so not every endpoint has to handle it individually.
Avoid adding transactions below min relay feerate because, even if they
were bumped through CPFP when entering the mempool, we do not have a
DoS-resistant way of ensuring they always remain bumped. In the future,
this rule can be relaxed (e.g. to allow packages to bump 0-fee
transactions) if we find a way to do so.
Simple example:
1) User_1 sends 0.1 btc to user_2 on a low fee transaction.
2) After few hours, the tx is still in the mempool, user_2
is not interested anymore, so user_1 decides to cancel
it by sending coins back to himself.
3) User_1 has the bright idea of opening the explorer and
copy the change output address of the transaction. Then
call bumpfee providing such output (in the "outputs" arg).
Currently, this is not possible. The wallet fails with
"Unable to create transaction. Transaction must have at least
one recipient" error.
The error reason is that we discard the provided output from
the recipients list and set it inside the coin control
so the process adds it later (when the change is calculated).
But.. there is no later if the tx has no outputs.
d52fa1b0a5 tests: Make sure that bumpfee feerate checks work when replacing outputs (Andrew Chow)
be177c15a4 bumpfee: Check the correct feerate when replacing outputs (Andrew Chow)
Pull request description:
When replacing the outputs of a transaction during `bumpfee`, it is possible to accidentally create a transaction that will not be accepted into the mempool as it does not meet the incremental relay fee requirements. This occurs because the size estimation used for checking the provided feerate does not account for the replaced outputs; it instead uses the original outputs. When the replaced outputs is significantly different from the original, there can be a large difference in estimated transaction sizes that can make a transaction miss the absolute fee requirements for the incremental relay fee. Unfortunately we do not currently inform the user when the bumped transaction fails to relay, so they could use `bumpfee` and think the transaction has been bumped when it actually has not.
This issue is resolved by replacing the outputs before doing the size estimation, and also updating the feerate checker to use the actual fee values when calculating the required minimum fee.
Also added a test for this scenario.
ACKs for top commit:
ishaanam:
reACK d52fa1b0a5
Xekyo:
reACK d52fa1b0a5
Tree-SHA512: d18301b587465322dd3fb1bb86496c3675265a56072047576e2baa5cf907dd3b54778f30721f662f0c235709a5568427c18542eb7efbfb6fdd9f481fe676c66b
4258c54f4e Merge bitcoin-core/secp256k1#1276: autotools: Don't regenerate Wycheproof header automatically
06c67dea9f autotools: Don't regenerate Wycheproof header automatically
3bab71cf05 Merge bitcoin-core/secp256k1#1268: release cleanup: bump version after 0.3.1
656c6ea8d8 release cleanup: bump version after 0.3.1
346a053d4c Merge bitcoin-core/secp256k1#1269: changelog: Fix link
6a37b2a5ea changelog: Fix link
ec98fcedd5 Merge bitcoin-core/secp256k1#1266: release: Prepare for 0.3.1
898e1c676e release: Prepare for 0.3.1
1d9a13fc26 changelog: Remove inconsistent newlines
0e091669a1 changelog: Catch up in preparation of 0.3.1
7b7503dac5 Merge bitcoin-core/secp256k1#1245: tests: Add Wycheproof ECDSA vectors
145078c418 Merge bitcoin-core/secp256k1#1118: Add x-only ecmult_const version with x specified as n/d
e5de454609 tests: Add Wycheproof ECDSA vectors
0f8642079b Add exhaustive tests for ecmult_const_xonly
4485926ace Add x-only ecmult_const version for x=n/d
a0f4644f7e Merge bitcoin-core/secp256k1#1252: Make position of * in pointer declarations in include/ consistent
4e682626a3 Merge bitcoin-core/secp256k1#1226: Add CMake instructions to release process
2d51a454fc Merge bitcoin-core/secp256k1#1257: ct: Use volatile "trick" in all fe/scalar cmov implementations
4a496a36fb ct: Use volatile "trick" in all fe/scalar cmov implementations
3d1f430f9f Make position of * in pointer declarations in include/ consistent
2bca0a5cbf Merge bitcoin-core/secp256k1#1241: build: Improve `SECP_TRY_APPEND_DEFAULT_CFLAGS` macro
afd8b23b27 Merge bitcoin-core/secp256k1#1244: Suppress `-Wunused-parameter` when building for coverage analysis
1d8f367515 Merge bitcoin-core/secp256k1#1250: No need to subtract 1 before doing a right shift
3e43041be6 No need to subtract 1 before doing a right shift
3addb4c1e8 build: Improve `SECP_TRY_APPEND_DEFAULT_CFLAGS` macro
0c07c82834 Add CMake instructions to release process
464a9115b4 Merge bitcoin-core/secp256k1#1242: Set ARM ASM symbol visibility to `hidden`
f16a709fd6 Merge bitcoin-core/secp256k1#1247: Apply Checks only in VERIFY mode.
70be3cade5 Merge bitcoin-core/secp256k1#1246: Typo
4ebd82852d Apply Checks only in VERIFY mode.
d1e7ca192d Typo
5bb03c2911 Replace `SECP256K1_ECMULT_TABLE_VERIFY` macro by a function
9c8c4f443c Merge bitcoin-core/secp256k1#1238: build: bump CMake minimum requirement to 3.13
0cf2fb91ef Merge bitcoin-core/secp256k1#1243: build: Ensure no optimization when building for coverage analysis
fd2a408647 Set ARM ASM symbol visibility to `hidden`
4429a8c218 Suppress `-Wunused-parameter` when building for coverage analysis
8e79c7ed11 build: Ensure no optimization when building for coverage analysis
96dd062511 build: bump CMake minimum requirement to 3.13
427bc3cdcf Merge bitcoin-core/secp256k1#1236: Update comment for secp256k1_modinv32_inv256
647f0a5cb1 Update comment for secp256k1_modinv32_inv256
5658209459 Merge bitcoin-core/secp256k1#1228: release cleanup: bump version after 0.3.0
28e63f7ea7 release cleanup: bump version after 0.3.0
git-subtree-dir: src/secp256k1
git-subtree-split: 4258c54f4ebfc09390168e8a43306c46b315134b
b5585ba5f9 p2p: skip netgroup diversity of new connections for tor/i2p/cjdns networks (stratospher)
Pull request description:
Follow up for #27264.
In order to make sure that our persistent outbound slots belong to different netgroups, distinct net groups of our peers are added to `setConnected`. We’d only open a persistent outbound connection to peers which have a different netgroup compared to those netgroups present in `setConnected`.
Current `GetGroup()` logic assumes route-based diversification behaviour for tor/i2p/cjdns addresses (addresses are public key based and not route-based). Distinct netgroups possible (according to the current `GetGroup()` logic) for:
1. tor => 030f, 031f, .. 03ff (16 possibilities)
2. i2p => 040f, 041f, .. 04ff (16 possibilities)
3. cjdns => 05fc0f, 05fc1f, ... 05fcff (16 possibilities)
`setConnected` is used in `ThreadOpenConnections()` before making [outbound](84f4ac39fd/src/net.cpp (L1846)) and [anchor](84f4ac39fd/src/net.cpp (L1805)) connections to new peers so that they belong to distinct netgroups.
**behaviour on master**
- if we run a node only on tor/i2p/cjdns
- we wouldn't be able to open more than 16 outbound connections(manual, block-relay-only anchor, outbound full relay, block-relay-only connections) because we run out of possible netgroups.
- see https://github.com/bitcoin/bitcoin/pull/27264#issuecomment-1481322628
- tested by changing `MAX_OUTBOUND_FULL_RELAY_CONNECTIONS` to 17 with `onlynet=onion` and observed how node wouldn't make more than 16 outbound connections.
**behaviour on PR**
- netgroup diversity checks are skipped for tor/i2p/cjdns addresses.
- we don't insert tor/i2p/cjdns address in `setConnected` and `GetGroup` doesn't get called on tor/i2p/cjdns(see #27369)
ACKs for top commit:
achow101:
ACK b5585ba5f9
mzumsande:
ACK b5585ba5f9
vasild:
ACK b5585ba5f9
Tree-SHA512: c120b3f9ca7f0be3f29ea665cd2f7dfb40cd1d7ec7058984252fb6e0295e414f736c5b4fba03c31188188a5ae4f543fb2654f6ee9776bad745c7ca72d23d5b9b
7ccdd741fe test: fix importmulti/importdescriptors assertion (Jon Atack)
19d888ce40 rpc: move WALLET_FLAG_CAVEATS to the compilation unit of its caller (Jon Atack)
01df011ca2 doc: release note for wallet RPCs "warning" field deprecation (Jon Atack)
9ea8b3739a test: createwallet "warning" field deprecation test (Jon Atack)
645d7f75ac rpc: deprecate "warning" field in {create,load,unload,restore}wallet (Jon Atack)
2f4a926e95 test: add test coverage for "warnings" field in createwallet (Jon Atack)
4a1e479ca6 rpc: add "warnings" field to RPCs {create,load,unload,restore}wallet (Jon Atack)
079d8cdda8 rpc: extract wallet "warnings" fields to a util helper (Jon Atack)
f73782a903 doc: fix/improve warning helps in {create,load,unload,restore}wallet (Jon Atack)
Pull request description:
Based on discussion and concept ACKed in #27138, add a `warnings` field to RPCs createwallet, loadwallet, unloadwallet, and restorewallet as a JSON array of strings to replace the `warning` string field in these 4 RPCs. The idea is to more gracefully handle multiple warning messages and for consistency with other wallet RPCs. Then, deprecate the latter fields, which represent all the remaining RPC `warning` fields.
The first commit f73782a903 implements https://github.com/bitcoin/bitcoin/pull/27138#issuecomment-1474789198 as an alternative to #27138. One of those two could potentially be backported to our currently supported releases.
ACKs for top commit:
achow101:
ACK 7ccdd741fe
1440000bytes:
utACK 7ccdd741fe
vasild:
ACK 7ccdd741fe
pinheadmz:
re-ACK 7ccdd741fe
Tree-SHA512: 314e0a4c41fa383d95e2817bfacf359d449e460529d235c3eb902851e2f4eacbabe646d9a5a4beabc4964cdfabf6397ed8301366a58d344a2f787f83b75e9d64
This is cleanup that doesn't change external behavior.
- Removes awkward `StringMap` intermediate representation
- Simplifies CWallet code, deals with used address and received request
serialization in walletdb.cpp
- Adds test coverage and documentation
- Reduces memory usage
This PR doesn't change externally observable behavior. Internally, only change
in behavior is that EraseDestData deletes directly from database because the
`StringMap` is gone. This is more direct and efficient because it uses a single
btree lookup and scan instead of multiple lookups
Motivation for this cleanup is making changes like #18550, #18192, #13756
easier to reason about and less likely to result in unintended behavior and
bugs
Co-authored-by: furszy <matiasfurszyfer@protonmail.com>
Instead of storing and passing around fixed strings for the purpose of
an address, use an enum.
This also rationalizes the CAddressBookData struct, documenting all fields and
making them public, and simplifying the representation to avoid bugs like
https://github.com/bitcoin/bitcoin/pull/26761#discussion_r1134615114 and make
it not possible to invalid address data like change addresses with labels.
Co-authored-by: Ryan Ofsky <ryan@ofsky.org>
68eed5df86 test,gui: add coverage for PSBT creation on legacy watch-only wallets (furszy)
306aab5bb4 test,gui: decouple widgets and model into a MiniGui struct (furszy)
2f76ac0383 test,gui: decouple chain and wallet initialization from test case (furszy)
cd98b71739 gui: 'getAvailableBalance', include watch only balance (furszy)
74eac3a82f test: add coverage for 'useAvailableBalance' functionality (furszy)
dc1cc1c359 gui: bugfix, getAvailableBalance skips selected coins (furszy)
Pull request description:
Fixes https://github.com/bitcoin-core/gui/issues/688 and https://github.com/bitcoin/bitcoin/issues/26687.
First Issue Description (https://github.com/bitcoin-core/gui/issues/688):
The previous behavior for `getAvailableBalance`, when the coin control had selected coins, was to return the sum of them. Instead, we are currently returning the wallet's available total balance minus the selected coins total amount.
Reason:
Missed to update the `GetAvailableBalance` function to include the coin control selected coins on #25685.
Context:
Since #25685 we skip the selected coins inside `AvailableCoins`, the reason is that there is no need to waste resources walking through the entire wallet's txes map just to get coins that could have gotten by just doing a simple `mapWallet.find`).
Places Where This Generates Issues (only when the user manually select coins via coin control):
1) The GUI balance check prior the transaction creation process.
2) The GUI "useAvailableBalance" functionality.
Note 1:
As the GUI uses a balance cache since https://github.com/bitcoin-core/gui/pull/598, this issue does not affect the regular spending process. Only arises when the user manually select coins.
Note 2:
Added test coverage for the `useAvailableBalance` functionality.
----------------------------------
Second Issue Description (https://github.com/bitcoin/bitcoin/issues/26687):
As we are using a cached balance on `WalletModel::getAvailableBalance`,
the function needs to include the watch-only available balance for wallets
with private keys disabled.
ACKs for top commit:
Sjors:
tACK 68eed5df86
achow101:
ACK 68eed5df86
theStack:
ACK 68eed5df86
Tree-SHA512: 674f3e050024dabda2ff4a04b9ed3750cf54a040527204c920e1e38bd3d7f5fd4d096e4fd08a0fea84ee6abb5070f022b5c0d450c58fd30202ef05ebfd7af6d3
3153e7d779 [fuzz] Add HeadersSyncState target (dergoegge)
53552affca [headerssync] Make m_commit_offset protected (dergoegge)
Pull request description:
This adds a fuzz target for the `HeadersSyncState` class.
I am unsure how well this is able to cover the logic since it is just processing unserialized CBlockHeaders straight from the fuzz input (headers are sometimes made continuous). However, it does manage to get to the redownload phase so i thought it is better then not having fuzzing at all.
It would also be nice to fuzz the p2p logic that is using `HeadersSyncState` (e.g. `TryLowWorkHeadersSync`, `IsContinuationOfLowWorkHeadersSync`) but that likely requires some more work (refactoring👻).
ACKs for top commit:
mzumsande:
ACK 3153e7d779
Tree-SHA512: 8a4630ceeeb30e4eeabaa8eb5491d98f0bf900efe7cda07384eaac9f2afaccfbcaa979cc1cc7f0b6ca297a8f5c17a7759f94809dd87eb87d35348d847c83e8ab
55c4795c57 [net processing] Use TxRelay::m_relay_txs over CNode::m_relays_txs (dergoegge)
Pull request description:
`CNode::m_relays_txs` is meant to only be used for the eviction logic in `net`. `TxRelay::m_relay_txs` will hold the same value and is meant to be used on the application layer to determine if we will/should relay transactions to a peer.
(Shameless plug: we should really better specify the interface for updating eviction data to avoid refactors like this in the future -> #25572)
ACKs for top commit:
MarcoFalke:
lgtm ACK 55c4795c57
Tree-SHA512: 59cfd23e32568fd96cda5570790e518242a6c76d4edf5b7d1a2a7f9724d590d2a38395504e05be0af4e98dd5c0056fc0be6568eab2818934692483a186e5181d
and add the walletutil.h include header for WALLET_FLAG_AVOID_REUSE that was
already missing before this change.
WALLET_FLAG_CAVEATS is only used in one RPC, so no need to encumber wallet.h and
wallet.cpp with it, along with all of the files that include wallet.h during
their compilation. Also apply clang-format per:
git diff -U0 HEAD~1.. | ./contrib/devtools/clang-format-diff.py -p1 -i -v
This new "warnings" field is a JSON array of strings intended to replace the
"warning" string field in these four RPCs, to better handle returning multiple
warning messages and for consistency with other wallet RPCs.
When doing the feerate check for bumped transactions that replace the
outputs, we need to consider that the size of the new outputs may be
different from the old outputs and calculate the minimum feerate accordingly.
Uses a min-effective-value heap, so we can remove the least valuable input/s
while the selected weight exceeds the maximum allowed weight.
Co-authored-by: Murch <murch@murch.one>
The simplest scenario where this is useful is on the 'check_max_weight' unit test
already:
We create 1515 UTXOs with 0.033 BTC each, and 1 UTXO with 50 BTC. Then perform
Coin Selection.
As the selection of the 1515 small UTXOs exceeds the max allowed tx size, the
expectation here is to receive a selection result that only contain the big
UTXO (which is not happening for the reasons stated below).
As knapsack returns a result that exceeds the max allowed transaction size, we
fallback to SRD, which selects coins randomly up until the target is met. So
we end up with a selection result with lot more coins than what is needed.
73f4eb511c Check that the Timestamp String is valid (John Moffett)
Pull request description:
Follow-up to https://github.com/bitcoin/bitcoin/pull/27233
The current `FormatISO8601DateTime` function will return an empty string if it encounters an error when converting the `int64_t` seconds-since-epoch to a formatted date time. In the unlikely case that happens, here `strStamped.pop_back()` would be undefined behavior.
ACKs for top commit:
MarcoFalke:
lgtm ACK 73f4eb511c
stickies-v:
ACK 73f4eb511c
Tree-SHA512: 089ed639c193deb98870a8385039b31b4baed821ea907937bfc6f65a5b0981bbf8284b2afec81b2d0a922e2340716b48cf55349640eb6b8c311ef7af25abc361
This makes it easier to handle descriptor strings manually. E.g. an RPC call that takes an array of descriptors can now use '["desc": ".../0h/..."]'.
Both markers can still be parsed. The default for new descriptors is changed to h. In normalized form h is also used. For private keys the chosen marker is preserved in a round trip.
The hdkeypath field in getaddressinfo is also impacted by this change.
The following cases were covered:
Case 1: No coin control selected coins.
- 'useAvailableBalance' should fill the amount edit box with the total available balance.
Case 2: With coin control selected coins.
- 'useAvailableBalance' should fill the amount edit box with the sum of the selected coins values.
The previous behavior for getAvailableBalance when coin control
has selected coins was to return the sum of them. Instead, we
are currently returning the wallet's available total balance minus
the selected coins total amount.
This turns into a GUI-only issue for the "use available balance"
button when the user manually select coins in the send screen.
Reason:
We missed to update the GetAvailableBalance function to include
the coin control selected coins on #25685.
Context:
Since #25685 we skip the selected coins inside `AvailableCoins`,
the reason is that there is no need to traverse the wallet's
txes map just to get coins that can directly be fetched by
their id.
00e9b97f37 refactor: Move fs.* to util/fs.* (TheCharlatan)
106b46d9d2 Add missing fs.h includes (TheCharlatan)
b202b3dd63 Add missing cstddef include in assumptions.h (TheCharlatan)
18fb36367a refactor: Extract util/fs_helpers from util/system (Ben Woosley)
Pull request description:
This pull request is part of the `libbitcoinkernel` project https://github.com/bitcoin/bitcoin/issues/24303https://github.com/bitcoin/bitcoin/projects/18 and more specifically its "Step 2: Decouple most non-consensus code from libbitcoinkernel". This commit was originally authored by empact and is taken from its parent PR #25152.
#### Context
There is an ongoing effort to decouple the `ArgsManager` used for command line parsing user-provided arguments from the libbitcoinkernel library (https://github.com/bitcoin/bitcoin/pull/25290, https://github.com/bitcoin/bitcoin/pull/25487, https://github.com/bitcoin/bitcoin/pull/25527, https://github.com/bitcoin/bitcoin/pull/25862, https://github.com/bitcoin/bitcoin/pull/26177, and https://github.com/bitcoin/bitcoin/pull/27125). The `ArgsManager` is defined in `system.h`. A similar pull request extracting functionality from `system.h` has been merged in https://github.com/bitcoin/bitcoin/pull/27238.
#### Changes
Next to providing better code organization, this PR removes some reliance of the tree of libbitcoinkernel header includes on `system.h` (and thus the `ArgsManager` definition) by moving filesystem related functions out of the `system.*` files.
There is already a pair of `fs.h` / `fs.cpp` in the top-level `src/` directory. They were not combined with the files introduced here, to keep the patch cleaner and more importantly because they are often included without the utility functions. The new files are therefore named `fs_helpers` and the existing `fs` files are moved into the util directory.
Further commits splitting more functionality out of `system.h` are still in #25152 and will be submitted in separate PRs once this PR has been processed.
ACKs for top commit:
hebasto:
ACK 00e9b97f37
Tree-SHA512: 31422f148d14ba3c843b99b1550a6fd77c77f350905ca324f93d4f97b652246bc58fa9696c64d1201979cf88733e40be02d262739bb7d417cf22bf506fdb7666
3fa4c54ac5 [net processing] Pass TxRelay to FindTxForGetData instead of Peer (dergoegge)
c85ee76a36 [net processin] Don't take cs_main in FindTxForGetData (dergoegge)
Pull request description:
Addresses left over feedback from #26140.
* https://github.com/bitcoin/bitcoin/pull/26140#discussion_r1153498543
* https://github.com/bitcoin/bitcoin/pull/26140#discussion_r1153499627
`mapRelay` is only accessed from the message processing thread and does not need to be kept in sync with anything validation specific, it is therfore perfectly fine to have it guarded by `g_msgproc_mutex`.
ACKs for top commit:
jnewbery:
utACK 3fa4c54ac5
hebasto:
ACK 3fa4c54ac5, I have reviewed the code and it looks OK, I agree it can be merged.
Tree-SHA512: 3ef84bfe4abfa8d991a7e65d9184221294d80e0df0bbb47f0270ab6ca1593266c98abf83c610f9f86b4d16c7a4b62bcf83f8856c68d3c2e10894bff6ed3e88cd
e414edd8fc qt: Update translation source file (Hennadii Stepanov)
b780095091 qt: Adjust plural forms for translations (Hennadii Stepanov)
6ae8a24009 GUI: Send: Make feerates translatable (Luke Dashjr)
bd42f5e1cd Bugfix: GUI: Send/PSBT: Correct virtual size unit and make translatable (Luke Dashjr)
1b0407f5f1 Bugfix: GUI: transactiondesc: Translate outlier "own address" and "watch-only" (Luke Dashjr)
170f3126f2 GUI: Use translated external signer errors for messagebox text (Luke Dashjr)
96989599d6 GUI: Make messages for copying unsigned PSBTs translatable (Luke Dashjr)
08b8b287d3 Bugfix: GUI: Debug info: Use correct "kB" case for small mempool sizes, and make translation-friendly (Luke Dashjr)
dacc322be1 GUI: PSBTOperationsDialog: Support translating window title (Luke Dashjr)
5a4fe55270 GUI: Intro: Support translating caption of data directory chooser (Luke Dashjr)
3868ba3a27 GUI: Support translating peer network names (Luke Dashjr)
f1f9811198 GUI: Support translating address type dropdown entries (Luke Dashjr)
Pull request description:
This PR updates the `src/qt/locale/bitcoin_en.xlf` translation source file according to [Release schedule for 25.0](https://github.com/bitcoin/bitcoin/issues/26549).
Some translation-related fixes have been picked from https://github.com/bitcoin-core/gui/pull/599 and https://github.com/bitcoin-core/gui/pull/716.
Note for reviewers: it is expected to get a zero diff after running `make -C src translate` locally.
ACKs for top commit:
jarolrod:
ACK e414edd8fc
Tree-SHA512: 5b0c70db1e2f5871067e84f43ebea4ee4f0027fc5f2be49bbcb1d04e162ae76607b2b038e9d0622bcb5b3658d0ede8c10c4421ddaa3343f0e0be54315ca7a4f5
1869310f3c refactor: remove unused param from legacy pubkey (Bushstar)
Pull request description:
Unused param present in legacy pubkey manager interface. This param will not be used and should be removed to prevent unintended usage.
ACKs for top commit:
Sjors:
ACK 1869310f3c
furszy:
ACK 1869310f3c
Tree-SHA512: 0fb41fc8f481f859262f2e8e9a93c990c1b4637e74fd9191ccc0b3c523d0e7d94217a3074bb357276e1941a10d29326f850f9b27eccc1eca57cf6b549353400c
Taking cs_main is no longer necessary since we moved
`m_recently_announced_invs` to `Peer` and `mapRelay` is actually only
accessed from the message processing thread.
Rewrite the same algo instead of reusing BlockAssembler because we have
a few extra requirements that would make the changes invasive and
difficult to review:
- Only operate on the relevant transactions rather than full mempool
- Remove transactions that will be replaced so they can't bump their ancestors
- Don't hold mempool lock outside of the constructor
- Skip things like max block weight and IsFinalTx
- Additionally calculate fees to bump remaining ancestor packages to target feerate
Co-authored-by: Murch <murch@murch.one>
faf8dc496e fuzz: Remove legacy int parse fuzz tests (MarcoFalke)
Pull request description:
The fuzz tests checked that the result of the new function was equal to the legacy function. (Side note: The checks were incomplete, as evident by the follow-up fix in commit b5c9bb5cb9).
Given that they haven't found any issues in years (beside missing the above issue, that they couldn't catch), it seems time to remove them.
They may come in handy in the rare case that someone would want to modify `LocaleIndependentAtoi()` or `Parse*Int*()`, however that seems unlikely. Also, appropriate checks can be added then.
ACKs for top commit:
fanquake:
ACK faf8dc496e
dergoegge:
ACK faf8dc496e
Tree-SHA512: 4ec88b9fa8ba49a923b0604016f0f471b3c9b9e0ba6c5c3dc4e20503c6994789921e7221d9ec467a2a37a73f21a70ba51ba3370ed5ad311dee989e218290b29a
cd0c8eeb09 [net] Pass nRecvFloodSize to CNode (dergoegge)
860402ef2e [net] Remove trivial GetConnectionType() getter (dergoegge)
b5a85b365a [net] Delete CNetMessage copy constructor/assignment op (dergoegge)
Pull request description:
Follow-up PR for #27257
* Deletes the copy constructor/assignment operator of `CNetMessage`
* Removes trivial getter for the connection type
* Avoids passing `nRecvFloodSize` to CNode methods by passing it to `CNode` on creation
ACKs for top commit:
jnewbery:
utACK cd0c8eeb09
theStack:
ACK cd0c8eeb09
Tree-SHA512: 673a758668617f69fba77e61f0eaa1538da27a4849c82c98742436692baa2d7f001129af3e7a66b160e599d12109dac08137a146f10ff9b9ebdc5c2237311d41
We limit GatherClusters’s result to a maximum of 500 transactions as
clusters can be made arbitrarily large by third parties.
Co-authored-by: Murch <murch@murch.one>
9a1d73fdff Fix segfault when shutdown during wallet open (John Moffett)
Pull request description:
Fixes#689
## Summary
If you open a wallet and send a shutdown signal during that process, you'll get a segfault when the wallet finishes opening. That's because the `WalletController` object gets deleted manually in bitcoin.cpp during shutdown, but copies of the pointer (and pointers to child objects) are dangling in various places and are accessed in queued events after the deletion.
## Details
The issue in #689 is caused by the following sequence of events:
1. Wallet open modal dialog is shown and worker thread does the actual work.
2. Every 200ms, the main event loop checks to see if a shutdown has been requested, but only if a modal is not being shown.
3. Request a shutdown while the modal window is shown.
4. The wallet open process completes, the modal window is dismissed, and various `finish` signals are sent.
5. During handling of one of the `finish` signals, `qApp->processEvents()` is [called](e9262ea32a/src/qt/sendcoinsdialog.cpp (L603)), which causes the main event loop to detect the shutdown (now that the modal window has been dismissed). The `WalletController` and all the `WalletModel`s are [deleted](65de8eeeca/src/qt/bitcoin.cpp (L394-L401)).
6. Control returns to the `finish` method, which eventually tries to send a [signal](e9262ea32a/src/qt/sendcoinsdialog.cpp (L167)) from a wallet model, but it's been deleted already (and the signal is sent from a now-[dangling](d8bdee0fc8/src/qt/walletview.cpp (L65)) pointer).
The simplest fix for that is to change the `qApp->processEvents()` into a `QueuedConnection` call. (The `qApp->processEvents() was a [workaround](https://github.com/bitcoin/bitcoin/pull/593#issuecomment-3050699) to get the GUI to scroll to the last item in a list that just got added, and this is just a safer way of doing that).
However, once that segfault is fixed, another segfault occurs due to some queued wallet events happening after the wallet controller object is deleted here:
65de8eeeca/src/qt/bitcoin.cpp (L394-L401)
Since `m_wallet_controller` is a copy of that pointer in `bitcoingui.cpp`, it's now dangling and `if(null)` checks won't work correctly. For instance, this line:
65de8eeeca/src/qt/bitcoingui.cpp (L413)
sets up a `QueuedConnection` to `setCurrentWallet`, but by the time control reaches that method (one event cycle after shutdown deleted `m_wallet_controller` in `bitcoin.cpp`), the underlying objects have been destroyed (but the pointers are still dangling).
Ideally, we'd use a `QPointer` or `std::shared_ptr / std::weak_ptr`s for these, but the changes would be more involved.
This is a minimal fix for the issues. Just set `m_wallet_controller` to `nullptr` in `bitcoingui.cpp`, check its value in a couple places, and avoid a use of `qApp->processEvents`.
ACKs for top commit:
hebasto:
ACK 9a1d73fdff, I have reviewed the code and it looks OK.
furszy:
ACK 9a1d73fdff
Tree-SHA512: a1b94676eb2fcb7606e68fab443b1565b4122aab93c35382b561842a049f4b43fecc459535370d67a64d6ebc4bcec0ebcda981fff633ebd41bdba6f7093ea540
Using swap() was rather wasteful because it had to copy the whole direct
memory data twice. Also, due to the swap() in move assignment the moved-from
object might hold on to unused memory for longer than necessary.
Move operations already are `noexcept`, so add the keyword to the methods.
This makes the `PrevectorFillVectorIndirect...` benchmarks about twice
as fast on my machine, because otherwise `std::vector` has to use a copy
when the vector resizes.
8c47d599b8 doc: improve -debuglogfile help to be a bit clearer (jonatack)
20d89d6802 bench: document expected results in logging benchmarks (jonatack)
d8deba8c36 bench: add LogPrintfCategory and LogPrintLevel benchmarks (Jon Atack)
102b203349 bench: order the logging benchmark code by output (Jon Atack)
4b3fdbf6fe bench: update logging benchmark naming for clarity (Jon Atack)
4684aa8733 bench: allow logging benchmarks to be order-independent (Larry Ruane)
Pull request description:
Update our logging benchmarks for evaluating ongoing work like #25203 and refactoring proposals like #26619 and #26697.
- make the logging benchmarks order-independent (Larry Ruane)
- add missing benchmarks for the `LogPrintLevel` and `LogPrintfCategory` macros that our logging is migrating to; at some later point it should be feasible to drop some of the previous logging benchmarks
- update the logging benchmark naming to be clear which benchmark corresponds to which log macro, and update the ordering to be the same as the output
- add clarifying documentation to the logging benchmarks
- improve the `-debuglogfile` config option help to be clearer; can be tested by running `./src/bitcoind -help | grep -A4 '\-debuglogfile'`
Reviewers can run the logging benchmarks with:
```bash
./src/bench/bench_bitcoin -filter='LogP*.*'
```
ACKs for top commit:
LarryRuane:
ACK 8c47d599b8
martinus:
code review & tested ACK 8c47d599b8, here are my benchmark results:
achow101:
ACK 8c47d599b8
Tree-SHA512: 705f8720c9ceaf14a1945039c7578a0c17a12215cbc44908099af4ac444561c3f95d833c5a91b325cdd4470737d8a01e2da64db2d542dd7c9a3747fbfdbf213e
The current `FormatISO8601DateTime` function will
return an empty string if it encounters an error
when converting the `int64_t` seconds since epoch
to a formatted date time. In the unlikely case that happens,
`strStamped.pop_back()` would be undefined behavior.
In my benchmarks, using this pool allocator for CCoinsMap gives about
20% faster `-reindex-chainstate` with -dbcache=5000 with practically the
same memory usage. The change in max RSS changed was 0.3%.
The `validation_flush_tests` tests need to be updated because
memory allocation is now done in large pools instead of one node at a
time, so the limits need to be updated accordingly.
This frees up all associated memory with the map, not only the nodes.
This is necessary in preparation for using the PoolAllocator for
CCoinsMap, which does not actually free any memory on clear().
A memory resource similar to std::pmr::unsynchronized_pool_resource, but
optimized for node-based containers.
Co-Authored-By: Pieter Wuille <pieter@wuille.net>
As per https://github.com/bitcoin/bitcoin/pull/26506#pullrequestreview-1211984059,
this function is no longer necessary and we can use UniValue::read() directly.
To avoid code duplication, we keep the function to throw on invalid input data
but rename it to Parse() and remove it from the header.
Preparation to deprecate ParseNonRFCJSONValue() but keep test coverage
on the underlying UniValue::read() unaffected. The test coverage on
AmountFromValue is no longer included, since that is already tested
in the rpc_parse_monetary_values test case.
Fuzzing coverage on ParseNonRFCJSONValue() was duplicated between string.cpp
and parse_univalue.cpp, only the one in parse_univalue.cpp is kept.
3566aa7d49 [net] Remove CNode friends (dergoegge)
3eac5e7cd1 [net] Add CNode helper for send byte accounting (dergoegge)
60441a3432 scripted-diff: [net] Rename CNode process queue members (dergoegge)
6693c499f7 [net] Make cs_vProcessMsg a non-recursive mutex (dergoegge)
23d9352654 [net] Make CNode msg process queue members private (dergoegge)
897e342d6e [net] Encapsulate CNode message polling (dergoegge)
cc5cdf8776 [net] Deduplicate marking received message for processing (dergoegge)
ad44aa5c64 [net] Add connection type getter to CNode (dergoegge)
Pull request description:
We should define clear interfaces between CNode, CConnman and PeerManager. This PR makes a small step in that direction by ending the friendship of CNode, CConnman and ConnmanTestMsg. CNode's message processing queue is made private in the process and its mutex is turned into a non-recursive mutex.
ACKs for top commit:
jnewbery:
utACK 3566aa7d49
vasild:
ACK 3566aa7d49
theStack:
re-ACK 3566aa7d49
brunoerg:
re-ACK 3566aa7d49
Tree-SHA512: 26b87da5054e32401b693b2904e9c5f40e35a53937c0b6cf44b8597034ad07bacf27d87cdffc54d3e7ccfebde4231ef30a38d326f88cc18133bbb34688ead567
f3221d373a test: add wallet too-long-mempool-chain error coverage (furszy)
acf0119d24 wallet: return error msg for too-long-mempool-chain failure (furszy)
Pull request description:
Fixes#23144.
We currently return a general "Insufficient funds" from Coin
Selection when we actually skipped unconfirmed UTXOs that
surpassed the mempool ancestors limit.
This PR make the error clearer by returning:
"Unconfirmed UTXOs are available, but spending them creates
a chain of transactions that will be rejected by the mempool"
Also, added an early return from Coin Selection if the sum of
the discarded coins decreases the available balance below the
target amount.
ACKs for top commit:
achow101:
ACK f3221d373a
S3RK:
Code review ACK f3221d373a
Xekyo:
ACK f3221d373a
Tree-SHA512: 13e5824b75ac302280ff894560a4ebf32a74f32fe49ef8281f2bc99c0104b92cef33d3b143c6e131f3a07eafe64533af7fc60abff585142c134b9d6e531a6a66
fa18504d57 rpc: Add submit option to generateblock (MarcoFalke)
fab9a08e14 refactor: Replace block_hash with block_out (MarcoFalke)
Pull request description:
When submit is turned off, a block can be generated and returned as hex, to be used for further tests. For example, it can be submitted on a different node, on a different interface (like p2p), or just never submitted and be used for other testing purposes.
ACKs for top commit:
instagibbs:
ACK fa18504d57
TheCharlatan:
tACK fa18504d57
Tree-SHA512: 1b2ab6b71bb7e155c6482d75f5373f4e77de6446cb16bc2dfd19e7a4075b3a6ad87d7ad7a049a9eed934cb71574acfd27202f54c8bb3b03fac869f2e95db7ee5
The fs.* files are already part of the libbitcoin_util library. With the
introduction of the fs_helpers.* it makes sense to move fs.* into the
util/ directory as well.
This is an extraction of filesystem related functions from util/system
into their own utility file.
The background of this commit is an ongoing effort to decouple the
libbitcoinkernel library from the ArgsManager defined in system.h.
Moving these functions out of system.h allows including them from a
separate source file without including the ArgsManager definitions from
system.h.
faf3f12424 refactor: Replace GetTimeMicros by SystemClock (MarcoFalke)
Pull request description:
It is unclear from the name that `GetTimeMicros` returns the system time. Also, it is not using the type-safe `std::chrono` types.
Fix both issues by replacing it with `SystemClock` in the only place it is used.
This refactor should not change behavior.
ACKs for top commit:
willcl-ark:
tACK faf3f1242
john-moffett:
ACK faf3f12424 changes, but left a comment for the existing code.
Tree-SHA512: 069e6ef26467a469f128b98a4aeb334f141742befd7880cb3a7d280480e9f0684dc0686fa6a828cdcb3d11943ae5c7f8ad5d9d9dab4c668be85e5d28c78cd489
fae349076d test: Remove unused Check* default constructors (MarcoFalke)
Pull request description:
They are no longer needed after the removal of `swap`, see https://github.com/bitcoin/bitcoin/pull/26749#discussion_r1144532693
Also, flatten a redundant `if` check.
ACKs for top commit:
hebasto:
ACK fae349076d
Tree-SHA512: c0bc0c16b5df0f16fc25e18d2414a2a3c4769da1aa30d53f8d267bc2e97dd79a0296db94c1e49cd1ca89bd42275d8c462f7bf47f03f105dfe867ebea6563454b
95ad70ab65 test: Default initialize `should_freeze` to `true` (Hennadii Stepanov)
cea50521fe refactor: Drop no longer used `swap` member functions (Hennadii Stepanov)
a87fb6bee5 clang-tidy: Fix modernize-use-default-member-init in `CScriptCheck` (Hennadii Stepanov)
b4bed5c1f9 refactor: Drop no longer used `CScriptCheck()` default constructor (Hennadii Stepanov)
d8427cc28e refactor: Use move semantics in `CCheckQueue::Loop` (Hennadii Stepanov)
9a0b524139 clang-tidy, test: Fix bugprone-use-after-move in `Correct_Queue_range()` (Hennadii Stepanov)
04831fee6d refactor: Make move semantics explicit for callers (Hennadii Stepanov)
6c2d5972f3 refactor: Use move semantics in `CCheckQueue::Add` (Hennadii Stepanov)
0682003214 test, refactor: Avoid `CScriptCheck::swap` in `transaction_tests` (Hennadii Stepanov)
15209d97c6 consensus, refactor: Avoid `CScriptCheck::swap` in `CheckInputScripts` (Hennadii Stepanov)
Pull request description:
This PR makes code more succinct and readable by using move semantics.
ACKs for top commit:
martinus:
re-ACK 95ad70ab65
achow101:
ACK 95ad70ab65
TheCharlatan:
re-ACK 95ad70ab65
MarcoFalke:
re-ACK 95ad70ab65🚥
Tree-SHA512: adda760891b12d252dc9b823fe7c41eed660364b6fb1a69f17607d7a31eb0bbb82a80d154a7acfaa241b5de37d42a293c2b6e059f26a8e92d88d3a87c99768fb
fa67b8181c Refactor: Remove unused FlatFilePos::SetNull (MarcoFalke)
Pull request description:
This is unused outside of tests and the default constructor. With C++11, it can be replaced by C++11 member initializers in the default constructor.
Beside removing unused code, this also makes it less fragile in light of uninitialized memory. (See also https://github.com/bitcoin/bitcoin/pull/26296#issuecomment-1477801767)
If new code needs to set this to null, it can use `std::optional`, or in the worst case re-introduce this method.
ACKs for top commit:
dergoegge:
Code review ACK fa67b8181c
TheCharlatan:
ACK fa67b8181c
john-moffett:
ACK fa67b8181c
Tree-SHA512: 465c5e3eb4625405c445695d33e09a1fc5185c7dd1e766ba06034fb093880bfc65441d5334f7d9b20e2e417c2075557d86059f59d9648ca0e62a54c699c029b9
2c3a90f663 log: on new valid header (James O'Beirne)
e5ce857634 log: net: new header over cmpctblock (James O'Beirne)
Pull request description:
Alternate to #27276.
Devs were [suprised to realize](https://twitter.com/jamesob/status/1637237917201383425) last night that we don't have definitive logging for when a given header was first received.
This logs to the main stream when new headers are received outside of IBD, as well as when headers come in over cmpctblocks. The rationale of not hiding these under log categories is that they may be useful to have widely available when debugging strange network activity, and the marginal volume is modest.
ACKs for top commit:
dergoegge:
Code review ACK 2c3a90f663
achow101:
ACK 2c3a90f663
Sjors:
tACK 2c3a90f663
josibake:
ACK 2c3a90f663
Tree-SHA512: 49fdcbe07799c8adc24143d7e5054a0c93fef120d2e9d5fddbd3b119550d895e2985be6ac10dd1825ea23a6fa5479c1b76d5518c136fbd983fa76c0d39dc354f
fabb95e7bf doc: add release note for 26899 (brunoerg)
c84c5f6e89 p2p: set `-dnsseed` and `-listen` false if `maxconnections=0` (brunoerg)
Pull request description:
If `maxconnections=0`, it means our possible connections are going to be manual (e.g via `addnode`). For this reason, we can skip DNS seeds and set `listen` false.
ACKs for top commit:
achow101:
ACK fabb95e7bf
vasild:
ACK fabb95e7bf
1440000bytes:
reACK fabb95e7bf
Tree-SHA512: 33919a784723a32450f39ee4f6de3e27cc7c7f4c6ab4b8ce673981d461df334197deaf43e3f882039fa1ac36b2fddc6c6ab4413512d6c393d4a6865302dd05e7
4b7aec2951 Add mempool tracepoints (virtu)
Pull request description:
This PR adds multiple mempool tracepoints.
| tracepoint | description |
| ------------- | ------------- |
| `mempool:added` | Is called when a transaction enters the mempool |
| `mempool:removed` | ... when a transaction is removed from the mempool |
| `mempool:replaced` | ... when a transaction is replaced in the mempool |
| `mempool:rejected` | ... when a transaction is rejected from entering the mempool |
The tracepoints are further documented in `docs/tracing.md`. Usage is demonstrated in the example script `contrib/tracing/mempool_monitor.py`. Interface tests are provided in `test/functional/interface_usdt_mempool.py`.
The rationale for passing the removal reason as a string instead of numerically is that the benefits of not having to maintain a redundant enum-string mapping seem to outweigh the small cost of string generation. The reject reason is passed as string as well, although in this instance the string does not have to be generated but is readily available.
ACKs for top commit:
0xB10C:
ACK 4b7aec2951
achow101:
ACK 4b7aec2951
Tree-SHA512: 6deb3ba2d1a061292fb9b0f885f7a5c4d11b109b838102d8a8f4828cd68f5cd03fa3fc64adc6fdf54a08a1eaccce261b0aa90c2b8c33cd5fd3828c8f74978958
Tracepoints for added, removed, replaced, and rejected transactions.
The removal reason is passed as string instead of a numeric value, since
the benefits of not having to maintain a redundant enum-string mapping
seem to outweigh the small cost of string generation. The reject reason
is passed as string as well, although here the string does not have to
be generated but is readily available.
So far, tracepoint PRs typically included two demo scripts: a naive
bpftrace script to show raw tracepoint data and a bcc script for a more
refined view. However, as some of the ongoing changes to bpftrace
introduce a certain degree of unreliability (running some of the
existing bpftrace scripts was not possible with standard kernels and
bpftrace packages on latest stable Ubuntu, Debian, and NixOS), this PR
includes only a single bcc script that fuses the functionality of former
bpftrace and bcc scripts.
e43a547a36 refactor: wallet, do not translate init arguments names (furszy)
Pull request description:
Simple, and not interesting, refactor that someone has to do sooner or later. We are translating some init arguments names when those shouldn't be translated.
ACKs for top commit:
achow101:
ACK e43a547a36
MarcoFalke:
lgtm ACK e43a547a36
ryanofsky:
Code review ACK e43a547a36. Just rebased since last review.
Tree-SHA512: c6eca98fd66d54d5510de03ab4e63c00ba2838af4237d2bb135d01c47f8ad8ca9aa7ae1e45cf668afcfb9dd958b075a1756cc887b3beef2cb494933d4d83eab0
72e8ffd7f8 p2p: Account for MANUAL conns when diversifying persistent outbound conns (Gleb Naumenko)
3faae99c3d p2p: Diversify connections only w.r.t *persistent* outbound peers (Gleb Naumenko)
Pull request description:
Revives #19860.
In order to make sure that our persistent outbound slots belong to different netgroups, distinct net groups of our peers are added to [`setConnected`](8c4958bd4c/src/net.cpp (L1716)). We’d only open a persistent outbound connection to peers which have a different netgroup compared to those netgroups present in `setConnected`.
**behaviour on master**
we open persistent outbound connections to peers which have different netgroups compared to outbound full relay, block relay, addrfetch and feeler connection peers.
**behaviour on PR**
netgroup diversity is based on outbound full relay, block relay and manual connection peers.
**rationale**
- addrfetch and feeler connections are short lived connections and shouldn’t affect how we select outbound peers from addrman.
- manual connections are like regular connections when viewed from addrman’s netgroup diversity point of view and should affect how we select outbound peers from addrman
ACKs for top commit:
amitiuttarwar:
code review ACK 72e8ffd7f8
vasild:
ACK 72e8ffd7f8
mzumsande:
Code Review ACK 72e8ffd7f8
brunoerg:
crACK 72e8ffd7f8
Tree-SHA512: 359451945a707b312ef6c2696a3a9d4256ab14dab9bd461cca4a52dae034db099012df6de3faef2f3fb38184b05996402ac280b681959483824419b6deb4db1a
Previously only the segwit utxos being spent by the psbt were looked for and
added to the psbt. Now, the full transaction corresponding to each of these
utxos (legacy and segwit) is looked for in the txindex and mempool and added
to the psbt. If txindex is disabled and the transaction is not in the mempool,
then we fall back to getting just the utxo (if segwit) from the utxo set.
the addrman select function will demonstrate it's worst case performance when
it is almost empty, because it might have to linearly search several buckets.
add a bench test to cover this case
Co-authored-by: Martin Zumsande <mzumsande@gmail.com>
to evaluate the worst case performance with the network parameter passed
through, fill the new table with addresses then add a singular I2P address to
retrieve
Co-authored-by: Martin Zumsande <mzumsande@gmail.com>
if an addr matching the network requirements is only on the new table and
select is invoked with new_only = false, ensure that the code selects the new
table.
in order to test this case, we use a non deterministic addrman. this means we
cannot have more than one address in any addrman table, or risk sporadic
failures when the second address happens to conflict.
if the code chose a table at random, the test would fail 50% of the time
Co-authored-by: Martin Zumsande <mzumsande@gmail.com>
this adds coverage for the 7 different cases of which table should be selected
when the network is specified. the different cases are the result of new_only
being true or false and whether there are network addresses on both, neither,
or one of new vs tried tables. the only case not covered is when new_only is
false and the only network addresses are on the new table.
Co-authored-by: Martin Zumsande <mzumsande@gmail.com>
Co-authored-by: Vasil Dimov <vd@FreeBSD.org>
Add an optional parameter to the addrman Select function that allows callers to
specify which network the returned address should be on. Ensure that the proper
table is selected with different cases of whether the new or tried table has
network addresses that match.
Co-authored-by: Martin Zumsande <mzumsande@gmail.com>
in preparation for consolidating the logic for searching the new and tried
tables, generalize the call paths for both
Co-authored-by: Martin Zumsande <mzumsande@gmail.com>
b3e78dc91d refactor: Don't use global chainparams in chainstatemanager method (TheCharlatan)
382b692a50 Split non/kernel chainparams (Carl Dong)
edabbc78a3 Add factory functions for Main/Test/Sig/Reg chainparams (Carl Dong)
d938098398 Remove UpdateVersionBitsParameters (Carl Dong)
84b85786f0 Decouple RegTestChainParams from ArgsManager (Carl Dong)
76cd4e7c96 Decouple SigNetChainParams from ArgsManager (Carl Dong)
Pull request description:
This pull request is part of the `libbitcoinkernel` project https://github.com/bitcoin/bitcoin/issues/24303https://github.com/bitcoin/bitcoin/projects/18 and more specifically its "Step 2: Decouple most non-consensus code from libbitcoinkernel". dongcarl is the original author of this patchset, these commits were taken from https://github.com/dongcarl/bitcoin/tree/2022-03-libbitcoinkernel-chainparams-args-only.
#### Context
The bitcoin kernel library currently relies on code containing user configurations through the `ArgsManager`. This is not optimal, since as a stand-alone library it should not rely on bitcoind's argument parsing logic. Instead, its interfaces should accept control and options structs that control the kernel library's desired configuration.
Similar work towards decoupling the `ArgsManager` from the kernel has been done in
https://github.com/bitcoin/bitcoin/pull/25290, https://github.com/bitcoin/bitcoin/pull/25487, https://github.com/bitcoin/bitcoin/pull/25527 and https://github.com/bitcoin/bitcoin/pull/25862.
#### Changes
By moving the `CChainParams` class definition into the kernel and giving it new factory functions `CChainParams::{RegTest,SigNet,Main,TestNet}`it can be constructed without an `ArgsManager` reference, unlike the current factory function `CreateChainParams`.
The first few commits remove uses of `ArgsManager` within `CChainParams`. Then the `CChainParams` definition is moved to a new file in the `kernel/` subdirectory.
ACKs for top commit:
MarcoFalke:
re-ACK b3e78dc91d🛁
ryanofsky:
Code review ACK b3e78dc91d. Only changes since last review were recent review suggestions.
ajtowns:
ACK b3e78dc91d
Tree-SHA512: 3835aca1d3e3c75cc3303dd584bab3a77e58f6c678724a5e359fe4b0e17e0763a00931ee6191f516b9fde50496f59cc691f0709c0254206db3863bbf7ab2cacd
1ff5d61dfd doc: add mempool/contents rest verbose and mempool_sequence args (Andrew Toth)
52a31dccc9 tests: mempool/contents verbose and mempool_sequence query params tests (Andrew Toth)
a518fff0f2 rest: add verbose and mempool_sequence query params for mempool/contents (Andrew Toth)
Pull request description:
The verbose mempool json response can get very large. This adds an option to return the non-verbose response of just the txids. It is identical to the rpc response so the diff here is minimal. This also adds the mempool_sequence parameter for rpc consistency. Verbose defaults to true to remain backwards compatible.
It uses query parameters to be compatible with the efforts in https://github.com/bitcoin/bitcoin/issues/25752.
ACKs for top commit:
achow101:
ACK 1ff5d61dfd
stickies-v:
re-ACK [1ff5d61](1ff5d61dfd)
pablomartin4btc:
tested ACK 1ff5d61dfd.
Tree-SHA512: 1bf08a7ffde2e7db14dc746e421feedf17d84c4b3f1141e79e36feb6014811dfde80e1d8dbc476c15ff705de2d3c967b3081dcd80536d76b7edf888f1a92e9d1
The chainstatemanager m_options.chainparams member variable gets its
value from the global chainparams in init.cpp. This allows
validation.cpp to only include the the kernel chainparams file.
Moves chainparams code not using the ArgsManager to the kernel.
Subsequently use the kernel chainparams header now where possible in
order to further decouple chainparams call sites from gArgs.
This normalizes the behavior of initializing Main/Test/Sig/Reg
chainparams with RegTest/SigNet chainparams. These factory functions can
also easily be used from a context without an instantiated ArgsManager,
e.g. from libbitcoin kernel code, unlike the existing CreateChainParams
method.
RegTest chain params can now be initialized by configuring a
RegTestOptions struct, or with ArgsManager. This offers an interface for
creating RegTestChainParams without a gArgs object.
SigNet chain params can now be initialized by configuring a
SigNetOptions struct, or with ArgsManager. This offers an interface for
creating SigNetChainParams without a gArgs object.
Previously, we would make connections to peer from the netgroups to which
our MANUAL outbound connections belong.
However, they should be seen as regular connections from Addrman when it comes to netgroup diversity check, since the same rationale can be applied.
Note, this has nothing to do with how we connect to MANUAL connections:
we connect to them unconditionally.
05eeba2c5f [test] Add manual prune startup test case (dergoegge)
4517419628 [util] Avoid integer overflow in CheckDiskSpace (dergoegge)
Pull request description:
Starting a fresh node with `-prune=1` causes an integer overflow to happen in `CheckDiskSpace` ([here](f7bdcfc83f/src/init.cpp (L1633-L1648))) because `nPruneTarget` is to the max `uint64_t` value.
```
node1 stderr util/system.cpp:138:51: runtime error: unsigned integer overflow: 52428800 + 18446744073709551615 cannot be represented in type 'unsigned long'
#0 0x564a482b5088 in CheckDiskSpace(fs::path const&, unsigned long) src/./src/util/system.cpp:138:51
#1 0x564a4728dc59 in AppInitMain(node::NodeContext&, interfaces::BlockAndHeaderTipInfo*) src/./src/init.cpp:1639:14
#2 0x564a47256e6a in AppInit(node::NodeContext&, int, char**) src/./src/bitcoind.cpp:221:43
#3 0x564a47256087 in main src/./src/bitcoind.cpp:265:13
#4 0x7fcb7cbffd8f (/lib/x86_64-linux-gnu/libc.so.6+0x29d8f) (BuildId: 69389d485a9793dbe873f0ea2c93e02efaa9aa3d)
#5 0x7fcb7cbffe3f in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x29e3f) (BuildId: 69389d485a9793dbe873f0ea2c93e02efaa9aa3d)
#6 0x564a471957f4 in _start (/tmp/cirrus-ci-build/ci/scratch/build/bitcoin-x86_64-pc-linux-gnu/src/bitcoind+0xca07f4) (BuildId: 035cb22302d37317a630900a15a26ecb326d395c)
SUMMARY: UndefinedBehaviorSanitizer: unsigned-integer-overflow util/system.cpp:138:51 in
```
I think side stepping the overflow for this specific case, is better than adding an exception to the UB suppresions file.
ACKs for top commit:
MarcoFalke:
ACK 05eeba2c5f🥝
john-moffett:
ACK 05eeba2c5f
Tree-SHA512: 1d8e6bcb49818139f04b5ab2cbef7f9b422bf0c38a804cd532b6bd0ba4c4fd07f959ba977e59896343f213086c8ecc48180f50d006638dc84649c66ec379d58a
error is a low-level function with a sole dependency on LogPrintf, which
is defined in logging.h
The background of this commit is an ongoing effort to decouple the
libbitcoinkernel library from the ArgsManager defined in system.h.
Moving the function out of system.h allows including it from a separate
source file without including the ArgsManager definitions from system.h.
This is a minimal extraction of a single function, but also the only use
of std::exception in util/system.
The background of this commit is an ongoing effort to decouple the
libbitcoinkernel library from the ArgsManager defined in system.h.
Moving the function out of system.h allows including it from a separate
source file without including the ArgsManager definitions from system.h.
763079a3f1 Squashed 'src/secp256k1/' changes from 21ffe4b22a9..bdf39000b9c (Pieter Wuille)
Pull request description:
This updates the libsecp256k1 subtree to [v0.3.0](https://github.com/bitcoin-core/secp256k1/releases/tag/v0.3.0). I don't believe there are code changes that are particularly important to Bitcoin Core, apart from the added CMake build system support.
ACKs for top commit:
jonasnick:
ACK e5c7fcb361
fanquake:
ACK e5c7fcb361
Tree-SHA512: eda42e44d6d4ae43e9fab8a15854e41c8d9e14b645945039dbc35402bec501d73caa5d293264bd03ec6a7fe4919b9a725560f1831a58a6364dc6edaf259145a0
fa3e9b420f refactor: Consistently use args over gArgs in init.cpp (MarcoFalke)
fa891120c8 refactor: Consistently use context args over gArgs in node/interfaces (MarcoFalke)
Pull request description:
Currently `node/interfaces.cpp` uses a mix of `gArgs` vs `m_context->args`. This is fine, because outside of tests those should be identical. However, it makes the code inconsistent and harder to use in tests.
Fix that by using `args` from the context consistently. Do the same in `init.cpp`, where `gArgs` and `args` are inconsistently used in the same scope or even line.
ACKs for top commit:
ryanofsky:
Code review ACK fa3e9b420f
Tree-SHA512: bcd52f176794ebb1ecb9e1922411f7b84d212ae13bad314a1961b85f3077f645fca71fb0124c0889ebfdd8b59a0903b99b9985b1a4fb8f152aa6d7f0126fe5c7
8fcbdadfad util: fix argsman dupe key error (willcl-ark)
Pull request description:
fixes#22638
Make GUI "Settings file could not be read. Do you want to reset settings to default values?" dialog actually clear all settings instead of partially keeping them when `settings.json` contains duplicate keys. This change has no effect on `bitcoind` because it treats a corrupt `settings.json` file as a hard error and doesn't attempt to modify it.
If we find a duplicate key and error, clear `values` before returning so that `WriteSettings()` will write an empty file, therefore clearing it.
This aligns with GUI behaviour added in 1ee6d0b.
The test added only checks that `values` is empty after a duplicate key is detected. This paves the way for the `abort` option in the GUI to properly clear `settings.json`, if the user selects the option, but the test does not currently check this entire mechanism (e.g. the file contents).
ACKs for top commit:
TheCharlatan:
Code review ACK 8fcbdadfad
ryanofsky:
Code review ACK 8fcbdadfad. Thanks for the fix! I would maybe update the PR description or add to it to describe behavior change of this PR:
Tree-SHA512: a5fd49b30ede0a24188623192825bccb952e427cc35f96ff9bfdc737361dcc35ac6480589ddf7f0ddeaebd34361bdaee31e7a91f2c0d857e4ff682614bb6bc04
If `maxconnections=0`, it means our possible connections are
going to be manual (e.g via `addnode`). For this reason, we
can skip DNS seeds and set `listen` false.
We currently return "Insufficient funds" which doesn't really
describe what went wrong; the tx creation failed because of
a long-mempool-chain, not because of a lack of funds.
Also, return early from Coin Selection if the sum of the
discarded coins decreases the available balance below the
target amount.
5c938e74cf Use string interpolation for default value of -listen (ekzyis)
Pull request description:
This is a refactoring change. So I have read the following and will try to answer why this change should be accepted
> * Refactoring changes are only accepted if they are required for a feature or
bug fix or **_otherwise improve developer experience significantly_**. For example,
most "code style" refactoring changes require a thorough explanation why they
are useful, what downsides they have and why they *significantly* improve
developer experience or avoid serious programming bugs. Note that code style
is often a subjective matter. Unless they are explicitly mentioned to be
preferred in the [developer notes](/doc/developer-notes.md), stylistic code
changes are usually rejected.
I have noticed in https://github.com/bitcoin/bitcoin/pull/26899#discussion_r1086731856 that the helper message for `-listen` does not use string interpolation.
That confused me and I wasn't sure what the reasons for that are. So it could be argued this confusion (by possibly many people in the past and in the future) may already be enough to accept this change.
However, not accepting this means that if `DEFAULT_LISTEN` is ever changed, this helper message will still use the old value (however unlikely that may be).
Therefore, this PR makes the helper message consistent with how other helper messages are implemented (using string interpolation) which leads to less confusion and prevents possibly wrong documentation in the future.
ACKs for top commit:
stratospher:
ACK 5c938e7.
vasild:
ACK 5c938e74cf
kristapsk:
cr utACK 5c938e74cf
Tree-SHA512: 8905f548ff399967966e67b29962821c28fd2620ff788c2b2bdfd088bbca75457dc63e933ad1985f93580508a65b91930fe4b2d97a2e0a7d83a3748b9ea2c26f
fe329dc936 test: Add test for getblockfrompeer on pruned nodes (Fabian Jahr)
cd761e6b2c rpc: Add note on guarantees to getblockfrompeer (Fabian Jahr)
Pull request description:
These are additions to `getblockfrompeer` that I already [suggested on the original PR](https://github.com/bitcoin/bitcoin/pull/20295#pullrequestreview-817157738).
The two commits do the following:
1. Add a test for `getblockfrompeer` usage on pruned nodes. This is important because many use-cases for `getblockfrompeer` are in a context of a pruned node.
2. Add some information on how long the users of pruned nodes can expect the block to be available after they have used the RPC. I think the behavior is not very intuitive for users and I would not be surprised if users expect the block to be available indefinitely.
ACKs for top commit:
Sjors:
re-utACK fe329dc936
MarcoFalke:
review ACK fe329dc936🍉
stratospher:
ACK fe329dc.
brunoerg:
re-ACK fe329dc936
Tree-SHA512: a686bd8955d9c3baf365db384e497d6ee1aa9ce2fdb0733fe6150f7e3d94bae19d55bc1b347f1c9f619e749e18b41a52b9f8c0aa2042dd311a968a4b5d251fac
fixes#22638
If we find a duplicate key and error, clear `values` before returning so that
WriteSettings will write an empty file, therefore clearing it.
This aligns with GUI behaviour added in 1ee6d0b.
4be57a5df1 gui: fix comments for BanTableModel and BanTablePriv::refreshBanlist() (Vasil Dimov)
a981af4e6f gui: use the stored CSubNet entry when unbanning (Vasil Dimov)
Pull request description:
The previous code visualized the `CSubNet` object as string, then parsed that string back to `CSubNet`. This is sub-optimal given that the original `CSubNet` object can be used directly instead.
This avoids calling `LookupSubNet()` from the GUI.
ACKs for top commit:
furszy:
utACK 4be57a5d
mzumsande:
Tested ACK 4be57a5df1
Tree-SHA512: b783c18c9d676aa9486cff2d27039dd5c5ef3f1cc67e5056a2be68e35930926f368f26dacdf4f3d394a1f73e3e28f42dc8a6936cd1765c6e6e60695c7b4d78af
bdf39000b9c Merge bitcoin-core/secp256k1#1223: release: prepare for 0.3.0
b40adf23604 release: prepare for 0.3.0
90b513aadad Merge bitcoin-core/secp256k1#1229: cmake: Rename project to "libsecp256k1"
8be82d43628 cmake: Rename project to "libsecp256k1"
ef4f8bd0259 Merge bitcoin-core/secp256k1#1227: readme: Use correct build type in CMake/Windows build instructions
756b61d451d readme: Use correct build type in CMake/Windows build instructions
3295aa149bd Merge bitcoin-core/secp256k1#1225: changelog: Add entry for CMake
92098d84cf7 changelog: Add entry for CMake
df323b5c146 Merge bitcoin-core/secp256k1#1113: build: Add CMake-based build system
e1eb33724c2 ci: Add "x86_64: Windows (VS 2022)" task
10602b0030e cmake: Export config files
5468d709644 build: Add CMake-based build system
6048e6c03e4 Merge bitcoin-core/secp256k1#1222: Remove redundant checks.
eb8749fcd0f Merge bitcoin-core/secp256k1#1221: Update Changelog
5d8f53e3129 Remove redudent checks.
9d1b458d5fb Merge bitcoin-core/secp256k1#1217: Add secp256k1_fe_add_int function
d232112fa7e Update Changelog
8962fc95bb0 Merge bitcoin-core/secp256k1#1218: Update overflow check
2ef1c9b3870 Update overflow check
57573187826 Merge bitcoin-core/secp256k1#1212: Prevent dead-store elimination when clearing secrets in examples
b081f7e4cbf Add secp256k1_fe_add_int function
5660c137552 prevent optimization in algorithms
09b1d466db7 Merge bitcoin-core/secp256k1#979: Native jacobi symbol algorithm
ce3cfc78a60 doc: Describe Jacobi calculation in safegcd_implementation.md
6be01036c8a Add secp256k1_fe_is_square_var function
1de2a01c2b2 Native jacobi symbol algorithm
04c6c1b1816 Make secp256k1_modinv64_det_check_pow2 support abs val
5fffb2c7af5 Make secp256k1_i128_check_pow2 support -(2^n)
cbd25559343 Merge bitcoin-core/secp256k1#1209: build: Add SECP256K1_API_VAR to fix importing variables from DLLs
1b21aa51752 Merge bitcoin-core/secp256k1#1078: group: Save a normalize_to_zero in gej_add_ge
e4330341bd6 ci: Shutdown wineserver whenever CI script exits
9a5a611a21f build: Suppress stupid MSVC linker warning
739c53b19a2 examples: Extend sig examples by call that uses static context
914276e4d27 build: Add SECP256K1_API_VAR to fix importing variables from DLLs
1cca7c1744b Merge bitcoin-core/secp256k1#1206: build: Add -Wreserved-identifier supported by clang
8c7e0fc1de0 build: Add -Wreserved-identifier supported by clang
8ebe5c52050 Merge bitcoin-core/secp256k1#1201: ci: Do not set git's `user.{email,name}` config options
5596ec5c2cf Merge bitcoin-core/secp256k1#1203: Do not link `bench` and `ctime_tests` to `COMMON_LIB`
ef39721ccce Do not link `bench` and `ctime_tests` to `COMMON_LIB`
9b60e3148d8 ci: Do not set git's `user.{email,name}` config options
e1817a6f54f Merge bitcoin-core/secp256k1#1199: ci: Minor improvements inspired by Bitcoin Core
1bff2005885 Merge bitcoin-core/secp256k1#1200: Drop no longer used Autoheader macros
9b7d18669dc Drop no longer used Autoheader macros
c2415866c7a ci: Don't fetch git history
0ecf3188515 ci: Use remote pull/merge ref instead of local git merge
2b77240b3ba Merge bitcoin-core/secp256k1#1172: benchmarks: fix bench_scalar_split
eb6bebaee39 scalar: restrict split_lambda args, improve doc and VERIFY_CHECKs
7f49aa7f2dc ci: add test job with -DVERIFY
620ba3d74be benchmarks: fix bench_scalar_split
5fbff5d348f Merge bitcoin-core/secp256k1#1170: contexts: Forbid destroying, cloning and randomizing the static context
233822d849d Merge bitcoin-core/secp256k1#1195: ctime_tests: improve output when CHECKMEM_RUNNING is not defined
ad7433b1409 Merge bitcoin-core/secp256k1#1196: Drop no longer used variables from the build system
e39d954f118 tests: Add CHECK_ILLEGAL(_VOID) macros and use in static ctx tests
2cd4e3c0a97 Drop no longer used `SECP_{LIBS,INCLUDE}` variables
613626f94c7 Drop no longer used `SECP_TEST_{LIBS,INCLUDE}` variables
61841fc9ee5 contexts: Forbid randomizing secp256k1_context_static
4b6df5e33e1 contexts: Forbid cloning/destroying secp256k1_context_static
b1579cf5fb4 Merge bitcoin-core/secp256k1#1194: Ensure safety of ctz_debruijn implementation.
8f51229e034 ctime_tests: improve output when CHECKMEM_RUNNING is not defined
d6ff738d5bb Ensure safety of ctz_debruijn implementation.
a01a7d86dc2 Merge bitcoin-core/secp256k1#1192: Switch to exhaustive groups with small B coefficient
a7a7bfaf3dc Merge bitcoin-core/secp256k1#1190: Make all non-API functions (except main) static
f29a3270923 Merge bitcoin-core/secp256k1#1169: Add support for msan instead of valgrind (for memcheck and ctime test)
ff8edf89e2e Merge bitcoin-core/secp256k1#1193: Add `noverify_tests` to `.gitignore`
ce60785b265 Introduce SECP256K1_B macro for curve b coefficient
4934aa79958 Switch to exhaustive groups with small B coefficient
d4a6b58df74 Add `noverify_tests` to `.gitignore`
88e80722d2a Merge bitcoin-core/secp256k1#1160: Makefile: add `-I$(top_srcdir)/{include,src}` to `CPPFLAGS` for precomputed
0f088ec1126 Rename CTIMETEST -> CTIMETESTS
74b026f05d5 Add runtime checking for DECLASSIFY flag
5e2e6fcfc0e Run ctime test in Linux MSan CI job
18974061a3f Make ctime tests building configurable
5048be17e93 Rename valgrind_ctime_test -> ctime_tests
6eed6c18ded Update error messages to suggest msan as well
8e11f89a685 Add support for msan integration to checkmem.h
8dc64079eb1 Add compile-time error to valgrind_ctime_test
0db05a770eb Abstract interactions with valgrind behind new checkmem.h
4f1a54e41d8 Move valgrind CPPFLAGS into SECP_CONFIG_DEFINES
cc3b8a4f404 Merge bitcoin-core/secp256k1#1187: refactor: Rename global variables in tests
9a93f48f502 refactor: Rename STTC to STATIC_CTX in tests
3385a2648d7 refactor: Rename global variables to uppercase in tests
e03ef865593 Make all non-API functions (except main) static
cbe41ac138b Merge bitcoin-core/secp256k1#1188: tests: Add noverify_tests which is like tests but without VERIFY
203760023c6 tests: Add noverify_tests which is like tests but without VERIFY
e862c4af0c5 Makefile: add -I$(top_srcdir)/src to CPPFLAGS for precomputed
0eb3000417f Merge bitcoin-core/secp256k1#1186: tests: Tidy context tests
39e8f0e3d7b refactor: Separate run_context_tests into static vs proper contexts
a4a09379b1a tests: Clean up and improve run_context_tests() further
fc90bb56956 refactor: Tidy up main()
f32a36f620e tests: Don't use global context for context tests
ce4f936c4fa tests: Tidy run_context_tests() by extracting functions
18e0db30cb4 tests: Don't recreate global context in scratch space test
b19806122e9 tests: Use global copy of secp256k1_context_static instead of clone
2a39ac162e0 Merge bitcoin-core/secp256k1#1185: Drop `SECP_CONFIG_DEFINES` from examples
2f9ca284e2a Drop `SECP_CONFIG_DEFINES` from examples
31ed5386e84 Merge bitcoin-core/secp256k1#1183: Bugfix: pass SECP_CONFIG_DEFINES to bench compilation
c0a555b2ae3 Bugfix: pass SECP_CONFIG_DEFINES to bench compilation
01b819a8c7d Merge bitcoin-core/secp256k1#1158: Add a secp256k1_i128_to_u64 function.
eacad90f699 Merge bitcoin-core/secp256k1#1171: Change ARG_CHECK_NO_RETURN to ARG_CHECK_VOID which returns (void)
3f57b9f7749 Merge bitcoin-core/secp256k1#1177: Some improvements to the changelog
c30b889f17e Clarify that the ABI-incompatible versions are earlier
881fc33d0c1 Consistency in naming of modules
665ba77e793 Merge bitcoin-core/secp256k1#1178: Drop `src/libsecp256k1-config.h`
75d7b7f5bae Merge bitcoin-core/secp256k1#1154: ci: set -u in cirrus.sh to treat unset variables as an error
7a746882013 ci: add missing CFLAGS & CPPFLAGS variable to print_environment
c2e0fdadebd ci: set -u in cirrus.sh to treat unset variables as an error
9c5a4d21bbe Do not define unused `HAVE_VALGRIND` macro
ad8647f548c Drop no longer relevant files from `.gitignore`
b627ba7050b Remove dependency on `src/libsecp256k1-config.h`
9ecf8149a19 Reduce font size in changelog
2dc133a67ff Add more changelog entries
ac233e181a5 Add links to diffs to changelog
cee8223ef6d Mention semantic versioning in changelog
9a8d65f07f1 Merge bitcoin-core/secp256k1#1174: release cleanup: bump version after 0.2.0
02ebc290f74 release cleanup: bump version after 0.2.0
b6b360efafc doc: improve message of cleanup commit
a49e0940ad6 docs: Fix typo
2551cdac903 tests: Fix code formatting
c635c1bfd54 Change ARG_CHECK_NO_RETURN to ARG_CHECK_VOID which returns (void)
cf66f2357c6 refactor: Add helper function secp256k1_context_is_proper()
d2164752053 test secp256k1_i128_to_i64
4bc429019dc Add a secp256k1_i128_to_u64 function.
e089eecc1e5 group: Further simply gej_add_ge
ac71020ebe0 group: Save a normalize_to_zero in gej_add_ge
git-subtree-dir: src/secp256k1
git-subtree-split: bdf39000b9c6a0818e7149ccb500873d079e6e85
b082f28101 rpc, wallet: use the same `next_index` in listdescriptors and importdescriptors (w0xlt)
Pull request description:
Currently `listdescriptors` RPC uses `next` key to represent `WalletDescriptor::next_index` while `importdescriptors` uses `next_index`. This creates two different descriptor formats.
This PR changes `listdescriptors` to use the same key as `importdescriptors`.
ACKs for top commit:
achow101:
ACK b082f28101
aureleoules:
reACK b082f28101
Tree-SHA512: c29ec59051878e614d749ed6dc85e5c14ad00db0e8fcbce3f5066d1aae85ef07ca70f02920299e48d191b7387024fe224b0054c4191a5951cb805106f7b8e37b
3e947d7117 doc: remove 'omitted...' doc for rpc getrawtransaction when verbose is 2 (dougEfish)
Pull request description:
Remove optional rpc doc for getrawtransaction when verbose is 2
ACKs for top commit:
stickies-v:
ACK 3e947d7117
Tree-SHA512: b9e970d6ef4a47ec7ca32f5ff1028cc901f1bfdc1571668208505d42f4160733530601b78e469de82a854d3b298a55a81d0a7916bc5db4a43ad6d6a299c55c9e
fa1b4e5c32 Use steady clock in FlushStateToDisk (MarcoFalke)
1111e2f8b4 Use steady clock in SeedStrengthen and FindBestImplementation (MarcoFalke)
Pull request description:
There may be a theoretical deadlock for the duration of the offset when the system clock is adjusted into a past time while executing `SeedStrengthen`.
Fix this by using steady clock.
Do the same in `FindBestImplementation`, which shouldn't be affected, because it discards outlier measurements. However, doing the same there for consistency seems fine.
Do the same in `FlushStateToDisk`, which should make the flushes more steady, if the system clock is adjusted by a large offset.
ACKs for top commit:
john-moffett:
ACK fa1b4e5c32
willcl-ark:
ACK fa1b4e5c3
Tree-SHA512: cc625e796b186accd53222bd64eb57d0512bc7e588312d254349b542bbc5e5daac348ff2b3b3f7dc5ae0bbbae2ec11fdbf3022cf2164211633765a4b0108e83e
2b373fe49d docs: update assumeutxo.md (James O'Beirne)
87a1108c81 test: add snapshot completion unittests (James O'Beirne)
d70919a88f refactor: make MempoolMutex() public (James O'Beirne)
7300ced9de log: add LoadBlockIndex() message for assumedvalid blocks (James O'Beirne)
d96c59cc5c validation: add ChainMan logic for completing UTXO snapshot validation (James O'Beirne)
f2a4f3376f move-only-ish: init: factor out chainstate initialization (James O'Beirne)
637a90b973 add Chainstate::HasCoinsViews() (James O'Beirne)
c29f26b47b validation: add CChainState::m_disabled and ChainMan::isUsable (James O'Beirne)
5ee22cdafd add ChainstateManager.GetSnapshot{BaseHeight,BaseBlock}() (James O'Beirne)
Pull request description:
This is part of the [assumeutxo project](https://github.com/bitcoin/bitcoin/projects/11) (parent PR: https://github.com/bitcoin/bitcoin/pull/15606)
Part two of replacing https://github.com/bitcoin/bitcoin/pull/24232.
---
When a user activates a snapshot, the serialized UTXO set data is used to create an "assumed-valid" chainstate, which becomes active in an attempt to get the node to network tip as quickly as possible. Simultaneously in the background, the already-existing chainstate continues "conventional" IBD to both accumulate full block data and serve as a belt-and-suspenders to validate the assumed-valid chainstate.
Once the background chainstate's tip reaches the base block of the snapshot used, we set `m_stop_use` on that chainstate and immediately take the hash of its UTXO set; we verify that this matches the assumeutxo value in the source code. Note that while we ultimately want to remove this background chainstate, we don't do so until the following initialization process, when we again check the UTXO set hash of the background chainstate, and if it continues to match, we remove the (now unnecessary) background chainstate, and move the (previously) assumed-valid chainstate into its place. We then reinitialize the chainstate in the normal way.
As noted in previous comments, we could do the filesystem operations "inline" immediately when the background validation completes, but that's basically just an optimization that saves disk space until the next restart. It didn't strike me as worth the risk of moving chainstate data around on disk during runtime of the node, though maybe my concerns are overblown.
The final result of this completion process is a fully-validated chain, where the only evidence that the user synced using assumeutxo is the existence of a `base_blockhash` file in the `chainstate` directory.
ACKs for top commit:
achow101:
ACK 2b373fe49d
Tree-SHA512: a204e1d6e6932dd83c799af3606b01a9faf893f04e9ee1a36d63f2f1ccfa9118bdc1c107d86976aa0312814267e6a42074bf3e2bf1dead4b2513efc6d955e13d
Also adjusts the previous snapshot chainstate init tests
to account for the fact that the init process is now attempting to
validate and complete background chainstates whose tip is at the
snapshot base block. We use a DisconnectTip() hack to preserve the
nature of the test.
Trigger completion when a background validation chainstate reaches the
same height as a UTXO snapshot, and handle cleaning up the chainstate
on subsequent startup.
802cc1ef53 Deduplicate bitcoind and bitcoin-qt init code (Ryan Ofsky)
d172b5c671 Add InitError(error, details) overload (Ryan Ofsky)
3db2874bd7 Extend bilingual_str support for tinyformat (Ryan Ofsky)
c361df90b9 scripted-diff: Remove double newlines after some init errors (Ryan Ofsky)
Pull request description:
Add common InitConfig function to deduplicate bitcoind and bitcoin-qt code reading config files and creating the datadir.
Noticed the duplicate code while reviewing #27073 and want to remove it because difference in bitcoin-qt and bitcoind behavior make it hard to evaluate changes like #27073
There are a few minor changes in behavior:
- In bitcoin-qt, when there is a problem reading the configuration file, the GUI error text has changed from "Error: Cannot parse configuration file:" to "Error reading configuration file:" to be consistent with bitcoind.
- In bitcoind, when there is a problem reading the settings.json file, the error text has changed from "Failed loading settings file" to "Settings file could not be read" to be consistent with bitcoin-qt.
- In bitcoind, when there is a problem writing the settings.json file, the error text has changed from "Failed saving settings file" to "Settings file could not be written" to be consistent with bitcoin-qt.
- In bitcoin-qt, if there datadir is not accessible (e.g. no permission to read), there is an normal error dialog showing "Error: filesystem error: status: Permission denied [.../settings.json]", instead of an uncaught exception.
ACKs for top commit:
Sjors:
Light review ACK 802cc1ef53
TheCharlatan:
ACK 802cc1ef53
achow101:
ACK 802cc1ef53
Tree-SHA512: 9c78d277e9ed595fa8ce286b97d2806e1ec06ddbbe7bd3434bd9dd7b456faf8d989f71231e97311f36edb9caaec645a50c730bd7514b8e0fe6e6f7741b13d981
and clarify the intention behind the -nodebuglogfile bench.
Co-authored-by: "kouloumos <kouloumosa@gmail.com>"
Co-authored-by: "Larry Ruane <larryruane@gmail.com>"
fa8481b05f util: Work around ParseHex gcc cross compiler bug (MarcoFalke)
Pull request description:
I fail to see how an explicit `ParseHex` template instantiation fails to also instantiate `TryParseHex`.
Nonetheless, to work around a compiler bug, change the explicit instantiation from `ParseHex` to `TryParseHex`. (`ParseHex` is inline anyway and will be instantiated by the compiler either way).
Fixes https://github.com/bitcoin/bitcoin/pull/25227#issuecomment-1456009757 :
```
CXXLD bitcoind
/usr/lib/gcc-cross/powerpc64le-linux-gnu/11/../../../../powerpc64le-linux-gnu/bin/ld: libbitcoin_node.a(libbitcoin_node_a-net_processing.o): in function `(anonymous namespace)::PeerManagerImpl::ProcessMessage(CNode&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, CDataStream&, std::chrono::duration<long, std::ratio<1l, 1000000l> >, std::atomic<bool> const&)':
net_processing.cpp:(.text+0x29660): undefined reference to `std::optional<std::vector<unsigned char, std::allocator<unsigned char> > > TryParseHex<unsigned char>(std::basic_string_view<char, std::char_traits<char> >)'
/usr/lib/gcc-cross/powerpc64le-linux-gnu/11/../../../../powerpc64le-linux-gnu/bin/ld: libbitcoin_node.a(libbitcoin_node_a-rest.o): in function `rest_getutxos(std::any const&, HTTPRequest*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)':
rest.cpp:(.text+0x83b4): undefined reference to `std::optional<std::vector<unsigned char, std::allocator<unsigned char> > > TryParseHex<unsigned char>(std::basic_string_view<char, std::char_traits<char> >)'
/usr/lib/gcc-cross/powerpc64le-linux-gnu/11/../../../../powerpc64le-linux-gnu/bin/ld: libbitcoin_node.a(libbitcoin_node_a-torcontrol.o): in function `std::vector<unsigned char, std::allocator<unsigned char> > ParseHex<unsigned char>(std::basic_string_view<char, std::char_traits<char> >)':
torcontrol.cpp:(.text._Z8ParseHexIhESt6vectorIT_SaIS1_EESt17basic_string_viewIcSt11char_traitsIcEE[_Z8ParseHexIhESt6vectorIT_SaIS1_EESt17basic_string_viewIcSt11char_traitsIcEE]+0x2c): undefined reference to `std::optional<std::vector<unsigned char, std::allocator<unsigned char> > > TryParseHex<unsigned char>(std::basic_string_view<char, std::char_traits<char> >)'
/usr/lib/gcc-cross/powerpc64le-linux-gnu/11/../../../../powerpc64le-linux-gnu/bin/ld: libbitcoin_common.a(libbitcoin_common_a-external_signer.o): in function `ExternalSigner::SignTransaction(PartiallySignedTransaction&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >&)':
external_signer.cpp:(.text+0x8d84): undefined reference to `std::optional<std::vector<unsigned char, std::allocator<unsigned char> > > TryParseHex<unsigned char>(std::basic_string_view<char, std::char_traits<char> >)'
collect2: error: ld returned 1 exit status
ACKs for top commit:
gruve-p:
ACK fa8481b05f
hebasto:
ACK fa8481b05f, tested on Ubuntu 22.04, gcc 11.3 for the `riscv64-linux-gnu` host.,
Tree-SHA512: 53efa424e7e18d85a2c9ac2267b9370ae3594d9be73da5135a3a79bf07ab50fcc5357cbde09dc0b2a9eb78d78ec37beae0c9f876333b568e678b9d0067bc9e4e
60978c8080 test: Reduce extended timeout on abortnode test (Fabian Jahr)
660bdbf785 http: Release server before waiting for event base loop exit (João Barbosa)
8c6d007c80 http: Track active requests and wait for last to finish (João Barbosa)
Pull request description:
This revives #19420. Since promag is not so active at the moment, I can support this to finally get it merged.
The PR is rebased and comments by jonatack have been addressed.
Once this is merged, I will also reopen#19434.
ACKs for top commit:
achow101:
ACK 60978c8080
stickies-v:
re-ACK [60978c8](60978c8080)
hebasto:
ACK 60978c8080
Tree-SHA512: eef0fe1081e9331b95cfafc71d82f2398abd1d3439dac5b2fa5c6d9c0a3f63ef19adde1c38c88d3b4e7fb41ce7c097943f1815c10e33d165918ccbdec512fe1c
6a302d40df wallet: single output groups filtering and grouping process (furszy)
bd91ed1cb2 wallet: unify outputs grouping process (furszy)
55962001da test: coinselector_tests refactor, use CoinsResult instead of plain std::vector (furszy)
34f54a0a3a wallet: decouple outputs grouping process from each ChooseSelectionResult (furszy)
461f0821a2 refactor: make OutputGroup::m_outputs field a vector of shared_ptr (furszy)
d8e749bb84 test: wallet, add coverage for outputs grouping process (furszy)
06ec8f9928 wallet: make OutputGroup "positive_only" filter explicit (furszy)
Pull request description:
The idea originates from https://github.com/bitcoin/bitcoin/pull/24845#issuecomment-1130310321.
Note:
For clarity, it's recommended to start reviewing from the end result to understand the structure of the flow.
#### GroupOutputs function rationale:
If "Avoid Partial Spends" is enabled, the function gathers outputs with the same script together inside a container. So Coin Selection can treats them as if them were just one possible input and either select them all or not select them.
#### How the Inputs Fetch + Selection process roughly works:
```
1. Fetch user’s manually selected inputs.
2. Fetch wallet available coins (walks through the entire wallet txes map) and insert them into a set of vectors (each vector store outputs from a single type).
3. Coin Selection Process:
Call `AttemptSelection` 8 times. Each of them expands the coin eligibility filter (accepting a larger subset of coins in the calculation) until it founds a solutions or completely fails if no solutions gets founds after the 8 rounds.
Each `AttemptSelection` call performs the following actions:
- For each output type supported by the wallet (P2SH, P2PK, P2WPKH, P2WSH and a combination of all of them):
Call ‘ChooseSelectionResult’ providing the respective, filtered by type, coins vector. Which:
I. Groups the outputs vector twice (one for positive only and a second one who includes the negative ones as well).
- GroupOutputs walks-through the entire inputted coins vector one time at least, + more if we are avoiding partial spends, to generate a vector of OutputGroups.
II. Then performs every coin selection algorithm using the recently created vector of OutputGroup: (1) BnB, (2) knapsack and (3) SRD.
III. Then returns the best solution out of them.
```
We perform the general operation of gathering outputs, with the same script, into a single container inside:
Each coins selection attempt (8 times —> each coin eligibility filter), for each of the outputs vector who were filtered by type (plus another one joining all the outputs as well if needed), twice (one for the positive only outputs effective value and a second one for all of them).
So, in the worst case scenario where no solution is found after the 8 Coin Selection attempts, the `GroupOutputs` function is called 80 times (8 * 5 * 2).
#### Improvements:
This proposal streamlines the process so that the output groups, filtered by coin eligibility and type, are created in a single loop outside of the Coin Selection Process.
The new process is as follows:
```
1. Fetch user’s manually selected inputs.
2. Fetch wallet available coins.
3. Group outputs by each coin eligibility filter and each different output type found.
4. Coin Selection Process:
Call AttemptSelection 8 times. Each of them expands the coin eligibility filter (accepting different output groups) until it founds a solutions or completely fails if no solutions gets founds after the 8 rounds.
Each ‘AttemptSelection’ call performs the following actions:
- For each output type supported by the wallet (P2SH, P2PK, P2WPKH, P2WSH and all of them):
A. Call ‘ChooseSelectionResult’ providing the respective, filtered by type, output group. Which:
I. Performs every coin selection algorithm using the provided vector of OutputGroup: (1) BnB, (2) knapsack and (3) SRD.
II. Then returns the best solution out of them.
```
Extra Note:
The next steps after this PR will be to:
1) Merge `AvailableCoins` and `GroupOutputs` processes.
2) Skip entire coin selection rounds if no new coins are added into the subsequent round.
3) Remove global feerates from the OutputGroup class.
4) Remove secondary "grouped" tx creation from `CreateTransactionInternal` by running Coin Selection results over the aps grouped outputs vs non-aps ones.
ACKs for top commit:
S3RK:
ReACK 6a302d4
achow101:
ACK 6a302d40df
theStack:
re-ACK 6a302d40df🥥
Tree-SHA512: dff849063be328e7d9c358ec80239a6db2cd6131963b511b83699b95b337d3106263507eaba0119eaac63e6ac21c6c42d187ae23d79d9220b90c323d44b01d24
The global logging object instance is not re-created for each run, so when
multiple logging benchmarks are run, each one after the first one still has
the logging categories enabled from the previous ones. This commit disables
all categories at the start of each benchmark.
e4ede64fe8 Expand scantxoutset help text to cover tr() and miniscript (Greg Sanders)
Pull request description:
ACKs for top commit:
achow101:
ACK e4ede64fe8
darosior:
webACK e4ede64fe8
Tree-SHA512: 6b5d9e7fccc8242f4534861c1b438ec40fb03fbf5968c5a4af5ddbced73df6d666812053c2e12a1e0a34057f6f4fe11987c8c59d8cdfa48ca7ab7d2720b51ef9
4163093d63 wallet: use Mutex for g_sqlite_mutex instead of GlobalMutex (Vasil Dimov)
Pull request description:
Using `Mutex` provides stronger guarantee than `GlobalMutex` wrt Clang's
thread safety analysis. Thus it is better to reduce the usage of
`GlobalMutex` in favor of `Mutex`.
Using `Mutex` for `g_sqlite_mutex` is ok because its usage is limited in
`wallet/sqlite.cpp` and it does not require propagating the negative
annotations to not relevant code.
ACKs for top commit:
achow101:
ACK 4163093d63
hebasto:
re-ACK 4163093d63
TheCharlatan:
ACK 4163093d63
Tree-SHA512: 4913bcb8437ecf0e6b6cb781d02a6d24ffb4bf3e2e1899fa60785eab41c4c65dbdd9600bcb696290c873661b873ad61e5a4c4f205b7e66fdef2ae17c676cd12f
The previous code visualized the `CSubNet` object as string, then
parsed that string back to `CSubNet`. This is sub-optimal given that
the original `CSubNet` object can be used directly instead.
This avoids calling `LookupSubNet()` from the GUI.
Co-authored-by: furszy <matiasfurszyfer@protonmail.com>
Optimizes coin selection by performing the "group outputs"
procedure only once, outside the "attempt selection" process.
Avoiding the repeated execution of the 'GroupOutputs' operation
that occurs on each coin eligibility filters (up to 8 of them);
then for every coin vector type plus one for all the coins together.
This also let us not perform coin selection over coin eligibility
filtered groups that don't add new elements.
(because, if the previous round failed, and the subsequent one has
the same coins, then this new round will fail again).
The 'GroupOutputs()' function performs the same
calculations for only-positive and mixed groups,
the only difference is that when we look for
only-positive groups, we discard negative utxos.
So, instead of wasting resources calling GroupOutputs()
for positive-only first, then call it again to include
the negative ones in the result, we can execute
GroupOutputs() only once, including in the response
both group types (positive-only and mixed).
The following scenarios are covered:
1) 10 UTXO with the same script:
partial spends is enabled --> outputs must not be grouped.
2) 10 UTXO with the same script:
partial spends disabled --> outputs must be grouped.
3) 20 UTXO, 10 one from scriptA + 10 from scriptB:
a) if partial spends is enabled --> outputs must not be grouped.
b) if partial spends is not enabled --> 2 output groups expected (one per script).
3) Try to add a negative output (value - fee < 0):
a) if "positive_only" is enabled --> negative output must be skipped.
b) if "positive_only" is disabled --> negative output must be added.
4) Try to add a non-eligible UTXO (due not fulfilling the min depth target for
"not mine" UTXOs) --> it must not be added to any group
5) Try to add a non-eligible UTXO (due not fulfilling the min depth target for
"mine" UTXOs) --> it must not be added to any group
6) Surpass the 'OUTPUT_GROUP_MAX_ENTRIES' size and verify that a second partial
group gets created.
Extract the logic that decides whether the new or the tried table is going to
be searched to the beginning of the function.
Co-authored-by: Martin Zumsande <mzumsande@gmail.com>
8847ce44e0 util: add missing include and fix function signature (Cory Fields)
Pull request description:
ping hebasto
Discovered while testing pre-compiled header support with CMake: https://github.com/theuni/bitcoin/commits/cmake-pch-poc. Compilation of that branch fails without this fix and succeeds with it.
Similar to the fix in #27144.
The problem of having a default argument in the definition was masked by the missing include. Using PCH forces that include, so we end up with the compiler error we should've been getting all along.
ACKs for top commit:
fanquake:
ACK 8847ce44e0
Tree-SHA512: 5eb9a6691ee37cbc5033a48aedcbf5c93af3b234614ae14c3fcc858f1ee505f630ad68f8bbb69ffa280080c8d0f91451362cb3819290b741ce906b2b3224a622
And not hide it inside the `OutputGroup::Insert` method.
This method does not return anything if insertion fails.
We can know before calling `Insert` whether the coin
will be accepted or not.
545ff924ab refactor: use string_view for RPC named argument values (stickies-v)
7727603e44 refactor: reduce unnecessary complexity in ParseNonRFCJSONValue (stickies-v)
1d02e59901 test: add cases to JSON parsing (stickies-v)
Pull request description:
Inspired by MarcoFalke's [comment](https://github.com/bitcoin/bitcoin/pull/26506#discussion_r1036149426). Main purpose of this PR is to minimize copying (potentially large) RPC named arguments when calling `.substr()` by using `std::string_view` instead of `std::string`. Furthermore, cleans up the code by removing unnecessary complexity in `ParseNonRFCJSONValue()` (done first to avoid refactoring required to concatenate `string` and `string_view`), updates some naming and adds a few test cases. Should not introduce any behaviour change.
## Questions
- ~Was there actually any merit to `ParseNonRFCJSONValue()` surrounding the value with brackets and then parsing it as an array? I don't see it, and the new approach doesn't fail any tests. Still a bit suspicious about it though.~
- Cleared up by https://github.com/bitcoin/bitcoin/pull/26506#pullrequestreview-1211984059
- If there are no objections to 7727603e44, I think we should follow up with a PR to rename `ParseNonRFCJSONValue()` to a local `Parse()` helper function (that throws if invalid), remove it from `client.h` and merge the test coverage we currently have on `ParseNonRFCJSONValue()` with the coverage we have on `UniValue::read()`.
ACKs for top commit:
ryanofsky:
Code review ACK 545ff924ab
MarcoFalke:
review ACK 545ff924ab📻
Tree-SHA512: b1c89fb010ac9c3054b023cac1acbba2a539a09cf39a7baffbd7f7571ee268d5a6d98701c7ac10d68a814526e8fd0fe96ac1d1fb072f272033e415b753f64a5c
56e37e71a2 Make miniscript fuzzers avoid script size limit (Pieter Wuille)
bcec5ab4ff Make miniscript fuzzers avoid ops limit (Pieter Wuille)
213fffa513 Enforce type consistency in miniscript_stable fuzz test (Pieter Wuille)
e1f30414c6 Simplify miniscript fuzzer NodeInfo struct (Pieter Wuille)
5abb0f5ac3 Do base type propagation in miniscript_stable fuzzer (Pieter Wuille)
Pull request description:
This adds a number of improvements to the miniscript fuzzers that all amount to rejecting invalid or overly big miniscripts early on:
* Base type propagation in the miniscript_stable fuzzers prevents constructing a large portion of miniscripts that would be illegal, with just a little bit of type logic in the fuzzer. The fuzzer input format is unchanged.
* Ops and script size tracking in GenNode means that too-large scripts (either due to script size limit or ops limit) will be detected on the fly during fuzz input processing, before actually constructing the scripts.
Closes#27147.
ACKs for top commit:
darosior:
re-ACK 56e37e71a2
dergoegge:
tACK 56e37e71a2
Tree-SHA512: 245584adf9a6644a35fe103bc81b619e5b4f5d467571a761b5809d08b1dec48f7ceaf4d8791ccd8208b45c6b309d2ccca23b3d1ec5399df76cd5bf88f2263280
Add common InitConfig function to deduplicate bitcoind and bitcoin-qt code
reading config files and creating the datadir.
There are a few minor changes in behavior:
- In bitcoin-qt, when there is a problem reading the configuration file, the
GUI error text has changed from "Error: Cannot parse configuration file:" to
"Error reading configuration file:" to be consistent with bitcoind.
- In bitcoind, when there is a problem reading the settings.json file, the
error text has changed from "Failed loading settings file" to "Settings
file could not be read" to be consistent with bitcoin-qt.
- In bitcoind, when there is a problem writing the settings.json file, the
error text has changed from "Failed saving settings file" to "Settings file
could not be written" to be consistent with bitcoin-qt.
- In bitcoin-qt, if there datadir is not accessible (e.g. no permission to read),
there is an normal error dialog showing "Error: filesystem error: status:
Permission denied [.../settings.json]", instead of an uncaught exception
This is only used in the current PR to avoid ugly
`strprintf(Untranslated("%s:\n%s"), str, MakeUnorderedList(details)`
boilerplate in init code. But in the future the function could be extended and
more widely used to include more details in GUI error messages or display them
in a more readable way, see code comment.
Previous bilingual_str tinyformat::format accepted bilingual format strings,
but not bilingual arguments. Extend it to accept both. This is useful when
embedding one translated string inside another translated string, for example:
`strprintf(_("Error: %s"), message)` which would fail previously if `message`
was a bilingual_str.
Some InitError calls had trailing \n characters, causing double newlines in
error output. After this change InitError calls consistently output one newline
instead of two. Appearance of messages in the GUI does not seem to be affected.
Can be tested with:
src/bitcoind -regtest -datadir=noexist
src/qt/bitcoin-qt -regtest -datadir=noexist
-BEGIN VERIFY SCRIPT-
git grep -l InitError src/ | xargs sed -i 's/\(InitError(.*\)\\n"/\1"/'
-END VERIFY SCRIPT-
75db62ba4c refactor: Move calculation logic out from `CheckSequenceLocksAtTip()` (Hennadii Stepanov)
3bc434f459 refactor: Add `CalculateLockPointsAtTip()` function (Hennadii Stepanov)
Pull request description:
This PR is follow up for bitcoin/bitcoin#22677 and bitcoin/bitcoin#23683.
On master (013daed9ac) it is not obvious that `CheckSequenceLocksAtTip()` function can modify its `LockPoints* lp` parameter which leads to https://github.com/bitcoin/bitcoin/pull/22677#discussion_r762040101.
This PR:
- separates the lockpoint calculate logic from `CheckSequenceLocksAtTip()` function into a new `CalculateLockPointsAtTip()` one
- cleans up the `CheckSequenceLocksAtTip()` function interface
- makes code easier to reason about (hopefully)
ACKs for top commit:
achow101:
ACK 75db62ba4c
stickies-v:
re-ACK 75db62b
Tree-SHA512: 072c3fd9cd1e1b0e0bfc8960a67b01c80a9f16d6778f374b6944ade03a020415ce8b8ab2593b0f5e787059c8cf90af798290b4c826785d41955092f6e12e7486
9a9d5da11f refactor: Stop using gArgs global in system.cpp (Ryan Ofsky)
b20b34f5b3 refactor: Use new GetConfigFilePath function (Ryan Ofsky)
Pull request description:
Most of the code in `util/system.cpp` that was hardcoded to use the global `ArgsManager` instance `gArgs` has been changed to stop using it (for example in https://github.com/bitcoin/bitcoin/pull/20092). But a few hardcoded references to `gArgs` remain. This commit removes the last ones so these functions aren't reading or writing global state.
Noticed these `gArgs` references while reviewing #27073
ACKs for top commit:
achow101:
ACK 9a9d5da11f
stickies-v:
ACK 9a9d5da11
willcl-ark:
tACK 9a9d5da11
Tree-SHA512: 2c74b0d5fc83e9ed2ec6562eb26ec735512f75db8876a11a5d5f04e6cdbe0cd8beec19894091aa2cbf29319194d2429ccbf8036f5520ecc394f6fe89a0079a7b
fb0dbe9423 docs: GetDataDirNet and GetDataDirBase don't create datadir (stickies-v)
Pull request description:
Since #27073, the behaviour of `GetDataDir()` [changed](https://github.com/bitcoin/bitcoin/pull/27073/files#diff-19427b0dd1a791adc728c82e88f267751ba4f1c751e19262cac03cccd2822216L435-L443) to only return the datadir path, but not create it if non-existent. This also changed the behaviour of `GetDataDirNet()` and `GetDataDirBase()` but the docs do not yet reflect that.
ACKs for top commit:
TheCharlatan:
ACK fb0dbe9423
theStack:
ACK fb0dbe9423
willcl-ark:
ACK fb0dbe942
Tree-SHA512: 3f10f4871df59882f3649c6d3b2362cae2f8a01ad0bd0c636c5608b0d177d279a2e8712930b819d6d3912e91fa6447b9e54507c33d8afe427f7f39002b013bfb
3141eab9c6 test: add functional test for ScanAndUnlinkAlreadyPrunedFiles (Andrew Toth)
e252909e56 test: add unit test for ScanAndUnlinkAlreadyPrunedFiles (Andrew Toth)
77557dda4a prune: scan and unlink already pruned block files on startup (Andrew Toth)
Pull request description:
There are a few cases where we can mark a block and undo file as pruned in our block index, but not actually remove the files from disk.
1. If we call `FindFilesToPrune` or `FindFilesToPruneManual` and crash before `UnlinkPrunedFiles`.
2. If on Windows there is an open file handle to the file somewhere else when calling `fs::remove` in `UnlinkPrunedFiles` (https://en.cppreference.com/w/cpp/filesystem/remove, https://learn.microsoft.com/en-us/windows/win32/api/fileapi/nf-fileapi-deletefilew#remarks). This could be from another process, or if we are calling `ReadBlockFromDisk`/`ReadRawBlockFromDisk` without having a lock on `cs_main` (which has been allowed since ccd8ef65f9).
This PR mitigates this by scanning all pruned block files on startup after `LoadBlockIndexDB` and unlinking them again.
ACKs for top commit:
achow101:
ACK 3141eab9c6
pablomartin4btc:
re-ACK with added functional test 3141eab9c6.
furszy:
Code review ACK 3141eab9
theStack:
Code-review ACK 3141eab9c6
Tree-SHA512: 6c73bc57838ad1b7e5d441af3c4d6bf4c61c4382e2b86485e57fbb74a61240710c0ceeceb8b4834e610ecfa3175c6955c81ea4b2285fee11ca6383f472979d8d
Use the same technique as is using in the FromString miniscript parser to
predict the final script size of the miniscript being generated in the
miniscript_stable and miniscript_smart fuzzers (by counting every unexplored
sub node as 1 script byte, which is possible because every leaf node always
adds at least 1 byte). This allows bailing out early if the script being
generated would exceed the maximum allowed size (before actually constructing
the miniscript, as that may happen only significantly later potentially).
Also add a self-check to make sure this predicted script size matches that
of generated scripts.
Keep track of the total number of ops the constructed script will have
during miniscript_stable and miniscript_smart fuzzers' GenNode, so it
can abort early if the 201 ops limit would be exceeded.
Also add a self-check that the final constructed node has the predicted
ops size limit, so we know the fuzzer's logic for keeping track of this
is correct.
Since we now keep track of all expected child node types (even if rudimentary)
in both miniscript_stable and miniscript_smart fuzzers, there is no need anymore
for the former shortcut NodeInfo constructors without sub types.
Keep track of which base type (B, K, V, or W) is desired in the miniscript_stable
ConsumeStableNode function. This allows aborting early if the constructed node
won't have the right type.
Note that this does not change the fuzzer format; the meaning of inputs in
ConsumeStableNode is unmodified. The only change is that often the fuzzer will
abort early.
The direct motivation is preventing recursing v: wrappers, which are the only
fragment type that does not otherwise increase the overall minimum possible script
size. In a later commit this will be exploited to prevent overly-large scripts from
being constructed.
Since #27073, the behaviour of GetDataDir changed to only return
the datadir path, but not create it. This also changed the behaviour
of GetDataDirNet and GetDataDirBase but the docs do not yet reflect
that.
c5825e14f8 doc: add explanation for fail_on_insufficient_dbcache (Ryan Ofsky)
7dff7da4f5 init: Return more fitting ChainStateLoadStatus if verification was interrupted (Martin Zumsande)
Pull request description:
This addresses two outstanding comments by ryanofsky from #25574:
* return `ChainstateLoadStatus::INTERRUPTED` instead of `ChainstateLoadStatus::SUCCESS` if verification was stopped by an interrupt. This would coincide with straightforward expectation, and it avoids a misleading [log entry](c5825e14f8/src/init.cpp (L1526)) in `init` for the block index load time (because that would include the verificiation, which didn't complete). It shouldn't affect node behavior otherwise because the shutdown signal would be caught in init anyway. In test, this would lead to an assert ([link](c5825e14f8/src/test/util/setup_common.cpp (L230))), which also makes more sense because benign interrupts are not expected there during init.
This can be tested by setting a large value for `-checkblocks`, interrupting the node during block verification and observing the log.
https://github.com/bitcoin/bitcoin/pull/25574#discussion_r1110050930
* add documentation for `require_full_verification` https://github.com/bitcoin/bitcoin/pull/25574#discussion_r1110031541
ACKs for top commit:
MarcoFalke:
thanks lgtm ACK c5825e14f8
Tree-SHA512: ca1c71a1b046d30083337dd9ef6d52e66fa1ac8c4ecd807716e4aa6a894179a81df41caee916fa30997fd6e0b284412a3c8f2919d19c29d826fb580ffb89fd73
Most of the code in util/system.cpp that was hardcoded to use the global
ArgsManager instance `gArgs` has been changed to work with explicit ArgsManager
instances (for example in https://github.com/bitcoin/bitcoin/pull/20092). But a
few hardcoded references to `gArgs` remain. This commit removes the last ones
so these functions aren't reading or writing global state.
807de2cebd wallet: skip R-value grinding for external signers (Sjors Provoost)
72b763e452 wallet: annotate bools in descriptor SPKM FillPSBT() (Sjors Provoost)
Pull request description:
When producing a dummy signature for the purpose of estimating the transaction fee, do not assume an external signer performs R-value grinding on the signature.
In particular, this avoids a scenario where the fee rate is 1 sat / vbyte and a transaction with a 72 byte signature is not accepted into our mempool.
Suggested testing:
1. On master, launch with `-signet` and create an external signer wallet using e.g. a Trezor and HWI, see [guide](https://github.com/bitcoin/bitcoin/blob/master/doc/external-signer.md#example-usage) (with the GUI it should "just work" once you have the HWI path configured).
2. Create a few addresses and fund them from the faucet: https://signet.bc-2.jp/ (wait for confirmation)
3. Create another address, and now send the entire wallet to it, set the fee to 1 sat/byte
4. Most likely this transaction never gets broadcast and you won't see it on the [signet explorer](https://explorer.bc-2.jp)
5. With this PR, try again.
6. Check the explorer and inspect the transaction. Each input witness starts with either `30440220` (R has 32 bytes) or `30440221` (R has 33 bytes). See this explainer for [DER encoding](https://bitcoin.stackexchange.com/questions/92680/what-are-the-der-signature-and-sec-format).
Fixes#26030
ACKs for top commit:
S3RK:
ACK 807de2cebd
achow101:
ACK 807de2cebd
furszy:
ACK 807de2ce
ishaanam:
utACK 807de2cebd
Tree-SHA512: 64f626a3030ef0ab1e43af86d8fba113151512561baf425e6e5182af53df3a64fa9e85c7f67bf4ed15b5ad6e5d5afc7fbba8b6e1f3bad388e48db51cb9446074
5da7c0b3e3 build: allow libitcoinkernel dll builds now that exports are fixed (Cory Fields)
130490aef9 build: always build bitcoin-chainstate against static libbitcoinkernel (Cory Fields)
545a74ef32 build: fix bitcoin-chainstate when libbitcoinkernel is static (Cory Fields)
9c253d2398 build: don't define DLL_EXPORT for windows (Cory Fields)
Pull request description:
Fixes#25008.
Fixes#19772.
1. Fixup the build defines so that exports are clean.
2. Work around a libtool issue wrt dependency calculation
3. Simplify everything by only ever building in-tree bitcoin-chainstate against a static libbitcoinkernel
4. Remove Windows-only hack that disabled dll creation
ACKs for top commit:
TheCharlatan:
ACK 5da7c0b3e3
Tree-SHA512: 61bab457e13842946387240da703d313509af30d4ca3371a19a26a5ef1716e4d7107b09567323041b549ab1fc97a064aa1d6992406936ab9c491a616bc7f4e7f
faab273e06 util: Return empty vector on invalid hex encoding (MarcoFalke)
fa3549a77b test: Add hex parse unit tests (MarcoFalke)
Pull request description:
Seems a bit confusing to happily accept random bytes and pretend they are hex encoded strings.
ACKs for top commit:
stickies-v:
re-ACK faab273e06
Tree-SHA512: a808135f744f50aece03d4bf5a71481c7bdca1fcdd0d5b113abdb0c8b382bf81cafee6d17c239041fb49b59f4e19970f24a475378e7f711c3a47d6438de2bdab
64c105442c util: make GetDataDir read-only & create datadir.. (willcl-ark)
56e370fbb9 util: add ArgsManager datadir helper functions (willcl-ark)
Pull request description:
Fixes#20070
Currently `ArgsManager::GetDataDir()` ensures it will always return a datadir by creating one if necessary. The function is shared between `bitcoind` `bitcoin-qt` and `bitcoin-cli` which results in the undesirable behaviour described in #20070.
This PR splits out the part of the function which creates directories and adds it as a standalone function, only called as part of `bitcoind` and `bitcoin-qt` init, but not `bitcoin-cli`.
`ReadConfigFiles`' behavior is changed to use the absolute path of the config file in error and warning messages instead of a relative path.
This was inadvertantly the form being tested [here](73966f75f6/test/functional/feature_config_args.py (L287)), whilst we were _not_ testing that a relative path was returned by the message even though we passed a relative path in as argument.
ACKs for top commit:
achow101:
ACK 64c105442c
hebasto:
re-ACK 64c105442c, only comments have been adjusted as requsted since my previous [review](https://github.com/bitcoin/bitcoin/pull/27073#pullrequestreview-1307435890).
TheCharlatan:
Re-ACK 64c105442c
ryanofsky:
Code review ACK 64c105442c. Only comment changes since last review
Tree-SHA512: b129501346071ad62551c9714492b21536d0558a94117d97218e255ef4e948d00df899a4bc2788faea27d3b1f20fc6136ef9d03e6a08498d926d9ad8688d6c96
f36d1d5b89 Use void* throughout support/lockedpool.h (Jeffrey Czyz)
Pull request description:
Replace uses of char* with void* in Arena's member variables. Instead,
cast to char* where needed in the implementation.
Certain compiler environments disallow std::hash<char*> specializations
to prevent hashing the pointer's value instead of the string contents.
Thus, compilation fails when std::unordered_map is keyed by char*.
Explicitly using void* is a workaround in such environments. For
consistency, void* is used throughout all member variables similarly to
the public interface.
Changes to this code are covered by src/test/allocator_tests.cpp.
ACKs for top commit:
achow101:
ACK f36d1d5b89
theStack:
Code-review ACK f36d1d5b89
jonatack:
ACK f36d1d5b89 review, debug build, unit tests, checked clang 15 raises "error: arithmetic on a pointer to void" without the conversions here from the generic void* pointer back to char*
Tree-SHA512: f9074e6d29ef78c795a512a6e00e9b591e2ff34165d09b73eae9eef25098c59e543c194346fcd4e83185a39c430d43744b6f7f9d1728a132843c67bd27ea5189
7013da07fb Add release note for PR#25943 (David Gumberg)
04f270b435 Add test for unspendable transactions and parameter 'maxburnamount' to sendrawtransaction. (David Gumberg)
Pull request description:
This PR adds a user configurable, zero by default parameter — `maxburnamount` — to `sendrawtransaction`. This PR makes bitcoin core reject transactions that contain unspendable outputs which exceed `maxburnamount`. closes#25899.
As a result of this PR, `sendrawtransaction` will by default block 3 kinds of transactions:
1. Those that begin with `OP_RETURN` - (datacarriers)
2. Those whose lengths exceed the script limit.
3. Those that contain invalid opcodes.
The user is able to configure a `maxburnamount` that will override this check and allow a user to send a potentially unspendable output into the mempool.
I see two legitimate use cases for this override:
1. Users that deliberately use `OP_RETURN` for datacarrier transactions that embed data into the blockchain.
2. Users that refuse to update, or are unable to update their bitcoin core client would be able to make use of new opcodes that their client doesn't know about.
ACKs for top commit:
glozow:
reACK 7013da07fb
achow101:
re-ACK 7013da07fb
Tree-SHA512: f786a796fb71a587d30313c96717fdf47e1106ab4ee0c16d713695e6c31ed6f6732dff6cbc91ca9841d66232166eb058f96028028e75c1507324426309ee4525
When producing a dummy signature for the purpose of estimating the transaction fee, do not assume an external signer performs R-value grinding on the signature.
In particular, this avoids a scenario where the fee rate is 1 sat / vbyte and a transaction with a 72 byte signature is not accepted into our mempool.
This commit also drops the nullptr default for CCoinControl arguments for functions that it touches. This is because having a boolean argument right next to an optional pointer is error prone.
Co-Authored-By: S3RK <1466284+S3RK@users.noreply.github.com>
.. only in bitcoind and bitcoin-qt
This changes behaviour of GetConfigFilePath which now always returns the
absolute path of the provided -conf argument.
Symbol visibility issues are not actually fixed yet because we have not yet
defined an api and exported symbols, but everything is now in place for that.
Building binaries against our uninstalled shared libs is impractical. Instead,
to test them, we'll need to work on a runtime shared-lib execution harness.
0af16e7134 doc: add release note for #25574 (Martin Zumsande)
57ef2a4812 validation: report if pruning prevents completion of verification (Martin Zumsande)
0c7785bb25 init, validation: Improve handling if VerifyDB() fails due to insufficient dbcache (Martin Zumsande)
d6f781f1cf validation: return VerifyDBResult::INTERRUPTED if verification was interrupted (Martin Zumsande)
6360b5302d validation: Change return value of VerifyDB to enum type (Martin Zumsande)
Pull request description:
`VerifyDB()` can fail to complete due to insufficient dbcache at the level 3 checks. This PR improves the error handling in this case in the following ways:
- The rpc `-verifychain` now returns false if the check can't be completed due to insufficient cache
- During init, we only log a warning if the default values for `-checkblocks` and `-checklevel` are taken and the check doesn't complete. However, if the user actively specifies one of these args, we return with an InitError if we can't complete the check.
This PR also changes `-verifychain` RPC to return `false` if the verification didn't finish due to missing block data (pruning) or due to being interrupted by the node being shutdown.
Previously, this PR also included a fix for a possible assert during verification - this was done in #27009 (now merged).
ACKs for top commit:
achow101:
ACK 0af16e7134
ryanofsky:
Code review ACK 0af16e7134. Only small suggested changes since the last review, like renaming some of the enum values. I did leave more suggestions, but they are not very important and could be followups
john-moffett:
ACK 0af16e7134
MarcoFalke:
lgtm re-ACK 0af16e7134 🎚
Tree-SHA512: 84b4f767cf9bfbafef362312757c9bf765b41ae3977f4ece840e40c52a2266b1457832df0cdf70440be0aac2168d9b58fc817238630b0b6812f3836ca950bc0e
4bbf5ddd44 Detailed error message for passphrases with null chars (John Moffett)
b4bdabc223 doc: Release notes for 27068 (John Moffett)
4b1205ba37 Test case for passphrases with null characters (John Moffett)
00a0861181 Pass all characters to SecureString including nulls (John Moffett)
Pull request description:
`SecureString` is a `std::string` specialization with a secure allocator. However, in practice it's treated like a C- string (no explicit length and null-terminated). This can cause unexpected and potentially insecure behavior. For instance, if a user enters a passphrase with embedded null characters (which is possible through Qt and the JSON-RPC), it will ignore any characters after the first null, potentially giving the user a false sense of security.
Instead of assigning to `SecureString` via `std::string::c_str()`, assign it via a `std::string_view` of the original. This explicitly captures the size and still doesn't make any extraneous copies in memory.
Note to reviewers, the following all compile identically in recent `GCC` (x86-64 and ARM64) with `-O2` (and `-std=c++17`):
```C++
std::string orig_string;
std::cin >> orig_string;
SecureString s;
s.reserve(100);
// The following all compile identically
s = orig_string;
s = std::string_view{orig_string};
s.assign(std::string_view{orig_string});
s.assign(orig_string.data(), orig_string.size());
```
So it's largely a matter of preference. However, one thing to keep in mind is that we want to avoid making unnecessary copies of any sensitive data in memory.
Something like `SecureString s{orig_string};` is still invalid and probably unwanted in our case, since it'd get treated as a short string and optimized away from the secure allocator. I presume that's the reason for the `reserve()` calls.
Fixes#27067.
ACKs for top commit:
achow101:
re-ACK 4bbf5ddd44
stickies-v:
re-ACK [4bbf5dd](4bbf5ddd44)
furszy:
utACK 4bbf5ddd
Tree-SHA512: 47a96905a82ca674b18076a20a388123beedf70e9de73e42574ea68afbb434734e56021835dd9b148cdbf61709926b487cc95e9021d9bc534a7c93b3e143d2f7
49d01f32c9 kernel: add missing include (Cory Fields)
Pull request description:
This syncs the cs_main definition/declaration.
Noticed when experimenting with the external visibility of `cs_main`.
Specifically, this is needed for the following to work as intended:
```c++
__attribute__ ((visibility ("default"))) extern RecursiveMutex cs_main;
```
ACKs for top commit:
fanquake:
ACK 49d01f32c9
Tree-SHA512: ea0dbcf81959566f949d76c7dcd1e33de53e613519500c863bfb0ac8209665b1c12cff2daa7890d03b76debc4d046339ee7b3231adb71b128e9d5a8fa3132b6c
3c1de032de i2p: use consistent number of tunnels with i2pd and Java I2P (Vasil Dimov)
801b405f85 i2p: lower the number of tunnels for transient sessions (Vasil Dimov)
b906b64eb7 i2p: reuse created I2P sessions if not used (Vasil Dimov)
Pull request description:
* Reuse an I2P transient session instead of discarding it if we failed to connect to the desired peer. This means we never used the generated address (destination), whose creation is not cheap. This does not mean that we will use the same address for more than one peer.
* Lower the number of tunnels for transient sessions.
* Explicitly specify the number of tunnels for persistent sessions instead of relying on the defaults which differ between I2P routers. This way we get consistent behavior with all routers.
Alleviates: https://github.com/bitcoin/bitcoin/issues/26754
(I have not tested this with i2pd, yet)
ACKs for top commit:
jonatack:
ACK 3c1de032de
mzumsande:
Light ACK 3c1de032de
Tree-SHA512: 477b4b9a5755e6a9a46bc0f7b268fa419dff4414e25445c750ae913f7552d9e2313f2aca4e3b70067b8390c2d0c2d68ec459f331765e939fc84139e454031cd4
9486509be6 wallet, rpc: Update migratewallet help text for encrypted wallets (Andrew Chow)
aaf02b5721 tests: Tests for migrating wallets by name, and providing passphrase (Andrew Chow)
7fd125b27d wallet: Be able to unlock the wallet for migration (Andrew Chow)
6bdbc5ff59 rpc: Allow users to specify wallet name for migratewallet (Andrew Chow)
dbfa345403 wallet: Allow MigrateLegacyToDescriptor to take a wallet name (Andrew Chow)
Pull request description:
`migratewallet` currently operates on wallets that are already loaded, however this is not necessarily required, and in the future, not possible once the legacy wallet is removed. So we need to also be able to give the wallet name to migrate.
Additionally, the passphrase is required when migrating a wallet. Since a wallet may not be loaded when we migrate, and as we currently unload wallets when migrating, we need the passphrase to be given to `migratewallet` in order to migrate encrypted wallets.
Fixes#27048
ACKs for top commit:
john-moffett:
reACK 9486509be6
pinheadmz:
ACK 9486509be6
furszy:
ACK 9486509b
Tree-SHA512: 35e2ba69a148e129a41e20d7fb99c4cab7947b1b7e7c362f4fd06ff8ac6e79e476e07207e063ba5b80e1a33e2343f4b4f1d72d7930ce80c34571c130d2f5cff4
Moves chainstate initialization into its own function. This is
necessary to later support a more readable way of handling
background-validation chainstate cleanup during init, since the
chainstate initialization functions may need to be repeated after
moving leveldb filesystem content around.
This commit isn't strictly necessary, but the alternative is to (ab)use
the `while` loop in init.cpp with a `continue` on the basis of a
specific ChainstateLoadingError return value from LoadChainstate. Not
only is this harder to read, but it can't be unittested.
The approach here lets us consolidate background-validation cleanup to
LoadChainstate, and therefore exercise it within tests.
This commit is most easily reviewed with
git diff --color-moved=dimmed-zebra
--color-moved-ws=ignore-space-change
and remove m_snapshot_validated. This state can now be inferred by the
number of isUsable chainstates.
m_disabled is used to signal that a chainstate should no longer be used
by validation logic; it is used as a sentinel when background validation
completes or if the snapshot chainstate is found to be invalid.
isUsable is a convenience method that incorporates m_disabled.
c1b7bd047f fuzz: avoid redundant dup key checks when creating Miniscript nodes (Antoine Poinsot)
Pull request description:
I thought i had done that already in #24149, but it must have slipped through the rebase. It's a 2x speed improvement against the existing corpora and will probably be much more as we extend them with larger nodes.
ACKs for top commit:
sipa:
ACK c1b7bd047f
Tree-SHA512: 9e6ceb6254183964b6c5538e21ba6321df95a68acb343a15a6ecfef5c51a1980d2627df5aeef9aef1db41656e18cc4f3bc96e6f24314d12fa60368b04a350001
Since users may have thought the null characters in their
passphrases were actually evaluated prior to this change,
they may be surprised to learn that their passphrases no
longer work. Give them feedback to explain how to remedy
the issue.
`SecureString` is a `std::string` specialization with
a secure allocator. However, it's treated like a C-
string (no explicit length and null-terminated). This
can cause unexpected behavior. For instance, if a user
enters a passphrase with an embedded null character
(which is possible through Qt and the JSON-RPC), it will
ignore any characters after the null, giving the user
a false sense of security.
Instead of assigning `SecureString` via `std::string::c_str()`,
assign it via a `std::string_view` of the original. This
explicitly captures the size and doesn't make any extraneous
copies in memory.
6a5b348f2e test: test rescanning encrypted wallets (ishaanam)
493b813e17 wallet: ensure that the passphrase is not deleted from memory when being used to rescan (ishaanam)
66a86ebabb wallet: keep track of when the passphrase is needed when rescanning (ishaanam)
Pull request description:
Wallet passphrases are needed to top up the keypool of encrypted wallets
during a rescan. The following RPCs need the passphrase when rescanning:
- `importdescriptors`
- `rescanblockchain`
The following RPCs use the information about whether or not the
passphrase is being used to ensure that full rescans are able to
take place (meaning the following RPCs should not be able to run
if a rescan requiring the wallet to be unlocked is taking place):
- `walletlock`
- `encryptwallet`
- `walletpassphrasechange`
`m_relock_mutex` is also introduced so that the passphrase is not
deleted from memory when the timeout provided in
`walletpassphrase` is up and the wallet is still rescanning.
Fixes#25702, #11249
Thanks to achow101 for coming up with the idea of using a new mutex to solve this issue and for answering related questions.
ACKs for top commit:
achow101:
ACK 6a5b348f2e
hernanmarino:
ACK 6a5b348f2e
furszy:
Tested ACK 6a5b348f
Tree-SHA512: 0b6db692714f6f94594fa47249f5ee24f85713bfa70ac295a7e84b9ca6c07dda65df7b47781a2dc73e5b603a8725343a2f864428ae20d3e126c5b4802abc4ab5
95f12de925 BIP341 txdata cannot be precomputed without spent outputs (Pieter Wuille)
Pull request description:
In `PrecomputedTransactionData::Init`, if `force` is set to `true`, `m_bip341_taproot_ready` is always set to true, suggesting that all its BIP341-relevant members (including `m_spent_amounts_single_hash`) are correct. If however no `spent` array of spent previous `CTxOut`s is provided, some of these members will be incorrect. This option was introduced in #21365.
That doesn't actually hurt, as without prevout data, it's fundamentally impossible to generate correct BIP341 signatures anyway, and f722a9bd13/src/script/sign.cpp (L71) should prevent the logic from being used anyway.
Still, don't set `m_bip341_taproot_ready` variable when we clearly don't have enough data to compute it.
Discovered by Russell O'Connor.
ACKs for top commit:
ajtowns:
ACK 95f12de925
achow101:
ACK 95f12de925
instagibbs:
ACK 95f12de925
Tree-SHA512: 90acd2bfa50a7a0bde75a15a9f6c1f5c40f48fb5b870b1bbc4082777e24a482c8282463ef7d1245e53201dbcb5c196ef0386352f8e380e68cdf00c2111633b77
14b4921a91 wallet: reuse change dest when recreating TX with avoidpartialspends (Matthew Zipkin)
Pull request description:
Closes https://github.com/bitcoin/bitcoin/issues/27051
When the wallet creates a transaction internally, it will also create an alternative that spends using destination groups and see if the fee difference is negligible. If it costs the user the same to send the grouped version, we send it (even if the user has `avoidpartialspends` set to `false` which is default). This patch ensures that the second transaction creation attempt re-uses the change destination selected by the first attempt. Otherwise, the first change address remains reserved, will not be used in the second attempt, and then will never be used by the wallet, leaving gaps in the BIP44 chain.
If the user had `avoidpartialspends` set to true, there is no second version of the created transaction and the change addresses are not affected.
I believe this behavior was introduced in https://github.com/bitcoin/bitcoin/pull/14582
ACKs for top commit:
achow101:
ACK 14b4921a91
Tree-SHA512: a3d56f251ff4b333fc11325f30d05513e34ab0a2eb703fadd0ad98d167ae074493df1a24068298336c6ed2da6b31aa2befa490bc790bbc260ed357c8f2397659
7554b1fd66 rpc: fix successful broadcast count in `submitpackage` error msg (Sebastian Falbesoner)
Pull request description:
If a `submitpackage` RPC call errors due to any of the individual tx broadcasts failing, the returned error message is supposed to contain the number of successful broadcasts so far:
4395b7f084/src/rpc/mempool.cpp (L848-L849)
Right now this is wrongly always shown as zero. Fix this by adding the missing increment of the counter. While touching that area, the variable is also renamed to better reflect its purpose (s/num_submitted/num_broadcast/; the submission has already happened at that point) and named arguments for the `BroadcastTransaction` call are added.
(Note that the error should be really rare, as all txs have already been submitted succesfully to the mempool. IIUC this code-path could only hit if somehow a tx is being removed from the mempool between `ProcessNewPackage` and the `BroadcastTransaction` calls, e.g. if a new block is received which confirms any of the package's txs.)
ACKs for top commit:
glozow:
utACK 7554b1fd66, thanks!
Tree-SHA512: e362e93b443109888e28d6facf6f52e67928e8baaa936e355bfdd324074302c4832e2fa0bd8745309a45eb729866d0513b928ac618ccc9432b7befc3aa2aac66
73ec4b2a83 tests: decodescript can infer descriptors for scripts >520 bytes (Andrew Chow)
7cc7822371 rpc: Use FlatSigningProvider in decodescript (Andrew Chow)
Pull request description:
`FillableSigningProvider` limits scripts to 520 bytes even though segwit allows scripts to be larger than that. We can avoid this limit by using a `FlatSigningProvider` so that such larger scripts can be decoded.
Fixes#27111
ACKs for top commit:
instagibbs:
ACK 73ec4b2a83
Tree-SHA512: c0e6d21025e2da864471989ac94c54e127d05459b9b048f34a0da8d76d8e372d5472a2e667ba2db74d6286e3e6faa55486ffa9232a068b519afa676394031d5a
6a5e88e5cf miner: don't re-apply default Options value if argument is unset (stickies-v)
ea72c3d9d5 refactor: avoid duplicating BlockAssembler::Options members (stickies-v)
cba749a9b7 refactor: rename local gArgs to args (stickies-v)
Pull request description:
Two follow-ups for #26695, both refactoring and no observed (*) behaviour change:
- Rename `gArgs` to `args` because it's not actually a global
- Add `BlockAssembler::Options` as a (private) member to `BlockAssembler` to avoid having to assign all the options individually, essentially duplicating them
Reduces LoC and makes the code more readable, in my opinion.
---
(*) as [pointed out by ajtowns](https://github.com/bitcoin/bitcoin/pull/26883#discussion_r1068247937), this PR changes the interface of `ApplyArgsManOptions()`, making this not a pure refactoring PR. In practice, `ApplyArgsManOptions()` is never called in such a way that this leads to observed behaviour change. Regardless, I've carved out the potential behaviour change into a separate commit and would be okay with dropping it, should it turn out to be controversial.
ACKs for top commit:
glozow:
ACK 6a5e88e5cf
TheCharlatan:
Light code review ACK 6a5e88e5cf
Tree-SHA512: 15c30442ff0e070b1a58dc4c9615550d619ce35b4a2596b2c0a9d790259bbf987cab708f7cbb1057a8cf8b4c3226f3ad981282d3499ac442094806492a5f68ce
If a `submitpackage` RPC call errors due to any of the individual tx
broadcasts failing, the returned error message is supposed to contain
the number of successful broadcasts so far. Right now this is wrongly
always shown as zero. Fix this by adding the missing counting.
(Note though that the error should be really rare, as all txs have
already been submitted succesfully to the mempool.)
621cfb7722 random: consolidate WIN32 #ifdefs (fanquake)
75ec6275e6 random: remove compat.h include (fanquake)
4dc12816ac random: use int for MAX_TRIES (fanquake)
Pull request description:
This change is related to removing the use of `compat.h` as a miscellaneous catch-all for unclear/platform specific includes. Somewhat prompted by IWYU-related discussion here: https://github.com/bitcoin/bitcoin/pull/26763/files#r1058861693.
The only reason `compat.h` is required in random.cpp for Windows (note the `#ifdef WIN32`), is for `ssize_t` and an "indirect" inclusion of `windows.h`. I say indirect, because `windows.h` isn't actually included in compat.h either, it's dragged in as a side-effect of other windows includes there, i.e `winsock2.h`.
Remove this coupling by replacing `ssize_t` with int, just including `windows.h` and removing compat.h.
ACKs for top commit:
hebasto:
re-ACK 621cfb7722, rebased only since my [recent](https://github.com/bitcoin/bitcoin/pull/26814#pullrequestreview-1237312144) review. Verified with:
john-moffett:
ACK 621cfb7722
Tree-SHA512: 31e1ed2e7ff7daf6c3ee72e6a908def52f7addf8305ba371c5032f1927cbb8ef5d302785e8de42b5c04a123052f04688cc9fd80decceb04738b5d9153f3d32d7
4275195606 De-duplicate add_coin methods to a test util helper (Jon Atack)
9d92c3d7f4 Create InsecureRandMoneyAmount() test util helper (Jon Atack)
81f5ade2a3 Move random test util code from setup_common to random (Jon Atack)
Pull request description:
- Move random test utilities from `setup_common` to a new `random` file, as many tests don't use this code.
- Create a helper to generate semi-random CAmounts up to `MONEY_RANGE` rather than only uint32, and use the helper in the unit tests.
- De-duplicate a shared `add_coin` method by extracting it to a `coins` test utility.
ACKs for top commit:
pinheadmz:
ACK 4275195606
achow101:
ACK 4275195606
john-moffett:
ACK 4275195606
Tree-SHA512: 3ed974251149c7417f935ef2f8865aa0dcc33b281b47522b0f96f1979dff94bb8527957f098fe4d210f40d715c00f29512f2ffe189097102229023b7284a3a27
aadd7c5b9b refactor, validation: Add ChainstateManagerOpts db options (Ryan Ofsky)
0352258148 refactor, txdb: Use DBParams struct in CBlockTreeDB (Ryan Ofsky)
c00fa1a734 refactor, txdb: Add CoinsViewOptions struct (Ryan Ofsky)
2eaeded37f refactor, dbwrapper: Add DBParams and DBOptions structs (Ryan Ofsky)
Pull request description:
Code in the libbitcoin_kernel library should not be calling `ArgsManager` methods or trying to read options from the command line. Instead it should just get options values from simple structs and function arguments that are passed in externally. This PR removes `gArgs` accesses from `dbwrapper` and `txdb` modules by defining appropriate options structs, and is a followup to PR's #25290#25487#25527 which remove other `ArgsManager` calls from kernel modules.
This PR does not change behavior in any way. It is a simpler alternative to #25623 because the only thing it does is remove `gArgs` references from kernel code. It avoids other unnecessary changes like adding options to the kernel API (they can be added separately later).
ACKs for top commit:
TheCharlatan:
Code review ACK aadd7c5b9b
achow101:
ACK aadd7c5b9b
furszy:
diff ACK aadd7c5b
Tree-SHA512: 46dfd5d99ab3110492e7bba97a87122c831b8344caaf7dd2ebdb6e0ad6aa9174d4d1832d6f3a7465eda9294fe50defaa3c000afbbddc4e72838687df09a63ffd
2555a3950f p2p: ProcessAddrFetch(-seednode) is unnecessary if -connect is specified (Dhruv Mehta)
Pull request description:
If the user runs: `bitcoind -connect=X -seednode=Y`, I _think_ it is safe to ignore `-seednode`. A more populated `addrman` (via `getaddr` calls to peers in `-seednode`) is not useful in this configuration: `addrman` entries are used to initiate new outbound connections when slots are open, or to open feeler connections and keep `addrman` from getting stale. This is all done in a part of `ThreadOpenConnections` (below [this line](https://github.com/bitcoin/bitcoin/blob/master/src/net.cpp#L1803)) which is never executed when `-connect` is supplied. With `-connect`, `ThreadOpenConnections` will run [this loop](https://github.com/bitcoin/bitcoin/blob/master/src/net.cpp#L1785) and exit thread execution when interrupted.
Reviewers may also find it relevant that when `-connect` is used, we [soft disable](https://github.com/bitcoin/bitcoin/blob/master/src/init.cpp#L800) `-dnsseed` in init.cpp perhaps for the same reason i.e. seeding is not useful with `-connect`.
Running `ProcessAddrFetch` does not seem to have downside except developer confusion AFAICT. I was confused by this and felt it might affect other new bitcoiners too. If there is strong preference to not remove the line, I'd also be happy to just leave a comment there mentioning `ADDR_FETCH`/`-seednode` is irrelevant when used with `-connect`.
If this change is accepted, the node will still make `getaddr` calls to peers in `-connect` and expand `addrman`. However, disabling those `getaddr` calls would leak information about the node's configuration.
ACKs for top commit:
mzumsande:
Code Review ACK 2555a3950f
achow101:
ACK 2555a3950f
vasild:
ACK 2555a3950f
Tree-SHA512: 9187a0cff58db8edeca7e15379b1c121e7ebe8c38fb82f69e3dae8846ee94c92a329d79025e0f023c7579b2d86e7dbf756e4e30e90a72236bfcd2c00714180b3
c9d548c91f net: remove CService::ToStringPort() (Vasil Dimov)
fd4f0f41e9 gui: simplify OptionsDialog::updateDefaultProxyNets() (Vasil Dimov)
96c791dd20 net: remove CService::ToString() use ToStringAddrPort() instead (Vasil Dimov)
944a9de08a net: remove CNetAddr::ToString() and use ToStringAddr() instead (Vasil Dimov)
043b9de59a scripted-diff: rename ToStringIP[Port]() to ToStringAddr[Port]() (Vasil Dimov)
Pull request description:
Before this PR we had the somewhat confusing combination of methods:
`CNetAddr::ToStringIP()`
`CNetAddr::ToString()` (duplicate of the above)
`CService::ToStringIPPort()`
`CService::ToString()` (duplicate of the above, overrides a non-virtual method from `CNetAddr`)
`CService::ToStringPort()`
Avoid [overriding non-virtual methods](https://github.com/bitcoin/bitcoin/pull/25349/#issuecomment-1185226396).
"IP" stands for "Internet Protocol" and while sometimes "IP addresses" are called just "IPs", it is incorrect to call Tor or I2P addresses "IPs". Thus use "Addr" instead of "IP".
Change the above to:
`CNetAddr::ToStringAddr()`
`CService::ToStringAddrPort()`
The changes touch a lot of files, but are mostly mechanical.
ACKs for top commit:
sipa:
utACK c9d548c91f
achow101:
ACK c9d548c91f
jonatack:
re-ACK c9d548c91f only change since my previous reviews is rebase, but as a sanity check rebased to current master and at each commit quickly re-reviewed and re-verified clean build and green unit tests
LarryRuane:
ACK c9d548c91f
Tree-SHA512: 633fb044bdecf9f551b5e3314c385bf10e2b78e8027dc51ec324b66b018da35e5b01f3fbe6295bbc455ea1bcd1a3629de1918d28de510693afaf6a52693f2157
52f4d567d6 refactor: remove <util/system.h> include from wallet.h (furszy)
6c9b342c30 refactor: wallet, remove global 'ArgsManager' access (furszy)
d8f5fc4462 wallet: set '-walletnotify' script instead of access global args manager (furszy)
3477a28dd3 wallet: set keypool_size instead of access global args manager (furszy)
Pull request description:
Structurally, the wallet class shouldn't access the global `ArgsManager` class, its internal behavior shouldn't be coupled to a global command line args parsing object.
So this PR migrates the only two places where we depend on it: (1) the keypool size, and (2) the "-walletnotify" script. And cleans up the, now unneeded, wallet `ArgsManager` ref member.
Extra note:
In the process of removing the args ref member, discovered and fixed files that were invalidly depending on the wallet header including `util/system.h`.
ACKs for top commit:
achow101:
ACK 52f4d567d6
TheCharlatan:
Re-ACK 52f4d567d6
hebasto:
re-ACK 52f4d567d6
Tree-SHA512: 0cffd99b4dd4864bf618aa45aeaabbef2b6441d27b6dbb03489c4e013330877682ff17b418d07aa25fbe1040bdf2c67d7559bdeb84128c5437bf0e6247719016
We no-longer need ssize_t.
Add windows.h, which was being indirectly included via compat.h. It isn't
actually included in compat.h itself, but was being included as a side-effect
of other includes, like winsock2.h.
Check it only once on the top level node.
Running libfuzzer with -runs=0 against the qa-assets corpus (1b9ddc96586769d92b1b62775f397b7f1a63f142).
Without this patch:
miniscript_stable: Done 6616 runs in 118 second(s)
miniscript_smart: Done 13182 runs in 253 second(s)
With this patch:
miniscript_stable: Done 6616 runs in 57 second(s)
miniscript_smart: Done 13182 runs in 124 second(s)
30a3230e86 script: remove out-of-date snprintf TODO (Jon Atack)
0e015146bd net: remove orphaned CSubNet::SanityCheck() (Jon Atack)
Pull request description:
`CSubNet::SanityCheck()` was added in #20140, and not removed in #22570 when it became orphaned code.
Also, remove an out-of-date `snprintf` TODO that was resolved in #27036, and fix up 2 words to make the spelling linter green again.
ACKs for top commit:
fanquake:
ACK 30a3230e86
pinheadmz:
ACK 30a3230e86
brunoerg:
crACK 30a3230e86
Tree-SHA512: f91a2a5af902d3b82ab496f19deeac17d58dbf72a8016e880ea61ad858b66e7ea0ae70b964c4032018eb3252cc34ac5fea163131c6a7f1baf87fc9ec9b5833d8
The rpc command verifychain now fails if the dbcache was not sufficient
to complete the verification at the specified level and depth.
In the same situation, the VerifyDB check during Init will now fail (and lead to
an early shutdown) if the user has explicitly specified -checkblocks or
-checklevel but the check couldn't be executed because of the limited
cache. If the user didn't change any of the two and is using the defaults, log a warning
but don't prevent the node from starting up.
This does not change behavior. It is in preparation for
special handling of the case where VerifyDB doesn't finish
for various reasons, but doesn't fail.
4c8ecccdcd test: add tests for `outputs` argument to `bumpfee`/`psbtbumpfee` (Seibart Nedor)
c0ebb98382 wallet: add `outputs` arguments to `bumpfee` and `psbtbumpfee` (Seibart Nedor)
a804f3cfc0 wallet: extract and reuse RPC argument format definition for outputs (Seibart Nedor)
Pull request description:
This implements a modification of the proposal in #22007: instead of **adding** outputs to the set of outputs in the original transaction, the outputs given by `outputs` argument **completely replace** the outputs in the original transaction.
As noted below, this makes it easier to "cancel" a transaction or to reduce the amounts in the outputs, which is not the case with the original proposal in #22007, but it seems from the discussion in this PR that the **replace** behavior is more desirable than **add** one.
ACKs for top commit:
achow101:
ACK 4c8ecccdcd
1440000bytes:
Code Review ACK 4c8ecccdcd
ishaanam:
reACK 4c8ecccdcd
Tree-SHA512: 31361f4a9b79c162bda7929583b0a3fd200e09f4c1a5378b12007576d6b14e02e9e4f0bab8aa209f08f75ac25a1f4805ad16ebff4a0334b07ad2378cc0090103
Since migration reloads the wallet, the wallet will always be locked
unless the passphrase is given. migratewallet can now take the
passphrase in order to unlock the wallet for migration.
An overload of MigrateLegacyToDescriptor is added which takes the wallet
name. The original that took a wallet pointer is still available, it
just gets the name, closes the wallet, and calls the new overload.
Using a FillableSigningProvider results in decodescript being unable to
infer descriptors for scripts larger than 520 bytes. Using a
FlatSigningProvider resolves this.
5669afb80e fs: drop old WSL1 hack. (sinetek)
Pull request description:
Following discussion, the WSL1 patch will be removed, as WSL1 is no longer being developed by Microsoft. Instead, please upgrade to a mainstream WSL2 version. More information can be found on [the official website](https://docs.microsoft.com/en-us/windows/wsl/).
ACKs for top commit:
1440000bytes:
ACK 5669afb80e
fanquake:
ACK 5669afb80e - seems ok as-is.
Tree-SHA512: 256c13985f6dd3453caf39c7ef1c951dbdfa8457a18cd05e4624db36d8ed8a4f809bb78a7b3c82c72997e9ed3823d5566a5c2d0812d2501aba2e54bc5e6eec79
6c7a17a8e0 psbt: support externally provided preimages for Miniscript satisfaction (Antoine Poinsot)
840a396029 qa: add a "smart" Miniscript fuzz target (Antoine Poinsot)
17e3547241 qa: add a fuzz target generating random nodes from a binary encoding (Antoine Poinsot)
611e12502a qa: functional test Miniscript signing with key and timelocks (Antoine Poinsot)
d57b7f2021 refactor: make descriptors in Miniscript functional test more readable (Antoine Poinsot)
0a8fc9e200 wallet: check solvability using descriptor in AvailableCoins (Antoine Poinsot)
560e62b1e2 script/sign: signing support for Miniscripts with hash preimage challenges (Antoine Poinsot)
a2f81b6a8f script/sign: signing support for Miniscript with timelocks (Antoine Poinsot)
61c6d1a844 script/sign: basic signing support for Miniscript descriptors (Antoine Poinsot)
4242c1c521 Align 'e' property of or_d and andor with website spec (Pieter Wuille)
f5deb41780 Various additional explanations of the satisfaction logic from Pieter (Pieter Wuille)
22c5b00345 miniscript: satisfaction support (Antoine Poinsot)
Pull request description:
This makes the Miniscript descriptors solvable.
Note this introduces signing support for much more complex scripts than the wallet was previously able to solve, and the whole tooling isn't provided for a complete Miniscript integration in the wallet. Particularly, the PSBT<->Miniscript integration isn't entirely covered in this PR.
ACKs for top commit:
achow101:
ACK 6c7a17a8e0
sipa:
utACK 6c7a17a8e0 (to the extent that it's not my own code).
Tree-SHA512: a71ec002aaf66bd429012caa338fc58384067bcd2f453a46e21d381ed1bacc8e57afb9db57c0fb4bf40de43b30808815e9ebc0ae1fbd9e61df0e7b91a17771cc
906631450d s/transcation/transaction/ (Greg Sanders)
Pull request description:
ACKs for top commit:
fanquake:
ACK 906631450d - looks like other comments are being addressed elsewhere.
Tree-SHA512: c835a14db2e0cf5e0317c95c8c7441df1f7c6cb14be7809fd947e07ea9d23f1f171f111429aabd0509b7f17601bc742041316b18e1135e547a966961f2c65038
9fa43b5af6 refactor: Disable unused special members functions in `UnlockContext` (Hennadii Stepanov)
Pull request description:
Also `UnlockContext::valid` and `UnlockContext::relock` are `const` now.
ACKs for top commit:
achow101:
ACK 9fa43b5af6
john-moffett:
ACK 9fa43b5af6
furszy:
ACK 9fa43b5a
Tree-SHA512: 6d9fa8208676b9bd5d85b73cb2d3136e7f28ef59e68ee34915ec598458868e302a80b9ef1384c0bf7a4c42f936830c3add9662ca0bae73860a55a25cc374b699
that was resolved in PR27036 "test: Remove last uses of snprintf and simplify"
and while here, fix up 2 words in docs to make the spelling linter green again.
Since we no longer store a ref to the global `ArgsManager`
inside the wallet, we can move the util/system.h
include to the cpp.
This dependency removal opened a can of worms, as few
other places were, invalidly, depending on the wallet's
header including it.
691eaf8873 Pass MSG_MORE flag when sending non-final network messages (Matt Whitlock)
Pull request description:
**N.B.:** This is my second attempt at introducing this optimization. #12519 (2018) was closed in deference to switching to doing gathering socket writes using `sendmsg(2)`, which I agree would have superior performance due to fewer syscalls, but that work was apparently abandoned in late 2018. Ever since, Bitcoin Core has continued writing tons of runt packets to the wire. Can we proceed with my halfway solution for now?
----
Since Nagle's algorithm is disabled, each and every call to `send(2)` can potentially generate a separate TCP segment on the wire. This is especially inefficient when sending the tiny header preceding each message payload.
Linux implements a `MSG_MORE` flag that tells the kernel not to push the passed data immediately to the connected peer but rather to collect it in the socket's internal transmit buffer where it can be combined with data from successive calls to `send(2)`. Where available, specify this flag when calling `send(2)` in `CConnman::SocketSendData(CNode &)` if the data buffer being sent is not the last one in `node.vSendMsg`.
ACKs for top commit:
sipa:
ACK 691eaf8873
vasild:
ACK 691eaf8873
Tree-SHA512: 9a7f46bc12edbf78d488f05d1c46760110a24c95af74b627d2604fcd198fa3f511c5956bac36d0034e88c632d432f7d394147e667a11b027af0a30f70a546d70
511aa4f1c7 Add unit test for ChaCha20's new caching (Pieter Wuille)
fb243d25f7 Improve test vectors for ChaCha20 (Pieter Wuille)
93aee8bbda Inline ChaCha20 32-byte specific constants (Pieter Wuille)
62ec713961 Only support 32-byte keys in ChaCha20{,Aligned} (Pieter Wuille)
f21994a02e Use ChaCha20Aligned in MuHash3072 code (Pieter Wuille)
5d16f75763 Use ChaCha20 caching in FastRandomContext (Pieter Wuille)
38eaece67b Add fuzz test for testing that ChaCha20 works as a stream (Pieter Wuille)
5f05b27841 Add xoroshiro128++ PRNG (Martin Leitner-Ankerl)
12ff72476a Make unrestricted ChaCha20 cipher not waste keystream bytes (Pieter Wuille)
6babf40213 Rename ChaCha20::Seek -> Seek64 to clarify multiple of 64 (Pieter Wuille)
e37bcaa0a6 Split ChaCha20 into aligned/unaligned variants (Pieter Wuille)
Pull request description:
This is an alternative to #25354 (by my benchmarking, somewhat faster), subsumes #25712, and adds additional test vectors.
It separates the multiple-of-64-bytes-only "core" logic (which becomes simpler) from a layer around which performs caching/slicing to support arbitrary byte amounts. Both have their uses (in particular, the MuHash3072 code can benefit from multiple-of-64-bytes assumptions), plus the separation results in more readable code. Also, since FastRandomContext effectively had its own (more naive) caching on top of ChaCha20, that can be dropped in favor of ChaCha20's new built-in caching.
I thought about rebasing #25712 on top of this, but the changes before are fairly extensive, so redid it instead.
ACKs for top commit:
ajtowns:
ut reACK 511aa4f1c7
dhruv:
tACK crACK 511aa4f1c7
Tree-SHA512: 3aa80971322a93e780c75a8d35bd39da3a9ea570fbae4491eaf0c45242f5f670a24a592c50ad870d5fd09b9f88ec06e274e8aa3cefd9561d623c63f7198cf2c7
9d3127b11e Add settings.json prune-prev, proxy-prev, onion-prev settings (Ryan Ofsky)
Pull request description:
With #602, if proxy and pruning settings are disabled in the GUI and the GUI is restarted, proxy and prune values are not stored anywhere. So if these settings are enabled in the future, default values will be shown, not previous values.
This PR stores previous values so they will preserved across restarts. I'm not sure I like this behavior because showing default values seems simpler and safer to me. Previous values may just have been set temporarily and may have never actually worked, and it adds some code complexity to store them.
This PR is one way of resolving #596. Other solutions are possible and could be implemented as alternatives.
ACKs for top commit:
hebasto:
ACK 9d3127b11e, tested on Ubuntu 22.04.
vasild:
ACK 9d3127b11e
jarolrod:
tACK 9d3127b11e
Tree-SHA512: 1778d1819443490c880cfd5c1711d9c5ac75ea3ee8440e2f0ced81d293247163a78ae8aba6027215110aec6533bd7dc6472aeead6796bfbd51bf2354e28f24a9
77192c9598 cli: include local ("unreachable") peers in -netinfo table (Matthew Zipkin)
Pull request description:
Closes https://github.com/bitcoin/bitcoin/issues/26579
The `-netinfo` dashboard did not list peers that were connected via "unroutable" networks. This included local peers including local-network peers. Personally, I run one bitcoind instance on my network that is used by other services like Wasabi Wallet and LND running on other machines.
This PR adds an "npr" (not publicly routable) column to the table of networks (ipv4, ipv6, onion, etc) so that every connection to the node is listed, and the totals are accurate as they relate to max inbound and max outbound limits.
Example connecting in regtest mode to one local and one remote peer:
```
Bitcoin Core client v24.99.0-151ce099ea8f-dirty regtest - server 70016/Satoshi:24.99.0/
<-> type net mping ping send recv txn blk hb addrp addrl age id address version
in npr 0 0 90 90 1 1 127.0.0.1:59180 70016/Satoshi:24.99.0/
out manual ipv4 63 63 84 84 3 3 0 143.244.175.41 70016/Satoshi:24.0.1/
ms ms sec sec min min min
ipv4 ipv6 npr total block manual
in 0 0 1 1
out 1 0 0 1 0 1
total 1 0 1 2
Local addresses: n/a
```
ACKs for top commit:
jonatack:
Re-tested ACK 77192c9598
Tree-SHA512: 78aa68bcff0dbaadb5f0604bf023fe8fd921313bd8276d12581f7655c089466a48765f9e123cb31d7f1d294d5ca45fdefdf8aa220466ff738f32414f41099c06
`m_relock_mutex` is introduced so that the passphrase is not
deleted from memory when the timeout provided in
`walletpassphrase` is up, but the wallet is still rescanning.
Wallet passphrases are needed to top up the keypool during a
rescan. The following RPCs need the passphrase when rescanning:
- `importdescriptors`
- `rescanblockchain`
The following RPCs use the information about whether or not the
passphrase is being used to ensure that full rescans are able to
take place:
- `walletlock`
- `encryptwallet`
- `walletpassphrasechange`
3a11adc700 Zero out wallet master key upon lock (John Moffett)
Pull request description:
When an encrypted wallet is locked (for instance via the RPC `walletlock`), the documentation indicates that the key is removed from memory:
b92d609fb2/src/wallet/rpc/encrypt.cpp (L157-L158)
However, the vector (a `std::vector<unsigned char, secure_allocator<unsigned char>>`) is merely _cleared_. As it is a member variable, it also stays in scope as long as the wallet is loaded, preventing the secure allocator from deallocating. This allows the key to persist indefinitely in memory. I confirmed this behavior on my macOS machine by using an open-source third party memory inspector ("Bit Slicer"). I was able to find my wallet's master key in Bit Slicer after unlocking and re-locking my encrypted wallet. I then confirmed the key data was at the address in LLDB.
This PR manually fills the bytes with zeroes before calling `clear()` by using our `memory_cleanse` function, which is designed to prevent the compiler from optimizing it away. I confirmed that it does remove the data from memory on my machine upon locking.
Note: An alternative approach could be to call `vMasterKey.shrink_to_fit()` after the `clear()`, which would trigger the secure allocator's deallocation. However, `shrink_to_fit()` is not _guaranteed_ to actually change the vector's capacity, so I think it's unwise to rely on it.
## Edit: A little more clarity on why this is an improvement.
Since `mlock`ed memory is guaranteed not to be swapped to disk and our threat model doesn't consider a super-user monitoring the memory in realtime, why is this an improvement? Most importantly, consider hibernation. Even `mlock`ed memory may get written to disk. From the `mlock` [manpage](https://man7.org/linux/man-pages/man2/mlock.2.html):
> (But be aware that the suspend mode on laptops and some desktop computers will save a copy of the system's RAM to disk, regardless of memory locks.)
As far as I can tell, this is true of [Windows](https://web.archive.org/web/20190127110059/https://blogs.msdn.microsoft.com/oldnewthing/20140207-00/?p=1833#:~:text=%5BThere%20does%20not%20appear%20to%20be%20any%20guarantee%20that%20the%20memory%20won%27t%20be%20written%20to%20disk%20while%20locked.%20As%20you%20noted%2C%20the%20machine%20may%20be%20hibernated%2C%20or%20it%20may%20be%20running%20in%20a%20VM%20that%20gets%20snapshotted.%20%2DRaymond%5D) and macOS as well.
Therefore, a user with a strong OS password and a strong wallet passphrase could still have their keys stolen if a thief takes their (hibernated) machine and reads the permanent storage.
ACKs for top commit:
S3RK:
Code review ACK 3a11adc700
achow101:
ACK 3a11adc700
Tree-SHA512: c4e3dab452ad051da74855a13aa711892c9b34c43cc43a45a3b1688ab044e75d715b42843c229219761913b4861abccbcc8d5cb6ac54957d74f6e357f04e8730
561848aaf2 Exercise non-DIRTY spent coins in caches in fuzz test (Pieter Wuille)
59e6828bb5 Add deterministic mode to CCoinsViewCache (Pieter Wuille)
b0ff310840 Add CCoinsViewCache::SanityCheck() and use it in fuzz test (Pieter Wuille)
3c9cea1340 Add simulation-based CCoinsViewCache fuzzer (Pieter Wuille)
Pull request description:
The fuzzer goes through a sequence of operations that get applied to both a real stack of `CCoinsViewCache` objects, and to simulation data, comparing the two at the end.
ACKs for top commit:
jamesob:
re-ACK 561848aaf2
dergoegge:
Code review ACK 561848aaf2
Tree-SHA512: 68634f251fdb39436b128ecba093f651bff12ac11508dc9885253e57fd21efd44edf3b22b0f821c228175ec507df7d46c7f9f5404fc1eb8187fdbd136a5d5ee2
This is a "dumb" way of randomly generating a Miniscript node from
fuzzer input. It defines a strict binary encoding and will always generate
a node defined from the encoding without "helping" to create valid nodes.
It will cut through as soon as it encounters an invalid fragment so
hopefully the fuzzer can tend to learn the encoding and generate valid
nodes with a higher probability.
On a valid generated node a number of invariants are checked, especially
around the satisfactions and testing them against the Script
interpreter.
The node generation and testing is modular in order to later introduce
other ways to generate nodes from fuzzer inputs with minimal code.
Co-Authored-By: Pieter Wuille <pieter@wuille.net>
This is a workaround for Miniscript descriptors containing hash
challenges. For those we can't mock the signature creator without making
OP_EQUAL mockable in the interpreter, so CalculateMaximumInputSize will
always return -1 and outputs for these descriptors would appear
unsolvable while they actually are.
Try to solve a script using the Miniscript satisfier if the legacy
solver fails under P2WSH context. Only solve public key and public key
hash challenges for now.
We don't entirely replace the raw solver and especially rule out trying to
solve CHECKMULTISIG-based multisigs with the Miniscript satisfier since
some features, such as the transaction input combiner, rely on the
specific behaviour of the former.
Cherry-picked and squashed from
https://github.com/sipa/bitcoin/commits/202302_miniscript_improve.
- Explain thresh() and multi() satisfaction algorithms
- Comment on and_v dissatisfaction
- Mark overcomplete thresh() dissats as malleable and explain
- Add comment on unnecessity of Malleable() in and_b dissat
When an encrypted wallet is locked (for instance via the
RPC `walletlock`), the docs indicate that the key is
removed from memory. However, the vector (with a secure
allocator) is merely cleared. This allows the key to persist
indefinitely in memory. Instead, manually fill the bytes with
zeroes before clearing.
2d955ff006 net: add `Ensure{any}Banman` (brunoerg)
Pull request description:
This PR adds `Ensure{any}Banman` functions to avoid code repetition and make it cleaner. Same approach as done with argsman, chainman, connman and others.
ACKs for top commit:
davidgumberg:
ACK [2d955ff](2d955ff006)
Tree-SHA512: 0beb7125312168a3df130c1793a1412ab423ef0f46023bfe2a121630c79df7e55d3d143fcf053bd09e2d96e9385a7a04594635da3e5c6be0c5d3a9cafbe3b631
Use DBParams struct to remove ArgsManager uses from txdb.
To reduce size of this commit, this moves references to gArgs variable out of
txdb.cpp to calling code in chainstate.cpp. But these moves are temporary. The
gArgs references in chainstate.cpp are moved out to calling code in init.cpp in
later commits.
This commit does not change behavior.
Add CoinsViewOptions struct to remove ArgsManager uses from txdb.
To reduce size of this commit, this moves references to gArgs variable out of
txdb.cpp to calling code in validation.cpp. But these moves are temporary. The
gArgs references in validation.cpp are moved out to calling code in init.cpp in
later commits.
This commit does not change behavior.
Add DBParams and DBOptions structs to remove ArgsManager uses from dbwrapper.
To reduce size of this commit, this moves references to gArgs variable out of
dbwrapper.cpp to calling code in txdb.cpp. But these moves are temporary. The
gArgs references in txdb.cpp are moved out to calling code in init.cpp in later
commits.
This commit does not change behavior.
it adds `Ensure{any}Banman` functions to avoid
code repetition and make it cleaner. Similar
approach as done with argsman, chainman, connman
and others.
4de02def84 qt: Persist Mask Values option (Andrew Chow)
Pull request description:
The mask values option is memory only. If a user has enabled this option, it's reasonable to expect that they would want to have it enabled on the next start.
ACKs for top commit:
RandyMcMillan:
tACK 4de02def84
jarolrod:
tACK 4de02def84
pablomartin4btc:
> tACK [4de02de](4de02def84)
john-moffett:
tACK 4de02def84
Tree-SHA512: 247deb78df4911516625bf8b25d752feb480ce30eb31335cf9baeb07b7c6c225fcc37d5c45de62d6e6895ec10c7eefabb15527e3c9723a3b8ddda1e12ebbf46b
faff2ba4f8 Remove reindex special case from the progress bar label (MarcoFalke)
Pull request description:
The user knows which option they passed to the program, so it seems overly verbose to offer the user feedback whether or not they passed `-reindex`. Treat it as `DISK`, like all other cases that are treated as `DISK`:
* `-reindex-chainstate`
* `-loadblock`
ACKs for top commit:
john-moffett:
Re-ACK faff2ba4f8
hebasto:
ACK faff2ba4f8, I have reviewed the code and it looks OK, I agree it can be merged.
Tree-SHA512: 7f110c4beb1451d26f32da3a60150dac91c8a7b8d1c01749017204712b73cc1b77578af492930e4b6704097a73ed051f77bc39d8f60e0ff15a797a201805312e
c9ba4f9ecb test: Add test for file system permissions (Hennadii Stepanov)
581f16ef34 Apply default umask in `SetupEnvironment()` (Hennadii Stepanov)
8a6219e543 Remove `-sysperms` option (Hennadii Stepanov)
Pull request description:
On master (1e7564eca8) docs say:
```
$ ./src/bitcoind -help | grep -A 3 sysperms
-sysperms
Create new files with system default permissions, instead of umask 077
(only effective with disabled wallet functionality)
```
Basing on that, one could expect that running `bitcoind` first time will create data directory and `wallets/` subdirectory with safe 0700 permissions.
But that is not the case:
```
$ stat .bitcoin | grep id
Access: (0775/drwxrwxr-x) Uid: ( 1000/ hebasto) Gid: ( 1000/ hebasto)
$ stat .bitcoin/wallets | grep id
Access: (0775/drwxrwxr-x) Uid: ( 1000/ hebasto) Gid: ( 1000/ hebasto)
```
Both directories, in fact, are created with system default permissions.
With this PR:
```
$ stat .bitcoin/wallets | grep id
Access: (0700/drwx------) Uid: ( 1000/ hebasto) Gid: ( 1000/ hebasto)
$ stat .bitcoin/wallets | grep id
Access: (0700/drwx------) Uid: ( 1000/ hebasto) Gid: ( 1000/ hebasto)
```
---
This PR:
- is alternative to bitcoin/bitcoin#13389
- fixesbitcoin/bitcoin#15902
- fixesbitcoin/bitcoin#22595
- closesbitcoin/bitcoin#13371
- reverts bitcoin/bitcoin#4286
Changes in behavior: removed `-sysperms` command-line argument / configure option. The related discussions are here:
- https://github.com/bitcoin/bitcoin/pull/13389#issuecomment-395306690
- https://github.com/bitcoin/bitcoin/pull/13389#issuecomment-539906114
- https://github.com/bitcoin/bitcoin/pull/13389#discussion_r279160472
If users rely on non-default access permissions, they could use `chmod`.
ACKs for top commit:
john-moffett:
ACK c9ba4f9ecb
willcl-ark:
ACK c9ba4f9ecb
Tree-SHA512: 96c745339e6bd0e4d7bf65daf9a721e2e1945b2b0ab74ca0f66576d0dc358b5de8eb8cdb89fe2160f3b19c39d2798bb8b291784316085dc73a27102d3415bd57
935acdcc79 refactor: modernize the implementation of uint256.* (pasta)
Pull request description:
- Constructors of uint256 to utilize Span instead of requiring a std::vector
- converts m_data into a std::array
- Prefers using `WIDTH` instead of `sizeof(m_data)`
- make all the things constexpr
- replace C style functions with c++ equivalents
- memset -> std::fill
This may also be replaced by std::memset, but I think that std::fill is more idiomatic of modern c++ and readable.
- memcpy -> std::copy
Note: In practice, implementations of std::copy avoid multiple assignments and use bulk copy functions such as std::memmove if the value type is TriviallyCopyable and the iterator types satisfy LegacyContiguousIterator. (https://en.cppreference.com/w/cpp/algorithm/copy)
This could also likely be replaced by std::memcpy, but as said above, I believe the using std::copy is the more c++ way to do anything and is almost guaranteed to compile to the same asm
- memcmp -> std::memcmp
ACKs for top commit:
achow101:
ACK 935acdcc79
hebasto:
Approach ACK 935acdcc79.
aureleoules:
reACK 935acdcc79
john-moffett:
ACK 935acdcc79
stickies-v:
Approach ACK 935acdcc7
Tree-SHA512: 4f1ba54ff2198eea0e505d41e73d552c84c60f6878d5c85a94a8ab57f39afc94ef8d79258e7afd01fa84ec2a99f4404bb877eecd671f65e1ee9273f3129fc650
b8032293e6 Remove use of snprintf and simplify (John Moffett)
Pull request description:
These are the only remaining uses of `snprintf` in our project, and they can cause unexpected issues -- for example, see https://github.com/bitcoin/bitcoin/issues/27014. Change them to use our `ToString` (which uses a locale-independent version of `std::to_string`) to convert an `int` to `std::string`. Also remove resulting unused parts of `StringContentsSerializer`.
Closes https://github.com/bitcoin/bitcoin/issues/27014
ACKs for top commit:
Sjors:
tACK b8032293e6, fixes#27014.
Tree-SHA512: c903977e654711929decafe8887d0de13b38a340d7082875acc5d41950d834dcfde074e9cabecaf5f9a760f62c34322297b4b156af29761650ef5803b1a54b59
82f895d7b5 Update nanobench to version v4.3.10 (Martin Leitner-Ankerl)
Pull request description:
Nothing has changed that would affect Bitcoin's usage of nanobench.
Here is a detailed list of the changes
* Plenty of clang-tidy updates
* documentation updates
* faster Rng::shuffle
* Enable perf counters on older kernels
* Raise default minimum epoch time to 1ms (doesn't effect bitcoin's usage)
* Add support for custom information per benchmark
ACKs for top commit:
hebasto:
ACK 82f895d7b5, I've reviewed the code, all related changes from #26642 have been implemented.
Tree-SHA512: 942518398809a2794617a347ab8182b784a8e822e84de5af078b2531eabb438412d687cac22a21936585e60e07138a89b41c28c9750744c05a3d1053f55cad01
fe683f3524 log: Log VerifyDB Progress over multiple lines (Martin Zumsande)
61431e3a57 validation: Skip VerifyDB checks of level >=3 if dbcache is too small (Martin Zumsande)
Pull request description:
This is the first two commits from #25574, leaving out all changes to `-verifychain` error-handling :
- The Problem of [25563](https://github.com/bitcoin/bitcoin/issues/25563) is that when we skip blocks at level 3 due to an insufficient dbcache (skipping some `DisconnectBlock()` calls), we would still attempt the level 4 checks, attempting to reconnect a block that was never disconnected, leading to an assert in `ConnectBlock()`.
Fix this by not attempting level 4 checks in this case.
- Logging of verification progress is now split over multiple lines. This is more verbose, but now each update has its own timestamp, and other threads logging concurrently will no longer lead to mangled output.
This can be tested with a small `dbcache` value, for example:
`bitcoind -signet -dbcache=10`
`bitcoin-cli -signet verifychain 4 1000`
Fixes#25563
ACKs for top commit:
MarcoFalke:
review ACK fe683f3524 🗄
john-moffett:
ACK fe683f3524
Tree-SHA512: 3e2e0f8b73cbc518a0fa17912c1956da437787aab95001c110b01048472e0dfe4783c44df22bd903d198069dd2f6b02bfdf74e0b934c7a776f144c2e86cb818a
Silently emitting an error makes it a bit harder to debug. Instead,
print a helpful log message to point the developer in the right
direction.
Alternatively this could have been implemented by just removing the
recipe echo suppression (@), but the subsequent make output became too
noisy.
When generating new files as part of the Makefile the recipe is
sometimes suppressed with $(AM_V_GEN) and sometimes with `@`. We should
prefer $(AM_V_GEN), since this also prints the lines in silent mode.
This is arguably more in style with the current recipe echoing.
Before:
Generated test/data/script_tests.json.h
Now:
GEN test/data/script_tests.json.h
A side effect of this change is that the recipe for generating build.h
is now echoed on each make run. Arguably this makes its generation more
transparent.
6699d850e4 doc: release notes for #27037 (Antoine Poinsot)
dfc9acbf01 rpc: decode Miniscript descriptor when possible in decodescript (Antoine Poinsot)
Pull request description:
The descriptor inference logic would previously always use a dummy signing provider and would never analyze the witness script of a P2WSH scriptPubKey.
It's often not possible to infer a Miniscript only from the onchain Script, but it was such a low hanging fruit that it's probably worth having it?
Fixes https://github.com/bitcoin/bitcoin/issues/27007. I think it also closes https://github.com/bitcoin/bitcoin/issues/25606.
ACKs for top commit:
instagibbs:
ACK 6699d850e4
achow101:
ACK 6699d850e4
sipa:
utACK 6699d850e4
Tree-SHA512: e592bf1ad45497e7bd58c26b33cd9d05bb3007f1e987bee773d26013c3824e1b394fe4903809d80997d5ba66616cc79d77850cd7e7f847a0efb2211c59466982
fdb8dc8a5a gui: Show watchonly balance only for Legacy wallets (Andrew Chow)
Pull request description:
Descriptor wallets do not have a watchonly balance as wallets are designated watchonly or not. Thus we should not be displaying the empty watchonly balance for descriptor wallets.
The result is that instead of the send page showing "Watch-only balance: 0.00000000 BTC" for watchonly descriptor wallets, we see the actual balance as "Balance: 10.00000000 BTC"
ACKs for top commit:
johnny9:
tACK fdb8dc8a5a
furszy:
ACK fdb8dc8a
hebasto:
ACK fdb8dc8a5a
Tree-SHA512: e5c0703a62d25c881c8dadfb9cffd482791f3d437a4ec5ae0088ce1a2069c2455ad6d3ec6c95a4404a3b55fbd727f92694529c35052236951553ca90c4ed31b5
c497a198db Fix comment about how wallet txs are sorted (John Moffett)
Pull request description:
The wallet transactions in the node are not sorted by txid (or any hash) since https://github.com/bitcoin/bitcoin/pull/24699.
This is how they're stored in memory now:
835212cd1d/src/wallet/wallet.h (L397-L399)
ACKs for top commit:
achow101:
ACK c497a198db
jarolrod:
ACK c497a198db
Tree-SHA512: e72559991688452ef254474d4235dc75fac655bce04909c3a0eece907360f4c6f57707db9b4373a4bd2271b23c57e863684c33e0728adf48e477c5499cdfdad7
One test case uses snprintf to convert an
int to a string. Change it to use ToString
(which uses a locale-independent version of
std::to_string). Also remove unnecessary
parts of StringContentsSerializer.
The descriptor inference logic would previously always use a dummy
signing provider and would never analyze the witness script of a P2WSH
scriptPubKey.
Note even a valid Miniscript might not always be decodable from Script
without more contextual information (for instance the key preimage for a
pk_h).
b093f5619f Fill out dust limit unit test for known types except bare multisig (Greg Sanders)
Pull request description:
Having the constants checked explicitly in a single spot helps with possible regressions and also useful for documentation.
In addition, add a check for undefined v1 witness programs.
ACKs for top commit:
theStack:
Code-review ACK b093f5619f
MarcoFalke:
review ACK b093f5619f🥉
Tree-SHA512: 1421f75471739d29b9ef59b0a925b6b07e4e9af92822dbe56eedfb590be9a00fb0c34312146c7c1b5211906461ed00bfa2eb53c88595c6e5a27694b2dc21df38
Nothing has changed that would affect Bitcoin's usage of nanobench. Here is a detailed list of the changes
* Plenty of clang-tidy updates
* documentation updates
* faster Rng::shuffle
* Enable perf counters on older kernels
* Raise default minimum epoch time to 1ms (doesn't effect bitcoin's usage)
* Add support for custom information per benchmark
fab9f7d1bd test: Use std::unique_ptr over manual delete in coins_tests (MarcoFalke)
Pull request description:
Makes the code smaller and easier to read
ACKs for top commit:
stickies-v:
ACK fab9f7d1bd
john-moffett:
ACK fab9f7d1bd
Tree-SHA512: 30d2d2097906e61fdef47a52fc6a0c5ce2417bc41c3c82dafc1b216c655f31dabf9c1c13759575a696f61bbdfdba3f442be032d5e5145b7a54fae2a927824621
576f7b8614 Fix misleading RPC console wallet message (John Moffett)
Pull request description:
## Misleading message from RPCConsole window ##
In certain circumstances, the GUI console will display the message 'Executing command without any wallet' when it is, in fact, using the currently loaded wallet. For instance:
In RPC calls, if no wallet is explicitly selected and there is exactly one wallet loaded, the [default](39363a4b94/src/wallet/rpc/util.cpp (L71-L93)) is to act on that loaded wallet.
The GUI console acts that way in reality, but sometimes erroneously reports that it's not acting on any particular wallet. The root issue is due to the logic that prevents changing the selected wallet if the RPCConsole is visible:
39363a4b94/src/qt/rpcconsole.cpp (L783-L786)
This PR removes that unnecessary logic. This does have some ramifications. Prior to this PR, if a user opened the console window without any wallets loaded, then opened two or more wallets, the RPC console would select "None" of the wallets and any wallet-specific RPCs would fail. However, the behavior was different if the user hadn't had the console window open. In that case, if they opened the RPC Console window _after_ loading at least the first wallet, it would select the first-loaded wallet. This context-dependent behavior is (IMO) undesirable, and this PR changes it to be consistent.
ACKs for top commit:
hebasto:
ACK 576f7b8614, tested on Ubuntu 22.04 (Qt 5.15.3).
Tree-SHA512: 627da186025ba4f4e8df7fdd1b10363f923c4ecc50f023bbf2aece6e2593da65c45147c933effaca9040f813a6e46f034fc2d1ee2fb0f401000a3a6221a0e36e
08209c039f Correctly limit overview transaction list (John Moffett)
Pull request description:
Fixes#703
The way the main overview page limits the number of transactions displayed (currently 5) is not an appropriate use of Qt. Our subclassed transaction sort/filter proxy model returns a maximum of `5` in `rowCount()`. However, the model itself actually may hold significantly more. While this has _worked_, it breaks the contract of `rowCount()`.
If `bitcoin-qt` is run with a DEBUG build of Qt, it'll result in an assert-crash in certain relatively common situations (see #703 for details). Instead of artificially limiting the `rowCount()` in the subclassed filter, we can hide/unhide the rows in the displaying `QListView` upon any changes in the sorted proxy filter.
I loaded a wallet with 20,000 transactions and did not notice any performance differences between master and this branch.
For reference, this is the list I'm referring to:
<img width="934" alt="image" src="https://user-images.githubusercontent.com/116917595/214947304-3f289380-3510-487b-80e8-d19428cf2f0f.png">
ACKs for top commit:
Sjors:
tACK 08209c039f
hebasto:
ACK 08209c039f, tested on Ubuntu 22.04.
Tree-SHA512: c2a7b1a2a6e6ff30694830d7c722274c4c47494a81ce9ef25f8e5587c24871b02343969f4437507693d4fd40ba7a212702b159cf54b3357d8d76c02bc8245113
fa47b28dfc refactor: Remove unused CDataStream SerializeMany constructor (MarcoFalke)
Pull request description:
Seems odd to have an unused method. Moreover, the function is fragile and dangerous, because one could have a `std::vector vec_a` and type `CDataStream{vec_a, 0, 0}.size()` and `CDataStream{0, 0, vec_a}.size()`, assuming they are the same thing, when in fact they are not. (The first takes over the memory as is, the second serializes the vector).
So my suggestion would be to remove the unused method and introduce a new method when this functionality is needed. For example: `static DataStream FromMany(Args&&... args)`.
ACKs for top commit:
stickies-v:
ACK fa47b28dfc
Tree-SHA512: 9593a034b997e33a0794f779f76f02425b1097b218cf8cb1facb7f874fa69da328ce567a79138015baeebe004ae7d103dda4f64f83e8ad375b6dae6b66d3d950
fad7af700e Use steady clock for logging timer (MarcoFalke)
Pull request description:
The logging timer has many issues:
* The underlying clock is mockable, meaning that benchmarks are useless when mocktime was set at the beginning or end of the benchmark.
* The underlying clock is not monotonic, meaning that benchmarks are useless when the system time was changed during the benchmark.
Fix all issues in this patch.
ACKs for top commit:
stickies-v:
Approach ACK fad7af700e
john-moffett:
ACK fad7af700e
Tree-SHA512: bec8da0f338ed4611e1807937575e1b2afda25139d88015b1c29fa7d13946fbfbc4ee589b576c0508d505df5e5fafafcbc07d63ce4bab4b01475260d9d5d2107
The fuzzer goes through a sequence of operations that get applied to both a
real stack of CCoinsViewCache objects, and to simulation data, comparing
the two at the end.
6d31900e52 wallet: migrate wallet, exit early if no legacy data exist (furszy)
Pull request description:
The process first creates a backup file then return an error,
without removing the recently created file, when notices that
the db is already running sqlite.
ACKs for top commit:
john-moffett:
ACK 6d31900e52
achow101:
ACK 6d31900e52
ishaanam:
crACK 6d31900e52
Tree-SHA512: 9fb52e80de96e129487ab91bef13647bc4570a782003b1e37940e2a00ca26283fd24ad39bdb63a984ae0a56140b518fd0d74aa2fc59ab04405b2c179b7d3c54a
dc70c1eb08 addrman: Use std::nullopt instead of {} (Martin Zumsande)
59cc66abb9 test: Remove AddrMan unit test that fails consistency checks (Martin Zumsande)
Pull request description:
Two fixups for #26847:
* Now that `AddrMan::Size()` performs internal consistency tests (it didn't before), we can't call it in the `load_addrman_corrupted` unit tests, where we deal with an artificially corrupted `AddrMan`. This would fail the test when using `-checkaddrman=1` (leading to spurious CI fails). Therefore remove the tests assertion, which is not particularly helpful anyway (in production we abort init when peers.dat is corrupted instead of querying AddrMan in its corrupted state).
(See https://github.com/bitcoin/bitcoin/pull/26847#issuecomment-1411458339)
* Use `std::nullopt` instead of `{}` for default args (suggested in https://github.com/bitcoin/bitcoin/pull/26847#discussion_r1090643603)
ACKs for top commit:
MarcoFalke:
lgtm ACK dc70c1eb08
Tree-SHA512: dd8a988e23d71a66d3dd30560bb653c9ad17db6915abfa5f722818b0ab18921051ec9223bfbc75d967df8bcd204dfe473d680bf68e8a8e4e4998fbb91dc973c5
fa451d4b60 Fix clang-tidy readability-const-return-type violations (MarcoFalke)
Pull request description:
This comes up during review, so instead of wasting review cycles on this, just enforce it via CI
ACKs for top commit:
stickies-v:
utACK fa451d4b6
hebasto:
ACK fa451d4b60.
Tree-SHA512: 144a85612f00ec43f7ea1fdaa11901ca981a9f0465a8849745712d741b201b9c3307118172ee0b8efd12bebf25bc6f32a6e2c908495e371f9ada0a917994f44e
Now that Size() performs internal consistency checks,
it will rightfully fail (and assert) when dealing with
a corrupted AddrMan. Therefore remove this check.
87f11ef47f refactor: use `Hash` helper for double-SHA256 calculations (Sebastian Falbesoner)
Pull request description:
We have two helper templates `Hash(const T& in1)` and `Hash(const T& in1, const T& in2)` available for calculating the double-SHA256 hash of one object or two concatenated objects, respectively:
b5868f4b1f/src/hash.h (L74-L89)
This PR uses them in order to increase readability and simplify the code. As in #15294 (which inspired this PR, doing the same for RIPEMD160), the helper is not utilized in validation.cpp and script/interpreter.cpp to avoid touching consensus-relevant code.
ACKs for top commit:
john-moffett:
ACK 87f11ef47f
stickies-v:
ACK 87f11ef47f
MarcoFalke:
review ACK 87f11ef47f😬
Tree-SHA512: 11d7e3d00c89685107784010fbffb33ccafb4d1b6a76c4dceb937b29bb234ef4d54581b16bd0737c8d2994a90cf4fe10a9738c7cc5b6d085c6a819f06176dab9
7a820cee0e test, build: Separate `read_json` function into its own module (Hennadii Stepanov)
Pull request description:
Currently, 4 source files rely on the definition of the `read_json` function provided in `src/test/script_tests.cpp`.
This PR breaks this entanglement, improves code structure and maintainability.
ACKs for top commit:
fanquake:
ACK 7a820cee0e
Tree-SHA512: f1567989f76cb54ab86cc48927851a8c424b08a9483d02d4918b629e0c792108bad4ccf7fa341d57b0921d91e84bf8fa3b9c07e5fdf12c64d9d5da83e4e464fb
b0e916913c clang-tidy: Force to check all headers (Hennadii Stepanov)
96ee992ac3 clang-tidy: Fix `modernize-use-default-member-init` in headers (Hennadii Stepanov)
Pull request description:
This PR:
- fixes the only [remained](https://github.com/bitcoin/bitcoin/pull/26705#issuecomment-1353742082) check in headers, i.e., `modernize-use-default-member-init`
- forces `clang-tidy` check all headers
Closesbitcoin/bitcoin#26703.
ACKs for top commit:
MarcoFalke:
review ACK b0e916913c🍹
Tree-SHA512: 4d33fe873094914541ae81968cdb4e7a7a01b3fdd4f25bc6daa8a53f45dab80565a5b3607ddc338f122369ca5a0a2d0d09c8e78cabe1beb6bd50c115bc5c5210
dfc01ccd73 net: simplify the call to vProcessMsg.splice() (Vasil Dimov)
Pull request description:
At the time when
```cpp
pnode->vProcessMsg.splice(pnode->vProcessMsg.end(), pnode->vRecvMsg, pnode->vRecvMsg.begin(), it);
```
is called, `it` is certainly `pnode->vRecvMsg.end()` which makes the call equivalent to:
```cpp
pnode->vProcessMsg.splice(pnode->vProcessMsg.end(), pnode->vRecvMsg, pnode->vRecvMsg.begin(), pnode->vRecvMsg.end());
```
which is equivalent to:
```cpp
pnode->vProcessMsg.splice(pnode->vProcessMsg.end(), pnode->vRecvMsg);
```
Thus, use the latter. Further, maybe irrelevant, but the latter has constant complexity while the original code is `O(length of vRecvMsg)`.
ACKs for top commit:
theStack:
Code-review ACK dfc01ccd73
MarcoFalke:
review ACK dfc01ccd73🐑
jonatack:
Light review ACK dfc01ccd73
Tree-SHA512: 9f4eb61d1caf4af9a61ba2f54b915fcfe406db62c58ab1ec42f736505b6792e9379a83d0458d6cc04f289edcec070b7c962f94a920ab51701c3cab103152866f
80f39c99ef addrman, refactor: combine two size functions (Amiti Uttarwar)
4885d6f197 addrman, refactor: move count increment into Create() (Martin Zumsande)
c77c877a8e net: Load fixed seeds from reachable networks for which we don't have addresses (Martin Zumsande)
d35595a78a addrman: add function to return size by network and table (Martin Zumsande)
Pull request description:
AddrMan currently doesn't track the number of its entries by network, it only knows the total number of addresses. This PR makes AddrMan keep track of these numbers, which would be helpful for multiple things:
1. Allow to specifically add fixed seeds to AddrMan of networks where we don't have any addresses yet - even if AddrMan as a whole is not empty (partly fixing #26035). This is in particular helpful if the user abruptly changes `-onlynet` settings (such that addrs that used to be reachable are no longer and vice versa), in which case they currently could get stuck and not find any outbound peers. The second commit of this PR implements this.
1. (Future work): Add logic for automatic connection management with respect to networks - such as making attempts to have at least one connection to each reachable network as suggested [here](https://github.com/bitcoin/bitcoin/issues/26035#issuecomment-1249420209). This would involve requesting an address from a particular network from AddrMan, and expanding its corresponding function `AddrMan::Select()` to do this requires internal knowledge of the current number of addresses for each network and table to avoid getting stuck in endless loops.
1. (Future work): Perhaps display the totals to users. At least I would find this helpful to debug, the existing option (`./bitcoin-cli -addrinfo`) is rather indirect by doing the aggregation itself in each call, doesn't distinguish between new and tried, and being based on `AddrMan::GetAddr()` it's also subject to a quality filter which we probably don't want in this spot.
ACKs for top commit:
naumenkogs:
utACK 80f39c9
stratospher:
ACK 80f39c9
achow101:
ACK 80f39c99ef
vasild:
ACK 80f39c99ef
Tree-SHA512: 6359f2e3f4db7c120c0789d92d74cb7d87a2ceedb7d6a34b5eff20c7f55c5c81092d10ed94efe29afc1c66947820a0d9c14876ee0c8d1f8e068a6df4e1131927
6d58117a31 build: Build minisketch test in `make check`, not in `make` (Hennadii Stepanov)
Pull request description:
On master (d1e42659bb):
```
$ ./autogen.sh && ./configure --without-gui --disable-wallet && make clean
$ make 2>&1 | grep LD | grep -v .la
CXXLD bitcoind
CXXLD bitcoin-cli
CXXLD bitcoin-tx
CXXLD bitcoin-util
CXXLD test/test_bitcoin
CXXLD bench/bench_bitcoin
CXXLD minisketch/test
CXXLD test/fuzz/fuzz
CXXLD univalue/test/object
CXXLD univalue/test/unitester
$ make check 2>&1 | grep LD
CCLD exhaustive_tests
CCLD tests
```
With this PR:
```
$ ./autogen.sh && ./configure --without-gui --disable-wallet && make clean
$ make 2>&1 | grep LD | grep -v .la
CXXLD bitcoind
CXXLD bitcoin-cli
CXXLD bitcoin-tx
CXXLD bitcoin-util
CXXLD test/test_bitcoin
CXXLD bench/bench_bitcoin
CXXLD test/fuzz/fuzz
CXXLD univalue/test/object
CXXLD univalue/test/unitester
$ make check 2>&1 | grep LD
CXXLD minisketch/test
CCLD exhaustive_tests
CCLD tests
```
In fact, this PR restores behavior that was before bitcoin/bitcoin#22646, and that behavior looks more optimal.
As an outcome, the `contrib/guix/libexec/build.sh` does not spend resources to build binaries which are not a part of the release package.
ACKs for top commit:
TheCharlatan:
ACK 6d58117a31
Tree-SHA512: 4957c8f88a01aca005813bf4c1c26f433756bf68ea0c958481c638ead229fa8e23ecae3a8ac31ea555876ba6f2cc10ecd91caf2e2f664de5cb529ec05fb38fa7
a24e633339 refactor: rpc: set TxToJSON default verbosity to SHOW_DETAILS (stickies-v)
Pull request description:
`TxToJSON()` and `TxToUniv()` are only to be called when we want to decode the transaction (i.e. its details) into JSON. If `TxVerbosity` is `SHOW_TXID`, the function should not have been (and currently is not) called in the first place.
There is no behaviour change, current logic simply assumes anything less than `TxVerbosity::SHOW_DETAILS_AND_PREVOUT` equals `TxVerbosity::SHOW_DETAILS`. With this change, the assumptions and intent become more explicit.
ACKs for top commit:
w0xlt:
ACK a24e633339
Tree-SHA512: b97235adae49b972bdbe10aca1438643fb35ec66a4e57166b1975b3015bc5a06a711feebe4453a8fefe71781e484b21ef80847d8e8a33694a3abcc863accd4d7
The previous behavior, skipping some L3 DisconnectBlock calls,
but still attempting to reconnect these blocks at L4, makes
ConnectBlock assert.
The variable skipped_l3_checks is introduced because even with an
insufficient cache for the L3 checks, the L1/L2 checks in the same
loop should still be completed.
Fixes#25563.
Xoroshiro128++ is a fast non-cryptographic random generator.
Reference implementation is available at https://prng.di.unimi.it/
Co-Authored-By: Pieter Wuille <pieter@wuille.net>
faba08b5b4 refactor: Remove stray cs_main redundant declaration (MarcoFalke)
fa02591edf doc: Export threadsafety.h from sync.h (MarcoFalke)
Pull request description:
Looks like this was forgotten when introducing kernel/cs_main ?
Also, there is a commit to export threadsafety.h from sync.h.
ACKs for top commit:
hebasto:
ACK faba08b5b4
Tree-SHA512: 0aa58e7693b6fcd504f9da7339f8baa463a6407f67b27f68002db705f4642321ac3765f16c3d906c925ee24085591b79160a62fa5f4aaf6f2e5dcc788411800d
1d7935b45a test: add test for coins view flush behavior using Sync() (James O'Beirne)
2c3cbd6c00 test: add use of Sync() to coins tests (James O'Beirne)
6d8affca96 test: refactor: clarify the coins simulation (James O'Beirne)
79cedc36af coins: add Sync() method to allow flush without cacheCoins drop (James O'Beirne)
Pull request description:
This is part of the [assumeutxo project](https://github.com/bitcoin/bitcoin/projects/11):
Parent PR: #15606
Issue: #15605
Specification: https://github.com/jamesob/assumeutxo-docs/tree/master/proposal
---
In certain circumstances, we may want to flush chainstate data to disk without
emptying `cacheCoins`, which affects performance. UTXO snapshot
activation is one such case, as we populate `cacheCoins` with the snapshot
contents and want to persist immediately afterwards but also enter IBD.
See also #15265, which makes the case that under normal operation a
flush-without-erase doesn't necessarily add much benefit. I open this PR
even in light of the previous discussion because (i) flush-without-erase
almost certainly provides benefit in the case of snapshot activation (especially
on spinning disk hardware) and (ii) this diff is fairly small and gives us convenient
options for more granular cache management without changing existing policy.
See also #15218.
ACKs for top commit:
sipa:
ACK 1d7935b45a
achow101:
ACK 1d7935b45a
Sjors:
tACK 1d7935b45a
Tree-SHA512: 897583963e98661767d2d09c9a22f6019da24125558cd88770bfe2d017d924f23a9075b729e4b1febdec5b0709a38e8fa1ef94d62aa88650556b06cb4826c845
eeee61065f Use AutoFile and HashVerifier where possible (MarcoFalke)
fa961141f7 Add HashVerifier (MarcoFalke)
Pull request description:
This was done in the context of https://github.com/bitcoin/bitcoin/pull/25284 , but I think it also makes sense standalone.
The basic idea is that serialization type should not be initialized when it is not needed. Same for the serialization version.
So do this here for `AutoFile` and `HashVerifier`. `CAutoFile` and `CHashVerifier` remain in places where it is not yet possible.
ACKs for top commit:
stickies-v:
ACK eeee61065f
Tree-SHA512: 93786778c309ecfdc1ed43552d24ff9d966954d69a47f66faaa6de24daacd25c651f3f62bde5abbb362700298fb3c04ffbd3207a0dd13d0bd8bff7fd6d07dcf8
d51f0fa4b7 doc: add release notes for 26896 (fanquake)
2b248798d9 build: remove --enable-upnp-default from configure (fanquake)
02f5a5e7b5 build: remove --enable-natpmp-default from configure (fanquake)
25a0e8ba0b Remove configure-time setting of DEFAULT_UPNP (fanquake)
06562e5fa7 Remove configure-time setting of DEFAULT_NATPMP (fanquake)
Pull request description:
This PR removes the `--enable-upnp-default` and `--enable-natpmp-default` options from configure.
It's odd to me that we maintain configure-time options for setting the default port-forwarding runtime state (but no other similar options), and I'm not sure what use-case it satisfies, that can't be achieved by multiple other means. I also doubt that we'll ever restart using these in release builds, or turning on any of this by default.
I think the only scenario these options would be used is when you want to compile your own binaries (we don't use them in Guix), with port-forwarding on by default, but otherwise can't or don't want to use a `.conf` file, can't or don't want to pass command line options at runtime, and also don't want to modify the source code?
ACKs for top commit:
hebasto:
ACK d51f0fa4b7, rebased and comments have been addressed since my recent [review](https://github.com/bitcoin/bitcoin/pull/26896#pullrequestreview-1273910740).
TheCharlatan:
ACK d51f0fa4b7
Tree-SHA512: 481decd8bddd8b03b7319591e3acf189f7b6b96c9a9a8c5bc1a3f8ec00d0b8f9b52d2f5c28a298a2ec947cfe9611cfd184e393ccb2e4e21bfce86ca7d4de60d3
At the time when
```cpp
pnode->vProcessMsg.splice(pnode->vProcessMsg.end(), pnode->vRecvMsg, pnode->vRecvMsg.begin(), it);
```
is called, `it` is certainly `pnode->vRecvMsg.end()` which makes the
call equivalent to:
```cpp
pnode->vProcessMsg.splice(pnode->vProcessMsg.end(), pnode->vRecvMsg, pnode->vRecvMsg.begin(), pnode->vRecvMsg.end());
```
which is equivalent to:
```cpp
pnode->vProcessMsg.splice(pnode->vProcessMsg.end(), pnode->vRecvMsg);
```
Thus, use the latter. Further, maybe irrelevant, but the latter has
constant complexity while the original code is `O(length of vRecvMsg)`.
b0fa5989e1 test: Check that orphaned coinbase unconf spend is still abandoned (Andrew Chow)
9addbd7890 wallet: Automatically abandon orphaned coinbases and their children (Andrew Chow)
Pull request description:
When a block is reorged out of the main chain, any descendants of the coinbase will no longer be valid. Currently they are only marked as inactive, which means that our balance calculations will still include them. In order to be excluded from the balance calculation, they need to either be abandoned or conflicted. This PR goes with the abandoned method.
Note that even when they are included in balance calculations, coin selection will not select outputs belonging to these transactions because they are not in the mempool.
Fixes#14148
ACKs for top commit:
furszy:
ACK b0fa5989 with a not-blocking nit.
aureleoules:
reACK b0fa5989e1
ishaanam:
ACK b0fa5989e1
Tree-SHA512: 68f12e7aa8df392d8817dc6ac0becce8dbe83837bfa538f47027e6730e5b2e1b1a090cfcea2dc598398fdb66090e02d321d799f087020d7e1badcf96e598c3ac
b2a1e47744 net_processing: simplify logging statement (Martin Zumsande)
6548ba68e8 test: fix intermittent errors in p2p_ibd_stalling.py (Martin Zumsande)
Pull request description:
Two small fixups to #25880:
- Use `is_connected` instead of `num_test_p2p_connections` to avoid intermittent failures where the p2p MiniNode got disconnected but this info hasn't made it to python yet, so it fails a ping. (https://github.com/bitcoin/bitcoin/pull/25880#discussion_r1089217720)
- Simplify a logging statement (suggested in https://github.com/bitcoin/bitcoin/pull/25880#discussion_r1013738635)
ACKs for top commit:
MarcoFalke:
review ACK b2a1e47744🕧
Tree-SHA512: 337f0883bf1c94cc26301a80dfa649093ed1e211ddda1acad8449a2add5be44e5c12d6073c209df9c7aa1edb9da33ec1cfdcb0deafd76178ed78785843e80bc7
faf7b4f1fc Add BlockManager::IsPruneMode() (MarcoFalke)
fae71fe27e Add BlockManager::GetPruneTarget() (MarcoFalke)
fa0f0436d8 Add BlockManager::LoadingBlocks() (MarcoFalke)
Pull request description:
Requested in https://github.com/bitcoin/bitcoin/pull/25781#discussion_r1061323795, but adding getters seems unrelated from removing globals, so I split it out for now.
ACKs for top commit:
dergoegge:
Code review ACK faf7b4f1fc
brunoerg:
crACK faf7b4f1fc
Tree-SHA512: 204d0e9a0e8b78175482f89b4ce620fba0e65d8e49ad845d187af44d3843f4c733a01bac1ffe5a5319f524d8346123693a456778b69d6c75268c447eb8839642
39b93649c4 test: add functional test for IBD stalling logic (Martin Zumsande)
0565951f34 p2p: Make block stalling timeout adaptive (Martin Zumsande)
Pull request description:
During IBD, there is the following stalling mechanism if we can't proceed with assigning blocks from a 1024 lookahead window because all of these blocks are either already downloaded or in-flight: We'll mark the peer from which we expect the current block that would allow us to advance our tip (and thereby move the 1024 window ahead) as a possible staller. We then give this peer 2 more seconds to deliver a block (`BLOCK_STALLING_TIMEOUT`) and if it doesn't, disconnect it and assign the critical block we need to another peer.
Now the problem is that this second peer is immediately marked as a potential staller using the same mechanism and given 2 seconds as well - if our own connection is so slow that it simply takes us more than 2 seconds to download this block, that peer will also be disconnected (and so on...), leading to repeated disconnections and no progress in IBD. This has been described in #9213, and I have observed this when doing IBD on slower connections or with Tor - sometimes there would be several minutes without progress, where all we did was disconnect peers and find new ones.
The `2s` stalling timeout was introduced in #4468, when blocks weren't full and before Segwit increased the maximum possible physical size of blocks - so I think it made a lot of sense back then.
But it would be good to revisit this timeout now.
This PR makes the timout adaptive (idea by sipa):
If we disconnect a peer for stalling, we now double the timeout for the next peer (up to a maximum of 64s). If we connect a block, we half it again up to the old value of 2 seconds. That way, peers that are comparatively slower will still get disconnected, but long phases of disconnecting all peers shouldn't happen anymore.
Fixes#9213
ACKs for top commit:
achow101:
ACK 39b93649c4
RandyMcMillan:
Strong Concept ACK 39b93649c4
vasild:
ACK 39b93649c4
naumenkogs:
ACK 39b93649c4
Tree-SHA512: 85bc57093b2fb1d28d7409ed8df5a91543909405907bc129de7c6285d0810dd79bc05219e4d5aefcb55c85512b0ad5bed43a4114a17e46c35b9a3f9a983d5754
The functionality of the old size() is covered by the new Size()
when no arguments are specified, so this does not change behavior.
Co-authored-by: Martin Zumsande <mzumsande@gmail.com>
Previously, we'd only load fixed seeds if we'd not
know any addresses at all. This change makes it possible
to change -onlynet abruptly, e.g. from -onlynet=onion to
-onlynet=i2p and still find peers.
For now, the new functionality will be used in the context of
querying fixed seeds. Other possible applications for
future changes is the use in the context of making automatic
connections to specific networks, or making more detailed info
about addrman accessible via rpc.
To directly return a CRIPEMD160 hash from data.
Incidentally, decoding this acronym:
* RIPEMD -> RIPE Message Digest
* RIPE -> RACE Integrity Primitives Evaluation
* RACE -> Research and Development in Advanced Communications Technologies in Europe
The way that the main overview page limits the number
of transactions displayed (currently 5) is not
an appropriate use of Qt. If it's run with a DEBUG
build of Qt, it'll result in a segfault in certain
relatively common situations. Instead of artificially
limiting the rowCount() in the subclassed proxy
filter, we hide/unhide the rows in the displaying
QListView upon any changes in the sorted proxy filter.
`TxToJSON()` and `TxToUniv()` are only to be called when we want to
decode the transaction (i.e. its details) into JSON. If `TxVerbosity` is
`SHOW_TXID`, the function should not have been (and currently is not)
called in the first place.
There is no behaviour change, current logic simply assumes anything less
than `TxVerbosity::SHOW_DETAILS_AND_PREVOUT` equals
`TxVerbosity::SHOW_DETAILS`. With this change, the assumptions and
intent become more explicit.
fa035fe2d6 Remove unused CDataStream::SetType (MarcoFalke)
fa29e73cda Use DataStream where possible (MarcoFalke)
fa9becfe1c streams: Add DataStream without ser-type and ser-version (MarcoFalke)
Pull request description:
This was done in the context of https://github.com/bitcoin/bitcoin/pull/25284 , but I think it also makes sense standalone.
The basic idea is that serialization type should not be initialized when it is not needed. Same for the serialization version.
So do this here for `DataStream`. `CDataStream` remains in places where it is not yet possible.
ACKs for top commit:
stickies-v:
re-ACK [fa035fe](fa035fe2d6)
aureleoules:
diff re-ACK fa035fe2d6fa0e6640ba..fa035fe2d6
Tree-SHA512: cb5e53d0df7c94319ffadc6ea1d887fc38516decaf43f0673396d79cc62d450a1a61173654a91b8c2b52d2cecea53fe4a500b8f6466596f35731471163fb051c
c58c249a5b net_processing: indicate more work to do when orphans are ready to reconsider (Anthony Towns)
ecb0a3e425 net_processing: Don't process tx after processing orphans (Anthony Towns)
c583775706 net_processing: only process orphans before messages (Anthony Towns)
be2304676b txorphange: Drop redundant originator arg from GetTxToReconsider (Anthony Towns)
a4fe09973a txorphanage: index workset by originating peer (Anthony Towns)
Pull request description:
We currently process orphans by assigning them to the peer that provided a missing parent; instead assign them to the peer that provided the orphan in the first place. This prevents a peer from being able to marginally delay another peer's transactions and also simplifies the internal API slightly. Because we're now associating orphan processing with the peer that provided the orphan originally, we no longer process orphans immediately after receiving the parent, but defer until a future call to `ProcessMessage`.
Based on #26295
ACKs for top commit:
naumenkogs:
utACK c58c249a5b
glozow:
ACK c58c249a5b
mzumsande:
Code Review ACK c58c249a5b
Tree-SHA512: 3186c346f21e60440266a2a80a9d23d7b96071414e14b2b3bfe50457c04c18b1eab109c3d8c2a7726a6b10a2eda1f0512510a52c102da112820a26f5d96f12de
fab958290b refactor: Remove c_str from util/check (MarcoFalke)
Pull request description:
Seems confusing and fragile to require calling code to call `c_str()` when passing a read-only view of a std::string.
Fix that by using std::string_view, which can be constructed from string literals and std::string.
Also, remove the now unused `c_str()` from `src/wallet/bdb.cpp`.
ACKs for top commit:
stickies-v:
ACK fab958290b
aureleoules:
ACK fab958290b
theStack:
ACK fab958290b
Tree-SHA512: ae39812c6bb8e2ef095e1b843774af2718f48404cb848c3e43b16d3c22240557d69d54a13a038a4a9c48b3ba0e4523e1f87abdd60f91486092f50fd43c0e8483
fafeddfe0e rpc: Throw more user friendly arg type check error (MarcoFalke)
Pull request description:
The arg type check error doesn't list which arg (position or name) failed. Fix that.
ACKs for top commit:
stickies-v:
ACK fafeddfe0e - although I think the functional test isn't in a logical place (but not blocking)
Tree-SHA512: 17425aa145aab5045940ec74fff28f0e3b2b17ae55f91c4bb5cbcdff0ef13732f8e31621d85964dc2c04333ea37dbe528296ac61be27541384b44e37957555c8
b9d5674541 init: Remove sensitive flag from rpcbind (Andrew Chow)
Pull request description:
`-rpcbind` is currently flagged as a sensitive option which means that its value will be masked when the command line args are written to the debug.log file. However this is not useful as if `rpcbind` is actually activated, the bound IP addresses will be written to the log anyways. The test `feature_config_args.py` did not catch this contradiction as the test node was not started with `rpcallowip` and so `rpcbind` was not acted upon.
This also brings `rpcbind` inline with `bind` as that is not flagged as sensitive either.
ACKs for top commit:
Sjors:
re-utACK b9d5674541
willcl-ark:
ACK b9d5674
theStack:
ACK b9d5674541
Tree-SHA512: 50ab5ad2e18ae70649deb1ac429d404b5f5c41f32a4943b2041480580152df22e72d4aae493379d0b23fcb649ab342376a82119760fbf6dfdcda659ffd3e244a
When PR#15644 made orphan processing interruptible, it also introduced a
potential 100ms delay between processing of the first and second newly
reconsiderable orphan, because it didn't check if the orphan work set
was non-empty after invoking ProcessMessage(). This adds that check, so
that ProcessMessages() will return true if there are orphans to process,
usually avoiding the 100ms delay in CConnman::ThreadMessageHandler().
If we made progress on orphans, consider that enough work for this peer
for this round of ProcessMessages. This also allows cleaning up the api
for TxOrphange:GetTxToReconsider().
Previously, when we processed a new tx we would attempt to ATMP any
orphans that considered the new tx a parent immediately, but would only
accept at most one such tx, leaving any others to be considered on a
future run of ProcessMessages(). With this patch, we don't attempt any
orphan processing immediately after receiving a tx, instead deferring
all of them until the next call to ProcessMessages().
9ab62d71fb [fuzz] Actually use mocked mempool in tx_pool target (dergoegge)
Pull request description:
The current tx_pool target uses the default mempool, making the target non-deterministic. This PR replaces the active chainstate's mempool (i.e. the node's default mempool) with the already present mocked mempool in the target.
ACKs for top commit:
fanquake:
ACK 9ab62d71fb
Tree-SHA512: fe9af3dbdd13cb569fdc2ddbb4290b5ce94206ae83d94267c6365ed0ee9bbe072fcfe7fd632a1a8522dce44608e89aba2f398c1e20bd250484bbadb78143320c
a1c36275b5 [fuzz] Assert that omitting missing transactions always fails block reconstruction (dergoegge)
a8ac61ab5e [fuzz] Add PartiallyDownloadedBlock target (dergoegge)
42bd4c7468 [block encodings] Avoid fuzz blocking asserts in PartiallyDownloadedBlock (dergoegge)
1429f83770 [block encodings] Make CheckBlock mockable for PartiallyDownloadedBlock (dergoegge)
Pull request description:
This PR adds a fuzz target for `PartiallyDownloadedBlock`, which we currently do not have any coverage for.
ACKs for top commit:
mzumsande:
Code Review ACK a1c36275b5
MarcoFalke:
re-ACK a1c36275b5🎼
Tree-SHA512: 01ae452fe457da0c8f2b28c72091d40807c56a9e5d0f80b55f166b67be50baf80a02f53d4cbe9736bb22424cca1758b87e4e471b8a24e756c22563a2640e9a5f
The mask values option is memory only. If a user has enabled this
option, it's reasonable to expect that they would want to have it
enabled on the next start.
4aebd832a4 db: Change DatabaseCursor::Next to return status enum (Andrew Chow)
d79e8dcf29 wallet: Have cursor users use DatabaseCursor directly (Andrew Chow)
7a198bba0a wallet: Introduce DatabaseCursor RAII class for managing cursor (Andrew Chow)
69efbc011b Move SafeDbt out of BerkeleyBatch (Andrew Chow)
Pull request description:
Instead of having database cursors be tied to a particular `DatabaseBatch` object and requiring its setup and teardown be separate functions in that batch, we can have cursors be separate RAII classes. This makes it easier to create and destroy cursors as well as having cursors that have slightly different behaviors.
Additionally, since reading data from a cursor is a tri-state, this PR changes the return value of the `Next` function (formerly `ReadAtCursor`) to return an Enum rather than the current system of 2 booleans. This greatly simplifies and unifies the code that deals with cursors as now there is no confusion as to what the function returns when there are no records left to be read.
Extracted from #24914
ACKs for top commit:
furszy:
diff ACK 4aebd83
theStack:
Code-review ACK 4aebd832a4
Tree-SHA512: 5d0be56a18de5b08c777dd5a73ba5a6ef1e696fdb07d1dca952a88ded07887b7c5c04342f9a76feb2f6fe24a45dc31f094f1f5d9500e6bdf4a44f4edb66dcaa1
b358bde020 randomenv: consolidate WIN32 #ifdefs (fanquake)
fff80cd248 random: remove windows-only compat.h include in randomenv (fanquake)
Pull request description:
Similar to #26814.
Having a windows-only include of compat.h is confusing, not-only because it's already included globally via util/time.h, but also because it's unclear why compat.h is included (neither of the required headers are included there).
This change is related to removing the use of compat.h as a miscellaneous catch-all for unclear/platform specific includes. Somewhat prompted by IWYU-related discussion here: https://github.com/bitcoin/bitcoin/pull/26763/files#r1058861693.
ACKs for top commit:
hebasto:
ACK b358bde020.
TheCharlatan:
ACK b358bde020
Tree-SHA512: d46dffe36a17ad0f9374a55e0ecaf2d60d0b473c8fc9ad6f3005142014c08a7c10bae4948856531abb443f5e0bd6062958fe574197e282dad22ae50134d71e5f
-BEGIN VERIFY SCRIPT-
sed -i -e "/Deprecated alias for OMITTED, can be removed/d" src/rpc/util.h src/rpc/util.cpp
sed -i -e "s/OMITTED_NAMED_ARG/OMITTED/g" $(git grep -l "OMITTED_NAMED_ARG" src/)
-END VERIFY SCRIPT-
8e85164e7d doc: release note on mempool size in -blocksonly (willcl-ark)
ae797463dc doc: Update blocksonly behaviour in reduce-memory (willcl-ark)
1134686ef9 mempool: Don't share mempool with dbcache in blocksonly (willcl-ark)
Pull request description:
Fixes#9526
When `-blocksonly` has been set reduce default mempool size to avoid surprising resource usage via sharing un-used mempool cache space with dbcache.
In comparison to https://github.com/bitcoin/bitcoin/pull/9569 which either set `maxmempool` size to 0 when `-blocksonly` was set or else errored on startup, this change will permit `maxmempool` options being set.
This preserves the current (surprising?) behaviour of having a functional mempool in `-blocksonly` mode, to permit whitelisted peer transaction relay, whilst reducing average runtime memory usage for blocksonly nodes which either use the default settings or have otherwise configured a `maxmempool` size.
To use the previous old defaults node operators can configure their node with: `-blocksonly -maxmempool=300`.
ACKs for top commit:
ajtowns:
ACK 8e85164e7d
stickies-v:
re-ACK 8e85164e7d
Tree-SHA512: 1c461c24b6f14ba02cfe4e2cde60dc629e47485db5701bca3003b8df79e3aa311c0c967979f6a1dca3ba69f5b1e45fa2db6ff83352fdf2d4349d5f8d120e740d
In certain circumstances, we may want to flush to disk without
emptying `cacheCoins`, which affects performance. UTXO snapshot
activation is one such case.
This method is currently unused and this commit does not
change any behavior.
Incorporates feedback from John Newbery.
Co-authored-by: Suhas Daftuar <sdaftuar@gmail.com>
fadeb6b103 Add missing includes to fix gcc-13 compile error (MarcoFalke)
Pull request description:
On current master:
```
CXX support/libbitcoin_util_a-lockedpool.o
support/lockedpool.cpp: In member function ‘void Arena::free(void*)’:
support/lockedpool.cpp:99:20: error: ‘runtime_error’ is not a member of ‘std’
99 | throw std::runtime_error("Arena: invalid or double free");
| ^~~~~~~~~~~~~
support/lockedpool.cpp:22:1: note: ‘std::runtime_error’ is defined in header ‘<stdexcept>’; did you forget to ‘#include <stdexcept>’?
21 | #include <algorithm>
+++ |+#include <stdexcept>
22 | #ifdef ARENA_DEBUG
support/lockedpool.cpp: In member function ‘void LockedPool::free(void*)’:
support/lockedpool.cpp:320:16: error: ‘runtime_error’ is not a member of ‘std’
320 | throw std::runtime_error("LockedPool: invalid address not pointing to any arena");
| ^~~~~~~~~~~~~
support/lockedpool.cpp:320:16: note: ‘std::runtime_error’ is defined in header ‘<stdexcept>’; did you forget to ‘#include <stdexcept>’?
ACKs for top commit:
hebasto:
ACK fadeb6b103.
fanquake:
ACK fadeb6b103 - tested this fixes compilation with GCC 13. I don't think theres a need to do anything else here, and that'd also just potentially complicate backporting.
Tree-SHA512: 99f79cf385c913138a9cf9fc23be0a3a067b0a28518b8bdc033a7220b85bbc5d18f5356c5bdad2f628c22abb87c18b232724f606eba6326c031518559054be31
3d1a4d8a45 RPC: make RPCResult::MatchesType return useful errors (Anthony Towns)
Pull request description:
Currently if you don't correctly update the description of the return value for an RPC call, you essentially just get an assertion failure with no useful information; this generates a description of the problems instead.
ACKs for top commit:
MarcoFalke:
re-ACK 3d1a4d8a45🌷
Tree-SHA512: cf0580b7046faab0128672a74f8cc5a1655dfdca6646a2e38b51f0fb5f672c98aad6cd4c5769454a2d644a67da639ccb1c8ff5d24d3d6b4446a082398a643722
dee89438b8 Abstract out ComputeTapbranchHash (Russell O'Connor)
8e3fc99427 Do not use CScript for tapleaf scripts until the tapleaf version is known (Russell O'Connor)
Pull request description:
While BIP-341 calls the contents of tapleaf a "script", only in the case that the tapleaf version is `0xc0` is this script known to be a tapscript. Otherwise the tapleaf "script" is simply an uninterpreted string of bytes.
This PR corrects the issue where the type `CScript` is used prior to the tapleaf version being known to be a tapscript. This prevents `CScript` methods from erroneously being called on non-tapscript data.
A second commit abstracts out the TapBranch hash computation in the same manner that the TapLeaf computation is already abstracted. These two abstractions ensure that the TapLeaf and TapBranch tagged hashes are always constructed properly.
ACKs for top commit:
ajtowns:
ACK dee89438b8
instagibbs:
ACK dee89438b8
achow101:
ACK dee89438b8
sipa:
ACK dee89438b8
aureleoules:
reACK dee89438b8 - I verified that there is no behavior change.
Tree-SHA512: 4a1d37f3e9a1890e7f5eadcf65562688cc451389581fe6e2da0feb2368708edacdd95392578d8afff05270d88fc61dce732d83d1063d84d12cf47b5f4633ec7e
5eabb61b23 addrdb: Only call Serialize() once (Martin Zumsande)
da6c7aeca3 hash: add HashedSourceWriter (Martin Zumsande)
Pull request description:
There have been various reports of corruption of `peers.dat` recently, see #26599.
As explained in [this post](https://github.com/bitcoin/bitcoin/issues/26599#issuecomment-1381082886) in more detail, the underlying issue is likely that we currently serialize `AddrMan` twice - once for the file stream, once for the hasher that helps create the checksum - and if `AddrMan` changes in between these two calls, the checksum doesn't match the data and the resulting `peers.dat` is corrupted.
This PR attempts to fix this by introducing and using `HashedSourceWriter` - a class that keeps a running hash while serializing data, similar to the existing `CHashVerifier` which does the analogous thing while unserializing data. Something like this was suggested before, see https://github.com/bitcoin/bitcoin/pull/10248#discussion_r120694343.
Fixes#26599 (not by changing the behavior in case of a crash, but by hopefully fixing the underlying cause of these corruptions).
ACKs for top commit:
sipa:
utACK 5eabb61b23
naumenkogs:
utACK 5eabb61b23
Tree-SHA512: f19ad37c37088d3a9825c60de2efe85cc2b7a21b79b9156024d33439e021443ef977a5f8424a7981bcc13d73d11e30eaa82de60e14d88b3568a67b03e02b153b
58c2bbdb55 [fuzz] Enable erlay in process_message(s) targets (dergoegge)
Pull request description:
The process_message(s) targets can't exercise the Erlay logic at the moment as the config setting is off by default and not switched on in the fuzz targets.
This PR enables the `-txreconciliation` setting in both targets.
ACKs for top commit:
fanquake:
ACK 58c2bbdb55
Tree-SHA512: a2754fd04549bdcac94d8225244c5c83fe4c26114c0c2fdf316257480625e05e4e6b1b791974e1f1021451d3f81cb59a109261fb73178ad03911f0a3db963077
d96d97ad30 doc: Add release note for shutdownnotify. (klementtan)
0bd73e2c45 util: Add -shutdownnotify option. (klementtan)
Pull request description:
**Description**: Similar to `-startupnotify`, this PR adds a new option to allow users to specify a command to be executed when Bitcoin Core shuts down.
**Note**: The `shutdownnotify` commands will not be executed if bitcoind shut down due to *unexpected* reasons (ie `killall -9 bitcoind`).
### Testing:
**Normal shutdown commands**
```
# start bitcoind with shutdownnotify optioin
./src/bitcoind -signet -shutdownnotify="touch foo.txt"
# shutdown bitcoind
./src/bitcoin-cli -signet stop
# check that foo.txt has been created
```
**Final RPC call**
Commands:
```
$ ./src/bitcoind -signet -nolisten -noconnect -shutdownnotify="./src/bitcoin-cli -signet getblockchaininfo > tmp.txt"
$ ./src/bitcoin-cli stop
$ cat tmp.txt
```
<details>
<summary>Screen Shot</summary>
</details>
ACKs for top commit:
achow101:
ACK d96d97ad30
1440000bytes:
ACK d96d97ad30
theStack:
re-ACK d96d97ad30
Tree-SHA512: 16f7406fd232e8b97aea5e58854c84755b0c35c88cb3ef9ee123b29a1475a376122b1e100da860cc336d4d657e6046a70e915fdb9b70c9fd097c6eef1b028161
This function is called from utxoupdatepsbt and will be modified
in a following commit to allow for updating inputs with the
`non_witness_utxo` as well.
a2ac6f9582 wallet: unify FindNonChangeParentOutput functions (furszy)
b3f4e82737 wallet: simplify ListCoins implementation (furszy)
Pull request description:
Focused on the following changes:
1) Removed the entire locked coins lookup that was inside `ListCoins` by including them directly on the `AvailableCoins` result (where we were skipping them before).
2) Unified both `FindNonChangeParentOutput` functions (only called from `ListCoins`)
ACKs for top commit:
achow101:
ACK a2ac6f9582
aureleoules:
ACK a2ac6f9582, LGTM
theStack:
Code-review ACK a2ac6f9582
Tree-SHA512: f72b21ee1600c5992559b5dcd8ff260527afac2ec696737f998343a0850b84d439e7f86ea52a14cc0cddabf132a30bf5b52fb34950578ac323083d4375b937f1
fad56f7dd6 doc: Properly report optional RPC args (MarcoFalke)
fa09cb6086 refactor: Introduce is_top_level_arg (MarcoFalke)
fab92a5a5a refactor: Remove const to fix performance-move-const-arg clang-tidy errors (MarcoFalke)
Pull request description:
`OMITTED_NAMED_ARG` and `OMITTED` are a confusing burden:
* It puts the burden on developers to pick the right one of the two
* They can be interchanged without introducing a compile failure or other error
* Picking the wrong one is leading to incorrect docs
* They are redundant, because the correct one can already be determined by the surrounding type
Fix all issues by making them an alias of each other; Pick the right one based on the outer type.
ACKs for top commit:
fanquake:
ACK fad56f7dd6
Tree-SHA512: 6e7193a05a852ba1618a9cb3261220c7ad3282bc5595325c04437aa811f529a88e2851e9c7dbf9878389b8aa42e98f8817b7eb0260fbb9e123da0893cbae6ca2
6d0ab07e81 refactor: use convenience fn to auto parse non-string parameters (stickies-v)
Pull request description:
Minimizes code duplication and improves function naming by having a single (overloaded) convenience function `ParseIfNonString` that both checks if the parameter is a non-string parameter and automatically parses the value if so.
ACKs for top commit:
aureleoules:
ACK 6d0ab07e81
Tree-SHA512: 8cbf68a17cfbdce1e31a19916f447a2965c139fdef00c19e32a9b679f4a4015dfe69ceea0bbe1723711e1c5033ea8d4005d1f4485dfbeea22226140f8cbe8aa3
1dc0e4bc6f rpc: remove optional from fStateStats fields (fanquake)
Pull request description:
These are no-longer optional after #26515, so remove the documentation, and no-op `fStateStats` checks.
ACKs for top commit:
dergoegge:
Code review ACK 1dc0e4bc6f
Tree-SHA512: 06d4550e866341b379bfdbc72d67d71a3b7ceceec06ebd4c5e6f178b75fe40cbf4aff51adba1bc52590e69e818cbdecb0366bf1528c59c5c3dff5bbdba8eac68
87a08cba43 build: move rpc/request from util lib to common (fanquake)
Pull request description:
This is JSON RPC related code that doesn't need to be in util, and should not be required by the kernel.
ACKs for top commit:
TheCharlatan:
ACK 87a08cba43
Tree-SHA512: 5f335be9f0f9ff02eff073af47558ecf505c1392c05f18ca24a065b12b8d92529ec3942d84978cc5028c38369c496ed0243653e1fa26d4db2fae26dfe55c3d65
The previous logic would call it once for serializing into the filestream,
and then again for serializing into the hasher. If AddrMan was changed
in between these calls by another thread, the resulting peers.dat would
be corrupt with non-matching checksum and data.
Fix this by using HashedSourceWriter, which writes the data
to the underlying stream and keeps track of the hash in one go.
The warnings look like:
src/rpc/util.h:192:19: error: std::move of the const variable 'name' has no effect; remove std::move() or make the variable non-const [performance-move-const-arg,-warnings-as-errors]
: m_names{std::move(name)},
^~~~~~~~~~ ~
This enables the type check and fixes the wrong docs.
Otherwise the enabled check would lead to test errors, such as:
> "wallet_labels.py", line 96, in run_test
> node.sendmany(
>
> test_framework.authproxy.JSONRPCException:
> JSON value of type null is not of expected type string (-3)
7b7cd11244 clang-tidy, qt: Force checks for headers in `src/qt` (Hennadii Stepanov)
69eacf2c5e clang-tidy, qt: Fix `modernize-use-default-member-init` in headers (Hennadii Stepanov)
Pull request description:
This PR split from bitcoin/bitcoin#26705 and contains only changes in `src/qt`.
Effectively, it fixes the clang-tidy's `modernize-use-default-member-init` errors, and forces clang-tidy checks for all headers in the `src/qt` directory.
ACKs for top commit:
jarolrod:
ACK 7b7cd11244
Tree-SHA512: 79525bb0f31ae7cad88c781e55091a21467c0485ddc1ed03ad62e051480fda3b3710619ea11af480437edba3c6e038f7c40edc6b373e3a37408c006d11b34686
fa9f6d7bcd rpc: Run type check against RPCArgs (MarcoFalke)
faf96721a6 test: Fix wrong types passed to RPCs (MarcoFalke)
Pull request description:
It seems brittle to require `RPCTypeCheck` being called inside the code logic. Without compile-time enforcement this will lead to places where it is forgotten and thus to inconsistencies and bugs. Fix this by removing the calls to `RPCTypeCheck` and doing the check internally.
The changes should be reviewed as refactoring changes. However, if they change behavior, it will be a bugfix. For example the changes here happen to also detect/fix bugs like the one fixed in commit 3b5fb6e77a.
ACKs for top commit:
ajtowns:
ACK fa9f6d7bcd
Tree-SHA512: fb2c0981fe6e24da3ca7dbc06898730779ea4e02ea485458505a281cf421015e44dad0221a04023fc547ea2c660d94657909843fc85d92b847611ec097532439
5ca7a7be76 rpc: Return accurate results for scanblocks (Aurèle Oulès)
Pull request description:
Implements #26322.
Adds a `filter_false_positives` mode to `scanblocks` to accurately verify results from blockfilters.
If the option is enabled, pre-results given by blockfilters will be filtered out again by checking vouts and vins of all transactions of the relevant blocks against the given descriptors.
### Master
```bash
./src/bitcoin-cli -testnet -named scanblocks action=start scanobjects='["addr(tb1qcxf2gv93c26s6mqz7y6etpqdf70zmn67dualgr)"]'
{
"from_height": 0,
"to_height": 2376055,
"relevant_blocks": [
"000000000001bc35077dec4104e0ab1f667ae27059bd907f9a8fac55c802ae36",
"00000000000120a9c50542d73248fb7c37640c252850f0cf273134ad9febaf61",
"0000000000000082f7af3835da8b6146b0bfb243b8842f09c495fa1e74d454ed",
"0000000000000094c32651728193bfbe91f6789683b8d6ac6ae2d22ebd3cb5d3"
]
}
```
### PR (without `filter_false_positives` mode)
Same as master
```bash
./src/bitcoin-cli -testnet -named scanblocks action=start scanobjects='["addr(tb1qcxf2gv93c26s6mqz7y6etpqdf70zmn67dualgr)"]' filter_false_positives=false
{
"from_height": 0,
"to_height": 2376055,
"relevant_blocks": [
"000000000001bc35077dec4104e0ab1f667ae27059bd907f9a8fac55c802ae36",
"00000000000120a9c50542d73248fb7c37640c252850f0cf273134ad9febaf61",
"0000000000000082f7af3835da8b6146b0bfb243b8842f09c495fa1e74d454ed",
"0000000000000094c32651728193bfbe91f6789683b8d6ac6ae2d22ebd3cb5d3"
]
}
```
### PR (with `filter_false_positives` mode)
```bash
./src/bitcoin-cli -testnet -named scanblocks action=start scanobjects='["addr(tb1qcxf2gv93c26s6mqz7y6etpqdf70zmn67dualgr)"]' filter_false_positives=true
{
"from_height": 0,
"to_height": 2376058,
"relevant_blocks": [
"0000000000000082f7af3835da8b6146b0bfb243b8842f09c495fa1e74d454ed",
"0000000000000094c32651728193bfbe91f6789683b8d6ac6ae2d22ebd3cb5d3"
]
}
```
Also adds a test to check that the blockhash of a transaction will be included in the `relevant_blocks` whether the `filter_false_positives` mode is enabled or not.
ACKs for top commit:
achow101:
ACK 5ca7a7be76
theStack:
re-ACK 5ca7a7be76
furszy:
Code review ACK 5ca7a7be
Tree-SHA512: e8f3cceddddd66f59509717b6314d89e2fef241e13cee81b18fd95e8362cbb95cc40f884342ce6cf892a86febd9e2d434afce05d51892240e67f72ae991852e7
cfe5aebc79 rpc: add minconf and maxconf options to sendall (ishaanam)
a07a413466 Wallet/RPC: Allow specifying min & max chain depth for inputs used by fund calls (Juan Pablo Civile)
Pull request description:
This PR adds a "minconf" option to `fundrawtransaction`, `walletcreatefundedpsbt`, and `sendall`.
Alternative implementation of #14641Fixes#14542
Edit: This PR now also adds this option to `send`
ACKs for top commit:
achow101:
ACK cfe5aebc79
Xekyo:
ACK cfe5aebc79
furszy:
diff ACK cfe5aebc, only a non-blocking nit.
Tree-SHA512: 836e610926eec3a62308fba88ddbd6a13d8f4dac37352d0309599f893cde9c1df5e9c298fda6e076493068e4d213e4afa7290a9e3bdb5a95a5d507da3f7b59e8
ApplyArgsManOptions does not need to set default values for missing
arguments, these are already defined in the BlockAssembler::Options.
This commit changes the interface of ApplyArgsManOptions(). If
ApplyArgsManOptions() is called again after a option is changed,
this option will no longer be reset to the default value.
There is no observed behaviour change due to how
ApplyArgsManOptions() is currently used, and the new interface is
consistent with e.g. ValidationCacheSizes and MemPoolLimits.
Add Options as a member to BlockAssembler to avoid having to assign
all the options individually.
Additionally brings the struct more in line with how we typically
define default and ArgManager values, as e.g. with
ChainstateManager::Options and and CTxMemPool::Options
282019cd3d refactor: add kernel/cs_main.* (fanquake)
Pull request description:
One place to find / include `cs_main`.
No more:
> // Actually declared in validation.cpp; can't include because of circular dependency.
> extern RecursiveMutex cs_main;
Ultimately, no more need to include `validation.h` (which also includes (heavy/boost filled) `txmempool.h`) everywhere for `cs_main`. See #26087 for another example of why that is useful.
ACKs for top commit:
ajtowns:
ACK 282019cd3d
Tree-SHA512: 142835b794873e7a09c3246d6101843ae81ec0c6295e6873130c98a2abfa5f7282748d0f1a37237a779cc71c3bc0a75d03b20313ef5398c83d4814215cbc8287
f4a11d7baf gui: bugfix, catch invalid networks combination crash (furszy)
Pull request description:
The app currently crashes if a network is set inside bitcoin.conf and
another one is provided as param.
The reason is an uncaught runtime_error.
ACKs for top commit:
jarolrod:
tACK f4a11d7baf
johnny9:
tACK f4a11d7baf
john-moffett:
ACK f4a11d7baf
pablomartin4btc:
Tested ACK f4a11d7baf.
hebasto:
ACK f4a11d7baf, tested on Ubuntu 22.04 (Qt 5.15.3).
Tree-SHA512: fc5e26ae0a361e37d53d904cc122d07f064f261b309629c6386cb046ab1b3d2c805cbfe0db8ed3e934af52c6cf0ebb0bef9df9117b4330d9b0ea40c76f9270f9
2022917223 Add secp256k1_selftest call (Pieter Wuille)
3bfca788b0 Remove explicit enabling of default modules (Pieter Wuille)
4462cb0498 Adapt to libsecp256k1 API changes (Pieter Wuille)
9d47e7b71b Squashed 'src/secp256k1/' changes from 44c2452fd3..21ffe4b22a (Pieter Wuille)
Pull request description:
Now that libsecp256k1 has a release (https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2022-December/021271.html), update the subtree to match it.
The changes themselves are not very impactful for Bitcoin Core, but include:
* It's no longer needed to specify whether contexts are for signing or verification or both (all contexts support everything), so make use of that in this PR.
* Verification operations can use the static context now, removing the need for some infrastructure in pubkey.cpp to make sure a context exists.
* Most modules are now enabled by default, so we can drop explicit enabling for them.
* CI improvements (in particular, MSVC and more recent MacOS)
* Introduction of an internal int128 type, which has no effect for GCC/Clang builds, but enables 128-bit multiplication in MSVC, giving a ~20% speedup there (but still slower than GCC/Clang).
* Release process changes (process documentation, changelog, ...).
ACKs for top commit:
Sjors:
ACK 2022917223, but 4462cb0498 could use more eyes on it.
achow101:
ACK 2022917223
jonasnick:
utACK 2022917223
Tree-SHA512: 8a9fe28852abe74abd6f96fef16a94d5a427b1d99bff4caab1699014d24698aab9b966a5364a46ed1001c07a7c1d825154ed4e6557c7decce952b77330a8616b
f9ce0eadf4 For feebump, ignore abandoned descendant spends (John Moffett)
Pull request description:
Closes#26667
To be eligible for fee-bumping, a transaction must not have any of its outputs (eg - change) spent in other unconfirmed transactions in the wallet. This behavior is currently [enforced](9e229a542f/src/wallet/feebumper.cpp (L25-L28)) and [tested](9e229a542f/test/functional/wallet_bumpfee.py (L270-L286)).
However, this check shouldn't apply to spends in abandoned descendant transactions, as explained by #26667.
`CWallet::IsSpent` already carves out an exception for abandoned transactions, so we can just use that.
I've also added a new test to cover this case.
ACKs for top commit:
Sjors:
re-utACK f9ce0eadf4
achow101:
ACK f9ce0eadf4
furszy:
ACK f9ce0ead
Tree-SHA512: 19d957d1cf6747668bb114e27a305027bfca5a9bed2b1d9cc9e1b0bd4666486c7c4b60b045a7fe677eb9734d746f5de76390781fb1e9e0bceb4a46d20acd1749
04528054fc [bench] BlockAssembler with mempool packages (glozow)
6ce265acf4 [test util] lock cs_main before pool.cs in PopulateMempool (glozow)
8791410662 [test util] randomize fee in PopulateMempool (glozow)
cba5934eb6 [miner] allow bypassing TestBlockValidity (glozow)
c058852308 [refactor] parameterize BlockAssembler::Options in PrepareBlock (glozow)
a2de971ba1 [refactor] add helper to apply ArgsManager to BlockAssembler::Options (glozow)
Pull request description:
Performance of block template building matters as miners likely want to be able to start mining on a block with transactions asap after a block is found. We would want to know if a mempool PR accidentally caused, for example, a 100x slowdown. An `AssembleBlock()` bench exists, but it operates on a mempool with 101 transactions, each with 0 ancestors or descendants and with the same fee. Adding a bench with a more complex mempool is useful because (1) it's more realistic (2) updating packages can potentially cause the algorithm to take a long time.
ACKs for top commit:
kevkevinpal:
Tested ACK [0452805](04528054fc)
achow101:
ACK 04528054fc
stickies-v:
ACK 04528054f
Tree-SHA512: 38c138d6a75616651f9b1faf4e3a1cd833437a486f4e84308fbee958e8462bb570582c88f7ba7ab99d80191e97855ac2cf27c43cc21585d3e4b0e227effe2fb5
