Commit Graph

6144 Commits

Author SHA1 Message Date
Pieter Wuille
ff52aa4a22 net: enable v2transport by default
Github-Pull: #29347
Rebased-From: 0bef1042ce
2024-03-04 17:50:54 +00:00
Luke Dashjr
5efeaaa5ca Merge remote-tracking branch 'furszy/2023_rpc_getblockfileinfo' into rpc_getblockfileinfo-26+knots 2024-03-04 17:50:04 +00:00
Andrew Poelstra
8721d6f228 codex32: add functional test for seed import
Github-Pull: #27351
Rebased-From: 91771366a3df22b9e8b4b7152d368d739bfeda94
2024-03-04 17:48:21 +00:00
Luke Dashjr
023807a8cc Diff-minimise 2024-03-04 17:45:54 +00:00
Luke Dashjr
5c487b614e Merge remote-tracking branch 'furszy/2022_rpc_importaddress_descriptors_compatible' into rpc_importaddr_for_descwallet-26+k 2024-03-04 17:45:49 +00:00
Luke Dashjr
d29d7580b1 QA: wallet_reindex: Use importdescriptors directly rather than abusing test_framework's importaddress wrapper
`importdescriptors` is expected to initialise wallet birth time to "now", but `importaddress` initialises it to 1 instead.
2024-03-04 17:45:43 +00:00
Lawrence Nahum
1f433a86b7 add support to save fee estimates without shutting down the node 2024-03-04 17:39:26 +00:00
Roman Zeyde
2ee1991c7d RPC: Add getblocklocations call
This RPC allows the client to retrieve the file system locations
of the confirmed blocks and their undo data, to allow building
efficient indexes outside of Bitcoin Core.

An example usage is described here:
https://github.com/romanz/electrs/issues/308

By using the new RPC, it is possible to build an address-based
index taking ~24GB and a txindex taking ~6GB (as of Dec. 2020).
2024-03-04 17:39:06 +00:00
Luke Dashjr
003720ebd5 Allow acceptstalefeeestimates on all networks 2024-03-04 17:26:48 +00:00
Martin Zumsande
255004a460 test: enable v2 transport for p2p_timeouts.py
by skipping the part where we send a non-version message
before the version - this message would be interpreted as
part of the v2 handshake.

Github-Pull: #28805
Rebased-From: 35fb9930ad
2024-03-04 17:20:58 +00:00
Sebastian Falbesoner
5f0f3f942c test: enable v2 transport for rpc_net.py
- "transport_protocol_type" of inbound peer before version handshake
  is "detecting" on p2p v2 nodes (as opposed to "v1" for p2p v1)
- size of a ping/pong message is 29 bytes (as opposed to 32 for p2p v1)
- for the sendmsgtopeer RPC sub-test, enforce p2p v1 connection to
  have a peer id of zero

Github-Pull: #28805
Rebased-From: 2c1669c37a
2024-03-04 17:20:58 +00:00
Luke Dashjr
1dc0afea1b test: enable v2 transport for p2p_node_network_limited.py
Github-Pull: #28805
Rebased-From: cc961c2695
2024-03-04 17:20:58 +00:00
Martin Zumsande
9d6c08ec30 test: enable --v2transport in combination with --usecli
By renaming the "command" send_cli arg. The old name was unsuitable
because the "addnode" RPC has its own "command" arg, leading to
ambiguity when included in kwargs.
Can be tested with
"python3 wallet_multiwallet.py --usecli --v2transport"
which fails on master because of this (python throws a TypeError).

Github-Pull: #28805
Rebased-From: 3598a1b5c9
2024-03-04 17:20:58 +00:00
Luke Dashjr
1d0a7a8198 test: persist -v2transport over restarts and respect -v2transport=0
Before, a global -v2transport provided to the test would be dropped
when restarting the node within a test and specifying any extra_args.

Fix this by adding "v2transport=1" to args (not extra_args) based
on the global parameter, and deciding for each (re)start of the node
based on this default and test-specific extra_args
(which take precedence over args) whether v2 should be used.

Github-Pull: #28805
Rebased-From: 68a9001751
2024-03-04 17:20:58 +00:00
Luke Dashjr
08647db809 Skip changing permissions entirely if -rpccookieperms=0 specified 2024-02-29 01:31:30 +00:00
Luke Dashjr
24ea5d9724 rpccookieperms: Allow setting setxid/sticky bits
Simply for backward compatibility
2024-02-29 01:31:30 +00:00
willcl-ark
c3bf4ef27d test: add rpccookieperms test
Tests various perms on non-Windows OSes

Github-Pull: #28167
Rebased-From: ce9df2aba3e98ba7f2a576d587ba8e59bf341083
2024-02-29 01:31:30 +00:00
Luke Dashjr
546e462f0a Merge branch 'rpcauthfile-26+knots' into rpccookieperms-26+knots 2024-02-29 01:31:16 +00:00
willcl-ark
cc3dd62493 init: don't delete PID file if it was not generated
Previously, starting a second bitcoind using the same datadir would
correctly fail to init and shutdown. However during shutdown the PID
file belonging to the first instance would be erroneously removed by
the second process shutting down.

Fix this to only delete the PID file if we created it.

Github-Pull: #28946
Rebased-From: 8f6ab31863
2024-02-29 01:10:10 +00:00
Luke Dashjr
abf71d1a53 Merge branch 'sendraw_force-26+knots' into unique_spk_mempool-26+knots 2024-02-28 17:48:12 +00:00
Luke Dashjr
2455b0f53c Merge branch 'whitelist_outgoing-mini-26+knots' into unique_spk_mempool-26+knots 2024-02-28 17:48:09 +00:00
Luke Dashjr
1a8cd10e82 net_permissions: Include "addr" permission on implicit whitelist entries 2024-02-28 17:47:52 +00:00
Luke Dashjr
44256fc2e9 net: Remove forcedinbound limit antifeature 2024-02-28 17:47:30 +00:00
Matthew Zipkin
9349155145 net: add forced_inbound to getpeerinfo
Github-Pull: #27600
Rebased-From: 8c2026848da910fdebff0a9f73e29f1f6ae81e43
2024-02-28 17:47:30 +00:00
Matthew Zipkin
07b24236a8 net: only allow 8 simultaneous forced inbound connections
Github-Pull: #27600
Rebased-From: 75868022a904c1f77871abf962bf9b88a9c5faf6
2024-02-28 17:47:30 +00:00
Matthew Zipkin
cee2e8c415 test: cover ForceInbound permission success even when connections are full
Github-Pull: #27600
Rebased-From: 6b6bcaf0b9ae0a7adaf96bf791aaafc6b5e9f000
2024-02-28 17:47:30 +00:00
brunoerg
f33cd8869d QA: p2p_invalid_messages: Check misbehaving with noban,out whitelisting
Github-Pull: #27114
Rebased-From: 1bae3b2ee6cb96c12fd7810c9362cfb7bdc165ad (partial)
2024-02-28 17:41:11 +00:00
brunoerg
8d290f4c57 test: add coverage for whitelisting manual connections
Github-Pull: #27114
Rebased-From: e6b8f19de9
2024-02-28 17:40:00 +00:00
Roman Zeyde
1e956439eb
rpc: keep .cookie if it was not generated
Otherwise, starting bitcoind twice may cause the `.cookie`
file generated by the first instance to be deleted by the
second instance shutdown (after failing to obtain a lock).

Github-Pull: bitcoin/bitcoin#28784
Rebased-From: 7cb9367157
2024-02-28 11:19:27 +01:00
Luke Dashjr
364ef48bbf QA: Functional test for checkpoint enforcement
Prior to receiving the correct chain's headers, an alternate chain should be stopped prior to the first checkpoint-violating block.
2024-02-26 16:36:08 +00:00
Luke Dashjr
d206f2b726 QA: wallet_taproot: Ensure transaction created by sendall has enough fee to get into the mempool
Fixes intermittent issue exposed by "Wallet: Increase default confirmation target to 144"
2024-02-23 23:09:44 +00:00
Luke Dashjr
98ca8bc71e QA: wallet_taproot: Workaround fee estimation limitation for sendall cleanup as well
(This is necessary to work properly after increasing the default wallet confirmation target)
2024-02-23 23:08:58 +00:00
Luke Dashjr
0e3c426ac1 rpc: implement getgeneralinfo 2024-02-23 22:45:12 +00:00
Luke Dashjr
6f579314d2 Wallet: Remove deprecation warning for newly created legacy wallets
This effectively reverts commit 61152183ab.
2024-02-23 18:25:02 +00:00
Luke Dashjr
fc3652e536 Revert "wallet: Give deprecation warning when loading a legacy wallet"
This reverts commit 8fbb6e99bf.
2024-02-23 18:23:57 +00:00
Luke Dashjr
78ff49027f QA: rpc_getblockfrompeer: Test that fetch-without-header actually works 2024-02-23 17:39:46 +00:00
Luke Dashjr
397e882e34 RPC/blockchain: Allow using getblockfrompeer without already having the block header
Github-Pull: #20295
Rebased-From: d0b537458d90b92f2632f8578cfdaadc9a8f4ed0
2024-02-23 17:39:45 +00:00
Luke Dashjr
aebfd947d2 RPC/blockchain: Restore ability for pruned nodes to getblockfrompeer future blocks 2024-02-23 17:27:09 +00:00
Luke Dashjr
9eefcb9b3f Merge branch 'fix_getblockfrompeer_rereq_err' into rpc_getblockfrompeer_future 2024-02-23 17:27:01 +00:00
Luke Dashjr
2665d63e22 QA: feature_rbf: Test full-RBF service bit 2024-02-23 16:49:59 +00:00
Luke Dashjr
587f6b7c27 RPC/Mempool: Add "rbf_policy" to getmempoolinfo result 2024-02-23 16:49:59 +00:00
Luke Dashjr
c757b00501 QA: feature_rbf: Test full RBF mode 2024-02-23 16:11:26 +00:00
Luke Dashjr
9da96bcd19 Restore -mempoolreplacement option to allow disabling opt-in RBF
This partially reverts commit 8053e5cdad.
2024-02-23 16:10:54 +00:00
Luke Dashjr
6678de6021 Merge branch 'sendraw_force-26+knots' into rbf_opts-26+knots 2024-02-23 15:41:23 +00:00
Luke Dashjr
eca77ec033 Merge branch 'fee_histogram+pr15836_api' into rbf_opts-26+knots 2024-02-23 15:41:16 +00:00
Luke Dashjr
8c0100012e Merge branch 'sendraw_force-26+knots' into mining_priority 2024-02-23 15:29:48 +00:00
Luke Dashjr
9bfaa5eedd Merge branch 'fee_histogram+pr15836_api' into mining_priority 2024-02-23 00:53:38 +00:00
Luke Dashjr
e16e6fec0d Merge branch 'uaappend' into mining_priority 2024-02-23 00:52:39 +00:00
Luke Dashjr
44577d68b2 Merge branch 'restore_blockmaxsize' into mining_priority 2024-02-23 00:52:33 +00:00
Luke Dashjr
35842db70d Allow disabling "non-standard" checks on mainnet 2024-02-23 00:47:23 +00:00
Luke Dashjr
26f729b5be QA: rpc_rawtransaction: Test ignore_rejects 2024-02-22 20:09:24 +00:00
Luke Dashjr
86f162157f Make bad-witness-nonstandard rejection more specific, and support overriding some 2024-02-22 20:08:19 +00:00
Luke Dashjr
3badcde965 AreInputsStandard: Return specific reject reasons 2024-02-22 20:08:19 +00:00
Jonas Schnelli
c470b72ba3 [RPC] Add interface to access mempool stats 2024-02-22 19:13:44 +00:00
Luke Dashjr
06aafaaa85 RPC/Wallet: Add "use_txids" to output of getaddressinfo
Github-Pull: #22693
Rebased-From: a00bc6f395
2024-02-21 15:58:47 +00:00
MarcoFalke
2d35cc3995 rpc: Fixed signed integer overflow for large feerates
Github-Pull: #29434
Rebased-From: fa2a4fdef7
2024-02-21 03:51:14 +00:00
Ava Chow
66d1d242a1 tests: Test that descriptors flag is set for migrated blank wallets
Github-Pull: #29367
Rebased-From: 3904123da9
2024-02-21 00:49:06 +00:00
Andrew Chow
c0560fccf3 tests: Test migration of blank wallets
Github-Pull: #28976
Rebased-From: c11c404281
2024-02-21 00:47:29 +00:00
Pieter Wuille
7caf2a8895 Make v2transport default for addnode RPC when enabled
Github-Pull: #29239
Rebased-From: 3ba815b42d
2024-01-27 16:05:56 +00:00
Torkel Rogstad
cfd1c957a5 rpc: validate conf_target is set alongside estimate_mode
Github-Pull: #29175
Rebased-From: be8ae64b82
2024-01-26 16:10:45 +00:00
Torkel Rogstad
b8e4d0a26a rpc: validate fee estimation mode case insensitive
Github-Pull: #29175
Rebased-From: 8d40addbd2
2024-01-26 16:10:44 +00:00
MarcoFalke
0aa26d600e test: Use blocks_path where possible
Github-Pull: #29262
Rebased-From: 5555d8db33
2024-01-24 03:44:58 +00:00
glozow
ac1b9a51db
[test] import descriptor wallet with reorged parent + IsFromMe child in mempool
Test that wallet rescans process transactions topologically, even if a
parent's entry into the mempool is later than that of its child.
This behavior is important because IsFromMe requires the ability to look
up a transaction's inputs.

Co-authored-by: furszy <matiasfurszyfer@protonmail.com>

Github-Pull: #29179
Rebased-From: df30247705
2024-01-19 16:13:22 +00:00
Gloria Zhao
ecb8ebc660
[test] rescan legacy wallet with reorged parent + IsFromMe child in mempool
Test that wallet rescans process transactions topologically, even if a
parent's entry into the mempool is later than that of its child.
This behavior is important because IsFromMe requires the ability to look
up a transaction's inputs.

Github-Pull: #29179
Rebased-From: c3d02be536
2024-01-19 16:13:14 +00:00
João Barbosa
3a4eecec88 wallet: Support %w in -walletnotify script, on Windows 2024-01-07 01:43:04 +00:00
Luke Dashjr
6086f6443b wallettool: Warn about dump commands not fully dumping/restoring BDB wallets
Fails to dump/restore wallet ID
2024-01-07 01:02:25 +00:00
Luke Dashjr
2ea48ee1aa wallettool: Print warnings to stderr instead of stdout 2024-01-07 00:59:56 +00:00
Luke Dashjr
cdc008f2d8 RPC: getrpcwhitelist: Return methods as a JSON Object for future expansion to sub-call permissions 2024-01-07 00:36:48 +00:00
Harris
b6988de098 RPC: Add getrpcwhitelist method 2024-01-07 00:36:47 +00:00
Jon Atack
3f4d06d61c test: add coverage for -getinfo total_balance
Github-Pull: #19092
Rebased-From: 414804e877f630a630c3d6226cb6714c0a6320e8
2024-01-07 00:28:49 +00:00
Luke Dashjr
6b571ac166 Merge branch 'cli_getinfo_mwbalances' into cli_getinfo_mw_total_balance 2024-01-07 00:25:29 +00:00
Luke Dashjr
2504a07906 Bugfix: RPC/Wallet: dumpwallet should include hdkeypath metadata outside of comment 2024-01-05 16:42:21 +00:00
Luke Dashjr
145a1ac244 QA: Exercise REST interface in feature_fee_estimation 2024-01-05 16:31:48 +00:00
Luke Dashjr
87c1d93eae rest: add endpoint for estimatesmartfee
* write REST interface for getting estimated fee
* update docs about REST interface for fee estimation
* add test
2024-01-05 16:31:47 +00:00
Luke Dashjr
e4acb761d4 dumpmasterprivkey command
RPC command to export the master private key from a wallet.
2024-01-05 00:46:01 +00:00
Luke Dashjr
026b25f99a Bugfix: QA/wallet_fundrawtransaction: Ensure segwit_inputs_only gets tested correctly
Co-authored-by: Aurèle Oulès <aurele@oules.com>
2024-01-04 22:25:57 +00:00
Aurèle Oulès
788a57d808 Witness-only option for fundrawtransaction
Github-Pull: #25183
Rebased-From: 1c5cfd84b3dc14ab886acd47c7891c11eb2457ec
2024-01-04 22:25:57 +00:00
Luke Dashjr
876a033cc2 RPC: createmultisig/addmultisigaddress: Support named args for options 2024-01-04 21:16:16 +00:00
Luke Dashjr
ad7f5858b5 QA: rpc_sort_multisig: Rename variables to avoid keyword conflict with sorted 2024-01-04 21:16:16 +00:00
Thomas Kerin
8488d9fc6a Add more tests to sort_multisig.py / wallet_labels.py
sort_multisig test: check uncompressed keys are disallowed
sort_multisig: add test demonstrating sorting
wallet_labels: test addmultisigaddress fails if sort=true and (wallet) address is uncompressed
2024-01-04 20:47:24 +00:00
Luke Dashjr
380d5f1e68 RPC: Use options object rather than adding a "sort" boolean for multisig methods 2024-01-04 20:46:11 +00:00
Thomas Kerin
e588e6d7bc RPC: addmultisigaddress / createmultisig: parameterize _createmultisig_redeemScript to allow sorting of public keys (BIP67) 2024-01-04 20:46:11 +00:00
furszy
b06b14e68d
rpc: getwalletinfo, return wallet 'birthtime'
And add coverage for it

Github-Pull: #28920
Rebased-From: 1ce45baed7
2024-01-04 16:21:37 +00:00
furszy
12834012c2
test: coverage for wallet birth time interaction with -reindex
Verifying the wallet updates the birth time accordingly when it
detects a transaction with a time older than the oldest descriptor
timestamp.
This could happen when the user blindly imports a descriptor with
'timestamp=now'.

Github-Pull: #28920
Rebased-From: 83c66444d0
2024-01-04 16:21:37 +00:00
Martin Zumsande
b15e2e2cec
test: add regression test for the getrawtransaction segfault
This fails on master without the previous commit.

Github-Pull: #29003
Rebased-From: 9075a44646
2024-01-04 16:21:36 +00:00
Luke Dashjr
f746c3a4b8 RPC/Wallet: Convert descriptorprocesspsbt to use options parameter 2023-12-30 22:05:18 +00:00
Andrew Chow
e5160731d2 RPC/Wallet: Convert walletprocesspsbt to use options parameter
Github-Pull: #24963
Rebased-From: f43f992b7318086859f710b920b4427e6c657fe8
2023-12-30 22:03:40 +00:00
Kashif Smith
f618887c1b test: fix node index bug when comparing peerinfo
Github-Pull: #28849
Rebased-From: 22e38080ea
2023-12-27 23:10:33 +00:00
Dan Benjamin
ea5e929be5 Added a field to the output of gettransaction/listtransactions to indicate whether the given transaction is in the mempool.
Github-Pull: #21260
Rebased-From: 46bf0b7b5d8c44bd7032c473f9878cfb59018161
2023-12-27 03:45:52 +00:00
Luke Dashjr
64dbce8455 QA: wallet_basic: Split wtx expected_fields over multiple lines to minimise merge conflicts 2023-12-27 03:45:09 +00:00
Luke Dashjr
a5d6b7c948 Ignore -rpcauthfile params if -norpcauth is used 2023-12-26 22:36:25 +00:00
Luke Dashjr
c092aa83d8 QA: rpc_users: Add tests for rpcauthfile 2023-12-26 22:10:28 +00:00
Luke Dashjr
acd7eeefea Merge branch 'fix_rpcauth_blank' into rpcauthfile-26+knots 2023-12-26 22:10:20 +00:00
Luke Dashjr
6b79de4f9a QA: rpc_users: Test behaviour of -norpcauth 2023-12-26 21:55:08 +00:00
Luke Dashjr
415734ebaf QA: rpc_users: Test blank rpcauth in combination with non-blank 2023-12-23 18:48:02 +00:00
Luke Dashjr
f956108274 Bugfix: QA: When testing setfeerate, check approximate fee range with a span able to tolerate normal signature length variation 2023-12-23 17:25:29 +00:00
Jon Atack
309b6be8a1 test: add more functional tests for setfeerate
Was: test: update functional tests from settxfee to setfeerate

Github-Pull: #20391
Rebased-From: d87f0f3a923dca2ce8d99aa700b0beda8e42d3ec (partial)
2023-12-23 17:25:29 +00:00
Jon Atack
5030cae7c9 test: add setfeerate functional coverage in wallet_bumpfee.py
Github-Pull: #20391
Rebased-From: c907f158a6bf3cad782d4441e02abcbda210265b
2023-12-23 17:25:28 +00:00
Jon Atack
b8b150a865 test: add setfeerate functional coverage in wallet_create_tx.py
Github-Pull: #20391
Rebased-From: 529bfc16ffb35c5356e27f61d59395fae6707bcd
2023-12-23 17:25:28 +00:00
Luke Dashjr
31874a0aee Bugfix: RPC: blockchain: Actually include "temporary" flag in listprunelocks result 2023-12-21 21:49:47 +00:00
Luke Dashjr
ecbbb74626 QA: Test prune locks via RPC 2023-12-19 23:35:25 +00:00
Luke Dashjr
11f3bca5f0 Move prune lock checking into BlockManager 2023-12-19 23:35:24 +00:00
Kashif Smith
fff4dc9760 tests: Add unit tests for bitcoin-tx replaceable command
Github-Pull: #29022
Rebased-From: 94feaf2b66
2023-12-17 22:09:23 +00:00
Luke Dashjr
ff459b5b55 Refactor to avoid conflicts in new p2p permissions 2023-11-25 00:06:56 +00:00
Luke Dashjr
fdcfa5ac36 Include "blockfilters" in NetPermissions::ToStrings as appropriate 2023-11-25 00:06:55 +00:00
Luke Dashjr
966644d79b QA: interface_rest: Check /mempool/info/with_fee_histogram matches RPC 2023-11-24 23:35:00 +00:00
Luke Dashjr
0bfca0cf32 Bugfix: RPC/blockchain: Actually round feerates down for getmempoolinfo fee histograms 2023-11-24 23:35:00 +00:00
Luke Dashjr
d53d29ce1b Bugfix: QA: Ensure mempool_fee_histogram expected feerates rounded down
As of #22949, fees are rounded up based on feerate, but going the opposite direction from fee to feerate still must round down.
2023-11-24 23:35:00 +00:00
Luke Dashjr
5a10b9ec5e RPC/blockchain: getmempoolinfo: Return fee_histogram in older format (only) 2023-11-24 23:35:00 +00:00
Kiminuo
9caa0f7595 RPC/mempool: Add "to" (end of range) field to fee histogram
Co-authored-by: Jonas Schnelli <dev@jonasschnelli.ch>
Co-authored-by: Jon Atack <jon@atack.com>

Github-Pull: #21422
Rebased-From: 0b87ba9bc3a2ada2839af0e1af868fcd5ddb9155
2023-11-24 23:35:00 +00:00
Luke Dashjr
e114acf9ef RPC/blockchain: getmempoolinfo: Enable specifying with_fee_histogram as a boolean to use a sensible default set of fee rate levels 2023-11-24 23:35:00 +00:00
Luke Dashjr
094e71f576 Bugfix: QA: Ensure mempool_fee_histogram can adapt to feerate rounding correctly
Caution: This implementation is for a post-#22949 codebase
2023-11-24 23:35:00 +00:00
Jonas Schnelli
6b1ec599d7 RPC/blockchain: Consider ancestor, descendant, and combined fee rates for histogram in getmempoolinfo
Test changes from: 0b6ba66238

Co-authored-by: João Barbosa <joao.paulo.barbosa@gmail.com>
Co-authored-by: Jon Atack <jon@atack.com>
2023-11-24 23:35:00 +00:00
Luke Dashjr
2def7cdf11 Merge remote-tracking branch 'origin-pull/21422/head' into fee_histogram+pr15836_api 2023-11-24 23:34:12 +00:00
Andrew Toth
74568a6d89 test: check for zmq block announcements with reorgs
Github-Pull: #26375
Rebased-From: 7b631dc9b1979afc99d9d1a3996492dcb415d3a3
2023-11-24 20:19:43 +00:00
Luke Dashjr
4d59d8bf48 RPC/Wallet: Hacky fix for getbalance bugs 2023-11-24 18:37:29 +00:00
Wladimir J. van der Laan
e2ac01c9ed http: Fail initialization when any bind fails
Currently the HTTP server initialization (`HTTPBindAddresses`) fails
only when *all* bindings fail. So if multiple binds are specified
(`127.0.0.1` and `::1` by defeault) and one succeeds and the other
fails, the latter is essentially ignored.

This commit changes the error behavior to fail *if not all* binds could
be performed, which I think is more in line with how software normally
handles this and what users expect.
2023-11-24 03:54:52 +00:00
pablomartin4btc
17415da9db gui: add validation to cli-side commmands
Add a validation for cli-side commands for the follwing cases:

* duplication of cli-command (and options: at the moment a user
  can specify -rpcwallet many times, only the last one will be
  taken into account,

* only 1 cli-command can run at a time (eg can't run -generate and
  -getinfo at the same time),

* no params starting with slash "-" will be accepted after a cli-command
  (this is the case for bitcoin-cli -generate 3 -rpcwallet=xyz).

Github-Pull: #26990
Rebased-From: 755320f75f2141909e84b62f420462c1c5b193e6
2023-11-09 23:01:58 +00:00
pablomartin4btc
efd7ea0f2e rpc, cli: improve error message on multiwallet mode
The primary objective is to provide users with clearer
and more informative error messages when encountering
the RPC_WALLET_NOT_SPECIFIED error, which occurs when
multiple wallets are loadad.

This commit also rectifies the error message consistency
by bringing the error message in line with the definition
established in protocol.h ("error when there are multiple
wallets loaded").

Github-Pull: #26990
Rebased-From: bf48c40c0ee62eb6343f8aff06a88322a33f2dd6
2023-11-09 23:01:57 +00:00
pablomartin4btc
fe57abd7e9
test: add coverage for snapshot chainstate not matching AssumeUTXO parameters
Co-authored-by: Russell Yanofsky <russ@yanofsky.org>
Co-authored-by: Sebastian Falbesoner <sebastian.falbesoner@gmail.com>

Github-Pull: #28698
Reabsed-From: 811067ca1c
2023-10-31 17:07:52 +00:00
fanquake
ab61087a7e
Merge bitcoin/bitcoin#28660: test: enable reindex readonly test on *BSD
5a0688a20d test: enable reindex readonly test on *BSD and macOS as root (Matthew Zipkin)

Pull request description:

  see https://github.com/bitcoin/bitcoin/pull/27850#discussion_r1349505585

  OpenBSD and FreeBSD don't have `chattr` but they do have `chflags`, use that method to make the block file immutable for the reindex_readonly test.

  Written and tested on a VPS running FreeBSD:
  ```
  FreeBSD freebsd-13-1 13.2-RELEASE-p4 FreeBSD 13.2-RELEASE-p4 GENERIC amd64
  ```

ACKs for top commit:
  maflcko:
    re-cr-lgtm-ACK 5a0688a20d
  jonatack:
    ACK 5a0688a20d tested on macOS only
  theStack:
    ACK 5a0688a20d

Tree-SHA512: 8c88d282d09c00355d22c4c504b779f60e420327a5e07bcf80fa77b97fefcb04952af9ceaf439d9033a0a2448cb26a02663fe6bddcd4a74792857cfbaf1c5162
2023-10-24 10:32:48 +01:00
Ryan Ofsky
d724bb5291
Merge bitcoin/bitcoin#28609: wallet: Reload watchonly and solvables wallets after migration
4814e4063e test: Check tx metadata is migrated to watchonly (Andrew Chow)
d616d30ea5 wallet: Reload watchonly and solvables wallets after migration (Andrew Chow)
118f2d7d70 wallet: Copy all tx metadata to watchonly wallet (Andrew Chow)
9af87cf348 test: Check that a failed wallet migration is cleaned up (Andrew Chow)

Pull request description:

  Some incomplete/incorrect state as a result of migration can be mitigated/cleaned up by simply restarting the migrated wallets. We already do this for a wallet when it is migrated, but we do not for the new watchonly and solvables wallets that may be created. This PR introduces this behavior, in addition to creating those wallets initially without an attached chain.

  While implementing this, I noticed that not all `CWalletTx` metadata was being copied over to the watchonly wallet and so some data, such as time received, was being lost. This PR fixes this as a side effect of not having a chain attached to the watchonly wallet. A test has also been added.

ACKs for top commit:
  ishaanam:
    light code review ACK 4814e4063e
  ryanofsky:
    Code review ACK 4814e4063e. Just implemented the suggested orderpos, copyfrom, and path set comments since last review
  furszy:
    ACK 4814e406

Tree-SHA512: 0b992430df9f452cb252c2212df8e876613f43564fcd1dc00c6c31fa497adb84dfff6b5ef597590f9b288c5f64cb455f108fcc9b6c9d1fe9eb2c39e7f2c12a89
2023-10-23 17:35:36 -04:00
Andrew Chow
da8e397e4a
Merge bitcoin/bitcoin#28685: coinstats, assumeutxo: fix hash_serialized2 calculation
4bfaad4eca chainparams, assumeutxo: Fix signet txoutset hash (Fabian Jahr)
a503cd0f0b chainparams, assumeutxo: Fix testnet txoutset hash (Fabian Jahr)
f6213929c5 assumeutxo: Check deserialized coins for out of range values (Fabian Jahr)
66865446a7 docs: Add release notes for #28685 (Fabian Jahr)
cb0336817e scripted-diff: Rename hash_serialized_2 to hash_serialized_3 (Fabian Jahr)
351370a1d2 coinstats: Fix hash_serialized2 calculation (Fabian Jahr)

Pull request description:

  Closes #28675

  The last commit demonstrates that theStack's analysis [here](https://github.com/bitcoin/bitcoin/issues/28675#issuecomment-1770389468) seems to be correct. There will be more changes needed for the rest of the test suite but the `feature_assumeutxo.py` with my additional tests pass.

ACKs for top commit:
  achow101:
    ACK 4bfaad4eca
  theStack:
    Code-review ACK 4bfaad4eca
  ryanofsky:
    Code review ACK 4bfaad4eca

Tree-SHA512: 2f6abc92b282f7c5da46391803cf0804d13978d191d541f2509b532c538abccd0a081e46cda23d80d47206a05fa2b5d41b7ab246e6a263db7a7461d6292116ef
2023-10-23 15:16:08 -04:00
Matthew Zipkin
5a0688a20d
test: enable reindex readonly test on *BSD and macOS as root 2023-10-23 10:58:54 -04:00
fanquake
0f15db0ec9
Merge bitcoin/bitcoin#28697: fuzz: Increase merge -rss_limit_mb
fa21535551 fuzz: Increase merge -rss_limit_mb (MarcoFalke)

Pull request description:

  For some reason, the limit is hit. (Presumably due to `-set_cover_merge=1` eating more memory, or by simply having more fuzz inputs).

  Fix it by increasing it for the merge operation.

ACKs for top commit:
  dergoegge:
    ACK fa21535551
  hebasto:
    ACK fa21535551, considering the discussion in https://github.com/bitcoin-core/qa-assets/pull/155.

Tree-SHA512: 4fed0f254eccc6fe0b53656bc345ff898b13811dc39387387317d34b521ab77cee03d82b0896dd92d253b7546b6a7e4bdcd478749f47064374ab44ad759ab9ff
2023-10-23 10:10:53 +01:00
Fabian Jahr
cb0336817e
scripted-diff: Rename hash_serialized_2 to hash_serialized_3
-BEGIN VERIFY SCRIPT-
sed -i 's/hash_serialized_2/hash_serialized_3/g' $( git grep -l 'hash_serialized_2' ./src ./contrib ./test )
-END VERIFY SCRIPT-
2023-10-20 22:53:06 +02:00
Fabian Jahr
351370a1d2
coinstats: Fix hash_serialized2 calculation
The legacy serialization was vulnerable to maleation and is fixed by
adopting the same serialization procedure as was already in use for
MuHash.

This also includes necessary test fixes where the hash_serialized2 was
hardcoded as well as correction of the regtest chainparams.

Co-authored-by: Sebastian Falbesoner <sebastian.falbesoner@gmail.com>
2023-10-20 22:53:05 +02:00
MarcoFalke
fa21535551
fuzz: Increase merge -rss_limit_mb 2023-10-20 18:16:59 +02:00
Fabian Jahr
d3223685b1
test: De-dublicate/optimize assumeutxo test for further extensions 2023-10-20 14:39:23 +02:00
Fabian Jahr
0a576d62fe
test: check au file with changed outpoint index 2023-10-20 14:39:21 +02:00
Andrew Chow
4814e4063e test: Check tx metadata is migrated to watchonly 2023-10-19 18:06:44 -04:00
Andrew Chow
d616d30ea5 wallet: Reload watchonly and solvables wallets after migration
When migrating, create the watchonly and solvables wallets without a
context. Then unload and reload them after migration completes, as we do
for the actual wallet.

There is also additional handling for a failed reload.
2023-10-19 18:06:43 -04:00
fanquake
091d29c495
Merge bitcoin/bitcoin#28617: test: Add Wallet Unlock Context Manager
004903ebad test: Add Wallet Unlock Context Manager (Brandon Odiwuor)

Pull request description:

  Fixes #28601, see https://github.com/bitcoin/bitcoin/pull/28403#discussion_r1325426430

  Add Context Manager to manage the locking and unlocking of locked wallets with a passphrase during testing.

ACKs for top commit:
  kevkevinpal:
    lgtm ACK [004903e](004903ebad)
  maflcko:
    lgtm ACK 004903ebad

Tree-SHA512: ab234c167e71531df0d974ff9a31d444f7ce2a1d05aba5ea868cc9452f139845eeb24ca058d88f058bc02482b762adf2d99e63a6640b872cc71a57a0068abfe8
2023-10-19 10:23:44 +01:00
fanquake
f4049eaf08
Merge bitcoin/bitcoin#28671: test: Fix failing time check in rpc_net.py
fa4c6836c9 test: Fix failing time check in rpc_net.py (MarcoFalke)

Pull request description:

  This check fails on slow runners, such as s390x qemu.

  Fix it by using mocktime.

  See https://github.com/bitcoin/bitcoin/pull/28523#discussion_r1357980527

ACKs for top commit:
  0xB10C:
    ACK fa4c6836c9
  pinheadmz:
    ACK fa4c6836c9
  brunoerg:
    crACK fa4c6836c9

Tree-SHA512: 83fb534682e11e97537dc89de8c16f206f38af1a892a2d5970c02684c05eaea8fc9adba3159f16b2440ca0b3871d513a0562a6f3a38f19a5574a47be0919e42f
2023-10-19 09:46:28 +01:00
furszy
5090771f32
test: rpc_getblockfrompeer.py, remove magic numbers usage
Instead of hardcoding the `pruneblockchain(<height>)` heights,
use 'getblockfileinfo' to obtain the highest block number of
each of the block files.

Making the test more robust and readable by stating which file
is being pruned at every point of time (the goal is to mimic
how the automatic pruning process work).
2023-10-18 11:12:15 -03:00
Ryan Ofsky
655dc716aa
Merge bitcoin/bitcoin#28666: test: assumeutxo file with unknown block hash
621db2f004 test: assumeutxo file with unknown block hash (Fabian Jahr)

Pull request description:

  Takes care of one of the open Todos in the assumeutxo functional test. Since an unknown block could be any hash, I simply chose one placeholder, it could also be a random string though.

ACKs for top commit:
  maflcko:
    lgtm ACK 621db2f004
  pablomartin4btc:
    cr ACK 621db2f004
  theStack:
    ACK 621db2f004
  ryanofsky:
    Code review ACK 621db2f004

Tree-SHA512: ee0438ce619f7348c6f88e39b0ea7ddddb8832956d9034ecc795c6033d5d905c09d11b7d0d5afc38231b2fd091ea7c1bd0a0be99d9c32c4e6357a25d76294142
2023-10-18 08:18:48 -04:00
MarcoFalke
fa4c6836c9
test: Fix failing time check in rpc_net.py 2023-10-18 10:35:08 +02:00
Andrew Chow
9af87cf348 test: Check that a failed wallet migration is cleaned up 2023-10-17 14:57:04 -04:00
Fabian Jahr
621db2f004
test: assumeutxo file with unknown block hash 2023-10-17 17:12:29 +02:00
MarcoFalke
fa68571566
test: Add assumeutxo test for wrong hash 2023-10-17 13:24:07 +02:00
fanquake
738ef44abb
Merge bitcoin/bitcoin#28652: assumeutxo: fail early if snapshot block hash doesn't match AssumeUTXO parameters
9620cb4493 assumeutxo: fail early if snapshot block hash doesn't match AssumeUTXO parameters (Sebastian Falbesoner)

Pull request description:

  Right now the `loadtxoutset` RPC call treats literally all files with a minimum size of 40 bytes (=size of metadata) as potential valid snapshot candidates and the waiting loop for seeing the metadata block hash in the headers chain is always entered, e.g.:
  ```
  $ ./src/bitcoin-cli loadtxoutset ~/.vimrc
  <wait>

  bitcoind log:
  ...
  2023-10-15T14:55:45Z [snapshot] waiting to see blockheader 626174207465730a7265626d756e207465730a656c62616e65207861746e7973 in headers chain before snapshot activation
  ...
  ```
  There is no point in doing any further action though if we already know from the start that the UTXO snapshot loading won't be successful. This PR adds an assumeutxo parameter check immediately after the metadata is read in, so we can fail immediately on a mismatch:
  ```
  $ ./src/bitcoin-cli loadtxoutset ~/.vimrc
  error code: -32603
  error message:
  Unable to load UTXO snapshot, assumeutxo block hash in snapshot metadata not recognized (626174207465730a7265626d756e207465730a656c62616e
  65207861746e7973)
  ```
  This way, users who mistakenly try to load files that are not snapshots don't have to wait 10 minutes (=the block header waiting timeout) anymore to get a negative response. If a file is loaded which is a valid snapshot (referencing to an existing block hash), but one which doesn't match the parameters, the feedback is also faster, as we don't have to wait anymore to see the hash in the headers chain before getting an error.

  This is also partially fixes #28621.

ACKs for top commit:
  maflcko:
    lgtm ACK 9620cb4493
  ryanofsky:
    Code review ACK 9620cb4493. This should fix an annoyance and bad UX.
  pablomartin4btc:
    tACK 9620cb4493

Tree-SHA512: f88b865e9d46254858e57c024463f389cd9d8760a7cb30c190aa1723a931e159987dfc2263a733825d700fa612e7416691e4d8aab64058f1aeb0a7fa9233ac9c
2023-10-17 10:20:08 +01:00
fanquake
e6c30834b4
Merge bitcoin/bitcoin#28656: fuzz: Allow multiple --m_dir args
faa5e061c2 fuzz: Allow multiple --m_dir args (MarcoFalke)

Pull request description:

  This allows to merge the result from several servers (or just several folders) at the same time, instead of having to iterate over them.

  This should also allow the fuzz engine (libFuzzer) to optimize the final merge result more, because all fuzz inputs from all folders are available at the same time.

ACKs for top commit:
  dergoegge:
    tACK faa5e061c2

Tree-SHA512: bf0da418b1f7b8a8af16bb7cc1e148b1ccd0f17062ce70758d1ca5b35c3eee77c0c30377d376befdd55480adfd1f1a1073cfc47118e7a710e6760e020abe24bb
2023-10-17 09:51:51 +01:00
fanquake
4caa10b580
Merge bitcoin/bitcoin#28628: lint: Include test_utxo_snapshots in lint_shell, fix linter errors
348e79f7c6 lint: Include test_utxo_snapshots in lint_shell (Fabian Jahr)

Pull request description:

  jamesob excluded `test_utxo_snapshots.sh` from the shell linter with this explanation: "Add the script to the shellcheck exception list since the quoted variables rule needs to be violated in order to get bitcoind to pick up on $EARLY_IBD_FLAGS." However, macrofake pointed out that single lines can be excluded from linting.

  This fixes one fixable rule violation, excludes the rest of the offending lines from the linter and then removes the exclusion of the `test_utxo_snapshots.sh` file. Also adds documentation.

ACKs for top commit:
  Empact:
    ACK 348e79f7c6
  maflcko:
    lgtm ACK 348e79f7c6
  pablomartin4btc:
    tACK 348e79f7c6

Tree-SHA512: a904cc1cc3c94488dfbd39ea69a3ef17259f991708a797009001669448fef81eed086ecbce1ec433988d88baef293849698e2e0eb86a969b949cc7ef93af7b4b
2023-10-17 09:48:50 +01:00
Luke Dashjr
cf1bda10ff Diff-minimise 2023-10-17 03:06:58 +00:00
Jameson Lopp
319e6c2bed Throw error if invalid parameters passed to getnetworkhashps RPC endpoint
Github-Pull: #28554
Rebased-From: 435ff29c5be0bd2ad2e89d9a37ecdb5da7c5d063
2023-10-17 03:06:52 +00:00
Sebastian Falbesoner
9620cb4493 assumeutxo: fail early if snapshot block hash doesn't match AssumeUTXO parameters 2023-10-16 17:20:59 +02:00
MarcoFalke
faa5e061c2
fuzz: Allow multiple --m_dir args 2023-10-16 16:04:02 +02:00
fanquake
22fa1f4702
Merge bitcoin/bitcoin#28565: rpc: getaddrmaninfo followups
e6e444c06c refactor: add and use EnsureAnyAddrman in rpc (stratospher)
bf589a50a0 doc: add release notes for #27511 (stratospher)
3931e6abc3 rpc: `getaddrmaninfo` followups (stratospher)

Pull request description:

  - make `getaddrmaninfo` RPC public since it's not for development purposes only and regular users might find it useful. [#26988 (comment)](https://github.com/bitcoin/bitcoin/pull/26988#issuecomment-1738371584)
  - add missing `all_networks` key to RPC help. [#27511 (comment)](https://github.com/bitcoin/bitcoin/pull/27511#discussion_r1335084087)
  - fix clang format spacing
  - add and use `EnsureAddrman` in RPC code. [#27511 (comment)](https://github.com/bitcoin/bitcoin/pull/27511#discussion_r1331501491)

ACKs for top commit:
  0xB10C:
    Code Review re-ACK e6e444c06c
  theStack:
    Code-review ACK e6e444c06c
  pablomartin4btc:
    tested ACK e6e444c06c

Tree-SHA512: c14090d5c64ff15e92d252578de2437bb2ce2e1e431d6698580241a29190f0a3528ae5b013c0ddb76a9ae538507191295c37cab7fd93469941cadbde44587072
2023-10-16 11:21:45 +02:00
fanquake
ab2f531b78
Merge bitcoin/bitcoin#28650: fuzz: Merge with -set_cover_merge=1
fa858d63a0 fuzz: Merge with -set_cover_merge=1 (MarcoFalke)

Pull request description:

  This should be less controversial than commit 151a2b189c. The overall size of the qa-assets repo is reduced further from 1.9GB to 1.6GB. Also, the runtime to iterate on the resulting folder is reduced further from ~1699s to ~1149s (N=1).

ACKs for top commit:
  murchandamus:
    crACK fa858d63a0
  dergoegge:
    ACK fa858d63a0

Tree-SHA512: e23fa93bd48f01d11c551b035004c678bd6d76bc24ac7d0d0a7883060804e6711763cbd0cd0ded3aad3e4c40da764decae81c2703388cc11961def3c89a4f9ba
2023-10-15 11:08:31 +02:00
MarcoFalke
fa858d63a0
fuzz: Merge with -set_cover_merge=1 2023-10-13 18:09:29 +02:00
Fabian Jahr
348e79f7c6
lint: Include test_utxo_snapshots in lint_shell 2023-10-13 14:06:06 +02:00
Sebastian Falbesoner
ac4caf3366 test: fix assert_debug_log call-site bugs, add type checks
Two recently added tests (PR #28625 / commit 2e31250027
and PR #28634 / commit 3bb51c29df)
introduced a bug by wrongly using the `assert_debug_log` helper.
Instead of passing the expected debug string in a list as expected, it
was passed as bare string, which is then interpretered as a list of
characters, very likely leading the debug log assertion pass even if the
intended message is not appearing.

In order to avoid bugs like this in the future, enforce that the
`{un}expected_msgs` parameters are lists.
2023-10-13 12:42:49 +02:00
fanquake
448790c00a
Merge bitcoin/bitcoin#28639: refactor: Remove unused nchaintx from SnapshotMetadata constructor, fix test, add test
fafde92f84 test: Check snapshot file with wrong number of coins (MarcoFalke)
faa90f6e7b refactor: Remove unused nchaintx from SnapshotMetadata constructor (MarcoFalke)

Pull request description:

  See commit messages

ACKs for top commit:
  Sjors:
    utACK fafde92f84
  theStack:
    ACK fafde92f84

Tree-SHA512: 9ed2720b50d1c0938f30543ba143e1a4c6af3a0ff166f8b3eb452e1d99ddee6e3443a4c99f77efe94b8c3eb2feff984bf5259807ee8085e1e0e1e0d1de98227e
2023-10-13 11:14:54 +02:00
MarcoFalke
faa190b1ef
test: Fuzz merge with -use_value_profile=0 for now 2023-10-12 16:44:41 +02:00
fanquake
06d469c26b
Merge bitcoin/bitcoin#28629: test: fix usdt undeclared function errors on mantis
4077e43bf6 test: fix usdt undeclared function errors on mantis (willcl-ark)

Pull request description:

  This is one way to fix #28600

  Recently usage of undeclared functions became an error rather than a warning, in C2x. https://reviews.llvm.org/D122983?id=420290

  This change has migrated into the build tools of Ubuntu 23.10 which now causes the USDT tests to fail to compile, see
  https://github.com/bitcoin/bitcoin/issues/28600

  I think there are various potential fixes:

  1. Manually declare the functions we use
  2. Fix imports so that manual declarations aren't needed
  3. Revert the new C2X behaviour and don't error on implicit function declarations

  I would have preferred solution 2, but I believe this will require changes to the upstream bcc package. Having played with the imports I can get things working in a standalone C program, using system headers, but when building the program from a python context as we do in the test it uses its own headers (bundled with the python lib) rather than the system ones, and manually importing (some) system headers results in definition mismatches. I also investigated explicitly importing required headers from the package, which use paths like `#import </virtual/bcc/bcc_helpers.h>`, but this seems more obtuse and brittle than simply ignoring the warning.

  Therefore I think that until the upstream python pacakge fixes their declarations, we should fix this by setting `-Wno-error=implicit-function-declaration` for the tracing programs.

  cc maflcko 0xB10C

ACKs for top commit:
  maflcko:
    lgtm ACK 4077e43bf6

Tree-SHA512: 8368bb1155e920a95db128dc893267f8dab64f1ae53f6d63c6d9294e2e4e92bef8515e3697e9113228bedc51c0afdbc5bbcf558c119bf0eb3293dc2ced86b435
2023-10-12 12:00:12 +02:00
fanquake
a927d5cabe
Merge bitcoin/bitcoin#27228: test: exempt previous release binaries from valgrind
850670e3d6 test: don't run old binaries under valgrind (Sjors Provoost)

Pull request description:

  Some, but not all, backward compatibility tests fail for me and it seems useless to run old release binaries under valgrind anyway.

  Can be tested by running `test/functional/feature_txindex_compatibility.py --valgrind --timeout-factor=10` with and without this PR.

  —
  The previous version of this PR disabled these test entirely under valgrind. The current version does run the test, but starts the old binaries without valgrind.

ACKs for top commit:
  maflcko:
    lgtm ACK 850670e3d6

Tree-SHA512: ebdf461083f1292528e6619963b910f486b60b4f6b183f0aea2c8bfcafa98caeb204d138700cd288450643bcec5e49e12b89f2f7537fccdf495a2a33acd9cea0
2023-10-12 11:53:15 +02:00
MarcoFalke
fafde92f84
test: Check snapshot file with wrong number of coins
Also, fix a bug in an assert_debug_log call.
2023-10-12 11:14:56 +02:00
fanquake
4a5aae9330
Merge bitcoin/bitcoin#28634: test: BIP324: add check for detection of missing garbage terminator
3bb51c29df test: BIP324: add check for missing garbage terminator detection (Sebastian Falbesoner)

Pull request description:

  This PR adds test coverage for the "missing garbage terminator" detection on incoming v2 transport (BIP324) connections:
  04265ba937/src/net.cpp (L1205-L1209)

  Note that this always happens at the same exact amount of bytes sent in (after 64 + 4095 + 16 = 4175 bytes), if at no point, the last 16 bytes of potential authentication data match the garbage, i.e. all the previous bytes after the ellswift pubkey. To keep it simple, we just send in zero-value bytes here and verify that the detection hits exactly after the last bytes is sent.

  AFAICT, with this PR all the v2 transport errors that can be triggered in this simple way of "just open a socket and send in a fixed byte-string" are covered. For more advanced test, we need BIP324 cryptography in the test framework in order to perform a v2 handshake etc. (PRs #28374, #24748).

ACKs for top commit:
  sipa:
    utACK 3bb51c29df
  laanwj:
    ACK 3bb51c29df

Tree-SHA512: f88275061c7c377a3d9f2608452671afc26deb6d5bd5be596de987c7e5042555153ffe681760c33bce2b921ae04e50f349ea0128a677e6443a95a079e52cdc5f
2023-10-12 09:35:07 +02:00
Sjors Provoost
850670e3d6
test: don't run old binaries under valgrind
This is unnecessary and caused test failures. The backward
compatibility tests are meant to find regressions in the
current codebase, not to detect bugs in older releases.
2023-10-12 09:25:44 +02:00
Andrew Chow
e3eb3aae30
Merge bitcoin/bitcoin#28625: test: check that loading snapshot not matching AssumeUTXO parameters fails
2e31250027 test: check that loading snapshot not matching AssumeUTXO parameters fails (Sebastian Falbesoner)

Pull request description:

  This PR adds test coverage for the failed loading of an AssumeUTXO snapshot in case the referenced block hash doesn't match the parameters in the chainparams. Right now, I expect this would be the most common error-case for `loadtxoutset` out in the wild, as for mainnet the `m_assumeutxo_data` map is empty and this error condition would obviously always be triggered for any (otherwise valid, correctly encoded) snapshot. Note that this test-case is the simplest scenario and doesn't cover any of the TODO ideas mentioned at the top of the functional test yet.

ACKs for top commit:
  jamesob:
    ACK 2e31250027
  Sjors:
    utACK 2e31250027
  achow101:
    ACK 2e31250027

Tree-SHA512: 8bcb2d525c95fbc95f87d3e978ad717d95bddb1ff67cbe7d3b06e4783f0f1ffba32b17ef451468c39c23bc1b3ef1150baa71148c145275c386f2d4822d790d39
2023-10-11 14:25:23 -04:00
fanquake
d98d88c779
Merge bitcoin/bitcoin#28392: test: Use pathlib over os path
bfa0bd632a  test: Use pathlib over os.path #28362 (ns-xvrn)

Pull request description:

  In reference to issue  #28362  refactoring of functional tests to use pathlib over os.path to reduce verbosity and increase the intuitiveness of managing file access.

ACKs for top commit:
  maflcko:
    re-ACK  bfa0bd632a  🐨
  willcl-ark:
    ACK bfa0bd632a

Tree-SHA512: fb0833c4039d09758796514e47567a93ac831cb0776ff1a7ed8299792ad132e83282ef80bea098a88ae1551d906f2a56093d3e8de240412f9080735d00d496d9
2023-10-11 20:06:36 +02:00
fanquake
744157ef1a
Merge bitcoin/bitcoin#28602: descriptors: Disallow hybrid and uncompressed keys when inferring
74c77825e5 test: Unit test for inferring scripts with hybrid and uncompressed keys (Andrew Chow)
f895f97014 test: Scripts with hybrid pubkeys are migrated to watchonly (Andrew Chow)
37b9b73477 descriptors: Move InferScript's pubkey validity checks to InferPubkey (Andrew Chow)
b7485f11ab descriptors: Check result of InferPubkey (Andrew Chow)

Pull request description:

  `InferDescriptor` was not always checking that the pubkey it was placing into the descriptor was an allowed pubkey. For example, given a P2WPKH script that uses an uncompressed pubkey, it would produce a `wpkh()` with the uncompressed key. Additionally, the hybrid key check was only being done for `pk()` scripts, where it should've been done for all scripts.

  This PR moves the key checking into `InferPubkey`. If the key is not valid for the context, then `nullptr` is returned and the inferring will fall through to the defaults of either `raw()` or `addr()`.

  This also resolves an issue with migrating legacy wallets that contain hybrid pubkeys as such watchonly scripts will become `raw()` or `addr()` and go to the watchonly wallet. Note that a legacy wallet cannot sign for hybrid pubkeys. A test has been added for the migration case.

  Also added unit tests for `InferDescriptor` itself as the edge cases with that function are not covered by the descriptor roundtrip test.

ACKs for top commit:
  furszy:
    ACK 74c77825
  Sjors:
    utACK 74c77825e5

Tree-SHA512: ed5f63e42a2e46120245a6b0288b90d2a6912860814c6c08fe393332add1cb364dc5eca72f16980352143570aef0c07bf1a91acd294099463bd028b6ce2fe40c
2023-10-11 12:50:43 +02:00
Sebastian Falbesoner
3bb51c29df test: BIP324: add check for missing garbage terminator detection 2023-10-11 11:58:07 +02:00
Brandon Odiwuor
004903ebad test: Add Wallet Unlock Context Manager
Add Context Manager to manage wallet locking/unlocking with passphrase
2023-10-10 18:12:31 +03:00
ns-xvrn
bfa0bd632a test: Use pathlib over os.path #28362
revert netutil chgs py3.8 compliant

fixes based on PR review
2023-10-10 09:26:56 -04:00
Sebastian Falbesoner
2e31250027 test: check that loading snapshot not matching AssumeUTXO parameters fails 2023-10-10 09:43:10 +02:00
willcl-ark
4077e43bf6
test: fix usdt undeclared function errors on mantis
Recently usage of undeclared functions became an error rather than a
warning, in C2x. https://reviews.llvm.org/D122983?id=420290

This change has migrated into the build tools of Ubuntu 23.10 which now
causes the USDT tests to fail to compile, see
https://github.com/bitcoin/bitcoin/issues/28600

Fix this by setting `-Wno-error=implicit-function-declaration` for the
tracing programs.
2023-10-09 21:01:38 +01:00
Andrew Chow
f895f97014 test: Scripts with hybrid pubkeys are migrated to watchonly
Descriptors disallows hybrid pubkeys. Anything with hybrid pubkeys
should becomes a raw() descriptor that shows up in the watchonly wallet.
2023-10-09 14:07:37 -04:00
fanquake
62346bc394
Merge bitcoin/bitcoin#28604: test: Use feerate higher than minrelay fee in wallet_fundraw
05af4dfa50 test: Use feerate higher than minrelay fee in wallet_fundraw (Andrew Chow)

Pull request description:

  The external input weight test in wallet_fundrawtransaction.py made transactions at the minimum relay fee. However due to ECDSA sometimes making a shorter signature than expected, the size estimate (and therefore the funded fee) ends up being a little bit too low, which results in the final transaction being under the min relay fee. We can compensate for this by just using a feerate higher than the minrelayfee as the actual feerate itself does not matter in this test.

  Fixes #28437

ACKs for top commit:
  glozow:
    utACK 05af4dfa50, seems right to me

Tree-SHA512: 3e08f052db32d891515d32b27b2d7c5bcbfc77d5ea457eefc75e8aa6d40960278e537c1e8bffe7ddc211949c78e14145a671a8d34e7e1bb15438773cb0d9e89d
2023-10-09 09:53:04 +01:00
Antoine Poinsot
128bc104ef
qa: bound testing for TapMiniscript
Make sure we can spend a maximum-sized Miniscript under Tapscript
context.
2023-10-08 02:43:26 +02:00
Antoine Poinsot
b917c715ac
qa: Tapscript Miniscript signing functional tests 2023-10-08 02:43:25 +02:00
Antoine Poinsot
5dc341dfe6
qa: list descriptors in Miniscript signing functional tests
This makes it more generalistic than just having the miniscripts since
we are going to have Taproot descriptors with (multiple) miniscripts in
them too.
2023-10-08 02:43:24 +02:00
Antoine Poinsot
bd4b11ee06
qa: functional test Miniscript inside Taproot descriptors 2023-10-08 02:43:23 +02:00
Fabian Jahr
1ff1c34656
test: Rename wait_until_helper to wait_until_helper_internal
Co-authored-by: MarcoFalke <falke.marco@gmail.com>
2023-10-06 19:43:32 +02:00
Fabian Jahr
4e915e926b
test: Improvements of feature_assumeutxo
- Remove usage of the internal wait_until_helper function
- Use framework self.no_op instead of new no_sync function

co-authored-by: Andrew Chow <github@achow101.com>
2023-10-06 18:39:02 +02:00
fanquake
1472df63f7
Merge bitcoin/bitcoin#28253: test: display abrupt shutdown errors in console output
0f83ab407e test: display abrupt shutdown errors in console output (furszy)

Pull request description:

  Making it easier to debug errors in the CI environment,
  particularly in scenarios where it's not immediately clear
  what happened nor which node crashed (or shutdown abruptly).

  A bit of context:
  Currently, the test framework redirects each node's stderr output
  stream to a different temporary file inside each node's data directory.
  While this is sufficient for storing the error, it isn't very helpful for
  understanding what happened just by reading the CI console output.

  Most of the time, reading the stderr file in the CI environment is not
  possible, because people don't have access to it.

  Testing Note:
  The displayed error difference can be observed by cherry-picking this
  commit 9cc5393c0f on top of this branch and running any
  functional test.

ACKs for top commit:
  maflcko:
    lgtm ACK 0f83ab407e
  theStack:
    ACK 0f83ab407e

Tree-SHA512: 83ce4d21d5316e8cb16a17d3fbe77b8649fced9e09410861d9674c233f6e9c34bcf573504e387e4f439c2841b2ee9855d0d35607fa13aa89eafe0080c45ee82d
2023-10-06 13:51:44 +01:00
Andrew Chow
05af4dfa50 test: Use feerate higher than minrelay fee in wallet_fundraw
The external input weight test in wallet_fundrawtransaction.py made
transactions at the minimum relay fee. However due to ECDSA sometimes
making a shorter signature than expected, the size estimate (and
therefore the funded fee) ends up being a little bit too low, which
results in the final transaction being under the min relay fee. We can
compensate for this by just using a feerate higher than the minrelayfee
as the actual feerate itself does not matter in this test.
2023-10-05 19:20:30 -04:00
Andrew Chow
54bdb6e074
Merge bitcoin/bitcoin#27609: rpc: allow submitpackage to be called outside of regtest
5b878be742 [doc] add release note for submitpackage (glozow)
7a9bb2a2a5 [rpc] allow submitpackage to be called outside of regtest (glozow)
5b9087a9a7 [rpc] require package to be a tree in submitpackage (glozow)
e32ba1599c [txpackages] IsChildWithParentsTree() (glozow)
b4f28cc345 [doc] parent pay for child in aggregate CheckFeeRate (glozow)

Pull request description:

  Permit (restricted topology) submitpackage RPC outside of regtest. Suggested in https://github.com/bitcoin/bitcoin/pull/26933#issuecomment-1510851570

  This RPC should be safe but still experimental - interface may change, not all features (e.g. package RBF) are implemented, etc. If a miner wants to expose this to people, they can effectively use "package relay" before the p2p changes are implemented. However, please note **this is not package relay**; transactions submitted this way will not relay to other nodes if the feerates are below their mempool min fee. Users should put this behind some kind of rate limit or permissions.

ACKs for top commit:
  instagibbs:
    ACK 5b878be742
  achow101:
    ACK 5b878be742
  dergoegge:
    Code review ACK 5b878be742
  ajtowns:
    ACK 5b878be742
  ariard:
    Code Review ACK  5b878be742. Though didn’t manually test the PR.

Tree-SHA512: 610365c0b2ffcccd55dedd1151879c82de1027e3319712bcb11d54f2467afaae4d05dca5f4b25f03354c80845fef538d3938b958174dda8b14c10670537a6524
2023-10-05 19:08:19 -04:00
Andrew Chow
cf553e3ab7
Merge bitcoin/bitcoin#28597: wallet: No BDB creation, unless -deprecatedrpc=create_bdb
fa071aeb61 wallet: No BDB creation, unless -deprecatedrpc=create_bdb (MarcoFalke)

Pull request description:

  With BDB being removed soon, it seems confusing and harmful to allow users to create fresh BDB wallets going forward, as it would load them with an additional burden of having to migrate them soon after.

  Also, it would be good to allow for one release for test (and external) scripts to adapt.

  Fix all issues by introducing the `-deprecatedrpc=create_bdb` setting.

ACKs for top commit:
  Sjors:
    tACK fa071aeb61
  achow101:
    ACK fa071aeb61
  furszy:
    utACK fa071aeb

Tree-SHA512: 37a4c3e4ba659e0ebe2382e71d9c80e42a895d9ad743f5dda7c110fbbb7d2a36f46769982552a9ac0c3a57203379ef164be97aa8033eb7674d6b4da030ba8f9b
2023-10-05 15:35:54 -04:00
Andrew Chow
0b2c93bc56
Merge bitcoin/bitcoin#28590: assumeutxo: change getchainstates RPC to return a list of chainstates
a9ef702a87 assumeutxo: change getchainstates RPC to return a list of chainstates (Ryan Ofsky)

Pull request description:

  Current `getchainstates` RPC returns "normal" and "snapshot" fields which are not ideal because it requires new "normal" and "snapshot" terms to be defined, and the definitions are not really consistent with internal code. (In the RPC interface, the "snapshot" chainstate becomes the "normal" chainstate after it is validated, while in internal code there is no "normal chainstate" and the "snapshot chainstate" is still called that temporarily after it is validated).

  The current `getchainstates` RPC is also awkward to use if you to want information about the most-work chainstate, because you have to look at the "snapshot" field if it exists, and otherwise fall back to the "normal" field.

  Fix these issues by having `getchainstates` just return a flat list of chainstates ordered by work, and adding a new chainstate "validated" field alongside the existing "snapshot_blockhash" field so it is explicit if a chainstate was originally loaded from a snapshot, and whether the snapshot has been validated.

  This change was motivated by comment thread in https://github.com/bitcoin/bitcoin/pull/28562#discussion_r1344154808

ACKs for top commit:
  Sjors:
    re-ACK a9ef702a87
  jamesob:
    re-ACK a9ef702
  achow101:
    ACK a9ef702a87

Tree-SHA512: b364e2e96675fb7beaaee60c4dff4b69e6bc2d8a30dea1ba094265633d1cddf9dbf1c5ce20c07d6e23222cf1e92a195acf6227e4901f3962e81a1e53a43490aa
2023-10-05 14:19:34 -04:00
fanquake
d9c1cc5f1f
Merge bitcoin/bitcoin#28027: test: Fixes and updates to wallet_backwards_compatibility.py for 25.0 and descriptor wallets
afd9a673c4 test: roundtrip wallet backwards compat downgrade (Andrew Chow)
bbf43c63b9 test: Add 25.0 to wallet backwards compatibiilty test (Andrew Chow)
538939ec39 test: Run upgrade test on all nodes (Andrew Chow)
6d4699028b test: Run downgrade test on descriptor wallets (Andrew Chow)
f158573be1 test: Add 0.21 tr() incompatibility test (Andrew Chow)
f41215c3f0 test: add logging 0.17 incompatibilities in wallet back compat (Andrew Chow)
71c03aeff7 test: Refactor v19 addmultisigaddress test to be distinct (Andrew Chow)
53f35d02cb test: Remove w1_v18 from wallet backwards compatibility (Andrew Chow)
313d665437 test: Fix 0.16 wallet paths and downgrade test (Andrew Chow)
5d8469362a test: Add helper functions for checking node versions (Andrew Chow)

Pull request description:

  It was somewhat surprising to me that wallet_backwards_compatibility.py did not catch #27915 since the purpose of the test is to find downgrade issues such as that. It turns out the test was deficient in several places when it came to testing descriptor wallets, as well as deficient in addition to failing to correctly test some releases.

  This PR fixes these test cases, adds more informative logging, slightly refactors the entire test in order to better test future versions, and adds a 25.0 node to the test.

  Notable changes:
  * The compatibility test with 0.16 should not have been passing. The wallets were being copied incorrectly for 0.16 and resulting in 0.16 creating new wallets rather than testing the target wallets.
  * The downgrade test will actually be run on descriptor wallets and it will test that downgrades are successful, and a subsequent upgrade is also successful. This catches #27915.
  * The upgrade and downgrade test will be run on all versions up to master, rather than just 0.16, 0.17, and 0.19.

ACKs for top commit:
  Sjors:
    re-ACK afd9a673c4
  furszy:
    ACK afd9a67

Tree-SHA512: dd2d85cab29a636da93020681c533534af4a9cda18d8550c9db9d8937719b3a225025966981c5d4d2f30486448a772b760f0e723a25ea6bc49df80387dc7b8b0
2023-10-05 16:51:15 +01:00
Andrew Chow
db19a7e89d
Merge bitcoin/bitcoin#28403: test: Bump walletpassphrase timeouts to avoid intermittent issues
fa28f5a381 test: Bump walletpassphrase timeouts to avoid intermittent issues (MarcoFalke)

Pull request description:

  This bumps all timeouts for all `walletpassphrase` to avoid intermittent issues in `valgrind` (or other sanitizers).

  As an idea for a follow-up, `walletpassphrase` could be changed to treat `0` as "no timeout" instead of "instant timeout".

  Example failure:

  ```
   node0 2023-09-03T22:44:38.374955Z [httpworker.3] [rpc/server.cpp:594] [RPCRunLater] [rpc] queue run of timer lockwallet(w6) in 60 seconds (using HTTP)
   test  2023-09-03T22:44:40.173000Z TestFramework.bitcoincli (DEBUG): Running bitcoin-cli ['-rpcwallet=w6', 'getnewaddress', '', 'legacy']
   node0 2023-09-03T22:44:59.810893Z [http] [httpserver.cpp:255] [http_request_cb] [http] Received a POST request for /wallet/w6 from 127.0.0.1:48928
   node0 2023-09-03T22:44:59.813132Z [httpworker.1] [rpc/request.cpp:181] [parse] [rpc] ThreadRPCServer method=getnewaddress user=__cookie__
   node0 2023-09-03T22:44:59.837183Z [httpworker.1] [wallet/sqlite.cpp:53] [TraceSqlCallback] [/ci_container_base/ci/scratch/test_runner/test_runner_₿_🏃_20230903_183350/wallet_createwallet_171/node0/regtest/w6/wallet.dat] SQLite Statement: INSERT or REPLACE into main values(?, ?)
   node0 2023-09-03T22:44:59.929735Z [httpworker.1] [wallet/sqlite.cpp:53] [TraceSqlCallback] [/ci_container_base/ci/scratch/test_runner/test_runner_₿_🏃_20230903_183350/wallet_createwallet_171/node0/regtest/w6/wallet.dat] SQLite Statement: INSERT or REPLACE into main values(?, ?)
   node0 2023-09-03T22:44:59.934484Z [httpworker.1] [wallet/sqlite.cpp:53] [TraceSqlCallback] [/ci_container_base/ci/scratch/test_runner/test_runner_₿_🏃_20230903_183350/wallet_createwallet_171/node0/regtest/w6/wallet.dat] SQLite Statement: INSERT or REPLACE into main values(?, ?)
   node0 2023-09-03T22:44:59.935467Z [httpworker.1] [wallet/sqlite.cpp:53] [TraceSqlCallback] [/ci_container_base/ci/scratch/test_runner/test_runner_₿_🏃_20230903_183350/wallet_createwallet_171/node0/regtest/w6/wallet.dat] SQLite Statement: INSERT or REPLACE into main values(?, ?)
   test  2023-09-03T22:45:02.328000Z TestFramework.bitcoincli (DEBUG): Running bitcoin-cli ['-rpcwallet=w6', 'signmessage', 'mqatqH4VQmrZ81nxUfrnfcLnxgbzhZb4PC', 'test']
   node0 2023-09-03T22:45:20.269375Z [http] [httpserver.cpp:255] [http_request_cb] [http] Received a POST request for /wallet/w6 from 127.0.0.1:44618
   node0 2023-09-03T22:45:20.270670Z [httpworker.2] [rpc/request.cpp:181] [parse] [rpc] ThreadRPCServer method=signmessage user=__cookie__
   test  2023-09-03T22:45:23.490000Z TestFramework.bitcoincli (DEBUG): Running bitcoin-cli ['-rpcwallet=w6', 'keypoolrefill', '1']
   node0 2023-09-03T22:45:40.244603Z [http] [httpserver.cpp:255] [http_request_cb] [http] Received a POST request for /wallet/w6 from 127.0.0.1:32854
   node0 2023-09-03T22:45:40.293021Z [httpworker.0] [rpc/request.cpp:181] [parse] [rpc] ThreadRPCServer method=keypoolrefill user=__cookie__
   test  2023-09-03T22:45:41.852000Z TestFramework (ERROR): JSONRPC error
                                     Traceback (most recent call last):
                                       File "/ci_container_base/ci/scratch/build/bitcoin-x86_64-pc-linux-gnu/test/functional/test_framework/test_framework.py", line 131, in main
                                         self.run_test()
                                       File "/ci_container_base/ci/scratch/build/bitcoin-x86_64-pc-linux-gnu/test/functional/wallet_createwallet.py", line 156, in run_test
                                         w6.keypoolrefill(1)
                                       File "/ci_container_base/ci/scratch/build/bitcoin-x86_64-pc-linux-gnu/test/functional/test_framework/test_node.py", line 732, in __call__
                                         return self.cli.send_cli(self.command, *args, **kwargs)
                                                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
                                       File "/ci_container_base/ci/scratch/build/bitcoin-x86_64-pc-linux-gnu/test/functional/test_framework/test_node.py", line 795, in send_cli
                                         raise JSONRPCException(dict(code=int(code), message=message))
                                     test_framework.authproxy.JSONRPCException: Error: Please enter the wallet passphrase with walletpassphrase first. (-13)

ACKs for top commit:
  achow101:
    ACK fa28f5a381

Tree-SHA512: 58caa569cec39acc121d4cc038a4190937af34e85d2696272ed4f2792fd386469b0cfefd2cb564438fedded97b21b23d8bf46ba27b5633671a277ed4679f0d5d
2023-10-05 11:18:31 -04:00
MarcoFalke
fa071aeb61
wallet: No BDB creation, unless -deprecatedrpc=create_bdb 2023-10-05 15:47:44 +02:00
Ryan Ofsky
a9ef702a87 assumeutxo: change getchainstates RPC to return a list of chainstates
Current getchainstates RPC returns "normal" and "snapshot" fields which are not
ideal because it requires new "normal" and "snapshot" terms to be defined, and
the definitions are not really consistent with internal code. (In the RPC
interface, the "snapshot" chainstate becomes the "normal" chainstate after it
is validated, while in internal code there is no "normal chainstate" and the
"snapshot chainstate" is still called that temporarily after it is validated).

The current getchainstatees RPC is also awkward to use if you to want
information about the most-work chainstate because you have to look at the
"snapshot" field if it exists, and otherwise fall back to the "normal" field.

Fix these issues by having getchainstates just return a flat list of
chainstates ordered by work, and adding new chainstate "validated" field
alongside the existing "snapshot_blockhash" so it is explicit if a chainstate
was originally loaded from a snapshot, and whether the snapshot has been
validated.
2023-10-05 09:41:43 -04:00
furszy
0f83ab407e
test: display abrupt shutdown errors in console output
Making it easier to debug errors in the CI environment,
particularly in scenarios where it's not immediately clear
what happened nor which node crashed (or shutdown abruptly).
2023-10-05 09:45:09 -03:00
MarcoFalke
fa28f5a381
test: Bump walletpassphrase timeouts to avoid intermittent issues 2023-10-05 12:57:33 +02:00
Sebastian Falbesoner
e1308967e1 test: BIP324: add checks for v1 prefix matching / wrong network magic detection 2023-10-05 00:00:09 +02:00
Andrew Chow
ab163b0fb5
Merge bitcoin/bitcoin#27823: init: return error when block index is non-contiguous, fix feature_init.py file perturbation
d27b9a2248 test: fix feature_init.py file perturbation (Martin Zumsande)
ad66ca1e47 init: abort loading of blockindex in case of missing height. (Martin Zumsande)

Pull request description:

  When the block index database is non-contiguous due to file corruption (i.e. it contains indexes of height `x-1` and `x+1`, but not `x`), bitcoind can currently crash with an assert in `BuildSkip()` / `GetAncestor()` during `BlockManager::LoadBlockIndex()`:
  ```
  bitcoind: chain.cpp:112: const CBlockIndex* CBlockIndex::GetAncestor(int) const: Assertion `pindexWalk->pprev' failed.
  ```
  This PR changes it such that we instead return an `InitError` to the user.

  I stumbled upon this because I noticed that the file perturbation in `feature_init.py`  wasn't working as intended, which is fixed in the second commit:
  * Opening the file twice in one `with` statement would lead to `tf_read` being empty, so the test wouldn't perturb anything but replace the file with a new one. Fixed by first opening for read, then for write.
  * We need to restore the previous state after perturbations, so that only the current perturbation is active and not a mix of the current and previous ones.
  * I also added `checkblocks=200` to the startup parameters so that corruption in earlier blocks of `blk00000.dat` is detected during init verification and not ignored.

  After fixing `feature_init.py` like that I'd run into the `assert` mentioned above (so running the testfix from the second commit without the first one is a way to reproduce it).

ACKs for top commit:
  achow101:
    ACK d27b9a2248
  furszy:
    Code ACK d27b9a224
  fjahr:
    Code review ACK d27b9a2248

Tree-SHA512: 2e54da6030c5813c86bd58f816401e090bb43c5b834764a5e3c0e55dbfe09e423f88042cab823db3742088204b274d4ad2abf58a3832a4b18328b11a30bf7094
2023-10-04 15:36:57 -04:00
James O'Beirne
7e40032260 tests: assumeutxo: accept final height from either chainstate 2023-10-04 11:32:36 -04:00
James O'Beirne
5bd2010f02 test: assumeutxo: avoid race in functional test 2023-10-04 11:05:27 -04:00
James O'Beirne
7005a01c19 test: add wait_for_connect to BitcoinTestFramework.connect_nodes 2023-10-04 11:05:03 -04:00
stratospher
3931e6abc3 rpc: getaddrmaninfo followups
- make `getaddrmaninfo` RPC public since it's not for development
  purposes only and regular users might find it useful
- add missing `all_networks` key to RPC help
- use clang format spacing
2023-10-04 08:34:30 +05:30
Andrew Chow
01bd9d7b99
Merge bitcoin/bitcoin#28523: rpc: add hidden getrawaddrman RPC to list addrman table entries
352d5eb2a9 test: getrawaddrman RPC (0xb10c)
da384a286b rpc: getrawaddrman for addrman entries (0xb10c)

Pull request description:

  Inspired by `getaddrmaninfo` (#27511), this adds a hidden/test-only `getrawaddrman` RPC. The RPC returns information on all addresses in the address manager new and tried tables. Addrman table contents can be used in tests and during development.

  The RPC result encodes the `bucket` and `position`, the internal location of addresses in the tables, in the address object's string key. This allows users to choose to consume or to ignore the location information. If the internals of the address manager implementation change, the location encoding might change too.

  ```
  getrawaddrman

  EXPERIMENTAL warning: this call may be changed in future releases.

  Returns information on all address manager entries for the new and tried tables.

  Result:
  {                                  (json object)
    "table" : {                      (json object) buckets with addresses in the address manager table ( new, tried )
      "bucket/position" : {          (json object) the location in the address manager table (<bucket>/<position>)
        "address" : "str",           (string) The address of the node
        "port" : n,                  (numeric) The port number of the node
        "network" : "str",           (string) The network (ipv4, ipv6, onion, i2p, cjdns) of the address
        "services" : n,              (numeric) The services offered by the node
        "time" : xxx,                (numeric) The UNIX epoch time when the node was last seen
        "source" : "str",            (string) The address that relayed the address to us
        "source_network" : "str"     (string) The network (ipv4, ipv6, onion, i2p, cjdns) of the source address
      },
      ...
    },
    ...
  }

  Examples:
  > bitcoin-cli getrawaddrman
  > curl --user myusername --data-binary '{"jsonrpc": "1.0", "id": "curltest", "method": "getrawaddrman", "params": []}' -H 'content-type: text/plain;' http://127.0.0.1:8332/
  ```

ACKs for top commit:
  willcl-ark:
    reACK 352d5eb2a9
  amitiuttarwar:
    reACK 352d5eb2a9
  stratospher:
    reACK 352d5eb.
  achow101:
    ACK 352d5eb2a9

Tree-SHA512: cc462666b5c709617c66b0e3e9a17c4c81e9e295f91bdd9572492d1cb6466fc9b6d48ee805ebe82f9f16010798370effe5c8f4db15065b8c7c0d8637675d615e
2023-10-03 11:38:20 -04:00
fanquake
4e78834ec1
Merge bitcoin/bitcoin#28304: doc: Remove confusing assert linter
fa6e6a3f03 doc: Remove confusing assert linter (MarcoFalke)

Pull request description:

  The `assert()` documentation and linter are redundant and confusing:

  * The source code already refuses to compile with `assert()` disabled.
  * They violate the assumptions about `Assert()`, which *requires* side effects.
  * The existing linter doesn't enforce the guideline, only checking for `++` and `--` side effects.

  Fix all issues by removing the docs and the linter. See also https://github.com/bitcoin/bitcoin/pull/26684#discussion_r1287370102

  Going forward everyone is free to use whatever code in this regard they think is the easiest to read. Also, everyone is still free to share style-nits, if they think it is a good use of their time and of the pull request author. Finally, the author is still free to dismiss or ignore this style-nit, or any other style-nit.

ACKs for top commit:
  hebasto:
    ACK fa6e6a3f03, I have reviewed the code and it looks OK.
  theStack:
    ACK fa6e6a3f03

Tree-SHA512: 686738d71e1316cc95e5d3f71869b55a02bfb137c795cc0875057f4410e564bc8eff03c985a2087b007fb08fc84551c7da1e8b30c7a9c3f2b14e5e44a5970236
2023-10-03 10:44:21 +01:00
Pieter Wuille
64ca7210f0 test: enable v2 transport between nodes in some functional tests 2023-10-02 18:11:11 -04:00
dhruv
05d19fbcc1 test: Functional test for opportunistic encryption
Co-authored-by: Pieter Wuille <bitcoin-dev@wuille.net>
2023-10-02 18:11:11 -04:00
Pieter Wuille
b815cce50e net: expose transport types/session IDs of connections in RPC and logs
Co-authored-by: Dhruv Mehta <856960+dhruv@users.noreply.github.com>
2023-10-02 18:11:11 -04:00
Pieter Wuille
62d21ee097 net: use V2Transport when NODE_P2P_V2 service flag is present
Co-authored-by: Dhruv Mehta <856960+dhruv@users.noreply.github.com>
2023-10-02 18:09:53 -04:00
Pieter Wuille
abf343b320 net: advertise NODE_P2P_V2 if CLI arg -v2transport is on
Co-authored-by: Dhruv Mehta <856960+dhruv@users.noreply.github.com>
2023-10-02 18:09:53 -04:00
Andrew Chow
e7b0004b37
Merge bitcoin/bitcoin#27596: assumeutxo (2)
edbed31066 chainparams: add signet assumeutxo param at height 160_000 (Sjors Provoost)
b8cafe3871 chainparams: add testnet assumeutxo param at height 2_500_000 (Sjors Provoost)
99839bbfa7 doc: add note about confusing HaveTxsDownloaded name (James O'Beirne)
7ee46a755f contrib: add script to demo/test assumeutxo (James O'Beirne)
42cae39356 test: add feature_assumeutxo functional test (James O'Beirne)
0f64bac603 rpc: add getchainstates (James O'Beirne)
bb05857794 refuse to activate a UTXO snapshot if mempool not empty (James O'Beirne)
ce585a9a15 rpc: add loadtxoutset (James O'Beirne)
62ac519e71 validation: do not activate snapshot if behind active chain (James O'Beirne)
9511fb3616 validation: assumeutxo: swap m_mempool on snapshot activation (James O'Beirne)
7fcd21544a blockstorage: segment normal/assumedvalid blockfiles (James O'Beirne)
4c3b8ca35c validation: populate nChainTx value for assumedvalid chainstates (James O'Beirne)
49ef778158 test: adjust chainstate tests to use recognized snapshot base (James O'Beirne)
1019c39982 validation: pruning for multiple chainstates (James O'Beirne)
373cf91531 validation: indexing changes for assumeutxo (James O'Beirne)
1fffdd76a1 net_processing: validationinterface: ignore some events for bg chain (James O'Beirne)
fbe0a7d7ca wallet: validationinterface: only handle active chain notifications (James O'Beirne)
f073917a9e validationinterface: only send zmq notifications for active (James O'Beirne)
4d8f4dcb45 validation: pass ChainstateRole for validationinterface calls (James O'Beirne)
1e59acdf17 validation: only call UpdatedBlockTip for active chainstate (James O'Beirne)
c6af23c517 validation: add ChainstateRole (James O'Beirne)
9f2318c76c validation: MaybeRebalanceCaches when chain leaves IBD (James O'Beirne)
434495a8c1 chainparams: add blockhash to AssumeutxoData (James O'Beirne)
c711ca186f assumeutxo: remove snapshot during -reindex{-chainstate} (James O'Beirne)
c93ef43e4f bugfix: correct is_snapshot_cs in VerifyDB (James O'Beirne)
b73d3bbd23 net_processing: Request assumeutxo background chain blocks (Suhas Daftuar)

Pull request description:

  - Background and FAQ: https://github.com/jamesob/assumeutxo-docs/tree/2019-04-proposal/proposal
  - Prior progress/project: https://github.com/bitcoin/bitcoin/projects/11
  - Replaces https://github.com/bitcoin/bitcoin/pull/15606, which was closed due to Github slowness. Original description and commentary can be found there.

  ---

  This changeset finishes the first phase of the assumeutxo project. It makes UTXO snapshots loadable via RPC (`loadtxoutset`) and adds `assumeutxo` parameters to chainparams. It contains all the remaining changes necessary to both use an assumedvalid snapshot chainstate and do a full validation sync in the background.

  This may look like a lot to review, but note that
  - ~200 lines are a (non-essential) demo shell script
  - Many lines are functional test, documentation, and relatively dilute RPC code.

  So it shouldn't be as burdensome to review as the linecount might suggest.

  - **P2P**: minor changes are made to `init.cpp` and `net_processing.cpp` to make simultaneous IBD across multiple chainstates work.
  - **Pruning**: implement correct pruning behavior when using a background chainstate
  - **Blockfile separation**: to prevent "fragmentation" in blockfile storage, have background chainstates use separate blockfiles from active snapshot chainstates to avoid interleaving heights and impairing pruning.
  - **Indexing**: some `CValidationInterface` events are given with an additional parameter, ChainstateRole, and all indexers ignore events from ChainstateRole::ASSUMEDVALID so that indexation only happens sequentially.
  - Have `-reindex` properly wipe snapshot chainstates.
  - **RPC**: introduce RPC commands `loadtxoutset` and (hidden) `getchainstates`.
  - **Release docs & first assumeutxo commitment**: add notes and a particular assumeutxo hash value for first AU-enabled release.
    - This will complete the project and allow use of UTXO snapshots for faster node bootstrap.

  The next phase, if it were to be pursued, would be coming up with a way to distribute the UTXO snapshots over the P2P network.

  ---

  ### UTXO snapshots

  Create your own with `./contrib/devtools/utxo_snapshot.sh`, e.g.
  ```shell
  ./contrib/devtools/utxo_snapshot.sh 788000 utxo.dat ./src/bitcoin-cli -datadir=$(pwd)/testdata`)
  ```
  or use the pre-generated ones listed below.

  - Testnet: **2'500'000** (Sjors):
    - torrent: `magnet:?xt=urn:btih:511e09f4bf853aefab00de5c070b1e031f0ecbe9&dn=utxo-testnet-2500000.dat&tr=udp%3A%2F%2Ftracker.bitcoin.sprovoost.nl%3A6969`
    - sha256: `79db4b025448cc0ac388d8589a28eab02de53055d181e34eb47391717aa16388`
  - Signet: **160'000** (Sjors):
    - torrent: `magnet:?xt=urn:btih:9da986cb27b3980ea7fd06b21e199b148d486880&dn=utxo-signet-160000.dat&tr=udp%3A%2F%2Ftracker.bitcoin.sprovoost.nl%3A6969`
    - sha256: `eeeca845385ba91e84ef58c09d38f98f246a24feadaad57fe1e5874f3f92ef8c`
  - Mainnet: **800'000** (Sjors):
    - Note: this needs the following commit cherry-picked in: 24deb2022b
    - torrent: `magnet:?xt=urn:btih:50ee955bef37f5ec3e5b0df4cf0288af3d715a2e&dn=utxo-800000.dat&tr=udp%3A%2F%2Ftracker.bitcoin.sprovoost.nl%3A6969`

  ### Testing

  #### For fun (~5min)

  If you want to do a quick test, you can run `./contrib/devtools/test_utxo_snapshots.sh` and follow the instructions. This is mostly obviated by the functional tests, though.

  #### For real (longer)

  If you'd like to experience a real usage of assumeutxo, you can do that too.
  I've cut a new snapshot at height 788'000 (http://img.jameso.be/utxo-788000.dat - but you can do it yourself with `./contrib/devtools/utxo_snapshot.sh` if you want). Download that, and then create a datadir for testing:
  ```sh
  $ cd ~/src/bitcoin  # or whatever

  # get the snapshot
  $ curl http://img.jameso.be/utxo-788000.dat > utxo-788000.dat

  # you'll want to do this if you like copy/pasting
  $ export AU_DATADIR=/home/${USER}/au-test # or wherever

  $ mkdir ${AU_DATADIR}
  $ vim ${AU_DATADIR}/bitcoin.conf

  dbcache=8000  # or, you know, something high
  blockfilterindex=1
  coinstatsindex=1
  prune=3000
  logthreadnames=1
  ```
  Obtain this branch, build it, and then start bitcoind:
  ```sh
  $ git remote add jamesob https://github.com/jamesob/bitcoin
  $ git fetch jamesob assumeutxo
  $ git checkout jamesob/assumeutxo

  $ ./configure $conf_args && make  # (whatever you like to do here)

  # start 'er up and watch the logs
  $ ./src/bitcoind -datadir=${AU_DATADIR}
  ```
  Then, in some other window, load the snapshot
  ```sh
  $ ./src/bitcoin-cli -datadir=${AU_DATADIR} loadtxoutset $(pwd)/utxo-788000.dat
  ```

  You'll see some log messages about headers retrieval and waiting to see the snapshot in the headers chain. Once you get the full headers chain, you'll spend a decent amount of time (~10min) loading the snapshot, checking it, and flushing it to disk. After all that happens, you should be syncing to tip in pretty short order, and you'll see the occasional `[background validation]` log message go by.

  In yet another window, you can check out chainstate status with
  ```sh
  $ ./src/bitcoin-cli -datadir=${AU_DATADIR} getchainstates
  ```
  as well as usual favorites like `getblockchaininfo`.

ACKs for top commit:
  achow101:
    ACK edbed31066

Tree-SHA512: 6086fb9a38dc7df85fedc76b30084dd8154617a2a91e89a84fb41326d34ef8e7d7ea593107afba01369093bf8cc91770621d98f0ea42a5b3b99db868d2f14dc2
2023-10-02 17:09:44 -04:00