fa875349e2 Fix iwyu (MacroFake)
faad673716 Fix issues when calling std::move(const&) (MacroFake)
Pull request description:
Passing a symbol to `std::move` that is marked `const` is a no-op, which can be fixed in two ways:
* Remove the `const`, or
* Remove the `std::move`
ACKs for top commit:
ryanofsky:
Code review ACK fa875349e2. Looks good. Good for univalue to support c++11 move optimizations
Tree-SHA512: 3dc5cad55b93cfa311abedfb811f35fc1b7f30a1c68561f15942438916c7de25e179c364be11881e01f844f9c2ccd71a3be55967ad5abd2f35b10bb7a882edea
416ceb8661 descriptor: check if `rawtr` has only one key. (w0xlt)
Pull request description:
If I understand `rawtr` descriptor correctly, it should only allow `rawtr(KEY)`, not `rawtr(KEY1, KEY2, ...)` or other concatenations.
On master branch, `rawtr(KEY1, KEY2, ...)` will produce the `rawtr(KEY1)` descriptor ignoring the `KEY2, ...` with no error messages or warnings.
For example, the code below will print `rawtr(tprv8ZgxMBicQKsPefef2Doobbq3xTCaVTHcDn6me82KSXY1vY9AJAWD5u7SDM4XGLfc4EoXRMFrJKpp6HNmQWA3FTMRQeEmMJYJ9RPqe9ne2hU/*)#lx9qryfh`
for the supposedly invalid descriptor
`rawtr(tprv8ZgxMBicQKsPefef2Doobbq3xTCaVTHcDn6me82KSXY1vY9AJAWD5u7SDM4XGLfc4EoXRMFrJKpp6HNmQWA3FTMRQeEmMJYJ9RPqe9ne2hU/*, tprv8ZgxMBicQKsPezQ2KGArMRovTEbCGxaLgBgaVcTvEx8mby8ogX2bgC4HBapH4yMwrz2FpoCuA17eocuUVMgEP6fnm83YpwSDTFrumw42bny/*)`
```python
self.nodes[1].createwallet(wallet_name="rawtr_multi", descriptors=True, blank=True)
rawtr_multi = self.nodes[1].get_wallet_rpc("rawtr_multi")
rawtr_multi_desc = "rawtr(tprv8ZgxMBicQKsPefef2Doobbq3xTCaVTHcDn6me82KSXY1vY9AJAWD5u7SDM4XGLfc4EoXRMFrJKpp6HNmQWA3FTMRQeEmMJYJ9RPqe9ne2hU/*, tprv8ZgxMBicQKsPezQ2KGArMRovTEbCGxaLgBgaVcTvEx8mby8ogX2bgC4HBapH4yMwrz2FpoCuA17eocuUVMgEP6fnm83YpwSDTFrumw42bny/*)#uv78hkt0"
result = rawtr_multi.importdescriptors([{"desc": rawtr_multi_desc, "active": True, "timestamp": "now"}])
print(rawtr_multi.listdescriptors(True))
```
This PR adds a check that prevents `rawtr` descriptors from being created if more than one key is entered, shows an error message, and adds a test for this case.
ACKs for top commit:
achow101:
ACK 416ceb8661
sipa:
ACK 416ceb8661
Tree-SHA512: a2009e91f1bca6ee79cc68f65811caa6a21fc8b80acd8dc58e283f424b41fe53b0db7ce3693b1c7e2184ff571e6d1fbb9f5ccde89b65d3026726f3393c492044
b16f93cadd script/sign: remove needless IsSolvable() utility (Antoine Poinsot)
c232ef20c0 outputtype: remove redundant check for uncompressed keys in AddAndGetDestinationForScript (Antoine Poinsot)
Pull request description:
Now that we have descriptors there is no need to try to sign for a scriptPubKey using dummy signatures, and using a mocked verification of this witness against the interpreter, just to make sure we know how to spend such a Script. Just try to infer a solvable descriptor: any scriptPubKey that we can sign for can be inferred as such.
This came up in #24149 but i think it's worth it on its own.
ACKs for top commit:
instagibbs:
ACK b16f93cadd
achow101:
re-ACK b16f93cadd
furszy:
ACK b16f93ca, only change is the `IsSolvable` helper function removal.
Tree-SHA512: 137068157ce90210b710b1bf9ac3c400e2ff5af1112f892094b69875ea473d6a899f52adb51e5030cb907dee517602059cd1661107808558efa5de842ba12b41
fb9faffae3 extended keys: fail to derive too large depth instead of wrapping around (Antoine Poinsot)
8dc6670ce1 descriptor: don't assert success of extended key derivation (Antoine Poinsot)
50cfc9e761 (pubk)key: mark Derive() as nodiscard (Antoine Poinsot)
0ca258a5ac descriptor: never ignore the return value when deriving an extended key (Antoine Poinsot)
d3599c22bd spkman: don't ignore the return value when deriving an extended key (Antoine Poinsot)
Pull request description:
We would previously silently wrap the derived child's depth back to `0`. Instead, explicitly fail when trying to derive an impossible depth, and handle the error in callers.
An extended fuzzing corpus of `descriptor_parse` triggered this behaviour, which was reported by MarcoFalke.
Fixes#25751.
ACKs for top commit:
achow101:
re-ACK fb9faffae3
instagibbs:
utACK fb9faffae3
Tree-SHA512: 9f75c23572ce847239bd15e5497df2960b6bd63c61ea72347959d968b5c4c9a4bfeee284e76bdcd7bacbf9eeb70feee85ffd3e316f353ca6eca30e93aafad343
544b4332f0 Add wallet tests for spending rawtr() (Pieter Wuille)
e1e3081200 If P2TR tweaked key is available, sign with it (Pieter Wuille)
8d9670ccb7 Add rawtr() descriptor for P2TR with unknown tweak (Pieter Wuille)
Pull request description:
It may be useful to be able to represent P2TR outputs in descriptors whose script tree and/or internal key aren't known. This PR does that, by adding a `rawtr(KEY)` descriptor, where the KEY represents the output key directly. If the private key corresponding to that output key is known, it also permits signing with it.
I'm not convinced this is desirable, but presumably "tr(KEY)" sounds more intended for direct use than "rawtr(KEY)".
ACKs for top commit:
achow101:
ACK 544b4332f0
sanket1729:
code review ACK 544b4332f0
w0xlt:
reACK 544b4332f0
Tree-SHA512: 0de08de517468bc22ab0c00db471ce33144f5dc211ebc2974c6ea95709f44e830532ec5cdb0128c572513d352120bd651c4559516d4500b5b0a3d257c4b45aca
Also:
- Make DEFAULT_MAX_SIG_CACHE_SIZE into constexpr
DEFAULT_MAX_SIG_CACHE_BYTES to utilize the compile-time integer
arithmetic overflow checking available to constexpr.
- Fix comment (MiB instead of MB) for DEFAULT_MAX_SIG_CACHE_BYTES.
- Pass in max_size_bytes parameter to InitS*Cache(), modify log line to
no longer allude to maxsigcachesize being split evenly between the two
validation caches.
- Fix possible integer truncation and add a comment.
[META] I've kept the integer types as int64_t in order to not introduce
unintended behaviour changes, in the next commit we will make
them size_t.
This fixes an potential overflow which existed prior to this patchset.
If CuckooCache::cache<Element, Hash>::setup_bytes is called with a
`size_t bytes` which, when divided by sizeof(Element), does not fit into
an uint32_t, the implicit conversion to uint32_t in the call to setup
will result in an overflow.
At least on x86_64, this overflow is possible:
static_assert(std::numeric_limits<size_t>::max() / 32 <= std::numeric_limits<uint32_t>::max());
static_assert(std::numeric_limits<size_t>::max() / 4 <= std::numeric_limits<uint32_t>::max());
This commit detects such cases and signals to callers that the `size_t
bytes` input is too large.
1. -maxsigcachesize is a DEBUG_ONLY option
2. Almost 7 years has passed since its semantics change in
830e3f3d02 from "number of entries" to
"number of mebibytes"
3. A std::new_handler was added to the codebase after the original PR
which introduced this limit, which will terminate immediately instead
of causing trouble by being caught somewhere unexpected.
Returning the approximate total size eliminates the need for
InitS*Cache() to do nElems*sizeof(uint256). The cuckoocache has a better
idea of this information.
The counter is an optimization over calling `ret.empty()`. It was
suggested that the compiler would realize `cnt` is only `0` on the first
iteration, and not actually emit the check and conditional.
This optimization was actually not triggered at all, since we
incremented `cnt` at the beginning of the first iteration. Fix it by
incrementing at the end instead.
This was reported by Github user "Janus".
Miniscript descriptors are defined under P2WSH context (either `wsh()`
or `sh(wsh())`).
Only sane Miniscripts are accepted, as insane ones (although valid by
type) can have surprising behaviour with regard to malleability
guarantees and resources limitations.
As Miniscript descriptors are longer and more complex than "legacy"
descriptors, care was taken in error reporting to help a user determine
for what reason a provided Miniscript is insane.
Co-authored-by: Pieter Wuille <pieter.wuille@gmail.com>
This is helpful for finer grained descriptor parsing error: when there
are multiple errors to report in a Miniscript descriptor start with the
"smallest" fragments: the ones closer to be a leaf.
Co-Authored-By: Pieter Wuille <pieter@wuille.net>
The taproot spenddata stored in a sigdata is the combination of data
existing previously (e.g. in a PSBT) and the data stored in a
SigningProvider. In order to use the external data when signing, we need
to be using the sigdata's spenddata.
GetSpendData needs to be finalized in order to be used. To avoid future
bugs, assert `!m_output_key.IsNull()` as m_output_key is only set during
Finalize.
TaprootSpendData can be gotten from TaprootBuilder, however for PSBT, we
also need TaprootBuilders directly (for the outputs). So we store the
TaprootBuilder in the FlatSigningProvider and when the TaprootSpendData
is needed, we generate it on the fly using the stored builder.
GetTreeTuples returns the leaves in DFS order as tuples of depth, leaf
version, and script. This is a representation of the tree that can be
serialized.
f3a50c9dfe miniscript: rename IsSane and IsSaneSubexpression to prevent misuse (Antoine Poinsot)
c5fe5163dc miniscript: nit: don't return after assert(false) (Antoine Poinsot)
7bbaca9d8d miniscript: explicit the threshold size computation in multi() (Antoine Poinsot)
8323e4249d miniscript: add an OpCode typedef for readability (Antoine Poinsot)
7a549c6c59 miniscript: mark nodes with duplicate keys as insane (Antoine Poinsot)
8c0f8bf7bc fuzz: add a Miniscript target for string representation roundtripping (Antoine Poinsot)
be34d5077b fuzz: rename and improve the Miniscript Script roundtrip target (Antoine Poinsot)
7eb70f0ac0 miniscript: tiny doc fixups (Antoine Poinsot)
5cea85f12c miniscript: split ValidSatisfactions from IsSane (Antoine Poinsot)
a0f064dc14 miniscript: introduce a CheckTimeLocksMix helper (Antoine Poinsot)
ed45ee3882 miniscript: use optional instead of bool/outarg (Antoine Poinsot)
1ab8d89fd1 miniscript: make equality operator non-recursive (Antoine Poinsot)
5922c662c0 scripted-diff: miniscript: rename 'nodetype' variables to 'fragment' (Antoine Poinsot)
c5f65db0f0 miniscript: remove a workaround for a GCC 4.8 bug (Antoine Poinsot)
Pull request description:
The Miniscript repository and the Miniscript integration PR here have been a moving target for the past months, and some final cleanups were done there that were not included here. I initially intended to add some small followup commits to #24148 but i think there are enough of them to be worth a followup PR on its own.
Some parts of the code did not change since it was initially written in 2019, and the code could use some modernization. (Use std::optional instead of out args, remove old compiler workarounds).
We refactored the helpers to be more meaningful, and also did some renaming. A new fuzz target was also added and both were merged in a single file. 2 more will be added in #24149 that will be contained in this file too.
The only behaviour change in this PR is to rule out Miniscript with duplicate keys from sane Miniscripts. In a P2WSH context, signatures can be rebounded (Miniscript does not use CODESEPARATOR) and it's reasonable to assume that reusing keys across the Script drops the malleability guarantees.
It was previously assumed such Miniscript would never exist in the first place since a compiler should never create them. We finally agreed that if one were to exist (say, written by hand or from a buggy compiler) it would be very confusing if an imported Miniscript descriptor (after #24148) with duplicate keys was deemed sane (ie, "safe to use") by Bitcoin Core. We now check for duplicate keys in the constructor.
This is (still) joint work with Pieter Wuille. (Actually he entirely authored the cleanups and code modernization.)
ACKs for top commit:
sipa:
utACK f3a50c9dfe (with the caveat that a lot of it is my own code)
sanket1729:
code review ACK f3a50c9dfe. Did not review the fuzz tests.
Tree-SHA512: c043325e4936fe25e8ece4266b46119e000c6745f88cea530fed1edf01c80f03ee6f9edc83b6e9d42ca01688d184bad16bfd967c5bb8037744e726993adf3deb
As stated on the website, duplicate keys make it hard to reason about
malleability as a single signature may unlock multiple paths.
We use a custom KeyCompare function instead of operator< to be explicit
about the requirement.
BIP 341 specifies constraints on the size of the control block _c_ used
to compute the taproot merkle root.
> The last stack element is called the control block _c_, and must have
> length _33 + 32m_, for a value of m that is an integer between 0 and
> 128, inclusive. Fail if it does not have such a length.
(See BIP-341 "Script Validation Rules" here: https://github.com/bitcoin/bips/blob/master/bip-0341.mediawiki#script-validation-rules)
Since the removal of NODISCARD in 81d5af42f4,
the only attributes def is LIFETIMEBOUND, and it's included in many more
places that it is used.
This removes all includes which do not have an associated use of LIFETIMEBOUND,
and adds it to the following files, due to their use of the same:
* src/validationinterface.h
* src/script/standard.h
This makes IsSane clearer. It is useful to differentiate between 'potential non-malleable satisfactions are valid' and 'such satisfactions exist' for testing.
Co-authored-by: Pieter Wuille <pieter.wuille@gmail.com>
The 'Fragment' type was previously named 'Nodetype'. For clarity, name
the variables the same.
-BEGIN VERIFY SCRIPT-
sed -i 's/nodetype/fragment/g' src/script/miniscript.*
-END VERIFY SCRIPT-
Co-authored-by: Pieter Wuille <pieter.wuille@gmail.com>
The value it leaves on the stack depends on the last element on the
stack. However, we can't make sure this element is OP_1 (which would
give us the 'u' property) without the MINIMALIF rule.
MINIMALIF is only policy for P2WSH, therefore giving 'd:' the 'u'
property breaks consensus soundness: it makes it possible (by consensus
but not policy) for instance to satisfy a thresh() without satisfying
at least k of its subs.
This bug was found and reported by Andrew Poelstra.
2da94a4c6f fuzz: add a fuzz target for Miniscript decoding from Script (Antoine Poinsot)
f8369996e7 Miniscript: ops limit and stack size computation (Pieter Wuille)
2e55e88f86 Miniscript: conversion from script (Pieter Wuille)
1ddaa66eae Miniscript: type system, script creation, text notation, tests (Pieter Wuille)
4fe29368c0 script: expose getter for CScriptNum, add a BuildScript helper (Antoine Poinsot)
f4e289f384 script: move CheckMinimalPush from interpreter to script.h (Antoine Poinsot)
31ec6ae92a script: make IsPushdataOp non-static (Antoine Poinsot)
Pull request description:
Miniscript is a language for writing (a subset of) Bitcoin Scripts in a structured way.
Miniscript permits:
- To safely extend the Output Descriptor language to many more scripting features thanks to the typing system (composition).
- Statical analysis of spending conditions, maximum spending cost of each branch, security properties, third-party malleability.
- General satisfaction of any correctly typed ("valid" [0]) Miniscript. The satisfaction itself is also analyzable.
- To extend the possibilities of external signers, because of all of the above and since it carries enough metadata.
Miniscript guarantees:
- That for any statically-analyzed as "safe" [0] Script, a witness can be constructed in the bounds of the consensus and standardness rules (standardness complete).
- That unless the conditions of the Miniscript are met, no witness can be created for the Script (consensus sound).
- Third-party malleability protection for the satisfaction of a sane Miniscript, which is too complex to summarize here.
For more details around Miniscript (including the specifications), please refer to the [website](https://bitcoin.sipa.be/miniscript/).
Miniscript was designed by Pieter Wuille, Andrew Poelstra and Sanket Kanjalkar.
This PR is an updated and rebased version of #16800. See [the commit history of the Miniscript repository](https://github.com/sipa/miniscript/commits/master) for details about the changes made since September 2019 (TL;DR: bugfixes, introduction of timelock conflicts in the type system, `pk()` and `pkh()` aliases, `thresh_m` renamed to `multi`, all recursive algorithms were made non-recursive).
This PR is also the first in a series of 3:
- The first one (here) integrates the backbone of Miniscript.
- The second one (#24148) introduces support for Miniscript in Output Descriptors, allowing for watch-only support of Miniscript Descriptors in the wallet.
- The third one (#24149) implements signing for these Miniscript Descriptors, using Miniscript's satisfaction algorithm.
Note to reviewers:
- Miniscript is currently defined only for P2WSH. No Taproot yet.
- Miniscript is different from the policy language (a high-level logical representation of a spending policy). A policy->Miniscript compiler is not included here.
- The fuzz target included here is more interestingly extended in the 3rd PR to check a script's satisfaction against `VerifyScript`. I think it could be further improved by having custom mutators as we now have for multisig (see https://github.com/bitcoin/bitcoin/issues/23105). A minified corpus of Miniscript Scripts is available at https://github.com/bitcoin-core/qa-assets/pull/85.
[0] We call "valid" any correctly-typed Miniscript. And "safe" any sane Miniscript, ie one whose satisfaction isn't malleable, which requires a key for any spending path, etc..
ACKs for top commit:
jb55:
ACK 2da94a4c6f
laanwj:
Light code review ACK 2da94a4c6f (mostly reviewed the changes to the existing code and build system)
Tree-SHA512: d3ef558436cfcc699a50ad13caf1e776f7d0addddb433ee28ef38f66ea5c3e581382d8c748ccac9b51768e4b95712ed7a6112b0e3281a6551e0f325331de9167
9b52672700 For descriptor pubkey parse errors, include context information (Ben Woosley)
Pull request description:
This adds readily-available context information to the error string, for further disambiguation.
This is a revival of #16123 which was largely addressed in #16542.
Note 'Multi:' is used rather than 'multi():' as it also encompasses 'sortedmulti():'
ACKs for top commit:
achow101:
ACK 9b52672700
theStack:
ACK 9b52672700
Tree-SHA512: 96533ea8c3ac7010f9b62e75b4bd20b65aff843030eb91c7a88312975acecaaf17909b7d1841f45edc86dbf7fa402d208adb85f0673bd79b857dbebacb8c9395
More information about Miniscript can be found at https://bitcoin.sipa.be/miniscript/ (the
website source is hosted at https://github.com/sipa/miniscript/).
This commit defines all fragments, their composition, parsing from
string representation and conversion to Script.
Co-Authored-By: Antoine Poinsot <darosior@protonmail.com>
Co-Authored-By: Sanket Kanjalkar <sanket1729@gmail.com>
Co-Authored-By: Samuel Dobson <dobsonsa68@gmail.com>
Some prep work for Miniscript. BuildScript is an efficient way to build
Scripts in a generic manner (by concatenating OPs, data, and other
Scripts).
Co-Authored-By: Pieter Wuille <pieter@wuille.net>
fae20e6b50 Revert "Avoid the use of P0083R3 std::set::merge" (MarcoFalke)
fab53b5fd4 ci/doc: Set minimum required clang/libc++ version to 8.0 (MarcoFalke)
Pull request description:
This is not for 23.0, but for 24.0. It comes with the following benefits:
* Can use C++17 P0083R3 std::set::merge from libc++ 8.0
* No longer need to provide support for clang-7, which already fails to compile on some architectures (https://github.com/bitcoin/bitcoin/issues/21294#issuecomment-998098483)
This should be fine, given that all supported operating systems ship with at least clang-10:
* CentOS 8: clang-12
* Stretch: https://packages.debian.org/stretch/clang-11
* Buster: https://packages.debian.org/buster-backports/clang-11
* Bionic: https://packages.ubuntu.com/bionic-updates/clang-10
* Focal: https://packages.ubuntu.com/focal/clang-10
ACKs for top commit:
fanquake:
ACK fae20e6b50 - I think this is fine to do. I would be surprised if in another 6 months time someone was stuck on a system we supported, needing to compile Core, and only had access to Clang 7 or older. As mentioned in the PR description, all systems we currently support, already support multiple newer versions of Clang.
hebasto:
ACK fae20e6b50, I have reviewed the code and it looks OK, I agree it can be merged.
Tree-SHA512: 3b4c6c130ff40dd7e84934af076863415e5dd661d823c72e3e3832566c65be6e877a7ef9164bbcf394bcea4b897fc29a48db0f231c22ace0e2c9b5638659a628
4828d53ecc Add (sorted)multi_a descriptors to doc/descriptors.md (Pieter Wuille)
b5f33ac1f8 Simplify wallet_taproot.py functional test (Pieter Wuille)
eb0667ea96 Add tests for (sorted)multi_a derivation/signing (Pieter Wuille)
c17c6aa08d Add signing support for (sorted)multi_a scripts (Pieter Wuille)
3eed6fca57 Add multi_a descriptor inference (Pieter Wuille)
79728c4a3d Add (sorted)multi_a descriptor and script derivation (Pieter Wuille)
25e95f9ff8 Merge/generalize IsValidMultisigKeyCount/GetMultisigKeyCount (Pieter Wuille)
Pull request description:
This adds a new `multi_a(k,key_1,key_2,...,key_n)` (and corresponding `sortedmulti_a`) descriptor for k-of-n policies inside `tr()`. Semantically it is very similar to the existing `multi()` descriptor, but with the following changes:
* The corresponding script is `<key1> OP_CHECKSIG <key2> OP_CHECKSIGADD <key3> OP_CHECKSIGADD ... <key_n> OP_CHECKSIGADD <k> OP_NUMEQUAL`, rather than the traditional `OP_CHECKMULTISIG`-based script, making it usable inside the `tr()` descriptor.
* The keys can optionally be specified in x-only notation.
* Both the number of keys and the threshold can be as high as 999; this is the limit due to the consensus stacksize=1000 limit
I expect that this functionality will later be replaced with a miniscript-based implementation, but I don't think it's necessary to wait for that.
Limitations:
* The wallet code will for not estimate witness size incorrectly for script path spends, which may result in a (dramatic) fee underpayment with large multi_a scripts.
* The multi_a script construction is (slightly) suboptimal for n-of-n (where a `<key1> OP_CHECKSIGVERIFY ... <key_n-1> OP_CHECKSIGVERIFY <key_n> OP_CHECKSIG` would be better). Such a construction is not included here.
ACKs for top commit:
achow101:
ACK 4828d53ecc
gruve-p:
ACK 4828d53ecc
sanket1729:
code review ACK 4828d53ecc
darosior:
Code review ACK 4828d53ecc
Tree-SHA512: 5dcd434b79585f0ff830f7d501d27df5e346f5749f47a3109ec309ebf2cbbad0e1da541eec654026d911ab67fd7cf7793fab0f765628d68d81b96ef2a4d234ce
fa5d2e678c Remove unused char serialize (MarcoFalke)
fa24493d63 Use spans of std::byte in serialize (MarcoFalke)
fa65bbf217 span: Add BytePtr helper (MarcoFalke)
Pull request description:
This changes the serialize code (`.read()` and `.write()` functions) to take a `Span` instead of a pointer and size. This is a breaking change for the serialize interface, so at no additional cost we can also switch to `std::byte` (instead of using `char`).
The benefits of using `Span`:
* Less verbose and less fragile code when passing an already existing `Span`(-like) object to or from serialization
The benefits of using `std::byte`:
* `std::byte` can't accidentally be mistaken for an integer
The goal here is to only change serialize to use spans of `std::byte`. If needed, `AsBytes`, `MakeUCharSpan`, ... can be used (temporarily) to pass spans of the right type.
Other changes that are included here:
* [#22167](https://github.com/bitcoin/bitcoin/pull/22167) (refactor: Remove char serialize by MarcoFalke)
* [#21906](https://github.com/bitcoin/bitcoin/pull/21906) (Preserve const in cast on CTransactionSignatureSerializer by promag)
ACKs for top commit:
laanwj:
Concept and code review ACK fa5d2e678c
sipa:
re-utACK fa5d2e678c
Tree-SHA512: 08ee9eced5fb777cedae593b11e33660bed9a3e1711a7451a87b835089a96c99ce0632918bb4666a4e859c4d020f88fb50f2dd734216b0c3d1a9a704967ece6f
SerializeToVector, UnserializeFromVector, DeserializeHDKeypaths, and SerializeHDKeypaths
were in sign.h where PSBT was originally implemented. Since all of the PSBT serialization
has moved to its own file, these functions should follow.
4868c9f1b3 Extract Taproot internal keyid with GetKeyFromDestination (Andrew Chow)
d8abbe119c Mention bech32m in -addresstype and -changetype help (Andrew Chow)
8fb57845ee Create a tr() descriptor bech32m DescriptorScriptPubKeyMan by default (Andrew Chow)
54b3699862 Store pubkeys in TRDescriptor::MakeScripts (Andrew Chow)
Pull request description:
Make a `tr()` descriptor by default in descriptor wallets so that users will be able to make and use segwit v1 bech32m addresses.
ACKs for top commit:
MarcoFalke:
Concept ACK 4868c9f1b3
Sjors:
re-utACK 4868c9f1b3
gruve-p:
ACK 4868c9f1b3
meshcollider:
Concept + code review ACK 4868c9f1b3
Tree-SHA512: e5896e665b8d559f1d759b6582d1bb24f70d4698a57307684339d9fdcdac28ae9bc17bc946a7efec9cb35c130a95ffc36e3961a335124ec4535d77b8d00e9631
libsecp256k1's secp256k1_schnorrsig_sign only follows BIP340 exactly
if an aux_rand32 argument is passed. When no randomness is used
(as is the case in the current codebase here), there is no impact
on security between not providing aux_rand32 at all, or providing
an empty one. Yet, for repeatability/testability it is simpler
to always use an all-zero one.
43cd6b8af9 doc: add release notes for removal of the -deprecatedrpc=addresses flag (Michael Dietz)
2b1fdc2c6c refactor: minor styling, prefer snake case and same line if (Michael Dietz)
d64deac7b8 refactor: share logic between ScriptPubKeyToUniv and ScriptToUniv (Michael Dietz)
8721638daa rpc: remove deprecated addresses and reqSigs from rpc outputs (Michael Dietz)
Pull request description:
Resolves#21797 now that we've branched-off to v23 ("addresses" and "reqSigs" deprecated) "ExtractDestinations" should be removed.
`-deprecatedrpc=addresses` was initially added in this PR #20286 (which resolved the original issue #20102).
Some chunks of code and logic are no longer used/necessary with the removal of this, and therefore some minor refactoring is done in this PR as well (separated commits)
ACKs for top commit:
MarcoFalke:
re-ACK 43cd6b8af9🐉
meshcollider:
Code review ACK 43cd6b8af9
jonatack:
ACK 43cd6b8af9 per `git range-diff a9d0cec 92dc5e9 43cd6b8`, also rebased to latest master, debug built + quick re-review of each commit to bring back context, and ran tests locally at the final commit
Tree-SHA512: fba83495e396d3c06f0dcf49292f14f4aa6b68fa758f0503941fade1a6e7271cda8378e2734af1faea550d1b43c85a36c52ebcc9dec0732936f9233b4b97901c
Improve readability of code, simplify future scripted diff cleanup PRs, and be
more consistent with naming for GetBoolArg.
This will also be useful for replacing runtime settings type checking
with compile time checking.
-BEGIN VERIFY SCRIPT-
git grep -l GetArg | xargs sed -i 's/GetArg(\([^)]*\( [0-9]\+\|-1\|port\|BaseParams().RPCPort()\|Params().GetDefaultPort()\|_TIMEOUT\|Height\|_WORKQUEUE\|_THREADS\|_CONNECTIONS\|LIMIT\|SigOp\|Bytes\|_VERSION\|_AGE\|_CHECKS\|Checks() ? 1 : 0\|_BANTIME\|Cache\|BLOCKS\|LEVEL\|Weight\|Version\|BUFFER\|TARGET\|WEIGHT\|TXN\|TRANSACTIONS\|ADJUSTMENT\|i64\|Size\|nDefault\|_EXPIRY\|HEIGHT\|SIZE\|SNDHWM\|_TIME_MS\)\))/GetIntArg(\1)/g'
-END VERIFY SCRIPT-
Co-authored-by: Hennadii Stepanov <32963518+hebasto@users.noreply.github.com>
The places where we need to lookup information for a XOnlyPubKey
currently implement a hack which makes both serializations of the full
pubkey in order to try the CKeyIDs for the lookup functions. Instead of
duplicating this everywhere it is needed, we can consolidate the CKeyID
generation into a function, and then have wrappers around GetPubKey,
GetKey, and GetKeyOrigin which takes the XOnlyPubKey, retrieves all of
the CKeyIDs (using the new GetKeyIDs() function in XOnlyPubKey), and
tries their respective underlying lookup function.
08f57a0057 Assert that IsComplete() in GetSpendData() (Pieter Wuille)
d8f4b976d5 Remove default nHashTypeIn arguments to MutableTransactionSignatureCreator (Pieter Wuille)
c7048aae95 Simplify SignTransaction precomputation loop (Pieter Wuille)
addb9b5a71 Improve comments in taproot signing logic (Pieter Wuille)
Pull request description:
This addresses a few review comments from #21365 that were left at the time of merge (as well as some from #22166 applying to the commit it shared with #21365).
I do not think any are blockers for a 22.0 release.
ACKs for top commit:
theStack:
re-ACK 08f57a0057🌴
Zero-1729:
crACK 08f57a0
jonatack:
Code review ACK 08f57a0057 per `git range-diff e9d6eb1 9336670 08f57a0` followed by re-code review per commit to swap context back into memory and debug build/run unit tests + feature_taproot.py as a sanity check
Tree-SHA512: 968750109ba8d6faf3016035a38f81c6aefb724c632a3cab0bbf43cf31b9d187b6b0fddd8772768f57338df11eb07ab9c4c6dacdf3cf35b71f29699c67a301ea
As XOnlyPubKey has a Span-based constructor, that can be used directly
without needing to first convert the byte sequence into a vector, only
to convert that to a uint256, which only then can then be passed as a
span to the constructor.
92993aa5cf Change SignTransaction's input_errors to use bilingual_str (Andrew Chow)
171366e89b Use bilingual_str for address fetching functions (Andrew Chow)
9571c69b51 Add bilingual_str::clear() (Andrew Chow)
Pull request description:
In a couple of places in the wallet, errors are `std::string`. In order for these errors to be translated, change them to use `bilingual_str`.
ACKs for top commit:
hebasto:
re-ACK 92993aa5cf, only rebased since my [previous](https://github.com/bitcoin/bitcoin/pull/22337#pullrequestreview-694542729) review, verified with
klementtan:
Code review ACK 92993aa5cf
meshcollider:
Code review ACK 92993aa5cf
Tree-SHA512: 5400e419dd87db8c49b67ed0964de2d44b58010a566ca246f2f0760ed9ef6a9b6f6df7a6adcb211b315b74c727bfe8c7d07eb5690b5922fa5828ceef4c83461f
c3e111a7da Reduced number of validations in `tx_validationcache_tests` to keep the run time reasonable. (lucash-dev)
Pull request description:
Following a suggestion in the comments, changed `ValidateCheckInputsForAllFlags` from testing all possible flag combinations to testing a random subset. Also created a new enum constant for the highest flag, so that this test doesn’t keep testing an incomplete subset in case a new flag is added.
Timing for `checkinputs_test`:
```
Before: 6.8s
After: 3.7s
----------------
Saved: 3.1s (45%)
```
This PR was split from #13050. Also see #10026.
ACKs for top commit:
leonardojobim:
tACK c3e111a7da.
kallewoof:
ACK c3e111a7da
theStack:
re-ACK c3e111a7da
Tree-SHA512: bef49645bdd4f61ec73cc77a9f028b95d9856db9446d2e7fc9a48867a6f0e94c2c9f150cb771a30fe852db0efb0a1bd15d38b00d712651793ccb59ff6157a7b4
007910388b [Refactor] Rename scriptPubKey -> exec_script (sanket1729)
Pull request description:
Rename scriptPubKey to witness_script in ExecuteWitnessScript() function to correctly reflect which script is being executed.
For example in segwitv0, this scriptPubKey refers to the script of the form `OP_0 <script_hash>`, but witness_script refers to the script that actually hashes to the `script_hash`.
If there is a reason why it's named this way, I would love to know
ACKs for top commit:
MarcoFalke:
review ACK 007910388b🖖
theStack:
ACK 007910388b
lsilva01:
Code Review 007910388b ACK
Tree-SHA512: 768e10e656b60b1293beb560fb7adbc2c1495e6db1f54f0c2c63109692ae0c579c856b194b33f72afd0d332159a9796c0e2bd99b79ea5c4b1803469a81301fd6
fa621ededd refactor: Pass script verify flags as uint32_t (MarcoFalke)
Pull request description:
The flags are cast to unsigned in the interpreter anyway, so avoid the confusion (and fuzz crashes) by just passing them as unsigned from the beginning.
Also, the flags are often inverted bit-wise with the `~` operator, which also works on signed integers, but might cause confusion as the sign bit is flipped.
Fixes#22233
ACKs for top commit:
theStack:
Concept and code review ACK fa621ededd
kristapsk:
ACK fa621ededd
jonatack:
ACK fa621ededd
Tree-SHA512: ea0720f32f823fa7f075309978672aa39773c6019d12b6c1c9d611fc1983a76115b7fe2a28d50814673bb6415c311ccc05b99d6e871575fb6900faf75ed17769
e6cf0ed92d wallet, rpc: listdescriptors does not need unlocked (Andrew Chow)
3280704886 Pass in DescriptorCache to ToNormalizedString (Andrew Chow)
7a26ff10c2 Change DescriptorImpl::ToStringHelper to use an enum (Andrew Chow)
75530c93a8 Remove priv option for ToNormalizedString (Andrew Chow)
74fede3b8b wallet: Upgrade existing descriptor caches (Andrew Chow)
432ba9e543 wallet: Store last hardened xpub cache (Andrew Chow)
d87b544b83 descriptors: Cache last hardened xpub (Andrew Chow)
cacc391098 Move DescriptorCache writing to WalletBatch (Andrew Chow)
0b4c8ef75c Refactor Cache merging and writing (Andrew Chow)
976b53b085 Revert "Cache parent xpub inside of BIP32PubkeyProvider" (Andrew Chow)
Pull request description:
Currently fetching a normalized descriptor requires the wallet to be unlocked as it needs the private keys to derive the last hardened xpub. This is not very user friendly as normalized descriptors shouldn't require and don't involve the private keys except for derivation. We solve this problem by caching the last hardened xpub (which has to be derived at some point when generating the address pool).
However the last hardened xpub was not already being cached. We only cached the immediate parent xpub and derived child keys. For example, with a descriptor derivation path of `/84'/0'/0'/0/*`, the parent xpub that is cached is `m/84'/0'/0'/0`, and the child keys of `m/84'/0'/0'/0/i` (note that child keys would not be cached in this case). This parent xpub is not suitable for the normalized descriptor form as we want the key at `m/84'/0'/0'`. So this PR adds another field to `DescriptorCache` to cache the last hardened xpub so that we can use them for normalized descriptors.
Since `DescriptorCache` is changing, existing descriptor wallets need to be upgraded to use this new cache. The upgrade will occur in the background either at loading time (if the wallet is not encrypted) or at unlocking time in the same manner that `UpgradeKeyMetadata` operates. It will use a new wallet flag `WALLET_FLAG_LAST_HARDENED_XPUB_CACHED` to indicate whether the descriptor wallet has the last hardened xpub cache.
Lastly `listdescriptors` will not require the wallet to be locked and `getaddressinfo`'s `parent_desc` will always be output (assuming the upgrade has occurred).
ACKs for top commit:
fjahr:
tACK e6cf0ed92d
S3RK:
reACK e6cf0ed
jonatack:
Semi ACK e6cf0ed92d reviewed, debug-built and ran unit tests and some of the descriptor functional tests at each commit. I'm not very familiar with this code and it could be clearer to the uninitiated IMHO, so I'm not confident enough to give a full ACK. Various minor suggestions follow, most of them for readability, feel free to pick and choose.
meshcollider:
Code review + functional test run ACK e6cf0ed92d
Tree-SHA512: ac27aade8644525cd65bfcaf27ff32afb974085b1451faf4ff68c6671a690bd6a41d4f39a33cbf461ae0fbe85995c0a4c08dbd36171da1c1d2a1d00053ad298d
f9e37f33ce doc: IsFinalTx comment about nSequence & OP_CLTV (Yuval Kogman)
Pull request description:
It's somewhat surprising that a transaction's `nLockTime` field is ignored
when all `nSequence` fields are final, so this change aims to clarify this
behavior and cross reference relevant details of `OP_CHECKLOCKTIMEVERIFY`.
ACKs for top commit:
MarcoFalke:
ACK f9e37f33ce
Tree-SHA512: 88460dacbe4b8115fb1948715f09b21d4f34ba1da9e88d52f0b774a969f845e9eddc5940e7fee66eacdd3062dc40d6d44c3f282b0e5144411fd47eb2320b44f5
Instead of having a large blob of cache merging code in TopUp, refactor
this into DescriptorCache so that it can merge and provide a diff
(another DescriptorCache containing just the items that were added).
Then TopUp can just write everything that was in the diff.
754f134a50 wallet: Add error message to GetReservedDestination (Andrew Chow)
87a0e7a3b7 Disallow bech32m addresses for legacy wallet things (Andrew Chow)
6dbe4d1072 Use BECH32M for tr() desc, WitV1Taproot, and WitUnknown CTxDests (Andrew Chow)
699dfcd8ad Opportunistically use bech32m change addresses if available (Andrew Chow)
0262536c34 Add OutputType::BECH32M (Andrew Chow)
177c15d2f7 Limit LegacyScriptPubKeyMan address types (Andrew Chow)
Pull request description:
Currently bech32m addresses are classfied as bech32. Because bech32m is incompatible with bech32, we need to define a new `OutputType` for it so that it can be handled correctly. This PR adds `OutputType::BECH32M`, updates all of the relevant `OutputType` classifications, and handle requests for bech32m addresses. There is now a `bech32m` address type string that can be used.
* `tr()` descriptors now report their output type as `OutputType::BECH32M`. `WtinessV1Taproot` and `WitnessUnknown` are also classified as `OutputType::BECH32M`.
* Bech32m addresses are completely disabled for legacy wallets. They cannot be imported (explicitly disallowed in `importaddress` and `importmulti`), will not be created when getting all destinations for a pubkey, and will not be added with `addmultisigaddress`. Additional protections have been added to `LegacyScriptPubKeyMan` to disallow attempting to retrieve bech32m addresses.
* Since Taproot multisigs are not implemented yet, `createmultisig` will also disallow the bech32m address type.
* As Taproot is not yet active, `DescriptorScriptPubKeyMan` cannot and will not create a `tr()` descriptor. Protections have been added to make sure this cannot occur.
* The change address type detection algorithm has been updated to return `bech32m` when there is a segwit v1+ output script and the wallet has a bech32m `ScriptPubKeyMan`, falling back to bech32 if one is not available.
ACKs for top commit:
laanwj:
re-review ACK 754f134a50
Sjors:
re-utACK 754f134: only change is switching to `bech32m` in two `wallet_taproot.py` test cases.
fjahr:
re-ACK 754f134a50
jonatack:
ACK 754f134a50
Tree-SHA512: 6ea90867d3631d0d438e2b08ce6ed930f37d01323224661e8e38f183ea5ee2ab65b5891394a3612c7382a1aff907b457616c6725665a10c320174017b998ca9f
We don't want the legacy wallet to ever have bech32m addresses so don't
allow importing them. This includes addmultisigaddress as that is a
legacy wallet only RPC
Additionally, bech32m multisigs are not available yet, so disallow them
in createmultisig.
At verification time, the to be precomputed data can be inferred from
the transaction itself. For signing, the necessary witnesses don't
exist yet, so just permit precomputing everything in that case.
This provides a means to pass in a PrecomputedTransactionData object to
the MutableTransactionSignatureCreator, allowing the prevout data to be
passed into the signature hashers. It is also more efficient.
This data structures stores all information necessary for spending a taproot
output (the internal key, the Merkle root, and the control blocks for every
script leaf).
It is added to signing providers, and populated by the tr() descriptor.
This adds a new descriptor with syntax e.g. tr(KEY,{S1,{{S2,S3},S4})
where KEY is a key expression for the internal key and S_i are
script expression for the leaves. They have to be organized in
nested {A,B} groups, with exactly two elements.
tr() only exists at the top level, and inside the script expressions
only pk() scripts are allowed for now.
That results in a much safer interface (making the tweak commit
to the key implicitly using a fixed tag means it can't be used for
unrelated tweaking).
This is just a small simplification to prepare for the follow-up instruction
of a CTxDestination variant for taproot outputs.
In the old code, WITNESS_V1_TAPROOT and WITNESS_UNKNOWN both produced
{version, program} as Solver() output. Change this so that WITNESS_V1_TAPROOT
produces just {program}, like WITNESS_V0_* do.
We were previously ruling out 17-20 pubkeys multisig, while they are
only invalid under P2SH context.
This makes multisigs with up to 20 keys be detected as valid by the
solver. This is however *not* a policy change as it would only apply
to bare multisigs, which are already limited to 3 pubkeys.
Note that this does not change the sigOpCount calculation (as it would
break consensus). Therefore 1-16 keys multisigs are counted as 1-16 sigops
and 17-20 keys multisigs are counted as 20 sigops.
Signed-off-by: Antoine Poinsot <darosior@protonmail.com>
It's somewhat surprising that a transaction's nLockTime field is ignored
when all nSequence fields are final, so this change aims to clarify this
behavior and cross reference relevant details of OP_CHECKLOCKTIMEVERIFY.
0b188b751f Clean up context dependent checks in descriptor parsing (Pieter Wuille)
33275a9649 refactor: move uncompressed-permitted logic into ParsePubkey* (Pieter Wuille)
17e006ff8d refactor: split off subscript logic from ToStringHelper (Pieter Wuille)
6ba5dda0c9 Account for key cache indices in subexpressions (Pieter Wuille)
4441c6f3c0 Make DescriptorImpl support multiple subscripts (Pieter Wuille)
a917478db0 refactor: move population of out.scripts from ExpandHelper to MakeScripts (Pieter Wuille)
84f3939ece Remove support for subdescriptors expanding to multiple scripts (Pieter Wuille)
Pull request description:
These are a few refactors and non-invasive improvements to the descriptors code to prepare for adding Taproot descriptors.
None of the commits change behavior in any way, except the last one which improves error reporting a bit.
ACKs for top commit:
S3RK:
reACK 0b188b7
Sjors:
re-ACK 0b188b7
achow101:
re-ACK 0b188b751f
Tree-SHA512: cb4e999134aa2bace0e13d4883454c65bcf1369e1c8585d93cc6444ddc245f3def5a628d58af7dab577e9d5a4a75d3bb46f766421fcc8cc5c85c01a11f148b3f
725d7ae049 Use PrecomputedTransactionData in signet check (Pieter Wuille)
497718b467 Treat amount<0 also as missing data for P2WPKH/P2WSH (Pieter Wuille)
3820090bd6 Make all SignatureChecker explicit about missing data (Pieter Wuille)
b77b0cc507 Add MissingDataBehavior and make TransactionSignatureChecker handle it (Pieter Wuille)
Pull request description:
Currently we have 2 levels of potentially-missing data in the transaction signature hashes:
* P2WPKH/P2WSH hashes need the spent amount
* P2TR hashes need all spent outputs (amount + scriptPubKey)
Missing amounts are treated as -1 (thus leading to unexpected signature failures), while missing outputs in P2TR validation cause assertion failure. This is hard to extend for signing support, and also quite ugly in general.
In this PR, an explicit configuration option to {Mutable,}TransactionSignatureChecker is added (MissingDataBehavior enum class) to either select ASSERT_FAIL or FAIL. Validation code passes ASSERT_FAIL (as at validation time all data should always be passed, and anything else is a serious bug in the code), while signing code uses FAIL.
The existence of the ASSERT_FAIL option is really just an abundance of caution. Always using FAIL should be just fine, but if there were for some reason a code path in consensus code was introduced that misses certain data, I think we prefer as assertion failure over silently introducing a consensus change.
Potentially useful follow-ups (not for this PR, in my preference):
* Having an explicit script validation error code for missing data.
* Having a MissingDataBehavior::SUCCEED option as well, for use in script/sign.cpp DataFromTransaction (if a signature is present in a witness, and we don't have enough data to fully validate it, we should probably treat it as valid and not touch it).
ACKs for top commit:
sanket1729:
reACK 725d7ae049
Sjors:
ACK 725d7ae049
achow101:
re-ACK 725d7ae049
benthecarman:
ACK 725d7ae049
fjahr:
Code review ACK 725d7ae049
Tree-SHA512: d67dc51bae9ca7ef6eb9acccefd682529f397830f77d74cd305500a081ef55aede0e9fa380648c3a8dd4857aa7eeb1ab54fe808979d79db0784ac94ceb31b657
