Retropex/bitcoin
1
0
Fork 0
mirror of https://github.com/Retropex/bitcoin.git synced 2025-05-16 21:20:43 +02:00
Code Issues Actions Packages Projects Releases Wiki Activity
42,157 Commits 27 Branches 11 Tags 274 MiB
98745e03ff
Commit Graph

461 Commits

Author SHA1 Message Date
brunoerg
027aab663a test, contrib, refactor: use with when opening a file 2022-04-27 20:04:33 -03:00
fanquake
9b0a13a289
tidy: Add include-what-you-use 2022-04-20 14:14:52 +01:00
fanquake
983e0a2058
contrib: use LIEF 0.12.0 for symbol and security checks 2022-03-28 10:31:12 +01:00
fanquake
e4e9dd3a28
contrib: fix implicit function decleration in win symbol check 
```bash
test3.c: In function 'main':
test3.c:6:21: warning: implicit declaration of function 'CoFreeUnusedLibrariesEx' [-Wimplicit-function-declaration]
    6 |                     CoFreeUnusedLibrariesEx(0,0);
```
 2022-03-07 11:51:11 +00:00
fanquake
5a89bed410
contrib: address gen-manpages feedback from #24263 
Co-authored-by: Carl Dong <contact@carldong.me>
 2022-02-22 15:36:19 +00:00
laanwj
42c202893b doc: Fix gen-manpages, rewrite in Python 
Rewrite the manual page generation script in Python.

This:
- Solves '-' stripping issue (fixes #22681)
- Makes that copyright footer is generated again
 2022-02-04 17:03:31 +01:00
fanquake
6fe55160dd
contrib: support arm64 darwin in security checks 2022-01-26 17:32:10 +08:00
fanquake
446e73cc0b
build: use macOS 11 SDK (Xcode 12.2) 
This should be sufficient to support building for Apple ARM when
cross-compiling.
 2022-01-26 17:28:16 +08:00
laanwj
792d0d8d51
Merge bitcoin/bitcoin#23839: Linux: build with and test for control flow instrumentation on x86_64 
5a8f907c93 scripts: add CONTROL_FLOW to ELF security checks (fanquake)
e13f8f775d build: build x86_64 Linux Boost with -fcf-protection=full (fanquake)
6ca5efa8ed script rename control flow check to MACHO specific (fanquake)

Pull request description:

  Closes #21888.

  TODO:
  * Duplication in security-check-tests

  Guix build:
  ```bash
  bash-5.1# find guix-build-$(git rev-parse --short=12 HEAD)/output/ -type f -print0 | env LC_ALL=C sort -z | xargs -r0 sha256sum
  8025e2e6859247eaf865a4a5009d0a39570ec5e8ab51739fa4da0d8ee4ab2117  guix-build-5a8f907c93f1/output/aarch64-linux-gnu/SHA256SUMS.part
  75576482994493262dbf7d3567b0450c0804bdc75c186d6b4c6a856bd0d8f0a3  guix-build-5a8f907c93f1/output/aarch64-linux-gnu/bitcoin-5a8f907c93f1-aarch64-linux-gnu-debug.tar.gz
  7cb89c9327cc540859334b597c041a2250156e3e83ce2aa7d16177376056302f  guix-build-5a8f907c93f1/output/aarch64-linux-gnu/bitcoin-5a8f907c93f1-aarch64-linux-gnu.tar.gz
  e264053e4c7a5d65340dd7bbd2c664bcd596292ab80f00a0bf0026bfe0c480bc  guix-build-5a8f907c93f1/output/arm-linux-gnueabihf/SHA256SUMS.part
  6c3ee9b3c0c4583620301b183226678f1618605dd07dbed0bbdf7d06d3591314  guix-build-5a8f907c93f1/output/arm-linux-gnueabihf/bitcoin-5a8f907c93f1-arm-linux-gnueabihf-debug.tar.gz
  f217cb8d8e233a8dbdbfde7dabf12c5d867d7de53e8c652b8ed1d4a55da82fd9  guix-build-5a8f907c93f1/output/arm-linux-gnueabihf/bitcoin-5a8f907c93f1-arm-linux-gnueabihf.tar.gz
  48b4b2a1b52b3098f4e92c11cb60f1e5e9696a2a960560cd6adea72277eaa4a4  guix-build-5a8f907c93f1/output/dist-archive/bitcoin-5a8f907c93f1.tar.gz
  40832db2446e129879caa9fbc9d682c53069876dbb2e0d4d76592e5dcb40bb12  guix-build-5a8f907c93f1/output/powerpc64-linux-gnu/SHA256SUMS.part
  26fdfa9b7b77d5db415ef34054cf6f6d020a5dab73239db6dd05539f654bc5d5  guix-build-5a8f907c93f1/output/powerpc64-linux-gnu/bitcoin-5a8f907c93f1-powerpc64-linux-gnu-debug.tar.gz
  3adbdb9d3eb1cb5f9adc38b29450054f286bd6d74cef8619adaee89494853605  guix-build-5a8f907c93f1/output/powerpc64-linux-gnu/bitcoin-5a8f907c93f1-powerpc64-linux-gnu.tar.gz
  8ec2baf82483a698350bfdabf530cd9b5241690c916f597c746210e95ac451de  guix-build-5a8f907c93f1/output/powerpc64le-linux-gnu/SHA256SUMS.part
  1797e75e1f66ec9068fa4e57e0108960475e863f8f054fbe854358b1f995c4df  guix-build-5a8f907c93f1/output/powerpc64le-linux-gnu/bitcoin-5a8f907c93f1-powerpc64le-linux-gnu-debug.tar.gz
  448bd289ef26c777a1fc4498e7ba7fb17d0f6a932dcac91b2f89cbba63704bb8  guix-build-5a8f907c93f1/output/powerpc64le-linux-gnu/bitcoin-5a8f907c93f1-powerpc64le-linux-gnu.tar.gz
  a23ee91eeae515c2a6a31eb25d659fab833839aaafa4676ccee364bdad2a468c  guix-build-5a8f907c93f1/output/riscv64-linux-gnu/SHA256SUMS.part
  82df1b6d5020d0af8268ecc8e823f752f20dec308277763b2dd675804dfa4bbd  guix-build-5a8f907c93f1/output/riscv64-linux-gnu/bitcoin-5a8f907c93f1-riscv64-linux-gnu-debug.tar.gz
  09a6098ce83896a6ee6d5c8aff12eaca51595bd724c8e0b2a6f90b6410dc168c  guix-build-5a8f907c93f1/output/riscv64-linux-gnu/bitcoin-5a8f907c93f1-riscv64-linux-gnu.tar.gz
  4fac2951f80eaa2bd1747a263be6be1b76282cac5062f7d86db631a2fb80f8db  guix-build-5a8f907c93f1/output/x86_64-apple-darwin/SHA256SUMS.part
  3392f417b09efca5916c384f0b2d0c177a72ec4921399c62e84484f0054cc8c4  guix-build-5a8f907c93f1/output/x86_64-apple-darwin/bitcoin-5a8f907c93f1-osx-unsigned.dmg
  bfb4f8ade6107996ec4bc9efdb53959151b8fb19b6790c34472fe218dd02383d  guix-build-5a8f907c93f1/output/x86_64-apple-darwin/bitcoin-5a8f907c93f1-osx-unsigned.tar.gz
  4de92e149bad46fc863efb3b650753d194aad96be991d020e0b859452cf27457  guix-build-5a8f907c93f1/output/x86_64-apple-darwin/bitcoin-5a8f907c93f1-osx64.tar.gz
  9329549a2f275a59e329afc8744ff8cbc297f3042e0ad03b47626805c7aae2e8  guix-build-5a8f907c93f1/output/x86_64-linux-gnu/SHA256SUMS.part
  d87deb1eca8a1d3780f02edde78350d27f700e13c7ff444be0bfb34e7369904a  guix-build-5a8f907c93f1/output/x86_64-linux-gnu/bitcoin-5a8f907c93f1-x86_64-linux-gnu-debug.tar.gz
  76f3bc2fad010d9373e854d941e8205b68fa6c9a8ecaff34c4978ae3ae76c806  guix-build-5a8f907c93f1/output/x86_64-linux-gnu/bitcoin-5a8f907c93f1-x86_64-linux-gnu.tar.gz
  2948631081c4bd475529da2b7bc2b32b5aa2e44de46dfdaa6cf9432b3c9fe869  guix-build-5a8f907c93f1/output/x86_64-w64-mingw32/SHA256SUMS.part
  c5d56fac163b73c00e9745aa7b0f9a0bd9fcac7517e39b677869b6e76faf7218  guix-build-5a8f907c93f1/output/x86_64-w64-mingw32/bitcoin-5a8f907c93f1-win-unsigned.tar.gz
  f12c7cd72511544c67f4934981c090cee0d9c17e931c059edbbbef6e843dd651  guix-build-5a8f907c93f1/output/x86_64-w64-mingw32/bitcoin-5a8f907c93f1-win64-debug.zip
  eb952cc4554f92ba6787353f4305d7cbcb1c6dafc4a3867b3088106252a573b8  guix-build-5a8f907c93f1/output/x86_64-w64-mingw32/bitcoin-5a8f907c93f1-win64-setup-unsigned.exe
  7bd2b071f1cdf9410535e6a10dd1da519f942bd2c1e47ed52c5b8c4f977e8f27  guix-build-5a8f907c93f1/output/x86_64-w64-mingw32/bitcoin-5a8f907c93f1-win64.zip
  ```

ACKs for top commit:
  laanwj:
    Code review ACK 5a8f907c93
  hebasto:
    ACK 5a8f907c93, I have reviewed the code and it looks OK, I agree it can be merged.

Tree-SHA512: 75702ac01175ccf08e73b5c3bce266cee9dd15ccf8fb38d46108cfada156de9a8c18e27d996f9343ae33f7a75a6904d335cbe25256d90af1f4ccbe72ce1788bb
 2022-01-25 16:01:21 +01:00
fanquake
3ccfba1c7f
guix: use GCC 10 (over GCC 8) to build releases 
This currently points to the version-1.4.0 branch.
 2022-01-05 10:32:19 +08:00
fanquake
5a8f907c93
scripts: add CONTROL_FLOW to ELF security checks 2022-01-04 22:45:45 +08:00
fanquake
6ca5efa8ed
script rename control flow check to MACHO specific 2022-01-04 22:45:44 +08:00
fanquake
b9898aeeaa
scripts: make security checks architecture independent 
This paves the way for using and checking for architecture dependent
flags like -fcf-protection on x86_64 Linux and -mbranch-protection on 64
bit ARM.
 2022-01-03 21:36:55 +08:00
Hennadii Stepanov
f47dda2c58
scripted-diff: Bump copyright headers 
-BEGIN VERIFY SCRIPT-
./contrib/devtools/copyright_header.py update ./
-END VERIFY SCRIPT-

Commits of previous years:
* 2020: fa0074e2d8
* 2019: aaaaad6ac9
 2021-12-30 19:36:57 +02:00
Hennadii Stepanov
c29105efdc
script: Fix copyright_header.py 
This change prevents updating copyright years from "2021" to
"2021-2017".
 2021-12-30 19:33:20 +02:00
fanquake
d1711a40b3
scripts: match on exe type over str in symbol-check.py 2021-12-18 11:41:49 +08:00
fanquake
feee308401
scripts: match on exe type over str in security-check.py 2021-12-18 11:41:36 +08:00
fanquake
c870027cc2
build: qt 5.15.2 2021-12-03 14:07:12 +08:00
fanquake
f31d4bd214
scripts: test for PE control flow instrumentation 2021-11-17 16:12:47 +08:00
fanquake
dbfca4a815
build: require glibc 2.18+ for release builds 
From what I can see the only platform this drops support for is CentOS
7. CentOS 7 reached the end of it's "full update" support at the end of
2020. It does receive maintenance updates until 2024, however I don't
think supporting glibc 2.17 until 2024 is realistic. Note that anyone
wanting to self-compile and target a glibc 2.17 runtime could build with
--disable-threadlocal.

glibc 2.18 was released in August 2013.
https://sourceware.org/legacy-ml/libc-alpha/2013-08/msg00160.html
 2021-11-16 07:51:24 +08:00
Hennadii Stepanov
9a1ad7bc0d
test: Enable SC2086 shellcheck rule 2021-11-13 16:54:56 +02:00
Gleb Naumenko
0659f12b13
Add minisketch dependency 2021-10-21 09:38:55 +08:00
josibake
a46f71bb70
lint: enable mypy checking for missing imports 
Achieve this by adding some ignore, and making data/ importable.

Co-authored-by: João Barbosa <joao.paulo.barbosa@gmail.com>
 2021-10-16 09:14:37 +08:00
Carl Dong
1527b7e8a1 symbol-check: Check requested ELF interpreter 
It is important that binaries request a standard interpreter location
where most distros would place the linker-loader. Otherwise, the user
would be met with a very confusing message:

    bash: <path>/<to>/bitcoind: No such file or directory

When really it's the interpreter that's not found.
 2021-10-13 08:39:48 -04:00
fanquake
ce69e18947
scripts: remove pixie.py 2021-10-12 08:36:21 +08:00
fanquake
00b85d0b13
scripts: only parse the binary once in security-check.py 2021-10-12 08:36:21 +08:00
fanquake
cad40a5b16
scripts: use LIEF for ELF checks in security-check.py 2021-10-12 08:36:21 +08:00
fanquake
8242ae230e
scripts: only parse the binary once in symbol-check.py 2021-10-12 08:36:21 +08:00
fanquake
309eac9019
scripts: use LIEF for ELF checks in symbol-check.py 
Co-authored-by: Carl Dong <contact@carldong.me>
 2021-10-12 08:36:15 +08:00
Carl Dong
610a8a8e39
test-*-check: Pass in *FLAGS and compile with them 
These test-*-check scripts should compile "test" binaries in a way that
is as close to what autotools would do, since the goal is to make sure
that if we run the *-check script, they can correctly detect flaws in
binaries which are compiled by our autotools-based system.

Therefore, we should emulate what happens when the binary is linked in
autotools, meaning that for C binaries, we need to supply the CFLAGS,
CPPFLAGS, and LDFLAGS flags in that order.

Note to future developers: perhaps it'd be nice to have these
test-*-check scripts be part of configure.ac to avoid having to manually
replicate autoconf-like behaviour every time we find a discrepancy. Of
course, that would also mean you'd have to write more m4...
 2021-10-12 08:28:25 +08:00
fanquake
a43b8e9555
build: set OSX_MIN_VERSION to 10.15 
This is required to use std::filesystem on macOS as support for it only
landed in the libc++ dylib shipped with 10.15.

See also: https://developer.apple.com/documentation/xcode-release-notes/xcode-11-release-notes

Clang now supports the C++17 <filesystem> library for iOS 13, macOS 10.15, watchOS 6, and tvOS 13.
 2021-09-16 17:50:19 +08:00
fanquake
ab9c34237a
release: remove gitian 2021-08-31 09:37:23 +08:00
fanquake
5449d44e37
scripts: prevent GCC optimising test symbols in test-symbol-check 2021-08-09 13:59:48 +08:00
fanquake
6cf3345297
scripts: adjust test-symbol-check for guix release environment 
Now that our release binaries are build in a glibc 2.24 and 2.27
environment, we can't use a symbol from glibc 2.28 to test our checks.
Replace renameat2() with nextup(), which was introduced in 2.24.

Note that this also means re-disabling the test for RISC-V, however
RISC-V is built in a glibc 2.27 environment, and our minimum required
glibc for that binary is 2.27.
 2021-07-07 20:29:01 +08:00
fanquake
1946b5f77c
scripts: more robustly test macOS symbol checks 2021-07-07 19:31:37 +08:00
Carl Dong
a8127b34bc
build: Use and test PE binutils with --reloc-section 
Also fix test-security-check.py to account for new PE PIE failure
indication.
 2021-07-07 19:31:37 +08:00
Carl Dong
9fdc8afe11
devtools: Improve *-check.py tool detection 
This is important to make sure that we're not testing tools different
from the one we're building with.

Introduce determine_wellknown_cmd, which encapsulates how we
should handle well-known tools specification (IFS splitting, env
override, etc.).
 2021-07-07 19:31:37 +08:00
fanquake
797b3ed909
script: remove gitian reference from symbol-check.py 2021-07-06 20:50:47 +08:00
W. J. van der Laan
a305a687e7
Merge bitcoin/bitcoin#22244: devtools: Correctly extract symbol versions in symbol-check 
e8cd3700ee devtools: Integrate ARCH_MIN_GLIBC_VER table into MAX_VERSIONS in symbol-check.py (W. J. van der Laan)
a33381acf5 devtools: Add xkb version to symbol-check (W. J. van der Laan)
19e598bab0 devtools: Fix verneed section parsing in pixie (W. J. van der Laan)

Pull request description:

  I misunderstood the ELF specification for version symbols (verneed): The `vn_aux` pointer is relative to the main verneed record, not the start of the section.

  This caused many symbols to not be versioned properly in the return value of `elf.dyn_symbols`. This was discovered in #21454.

  Fix it by correcting the offset computation.

  - xkb versions symbols (using the prefix `V`), as this library is used by bitcoin-qt, add it to the valid versions in `symbol-check.py`

  This unfortunately brings to light some symbols that have been introduced since and weren't caught (from a gitian compile of master):

  ```
  bitcoin-cli: symbol getrandom from unsupported version GLIBC_2.25
  bitcoin-cli: failed IMPORTED_SYMBOLS
  bitcoind: symbol getrandom from unsupported version GLIBC_2.25
  bitcoind: symbol log from unsupported version GLIBC_2.29
  bitcoind: symbol fcntl64 from unsupported version GLIBC_2.28
  bitcoind: symbol pow from unsupported version GLIBC_2.29
  bitcoind: symbol exp from unsupported version GLIBC_2.29
  bitcoind: failed IMPORTED_SYMBOLS
  bitcoin-qt: symbol exp from unsupported version GLIBC_2.29
  bitcoin-qt: symbol fcntl64 from unsupported version GLIBC_2.28
  bitcoin-qt: symbol log from unsupported version GLIBC_2.29
  bitcoin-qt: symbol pow from unsupported version GLIBC_2.29
  bitcoin-qt: symbol statx from unsupported version GLIBC_2.28
  bitcoin-qt: symbol getrandom from unsupported version GLIBC_2.25
  bitcoin-qt: symbol renameat2 from unsupported version GLIBC_2.28
  bitcoin-qt: symbol getentropy from unsupported version GLIBC_2.25
  bitcoin-qt: failed IMPORTED_SYMBOLS
  bitcoin-wallet: symbol exp from unsupported version GLIBC_2.29
  bitcoin-wallet: symbol log from unsupported version GLIBC_2.29
  bitcoin-wallet: symbol fcntl64 from unsupported version GLIBC_2.28
  bitcoin-wallet: failed IMPORTED_SYMBOLS
  test_bitcoin: symbol getrandom from unsupported version GLIBC_2.25
  test_bitcoin: symbol log from unsupported version GLIBC_2.29
  test_bitcoin: symbol fcntl64 from unsupported version GLIBC_2.28
  test_bitcoin: symbol pow from unsupported version GLIBC_2.29
  test_bitcoin: symbol exp from unsupported version GLIBC_2.29
  test_bitcoin: failed IMPORTED_SYMBOLS
  ```

ACKs for top commit:
  hebasto:
    ACK e8cd3700ee

Tree-SHA512: 8c15e3478eb642f01a1ddaadef03f80583f088f9fa8e3bf171ce16b0ec05ffb4675ec147d7ffc6a4360637ed47fca517c6ca2bac7bb30d794c03783cfb964b79
 2021-06-21 07:58:12 +02:00
W. J. van der Laan
e8cd3700ee devtools: Integrate ARCH_MIN_GLIBC_VER table into MAX_VERSIONS in symbol-check.py 
The (ancient) versions specified here were deceptive. Entries older than
MAX_VERSIONS['GLIBC'], which is 2.17, are ignored here. So reorganize
the code to avoid confusion for other people reading this code.
 2021-06-18 13:25:57 +02:00
W. J. van der Laan
a33381acf5 devtools: Add xkb version to symbol-check 
xkb versions symbols (using the prefix `V`), as this library is used by
bitcoin-qt, add it to the valid versions in `symbol-check.py`.
 2021-06-14 20:32:09 +02:00
W. J. van der Laan
19e598bab0 devtools: Fix verneed section parsing in pixie 
I misunderstood the ELF specification for version symbols (verneed):
The `vn_aux` pointer is relative to the main verneed record, not the
start of the section.

This caused many symbols to not be versioned properly in the return
value of `elf.dyn_symbols`. This was discovered in #21454.

Fix it by correcting the offset computation.
 2021-06-14 20:31:58 +02:00
fanquake
aa80b5759d
scripts: check macOS SDK version is set 
Clangs Darwin driver should infer the SDK version used during compilation, and
forward that through to the linker. Add a check that this has been done, and the
expected SDK version is set.

Should help prevent issues like #21771 in future.
 2021-06-10 15:43:50 +08:00
fanquake
c972345bac
scripts: check minimum required Windows version is set 
We use linker flags (-Wl,--major/minor-subsystem-version) to set the
minimum required version of Windows needed to run our binaries. This
adds a sanity check that the version is being set as expected.
 2021-06-10 10:40:53 +08:00
fanquake
29615aef52
scripts: check minimum required macOS vesion is set 
We use a compile flag (-mmacosx-version-min) to set the minimum required
version of macOS needed to run our binaries. This adds a sanity check
that the version is being set as expected.
 2021-06-10 10:40:53 +08:00
fanquake
42b589d18f
scripts: test for MACHO control flow instrumentation 2021-05-09 14:26:09 +08:00
fanquake
955140b326
contrib: consolidate PIE and NX security checks 2021-05-04 20:48:00 +08:00
fanquake
2aa1631822
contrib: use LIEF in PE symbol checks 2021-05-04 20:48:00 +08:00
fanquake
e93ac26b85
contrib: use LIEF in macOS symbol checks 2021-05-04 20:48:00 +08:00
fanquake
a632cbcee5
contrib: use f strings in symbol-check.py 2021-05-04 20:48:00 +08:00
fanquake
0f5d77c8e4
contrib: add PE PIE check to security checks 2021-05-04 20:48:00 +08:00
fanquake
8e1f40dd9a
contrib: use LIEF for PE security checks 2021-05-04 20:47:59 +08:00
fanquake
a25b2e965c
contrib: use LIEF for macOS security checks 2021-05-04 20:47:59 +08:00
fanquake
7e7eae7aa8
contrib: use f strings in security-check.py 2021-05-04 20:47:59 +08:00
wodry
ea76f4ac7d Doc: Tell howto install clang-format on Debian/Ubuntu 
Because only macOS wasy mentioned, I was unsure if this would be a macOS specific tool. I guess Linux is more used than Mac, so Linux guide should be there, too.
 2021-03-19 19:24:40 +01:00
Hennadii Stepanov
0fc0c00f7a
test: Drop unused get_machine function 
The removed get_machine function is no longer used since #21255.
 2021-03-13 18:29:08 +02:00
Hennadii Stepanov
61a0f8f9cc
test: Cleanup test files in test-{security,symbol}-check.py 2021-03-13 18:27:11 +02:00
fanquake
06cd0da21f
build: qt 5.12.10 
remove fix_configure_mac.patch
Fixed upstream: https://bugreports.qt.io/browse/QTBUG-67286

remove fix_riscv64_arch.patch
Was fixed upstream in 6a39e49a6cdeb28a04a3657bb6a22f848d5dfa9d

remove fix_rcc_determinism.patch
Fixed upstream in https://bugreports.qt.io/browse/QTBUG-62511

remove freetype_back_compat.patch
By the time we ship a release with Qt 5.12, we'll certainly no-longer be
supporting Ubuntu 14.04 and Ubuntu 16.04 ships with FreeType 2.6.1,
which is new enough that using the symbol is no-longer an issue.
The renaming of FT_Get_X11_Font_Format() happened in FreeType 2.6

remove xkb-default.patch
This was removed upstream in d5abf545971da717014d316127045fc19edbcd65

Co-authored-by: Hennadii Stepanov <32963518+hebasto@users.noreply.github.com>
 2021-03-10 12:57:22 +08:00
Hennadii Stepanov
3272e34f9c
build: Add xkbcommon 0.8.4 
Co-authored-by: fanquake <fanquake@gmail.com>
 2021-03-10 08:23:47 +08:00
fanquake
ccb7b6a976
contrib: run test-symbol check for RISCV 
Now that we are using Focal for Gitian building (glibc 2.31), we can
user a newer introduced  symbol, and include RISCV in this test.
 2021-02-22 08:35:06 +08:00
Wladimir J. van der Laan
b5e93f873a doc: Add manual page generation for bitcoin-util 
- Add `-version` option to `bitcoin-util`
- Add `bitcoin-util` call to `gen-manpages.sh`
- Add stub manual page `bitcoin-util.1`
- Add install of `bitcoin-util.1` to build system
 2021-01-12 14:09:21 +01:00
MarcoFalke
fa0074e2d8
scripted-diff: Bump copyright headers 
-BEGIN VERIFY SCRIPT-
./contrib/devtools/copyright_header.py update ./
-END VERIFY SCRIPT-
 2020-12-31 09:45:41 +01:00
fanquake
1ef2138c0d
lint: run mypy over contrib/devtools 2020-12-28 14:25:06 +08:00
Wladimir J. van der Laan
f1dbf92ff0
Merge #20434: contrib: Parse ELF directly for symbol and security checks 
a0a771843f contrib: Changes to checks for PowerPC64 (Luke Dashjr)
634f6ec4eb contrib: Parse ELF directly for symbol and security checks (Wladimir J. van der Laan)

Pull request description:

  Instead of the ever-messier text parsing of the output of the readelf tool (which is clearly meant for human consumption not to be machine parseable), parse the ELF binaries directly.

  Add a small dependency-less ELF parser specific to the checks.

  This is slightly more secure, too, because it removes potential ambiguity due to misparsing and changes in the output format of `elfread`. It also allows for stricter and more specific ELF format checks in the future.

  This removes the build-time dependency for `readelf`.

  It passes the test-security-check for me locally, ~~though I haven't checked on all platforms~~. I've checked that this works on the cross-compile output for all ELF platforms supported by Bitcoin Core at the moment, as well as PPC64 LE and BE.

Top commit has no ACKs.

Tree-SHA512: 7f9241fec83ee512642fecf5afd90546964561efd8c8c0f99826dcf6660604a4db2b7255e1afb1e9bb0211fd06f5dbad18a6175dfc03e39761a40025118e7bfc
 2020-12-18 12:16:40 +01:00
fanquake
ae9b48995b
contrib: add symbol check test for PE 2020-12-09 23:13:24 +08:00
Wladimir J. van der Laan
f3e17686b3
Merge #20468: build: warn when generating man pages for binaries built from a dirty branch 
6690adba08 Warn when binaries are built from a dirty branch. (Tyler Chambers)

Pull request description:

  - Adjusted `--version` flag behavior in bitcoind and bitcoin-wallet to have the same behavior.
  - Added `--version` flag to bitcoin-tx to match.
  - Added functionality in gen-manpages.sh to error when attempting to generate man pages for binaries built from a dirty branch.

  mitigates problem with  issue #20412

ACKs for top commit:
  laanwj:
    Tested ACK 6690adba08

Tree-SHA512: b5ca509f1a57f66808c2bebc4b710ca00c6fec7b5ebd7eef58018e28e716f5f2358e36551b8a4df571bf3204baed565a297aeefb93990e7a99add502b97ee1b8
 2020-12-07 10:51:45 +01:00
fanquake
ed1bbcefea contrib: add MACHO tests to symbol-check tests 2020-12-03 12:16:19 +01:00
Wladimir J. van der Laan
5bab08df17 contrib: Add test for ELF symbol-check 
Check both failure cases:
- Use a glibc symbol from a version that is too new
- Use a symbol from a library that is not in the allowlist

And also check a conforming binary.

Adding a similar check for Windows PE can be done in a separate PR.
 2020-12-03 12:15:12 +01:00
Tyler Chambers
6690adba08
Warn when binaries are built from a dirty branch. 
Adjusted version flag behavior in bitcoin-tx, bitcoin-wallet, and
bitcoind to match. Added functionality in gen-manpages.sh to warning when
attempting to generate man pages for binaries built from a dirty
branch.
 2020-11-28 18:43:55 -05:00
Luke Dashjr
a0a771843f contrib: Changes to checks for PowerPC64 
Changes from #14066.
 2020-11-22 11:11:32 +01:00
Wladimir J. van der Laan
634f6ec4eb contrib: Parse ELF directly for symbol and security checks 
Instead of the ever-messier text parsing of the output of the readelf
tool (which is clearly meant for human consumption not to be machine
parseable), parse the ELF binaries directly.

Add a small dependency-less ELF parser specific to the checks.

This is slightly more secure, too, because it removes potential
ambiguity due to misparsing and changes in the output format of `elfread`. It
also allows for stricter and more specific ELF format checks in the future.

This removes the build-time dependency for `readelf`.

It passes the test-security-check for me locally, though I haven't
checked on all platforms.
 2020-11-22 11:11:32 +01:00
Tyler Chambers
b6121edf70 swapped "is" for "==" in literal comparison 
update lint-python.sh to include check F632
 2020-11-09 10:21:51 -05:00
Hennadii Stepanov
bd897ce79f
scripted-diff: Move previous_release.py to test/get_previous_releases.py 
-BEGIN VERIFY SCRIPT-
OLD=contrib/devtools/previous_release.py
NEW=test/get_previous_releases.py
sed -i "s|$OLD|$NEW|g" $(git grep -l $OLD)
git mv $OLD $NEW
-END VERIFY SCRIPT-
 2020-08-29 11:26:25 +03:00
Wladimir J. van der Laan
4ebe2f6e75
Merge #18011: Replace current benchmarking framework with nanobench 
78c312c983 Replace current benchmarking framework with nanobench (Martin Ankerl)

Pull request description:

  Replace current benchmarking framework with nanobench

  This replaces the current benchmarking framework with nanobench [1], an
  MIT licensed single-header benchmarking library, of which I am the
  autor. This has in my opinion several advantages, especially on Linux:

  * fast: Running all benchmarks takes ~6 seconds instead of 4m13s on
    an Intel i7-8700 CPU @ 3.20GHz.

  * accurate: I ran e.g. the benchmark for SipHash_32b 10 times and
    calculate standard deviation / mean = coefficient of variation:

    * 0.57% CV for old benchmarking framework
    * 0.20% CV for nanobench

    So the benchmark results with nanobench seem to vary less than with
    the old framework.

  * It automatically determines runtime based on clock precision, no need
    to specify number of evaluations.

  * measure instructions, cycles, branches, instructions per cycle,
    branch misses (only Linux, when performance counters are available)

  * output in markdown table format.

  * Warn about unstable environment (frequency scaling, turbo, ...)

  * For better profiling, it is possible to set the environment variable
    NANOBENCH_ENDLESS to force endless running of a particular benchmark
    without the need to recompile. This makes it to e.g. run "perf top"
    and look at hotspots.

  Here is an example copy & pasted from the terminal output:

  |             ns/byte |              byte/s |    err% |        ins/byte |        cyc/byte |    IPC |       bra/byte |   miss% |     total | benchmark
  |--------------------:|--------------------:|--------:|----------------:|----------------:|-------:|---------------:|--------:|----------:|:----------
  |                2.52 |      396,529,415.94 |    0.6% |           25.42 |            8.02 |  3.169 |           0.06 |    0.0% |      0.03 | `bench/crypto_hash.cpp RIPEMD160`
  |                1.87 |      535,161,444.83 |    0.3% |           21.36 |            5.95 |  3.589 |           0.06 |    0.0% |      0.02 | `bench/crypto_hash.cpp SHA1`
  |                3.22 |      310,344,174.79 |    1.1% |           36.80 |           10.22 |  3.601 |           0.09 |    0.0% |      0.04 | `bench/crypto_hash.cpp SHA256`
  |                2.01 |      496,375,796.23 |    0.0% |           18.72 |            6.43 |  2.911 |           0.01 |    1.0% |      0.00 | `bench/crypto_hash.cpp SHA256D64_1024`
  |                7.23 |      138,263,519.35 |    0.1% |           82.66 |           23.11 |  3.577 |           1.63 |    0.1% |      0.00 | `bench/crypto_hash.cpp SHA256_32b`
  |                3.04 |      328,780,166.40 |    0.3% |           35.82 |            9.69 |  3.696 |           0.03 |    0.0% |      0.03 | `bench/crypto_hash.cpp SHA512`

  [1] https://github.com/martinus/nanobench

ACKs for top commit:
  laanwj:
    ACK 78c312c983

Tree-SHA512: 9e18770b18b6f95a7d0105a4a5497d31cf4eb5efe6574f4482f6f1b4c88d7e0946b9a4a1e9e8e6ecbf41a3f2d7571240677dcb45af29a6f0584e89b25f32e49e
 2020-07-30 15:34:17 +02:00
Wladimir J. van der Laan
65d0f1a533
devtools: Add security check for separate_code 
Check that sections are appropriately separated in virtual memory,
based on their (expected) permissions. This checks for missing
-Wl,-z,separate-code and potentially other problems.

Co-authored-by: fanquake <fanquake@gmail.com>
 2020-07-28 12:57:35 +08:00
MarcoFalke
facdf530c7
contrib: Clean up previous_releases.py 
* Replace curl single char options with their verbose counterpart
* Stricter check for tarballHash
 2020-07-21 11:03:35 +02:00
MarcoFalke
ea595d39f7
Merge #19205: script: previous_release.sh rewritten in python 
9c34aff393 Remove previous_release.sh (Brian Liotti)
e1e5960e10 script: Add previous_release.py (Brian Liotti)

Pull request description:

  Closes #18132

  Added functionality:
  1) checks file hash before untarring when using the binary download option

ACKs for top commit:
  fjahr:
    re-ACK 9c34aff393
  Sjors:
    tACK 9c34aff393

Tree-SHA512: 323f11828736a372a47f048592de8b027ddcd75b38f312dfc73f7b495d1e078bfeb384d9cdf434b3e70f2c6c0ce2da2df48e9a6460ac0e1967c6829a411c52d5
 2020-07-21 10:11:39 +02:00
David Reikher
4455949d6f Make test DoS_mapOrphans deterministic 
The RandomOrphan function and the function ecdsa_signature_parse_der_lax
in pubkey.cpp were causing non-deterministic test coverage.

Force seed in the beginning of the test to make it deterministic.
The seed is selected carefully so that all branches of the function
ecdsa_signature_parse_der_lax are executed. Prior to this fix, the test
was exhibiting non-deterministic coverage since none of the ECDSA
signatures that were generated during the test had leading zeroes in
either R, S, or both, resulting in some branches of said function not
being executed. The seed ensures that both conditions are hit.

Removed denialofservice_tests test entry from the list of non-deterministic
tests in the coverage script.
 2020-07-21 09:18:57 +03:00
Brian Liotti
9c34aff393 Remove previous_release.sh 2020-07-05 04:05:02 -04:00
Brian Liotti
e1e5960e10 script: Add previous_release.py 
closes #18132
added GPG verify for binaries

co-authored-by: bboot <bboot@cisco.com>
 2020-07-05 04:04:52 -04:00
fanquake
9fe71a57a6
test: use subprocess.run() in test-security-check.py 2020-06-16 19:52:30 +08:00
fanquake
968aaae940
tests: run test-security-check.py in CI 2020-06-16 19:52:30 +08:00
Martin Ankerl
78c312c983 Replace current benchmarking framework with nanobench 
This replaces the current benchmarking framework with nanobench [1], an
MIT licensed single-header benchmarking library, of which I am the
autor. This has in my opinion several advantages, especially on Linux:

* fast: Running all benchmarks takes ~6 seconds instead of 4m13s on
  an Intel i7-8700 CPU @ 3.20GHz.

* accurate: I ran e.g. the benchmark for SipHash_32b 10 times and
  calculate standard deviation / mean = coefficient of variation:

  * 0.57% CV for old benchmarking framework
  * 0.20% CV for nanobench

  So the benchmark results with nanobench seem to vary less than with
  the old framework.

* It automatically determines runtime based on clock precision, no need
  to specify number of evaluations.

* measure instructions, cycles, branches, instructions per cycle,
  branch misses (only Linux, when performance counters are available)

* output in markdown table format.

* Warn about unstable environment (frequency scaling, turbo, ...)

* For better profiling, it is possible to set the environment variable
  NANOBENCH_ENDLESS to force endless running of a particular benchmark
  without the need to recompile. This makes it to e.g. run "perf top"
  and look at hotspots.

Here is an example copy & pasted from the terminal output:

|             ns/byte |              byte/s |    err% |        ins/byte |        cyc/byte |    IPC |       bra/byte |   miss% |     total | benchmark
|--------------------:|--------------------:|--------:|----------------:|----------------:|-------:|---------------:|--------:|----------:|:----------
|                2.52 |      396,529,415.94 |    0.6% |           25.42 |            8.02 |  3.169 |           0.06 |    0.0% |      0.03 | `bench/crypto_hash.cpp RIPEMD160`
|                1.87 |      535,161,444.83 |    0.3% |           21.36 |            5.95 |  3.589 |           0.06 |    0.0% |      0.02 | `bench/crypto_hash.cpp SHA1`
|                3.22 |      310,344,174.79 |    1.1% |           36.80 |           10.22 |  3.601 |           0.09 |    0.0% |      0.04 | `bench/crypto_hash.cpp SHA256`
|                2.01 |      496,375,796.23 |    0.0% |           18.72 |            6.43 |  2.911 |           0.01 |    1.0% |      0.00 | `bench/crypto_hash.cpp SHA256D64_1024`
|                7.23 |      138,263,519.35 |    0.1% |           82.66 |           23.11 |  3.577 |           1.63 |    0.1% |      0.00 | `bench/crypto_hash.cpp SHA256_32b`
|                3.04 |      328,780,166.40 |    0.3% |           35.82 |            9.69 |  3.696 |           0.03 |    0.0% |      0.03 | `bench/crypto_hash.cpp SHA512`

[1] https://github.com/martinus/nanobench

* Adds support for asymptotes

  This adds support to calculate asymptotic complexity of a benchmark.
  This is similar to #17375, but currently only one asymptote is
  supported, and I have added support in the benchmark `ComplexMemPool`
  as an example.

  Usage is e.g. like this:

  ```
  ./bench_bitcoin -filter=ComplexMemPool -asymptote=25,50,100,200,400,600,800
  ```

  This runs the benchmark `ComplexMemPool` several times but with
  different complexityN settings. The benchmark can extract that number
  and use it accordingly. Here, it's used for `childTxs`. The output is
  this:

  | complexityN |               ns/op |                op/s |    err% |          ins/op |          cyc/op |    IPC |     total | benchmark
  |------------:|--------------------:|--------------------:|--------:|----------------:|----------------:|-------:|----------:|:----------
  |          25 |        1,064,241.00 |              939.64 |    1.4% |    3,960,279.00 |    2,829,708.00 |  1.400 |      0.01 | `ComplexMemPool`
  |          50 |        1,579,530.00 |              633.10 |    1.0% |    6,231,810.00 |    4,412,674.00 |  1.412 |      0.02 | `ComplexMemPool`
  |         100 |        4,022,774.00 |              248.58 |    0.6% |   16,544,406.00 |   11,889,535.00 |  1.392 |      0.04 | `ComplexMemPool`
  |         200 |       15,390,986.00 |               64.97 |    0.2% |   63,904,254.00 |   47,731,705.00 |  1.339 |      0.17 | `ComplexMemPool`
  |         400 |       69,394,711.00 |               14.41 |    0.1% |  272,602,461.00 |  219,014,691.00 |  1.245 |      0.76 | `ComplexMemPool`
  |         600 |      168,977,165.00 |                5.92 |    0.1% |  639,108,082.00 |  535,316,887.00 |  1.194 |      1.86 | `ComplexMemPool`
  |         800 |      310,109,077.00 |                3.22 |    0.1% |1,149,134,246.00 |  984,620,812.00 |  1.167 |      3.41 | `ComplexMemPool`

  |   coefficient |   err% | complexity
  |--------------:|-------:|------------
  |   4.78486e-07 |   4.5% | O(n^2)
  |   6.38557e-10 |  21.7% | O(n^3)
  |   3.42338e-05 |  38.0% | O(n log n)
  |   0.000313914 |  46.9% | O(n)
  |     0.0129823 | 114.4% | O(log n)
  |     0.0815055 | 133.8% | O(1)

  The best fitting curve is O(n^2), so the algorithm seems to scale
  quadratic with `childTxs` in the range 25 to 800.
 2020-06-13 12:24:18 +02:00
MarcoFalke
fa4cd1fdae
ci: Switch to bitcoincore.org download 2020-06-07 10:30:38 -04:00
fanquake
eacedfb023
scripts: add additional type annotations to security-check.py 2020-05-14 15:30:52 +08:00
fanquake
83d063e954
scripts: add run_command to security-check.py 
Deduplicate all the subprocess code as mentioned in 18713.
 2020-05-14 15:29:58 +08:00
fanquake
13f606b4f9
scripts: remove NONFATAL from security-check.py 2020-05-14 14:36:27 +08:00
fanquake
061acf62a1
scripts: no-longer check for 32 bit windows in security-check.py 2020-05-14 14:36:27 +08:00
MarcoFalke
fbd522721c
Merge #18885: contrib: Move optimize-pngs.py script to the maintainer repo 
fa13090d20 contrib: Remove optimize-pngs.py script, which lives in the maintainer repo (MarcoFalke)

Pull request description:

  Moved to https://github.com/bitcoin-core/bitcoin-maintainer-tools/blob/master/optimize-pngs.py

  Bitcoin Core should focus on the full node implementation, not on scripts to compress png images.

  This script is only used when new PNG files are added to the repo. This happens about once every two years. So fetching the script from the other repo should not be a burden, but removing it from this repo is going to cut down on the meta files we need to maintain in the main repo.

ACKs for top commit:
  practicalswift:
    ACK fa13090d20 -- `+0 lines, -82 lines` :)
  promag:
    ACK fa13090d20.
  hebasto:
    ACK fa13090d20, verified that script is already [moved](https://github.com/bitcoin-core/bitcoin-maintainer-tools/pull/56).

Tree-SHA512: 37d111adae769bcddc6ae88041032d5a2b8b228fec67f555c8333c38de3992f5138b30bea868d7d6d6b7f966a47133e5853134373b149ab23cba3b8b560ecb31
 2020-05-05 10:40:39 -04:00
MarcoFalke
fa13090d20
contrib: Remove optimize-pngs.py script, which lives in the maintainer repo 
https://github.com/bitcoin-core/bitcoin-maintainer-tools/blob/master/optimize-pngs.py
 2020-05-05 07:25:58 -04:00
fanquake
3e38023af7
scripts: add PE .reloc section check to security-check.py 2020-04-23 08:40:24 +08:00
fanquake
8334ee31f8
scripts: add MACHO LAZY_BINDINGS test to test-security-check.py 
I didn't add the relevant test in #18295.
 2020-04-21 11:32:06 +08:00
fanquake
7b99c7454c
scripts: add MACHO Canary check to security-check.py 2020-04-21 11:32:01 +08:00
MarcoFalke
fa488f131f
scripted-diff: Bump copyright headers 
-BEGIN VERIFY SCRIPT-
./contrib/devtools/copyright_header.py update ./
-END VERIFY SCRIPT-
 2020-04-16 13:33:09 -04:00
fanquake
5ca90f8b59
scripts: add MACHO lazy bindings check to security-check.py 2020-04-04 09:54:25 +08:00
fanquake
6ec42df32b
Merge #18426: scripts: previous_release: improve behaviour on failed download 
332f373a9d [scripts] previous_release: improve failed download error message (Sebastian Falbesoner)

Pull request description:

  Currently, if the earlier release build/fetch script `previous_release.sh` is invoked with the option `-b` (intending to fetch a binary package from `https://bitcoin.org`) and the download fails, the user sees the following confusing output:
  ```
  $ contrib/devtools/previous_release.sh -r -b v0.9.5
  [...]
  gzip: stdin: not in gzip format
  tar: Child returned status 1
  tar: Error is not recoverable: exiting now
  ```
  This implies that the download worked, but the archive is corrupted, when in reality the HTML document containing the delivery fail reason (most likely 404 Not Found) is saved and tried to get unpacked. In contrast to wget, curl is a bit stubborn and needs explicit instructions to react to server errors via the flag `-f` (outputs error message and returns error code, ideal for scripts): https://curl.haxx.se/docs/manpage.html#-f

  On the PR branch, the output on failed download looks now the following:
  ```
  $ contrib/devtools/previous_release.sh -r -b v0.9.5
  [...]
  curl: (22) The requested URL returned error: 404 Not Found
  Download failed.
  ```

ACKs for top commit:
  fanquake:
    ACK 332f373a9d

Tree-SHA512: 046c931ad9e78aeb2d13faa4866d46122ed325aa142483547c2b04032d03223ed2411783b00106fcab0cd91b2f78691531ac526ed7bb3ed7547b6e2adbfb2e93
 2020-04-03 18:10:28 +08:00
fanquake
7142d50ac3
scripts: rename test_64bit_PE to test_PE 2020-03-26 11:39:34 +08:00
fanquake
edaca2dd12
scripts: add MACHO NX check to security-check.py 2020-03-26 11:39:34 +08:00
fanquake
1a4e9f32ef
scripts: add MACHO tests to test-security-check.py 2020-03-26 11:39:34 +08:00
Sebastian Falbesoner
332f373a9d [scripts] previous_release: improve failed download error message 
before:
------------------------------------------------------------
$ contrib/devtools/previous_release.sh -r -b v0.9.5
[...]
gzip: stdin: not in gzip format
tar: Child returned status 1
tar: Error is not recoverable: exiting now
------------------------------------------------------------

now:
------------------------------------------------------------
$ contrib/devtools/previous_release.sh -r -b v0.9.5
[...]
curl: (22) The requested URL returned error: 404 Not Found
Download failed.
------------------------------------------------------------
 2020-03-25 09:06:48 +01:00
fanquake
1a0993ae35
scripts: add PE dylib checking to symbol-check.py 2020-03-22 10:47:38 +08:00
Sjors Provoost
c7ca630896
[scripts] support release candidates of earlier releases 2020-02-11 21:46:22 +01:00
Sjors Provoost
ae379cf7d1
[scripts] build earlier releases 2020-02-11 19:37:37 +01:00
Wladimir J. van der Laan
3a037d0067 test: Add crc32c exception to various linters and generation scripts 2020-01-28 17:01:48 +01:00
Sebastian Falbesoner
b3c4d9bac6 test: rename test suite name "tx_validationcache_tests" to match filename 
Quoting src/test/README.md, 'Adding test cases':
    "The file naming convention is `<source_filename>_tests.cpp`
     and such files should wrap their tests in a test suite
     called `<source_filename>_tests`."

Currently the unit test source file txvalidationcache_tests.cpp contains a unit
test suite with the name tx_validationcache_tests, which is fixed by this commit.
The following shell script shows that this is the only mismatch and for all other
unit test source files the test suite names are correct:

 #!/bin/bash
 shopt -s globstar
 for test_full_filename in **/*_tests.cpp; do
     test_name_file=`basename $test_full_filename .cpp`
     test_name_suite=`sed -n "s/^.*TEST_SUITE(\(.*_tests\).*$/\1/p" $test_full_filename`
     if [ $test_name_file != $test_name_suite ]; then
         echo "TestFilename: $test_name_file != TestSuitname: $test_name_suite"
     fi
 done
 2020-01-27 22:44:02 +01:00
Wladimir J. van der Laan
a3d198c93c
Merge #17863: scripts: Add MACHO dylib checks to symbol-check.py 
c491368d8c scripts: add MACHO dylib checking to symbol-check.py (fanquake)
76bf97213f scripts: fix check-symbols & check-security argument passing (fanquake)

Pull request description:

  Based on #17857.

  This adds dynamic library checks for MACHO executables to symbol-check.py. The script has been modified to function more like `security-check.py`. The error output is now also slightly different. i.e:
  ```bash
  # Linux x86
  bitcoin-cli: symbol operator new[](unsigned long) from unsupported version GLIBCXX_3.4
  bitcoin-cli: export of symbol vtable for std::basic_ios<char, std::char_traits<char> > not allowed
  bitcoin-cli: NEEDED library libstdc++.so.6 is not allowed
  bitcoin-cli: failed IMPORTED_SYMBOLS EXPORTED_SYMBOLS LIBRARY_DEPENDENCIES

  # RISCV (skips exported symbols checks)
  bitcoin-tx: symbol operator new[](unsigned long) from unsupported version GLIBCXX_3.4
  bitcoin-tx: NEEDED library libstdc++.so.6 is not allowed
  bitcoin-tx: failed IMPORTED_SYMBOLS LIBRARY_DEPENDENCIES

  # macOS
  Checking macOS dynamic libraries...
  libboost_filesystem.dylib is not in ALLOWED_LIBRARIES!
  bitcoind: failed DYNAMIC_LIBRARIES
  ```

  Compared to `v0.19.0.1` the macOS allowed dylibs has been slimmed down somewhat:
  ```diff
   src/qt/bitcoin-qt:
   /usr/lib/libSystem.B.dylib
  -/System/Library/Frameworks/DiskArbitration.framework/Versions/A/DiskArbitration
   /System/Library/Frameworks/IOKit.framework/Versions/A/IOKit
   /System/Library/Frameworks/Foundation.framework/Versions/C/Foundation
   /System/Library/Frameworks/CoreServices.framework/Versions/A/CoreServices
   /System/Library/Frameworks/AppKit.framework/Versions/C/AppKit
   /System/Library/Frameworks/ApplicationServices.framework/Versions/A/ApplicationServices
   /System/Library/Frameworks/CoreFoundation.framework/Versions/A/CoreFoundation
  -/System/Library/Frameworks/Security.framework/Versions/A/Security
  -/System/Library/Frameworks/SystemConfiguration.framework/Versions/A/SystemConfiguration
   /System/Library/Frameworks/CoreGraphics.framework/Versions/A/CoreGraphics
  -/System/Library/Frameworks/OpenGL.framework/Versions/A/OpenGL
  -/System/Library/Frameworks/AGL.framework/Versions/A/AGL
   /System/Library/Frameworks/Carbon.framework/Versions/A/Carbon
   /usr/lib/libc++.1.dylib
  -/System/Library/Frameworks/CFNetwork.framework/Versions/A/CFNetwork
   /System/Library/Frameworks/CoreText.framework/Versions/A/CoreText
   /System/Library/Frameworks/ImageIO.framework/Versions/A/ImageIO
   /usr/lib/libobjc.A.dylib
  ```

ACKs for top commit:
  laanwj:
    ACK c491368d8c

Tree-SHA512: f8624e4964e80b3e0d34e8d3cc33f3107938f3ef7a01c07828f09b902b5ea31a53c50f9be03576e1896ed832cf2c399e03a7943a4f537a1e1c705f3804aed979
 2020-01-22 20:33:44 +01:00
MarcoFalke
95ca6aeec7
Merge #17691: doc: Add missed copyright headers 
fac86ac7b3 scripted-diff: Add missed copyright headers (Hennadii Stepanov)
6fde9d5e47 script: Update EXLUDE list in copyright_header.py (Hennadii Stepanov)
1998152f15 script: Add empty line after C++ copyright (Hennadii Stepanov)
071f2fc204 script: Add ability to insert copyright to *.sh (Hennadii Stepanov)

Pull request description:

  This PR improves `contrib/devtools/copyright_header.py` script and adds copyright headers to the files in `src` and `test` directories with two exceptions:
  - [`src/reverse_iterator.h`](https://github.com/bitcoin/bitcoin/blob/master/src/reverse_iterator.h) (added to exceptions)
  - [`src/test/fuzz/FuzzedDataProvider.h`](https://github.com/bitcoin/bitcoin/blob/master/src/test/fuzz/FuzzedDataProvider.h) (added to exceptions)

  On master 5622d8f315:
  ```
  $ ./contrib/devtools/copyright_header.py report . | grep zero
    25 with zero copyrights
  ```

  With this PR:
  ```
  $ ./contrib/devtools/copyright_header.py report . | grep zero
     2 with zero copyrights
  ```

  ~I am uncertain about our copyright policy with `build_msvc` and `contrib` directories content, so they are out of scope of this PR.~

ACKs for top commit:
  MarcoFalke:
    ACK fac86ac7b3

Tree-SHA512: d7832c4a7a1a3b7806119775b40ec35d7982f49ff0e6199b8cee4c0e0a36e68d51728b6ee9924b1c161df4bc6105bd93391b79d42914357fa522f499cb113fa8
 2020-01-16 15:58:35 -05:00
MarcoFalke
e09c701e01 scripted-diff: Bump copyright of files changed in 2020 
-BEGIN VERIFY SCRIPT-
./contrib/devtools/copyright_header.py update ./
-END VERIFY SCRIPT-
 2020-01-15 02:18:00 +07:00
Hennadii Stepanov
fac86ac7b3
scripted-diff: Add missed copyright headers 
-BEGIN VERIFY SCRIPT-
s() { contrib/devtools/copyright_header.py insert "$1"; }

s build_msvc/bitcoin_config.h
s build_msvc/msvc-autogen.py
s build_msvc/testconsensus/testconsensus.cpp
s contrib/devtools/circular-dependencies.py
s contrib/devtools/gen-manpages.sh
s contrib/filter-lcov.py
s contrib/gitian-build.py
s contrib/install_db4.sh
s src/crypto/sha256_avx2.cpp
s src/crypto/sha256_sse41.cpp
s src/fs.cpp
s src/qt/test/addressbooktests.cpp
s src/qt/test/addressbooktests.h
s src/qt/test/util.cpp
s src/qt/test/util.h
s src/qt/test/wallettests.cpp
s src/qt/test/wallettests.h
s src/test/blockchain_tests.cpp
s test/functional/combine_logs.py
s test/lint/lint-locale-dependence.sh
sed -i '1G' test/lint/lint-shebang.sh
s test/lint/lint-shebang.sh
-END VERIFY SCRIPT-
 2020-01-04 20:18:28 +02:00
fanquake
c491368d8c
scripts: add MACHO dylib checking to symbol-check.py 2020-01-04 11:24:42 +08:00
Hennadii Stepanov
6fde9d5e47
script: Update EXLUDE list in copyright_header.py 2020-01-03 17:48:37 +02:00
Hennadii Stepanov
1998152f15
script: Add empty line after C++ copyright 2020-01-03 17:48:37 +02:00
Hennadii Stepanov
071f2fc204
script: Add ability to insert copyright to *.sh 2020-01-03 17:47:54 +02:00
fanquake
7c9e821c4e
scripts: add MACHO NOUNDEFS check to security-check.py 2020-01-02 14:42:23 +08:00
fanquake
4ca92dc6d3
scripts: add MACHO PIE check to security-check.py 2020-01-02 14:42:21 +08:00
MarcoFalke
aaaaad6ac9
scripted-diff: Bump copyright of files changed in 2019 
-BEGIN VERIFY SCRIPT-
./contrib/devtools/copyright_header.py update ./
-END VERIFY SCRIPT-
 2019-12-30 10:42:20 +13:00
Wladimir J. van der Laan
f6e42256fe build: Allow export of environ symbols 
This export was introduced in #17270 which added
```
//! Necessary on some platforms
extern char** environ;
```
 2019-11-23 10:30:41 +01:00
Wladimir J. van der Laan
16a7be1663
build: Bump minimum versions in symbol checker 
Debian 8 (Jessie) has:
- g++ version 4.9.2
- libc version 2.19

Ubuntu 16.04.4 (Xenial) has:
- g++ version 5.3.1
- libc version 2.23.0

CentOS 7 has:
- g++ version 4.8.5
- libc version 2.17

Taking the minimum of these as our target.
According to the GNU ABI document this corresponds to:

- GCC 4.8.5: GCC_4.8.0
- (glibc)    GLIBC_2_17

Co-Authored-By: fanquake <fanquake@gmail.com>
 2019-11-22 15:40:05 -05:00
Wladimir J. van der Laan
b77d5ad59f
build: Disallow dynamic linking against c++ library 
Ever since statically linking Qt, we've been linking the C++ library
statically too (-static-libstdc++). Take this into
account in the symbol checker.
 2019-11-22 15:38:52 -05:00
fanquake
ec89d2882a
build: remove libanl.so.1 from ALLOWED_LIBRARIES 2019-11-20 15:05:35 -05:00
James O'Beirne
c1ccbc3dde devtools: add utxo_snapshot.sh 
to allow easy (if not time-consuming) generation and verification of
snapshots.
 2019-11-05 13:36:04 -05:00
gchuf
dffae5a5ad scripts: update copyright_header script to include additional files 
Includes .sh and .bash-completion files in the script as well
 2019-10-04 17:47:55 +02:00
Ben Woosley
3d0a82cff8
devtools: Accomodate block-style copyright blocks 
Without this, `copyright_header.py report . verbose` reports:
-------------------------------------------------------------------------------
1 with unexpected copyright holder names
	./build_msvc/libsecp256k1_config.h
-------------------------------------------------------------------------------
 2019-09-03 14:41:43 -04:00
Ben Woosley
b21680baf5
test/contrib: Fix invalid escapes in regex strings 
Flagged by flake8 v3.6.0, as W605, plus a few others identified
incidentally, e.g. 59ffecf66cf4d08c4b431e457b083878d66a3fd6.

Note that r"\n" matches to "\n" under re.match/search.
 2019-09-03 14:38:38 -04:00
fanquake
3d50fe2c1f
scripts: move update-translations.py to maintainer-tools repo 2019-08-20 11:14:00 +08:00
fanquake
6c27fa0f09
scripts: remove github-merge.py 
This script has been moved to the bitcoin-core/bitcoin-maintainer-tools repository,
after discussion in a core dev meeting. The rationale being that it is also useful to
other projects, and thus should be moved to a more general repository.

Meeting log: http://www.erisian.com.au/bitcoin-core-dev/log-2019-08-15.html
See also: https://github.com/bitcoin-core/bitcoin-maintainer-tools/pull/28
 2019-08-19 19:16:57 +08:00
Carl Dong
65f8da08df
symbol-check: Disallow libX11-*.so.* shared libraries 
They should no longer be needed as we build QT without libX11/XLib
libraries now.
 2019-07-17 17:09:48 -04:00
fanquake
1088b90cba
Merge #16327: scripts and tools: Update ShellCheck linter 
1ac454a384 Enable ShellCheck rules (Hennadii Stepanov)

Pull request description:

  Enable some simple ShellCheck rules.

  Note for reviewers: `bash` and `shellcheck` on macOS are different from ones on Ubuntu.
  For local tests the latest `shellcheck` version 0.6.0 should be used (see #15166).

ACKs for top commit:
  practicalswift:
    utACK 1ac454a384
  dongcarl:
    utACK 1ac454a
  fanquake:
    ACK 1ac454a384

Tree-SHA512: 8d0a3a5c09fe1a0c22120178f5e6b80f81f746f8c3356b7701ff301c117acb2edea8fe08f08fb54ed73f94b1617515fb239fa28e7ab4121f74872e6494b6f20e
 2019-07-05 09:19:23 +08:00
Hennadii Stepanov
1ac454a384
Enable ShellCheck rules 
Enabled ShellCheck rules:
  SC1087
  SC2001
  SC2004
  SC2005
  SC2006
  SC2016
  SC2028
  SC2048
  SC2066 (note that IFS already contains only a line feed)
  SC2116
  SC2166
  SC2181
  SC2206
  SC2207
  SC2230
  SC2236
 2019-07-04 19:35:25 +03:00
Hennadii Stepanov
ca11606858
Fix: "Bitcoin Core" -> "The Bitcoin Core" 2019-06-29 18:58:52 +03:00
Hennadii Stepanov
621463d7df
Drop no-longer-relevant copyright holder name 2019-06-29 18:50:39 +03:00
Hennadii Stepanov
01fafe55a1
Include Objective-C source files 
The copyright_header.py script will process Objective-C source files 
(*.mm) as other ones.
 2019-06-29 18:35:45 +03:00
MarcoFalke
e115a21f79
Merge #16223: devtools: Fetch and display ACKs at sign-off time in github-merge 
0e01e4522e devtools: Fetch and display ACKs at sign-off time in github-merge (Wladimir J. van der Laan)

Pull request description:

  - Fetch the ACKs only at sign-off time. This makes sure that any last-minute ACKs are included (fixes #16200)
  - Show a list of ACKs that will be included and their author before signing off, and warn if there are none

  ![1](https://user-images.githubusercontent.com/126646/59605250-ad070980-910e-11e9-9f9a-d789c7f06ebb.png)
  ![2](https://user-images.githubusercontent.com/126646/59605255-b1332700-910e-11e9-80a5-d1e244f48264.png)

  There's a slight change to the merge commit format—before it was
  ```
      ACKs for commit 88884c:
  (list of ACKs, could be empty)
  ```
  now it is
  ```
  ACKs for top commit:
        jnewbery:
          ACK 5ebc6b0eb
      ... (list of ACKs cannot be empty)
  ```
  or
  ```
  Top commit has no ACKs.
  ```
  I don't think there's a reason to have the abbreviated commit ID there, after all the full commit id is already in the beginning of the merge commit message, and at least the abbreviated one is in every single ACK message.

ACKs for commit 0e01e4:
  fanquake:
    ACK 0e01e4522e

Tree-SHA512: 8576de016137d71cfc101747e9bb6779c13e0953cf2babee7afc9972bf2bd46f6912be4982b54fa5abf4d91e98e8fdae6b4ca3eef7d6892b7a5f04a7017b6882
 2019-06-24 15:09:24 -04:00
Wladimir J. van der Laan
0e01e4522e devtools: Fetch and display ACKs at sign-off time in github-merge 
- Fetch the ACKs only at sign-off time. This makes sure that any
  last-minute ACKs are included (fixes #16200)
- Show a list of ACKs and their author before signing off, and warn if
  there are none
 2019-06-17 14:54:35 +02:00
fanquake
47d981e827
Merge #16186: doc/lint: Fix spelling errors identified by codespell 1.15.0 
b748bf6f50 Fix spelling errors identified by codespell 1.15.0 (Ben Woosley)

Pull request description:

  Note all changes are to comments / documentation.

  After this commit, the only remaining output is:

  ```
    $ test/lint/lint-spelling.sh
    src/test/base32_tests.cpp:14: fo  ==> of, for
    src/test/base64_tests.cpp:14: fo  ==> of, for
    ^ Warning: codespell identified likely spelling errors. Any false positives? Add them to the list of ignored words in test/lint/lint-spelling.ignore-words.txt
  ```

  Note:
  * I ignore several valid alternative spellings ~, but changed homogenous
  to homogeneous as the latter is a more specific term according to the
  Google dictionary definitions I found~
  * homogenous is present in tinyformat, hence should be addressed upstream
  * process' is correct only if there are plural processes

ACKs for commit b748bf:
  practicalswift:
    ACK b748bf6f50
  fanquake:
    ACK b748bf6f50

Tree-SHA512: 9add7044643ce015e0a44d8b27a3f300d72c485ffff550fb6491a17f14528085289ec5caddfe02f291ea9b2cded38a0dd3079652a054e2d7fe2ff4f7b53db5d7
 2019-06-16 09:57:09 +08:00
fanquake
7524376a81
Merge #16201: devtools: Always use unabbreviated commit IDs in github-merge.py 
067fba563 devtools: Always use unabbreviated commit IDs in github-merge.py (Wladimir J. van der Laan)

Pull request description:

  Always put the unabbreviated commit IDs in the generated commit messages and other places. This prevents the developer's `core.abbrev` git setting from leaking through and is better against ambiguity too.

ACKs for commit 067fba:
  MarcoFalke:
    ACK 067fba5631 (replaces `h` with `H`, didn't test)
  promag:
    ACK 067fba5631, from the documentation https://git-scm.com/docs/pretty-formats:
  fanquake:
    ACK 067fba5631. Tested by merging this PR into master, then merging a second PR ontop and checking that full commit hashes were being used. Also checked documentation linked to above. Did not check that this works when a different `core.abbrev` is set locally.

Tree-SHA512: a851d10490cd8bcd8bca29094b08a6b9f883cfe1b0767ccda7ca789e4c8eff6260a4d82c33cb3d9bab01dd30ac8c9100cb7adbcb1911bb399d9385c1e1f15ecd
 2019-06-14 13:38:52 +08:00
Wladimir J. van der Laan
067fba5631 devtools: Always use unabbreviated commit IDs in github-merge.py 
Always put the unabbreviated commit IDs in the generated commit messages
and other places. This prevents the developer's `core.abbrev` git setting
from leaking through and is better against ambiguity too.
 2019-06-13 10:21:36 +02:00
Ben Woosley
b748bf6f50
Fix spelling errors identified by codespell 1.15.0 
After this commit, the only remaining output is:

  $ test/lint/lint-spelling.sh
  src/test/base32_tests.cpp:14: fo  ==> of, for
  src/test/base64_tests.cpp:14: fo  ==> of, for
  ^ Warning: codespell identified likely spelling errors. Any false positives? Add them to the list of ignored words in test/lint/lint-spelling.ignore-words.txt

Note:
* I ignore several valid alternative spellings
* homogenous is present in tinyformat, hence should be addressed upstream
* process' is correct only if there are plural processes
 2019-06-11 17:18:16 +02:00
MarcoFalke
6899ef3f0e
Merge #16143: tests: Mark unit test blockfilter_index_initial_sync as non-deterministic 
c061be1e2f tests: Mark unit test blockfilter_index_initial_sync as non-deterministic (practicalswift)

Pull request description:

  Mark unit test `blockfilter_index_tests/blockfilter_index_initial_sync` as non-deterministic.

  Before this PR:

  ```
  $ contrib/devtools/test_deterministic_coverage.sh 500
  [2019-06-04 09:58:57] Measuring coverage, run #1 of 500
  [2019-06-04 10:00:33] Measuring coverage, run #2 of 500
  [2019-06-04 10:02:19] Measuring coverage, run #3 of 500

  The line coverage is non-deterministic between runs. Exiting.

  The test suite must be deterministic in the sense that the set of lines executed at least
  once must be identical between runs. This is a necessary condition for meaningful
  coverage measuring.

  --- gcovr.run-1.txt     2019-06-04 10:00:33.389059973 +0000
  +++ gcovr.run-3.txt     2019-06-04 10:03:45.619491207 +0000
  @@ -72,7 +72,7 @@
   hash.h                                        54      33    61%   71,74-77,82,85-89,111,113,128,147-148,175,178-181
   httprpc.cpp                                  120       3     2%   31,34-35,38-40,46,49,52,54,56,58,70,73-74,76,78-79,81,83-84,89,91,94-95,97,99-101,103,106-107,111-112,117-119,121-122,125,128,130,132,134-136,138-139,142,145,148,151-153,156-160,163-166,171,173-175,180-182,185,187,189-190,192,195,198-199,201,203-204,212,215,217,219-222,224,227-228,230,232,237,239-240,243-245,247-251,254,256,259,261-264,266-267 [* 205-206,208-209]
   httpserver.cpp                               312       6     1%   46,49-50,53,55,80-81,90,92-93,96-98,101,104,106-109,111-112,114,118,120-122,126,128-129,153,155,157-158,164,166-178,180,182,184-188,192,194-196,198-199,201-202,204-205,207-208,213,216-221,225,228-232,236-239,243-244,247-254,256-258,264-267,270-271,274,279,281-282,286,288-290,292-293,297,299-300,303-307,309-310,312-317,322-328,330,332,335,339,341-342,346,352-353,355,358,360,364,368-369,375,378,381-384,388-391,393-394,398-400,402,404-406,409,411-412,414,416,426,428-431,433-434,438,440-441,443,445-446,449,451-455,457-459,463-464,466-469,471-473,475-477,479,482,484,487,490-493,496-497,499-500,502,504,506,508-509,511,513-514,517,519,521-522,527,529-533,535,538,540-543,550-555,558,560-562,570,572-574,577-582,585-590,594-597,600,602-604,606-609,611,614,616,619,621,625-626,628-629,631-632,634-635,640,642-643,646,648-651,653,655-656
  -index/base.cpp                               149      94    63%   20,22-25,28,66,98,102-103,117-118,140-141,145-146,155,163,175,177-178,181-182,184-185,200-201,203,212,214-215,219-221,228-229,234,236,240,243-244,247-249,258-260,262,270,292-294,308-309 [* 263]
  +index/base.cpp                               149      97    65%   20,22-25,28,66,98,102-103,117-118,140-141,145-146,155,163,175,177-178,181-182,184-185,200-201,203,212,214-215,219-221,228-229,234,236,240,243-244,247-249,258-260,262,270,308-309 [* 263]
   index/base.h                                   3       2    66%   77
   index/blockfilterindex.cpp                   199     134    67%   70,79,81,84-88,91,122,139,142,179-181,184-185,188-189,193-194,201-202,207,233,258,262-263,265-266,268,271-272,274,277,279,284,286,288-289,294,301-302,304,322,329,332-333,350,371,373,438,440-441,444,446,449,455-456,459,461,464,466 [* 162-163]
   index/blockfilterindex.h                       4       4   100%
  @@ -358,7 +358,7 @@
   util/validation.cpp                            5       1    20%   12,15-17
   validation.cpp                              2167     808    37%   291,293,297-300,302,330,332,340,348,355-357,359,362,364-365,368,371,380,382-383,385-386,388-389,396,398-402,406-413,415,417,419,422-425,439-440,442-443,446,449,455-458,461-464,467,469-470,472,474,476,492,494-495,502-503,505-507,511-513,515,517,523,526,528,533,535,540,542-544,550,552-556,558-560,564,574,578-583,586,590-591,594-596,601-602,607-608,611-612,616-617,619-621,635-636,638,640,647-648,651,657-658,660-662,665-667,673,675,677-678,682-683,690,693,700-701,703-705,709-710,713-714,716,719-720,724-727,733-735,737-739,741-743,747-748,751-752,754,757-764,771,773-774,776-779,785-788,793-794,796-800,815-816,818-822,825,827,830,835,838-839,841-843,846-848,850,853,859,864-867,875,877-879,884-885,887-891,895,899-900,904-906,908-909,911,930-931,933,936,942,944-950,952,959,962,965-968,972,978,982-984,990-991,994-996,999,1003-1004,1011,1013,1015-1019,1022-1023,1026-1032,1056,1065,1079,1091,1108,1112,1114-1118,1125,1127-1130,1133-1135,1138-1139,1147,1149,1151-1152,1155,1197,1199-1201,1206-1209,1211-1212,1226,1230,1232-1234,1236,1238-1241,1245-1246,1256,1258,1260-1262,1264-1266,1268,1278-1280,1282-1283,1286,1289,1291-1292,1294-1302,1305-1311,1319-1323,1330,1332-1333,1336-1339,1379,1383-1384,1395,1401,1405-1407,1411-1414,1423-1428,1438-1440,1451,1455,1458,1471,1480,1497,1503,1519,1525,1527-1530,1532-1533,1536,1538-1539,1549,1551,1553,1555,1559-1562,1571,1573,1578,1580,1582-1584,1588-1589,1594-1597,1601-1606,1613-1616,1619-1623,1630,1632,1635,1637,1639-1640,1642-1646,1658,1660,1675,1688,1711,1713-1715,1742,1755,1760,1765,1769,1811,1815,1817,1841-1845,1855,1942,1946-1947,1956,1984-1986,1991-1992,1994,1996-1999,2005-2007,2010-2012,2022-2023,2028-2031,2038-2039,2042,2044,2049,2058-2061,2064,2114-2115,2117-2118,2120-2124,2152-2153,2156,2159-2163,2165-2169,2171-2172,2176-2178,2187-2188,2191-2194,2199,2207-2211,2215-2220,2224,2227-2230,2235,2237-2238,2261-2263,2265,2274,2278,2286,2301,2303-2304,2306-2309,2311,2313-2318,2320,2322,2325,2327-2328,2330,2332-2334,2338,2340,2343-2344,2407-2410,2430,2445-2447,2507-2509,2511-2514,2518,2520-2521,2523-2524,2561,2564,2590,2592-2593,2595-2598,2603,2620,2626,2658,2719,2724,2773,2776-2777,2779,2781,2783,2785-2788,2791,2793-2795,2799,2801-2802,2805,2807-2809,2813,2816,2818-2821,2825-2826,2832-2834,2841-2845,2848,2854,2858-2859,2861,2865-2868,2872-2875,2880,2884-2885,2890-2891,2894-2895,2897,2900-2906,2908,2910,2912,2918-2922,2924,2928-2929,2940,3002-3005,3009-3010,3026-3028,3036-3037,3039-3040,3045,3053,3056,3077,3080,3090,3112,3118,3129,3133,3135-3136,3141-3142,3150,3190-3193,3259,3268,3273,3277,3282-3285,3303,3314,3321-3324,3338-3341,3345-3346,3348-3350,3360,3372,3392,3397,3403,3406,3408,3435-3441,3443,3468-3469,3485,3487-3488,3492-3493,3534-3536,3542,3547-3549,3552,3565-3566,3601-3602,3610,3628,3630,3632,3645,3647,3649-3651,3653,3657,3659,3661-3669,3675-3680,3686-3687,3691,3693-3697,3702,3704,3706-3708,3711-3718,3720,3724,3726-3729,3748,3750-3752,3754,3758-3759,3763,3765,3767,3772,3774,3777-3778,3780-3781,3783,3787-3788,3790,3792-3794,3798-3800,3823,3825,3828,3830,3832,3836-3838,3841-3843,3845,3848,3850,3854-3856,3858-3859,3861-3862,3864-3867,3870-3873,3875-3876,3879,3882-3883,3886-3893,3899,3901,3905-3909,3911-3915,3922-3924,3926-3928,3931,3933-3934,3940-3942,3945-3947,3952,3954-3955,3957,3960-3961,3964,3966,3968-3972,3975,3977,3980,3982,3985,3987-3988,3992-3996,3998-4006,4008-4009,4011-4012,4014,4016,4019,4021-4022,4024-4026,4028-4032,4037-4041,4043-4045,4047,4050,4053-4054,4057,4060-4064,4066-4067,4069-4075,4079-4080,4086,4089-4091,4094-4097,4101,4106,4108,4110,4112-4114,4116-4117,4119,4121,4123-4124,4126,4128-4130,4132-4134,4138-4142,4144-4147,4154,4158-4163,4166-4169,4172-4173,4177,4179-4180,4183,4185,4187-4189,4191-4193,4195,4197-4201,4207-4208,4212,4220-4223,4230,4232-4233,4237,4240,4243,4247,4249,4251,4253-4255,4265-4266,4277,4279,4282,4285-4287,4292-4293,4296,4298,4302,4305-4306,4310-4311,4315-4318,4360,4363-4367,4370,4377,4397,4412,4415-4416,4418,4421-4422,4424,4426-4429,4433-4437,4439-4441,4448-4452,4454-4456,4458,4460,4462-4467,4471-4475,4477,4480-4481,4486-4488,4493,4496-4503,4505,4507-4511,4513-4514,4517-4519,4529-4531,4546,4600,4638-4639,4647,4653,4662-4664,4696,4703-4704,4718,4720,4723,4725,4727,4730,4732-4733,4736,4738-4739,4742,4744-4745,4750,4752-4757,4761-4765,4769-4770,4774-4776,4779-4781,4783-4785,4787-4790,4793-4794,4800-4801,4803,4807,4809-4810,4812-4813,4815-4816,4823,4827,4829,4831-4832,4834-4835,4838-4840,4842,4845,4848-4849,4853,4855-4856,4858-4863,4866-4872,4877,4891,4907 [* 1085-1086,1140-1141,1513-1514,2201-2202,2428,3569-3570,4400-4401,4442,4453,4504,4522-4523,4526-4527,4818-4819,4873-4874]
   validation.h                                  19       5    26%   338,350-352,356-363,366,484
  -validationinterface.cpp                       81      50    61%   78-82,85-86,112-113,116,119-120,123-124,126-128,130,133-136,151-153,163-165,169-171
  +validationinterface.cpp                       83      60    72%   78-82,85-86,112-113,116,133-136,151-153,163-165,169-171
   validationinterface.h                          9       4    44%   94,105,112,118,135
   versionbits.cpp                               92      27    29%   33,35-36,38-39,48-50,52-54,56-57,61-62,67-71,73,75-76,80,82-83,91,98,100,102-103,105,109-110,113-118,121-122,124,127,129-130,134,137,141,149,151,153-155,159,177,179,184,194,196,199,201,204,206 [* 26]
   versionbits.h                                  1       1   100%
  @@ -400,5 +400,5 @@
   zmq/zmqpublishnotifier.h                       5       0     0%   12,31,37,43,49
   zmq/zmqrpc.cpp                                23       3    13%   16,18,20,23,33-35,37,40-47,51,62,64-65
   ------------------------------------------------------------------------------
  -TOTAL                                      52472    7784    14%
  +TOTAL                                      52474    7797    14%
   ------------------------------------------------------------------------------
  $
  ```

  After this PR:

  ```
  $ contrib/devtools/test_deterministic_coverage.sh 500
  [2019-06-03 14:45:25] Measuring coverage, run #1 of 500
  [2019-06-03 14:48:15] Measuring coverage, run #2 of 500
  [2019-06-03 14:50:49] Measuring coverage, run #3 of 500
  [2019-06-03 14:52:20] Measuring coverage, run #4 of 500
  [2019-06-03 14:53:49] Measuring coverage, run #5 of 500
  …
  [2019-06-04 09:04:58] Measuring coverage, run #496 of 500
  [2019-06-04 09:07:42] Measuring coverage, run #497 of 500
  [2019-06-04 09:10:32] Measuring coverage, run #498 of 500
  [2019-06-04 09:13:26] Measuring coverage, run #499 of 500
  [2019-06-04 09:16:32] Measuring coverage, run #500 of 500

  Coverage test passed: Deterministic coverage across 500 runs.
  $
  ```

ACKs for commit c061be:

Tree-SHA512: 00cd55b4371290d8587ab667c64249bc31d26cc9dc3dd519677eb91ddb9dbc5333dfbdef5e90c7a0d74eecd24757113e7ec3eda836859ddc033b1de715df81b6
 2019-06-11 07:37:03 -04:00
practicalswift
c061be1e2f tests: Mark unit test blockfilter_index_initial_sync as non-deterministic 2019-06-04 12:28:04 +02:00
Steven Roose
a352d2ae5f
Don't GPG sign intermediate commits with github-merge tool 2019-05-31 19:22:02 +01:00
MarcoFalke
faf666f814
Remove Windows 32 bit build 2019-05-03 13:41:27 -04:00
nkostoulas
942ff2054b
contrib: gh-merge: Use pagination to fetch all review comments 2019-04-17 20:21:45 +01:00
MarcoFalke
fa1c073154
contrib: gh-merge: Include review comments in merge commit 2019-03-22 12:03:04 -04:00
MarcoFalke
b83c6f7940
Merge #15444: [docs] Additional productivity tips 
ff7f31e07d [doc] productivity: more advanced git range-diff (Sjors Provoost)
3a21905a4e [doc] devtools: mention clang-format dependency (Sjors Provoost)
bf12093191 [doc] productivity: fix broken link (Sjors Provoost)

Pull request description:

  Fixes a broken link to `devtools/README.md`, points out the `clang-format` dependency and adds a `git range-diff` incantation that works even with rebases and squashes.

Tree-SHA512: 36e46282f1e28d1bf3f48ada995fbac548f61b7747091eb032b60919cf76c7bdad0fa8aecb0c47adbdaa9ef986d3ec7752b0bb94c63191401856e2ddeec48f3e
 2019-03-14 17:02:28 -04:00
Wladimir J. van der Laan
12408d33c6
Merge #15549: gitian: Improve error handling 
32da92bdf6 gitian: Improve error handling (Wladimir J. van der Laan)

Pull request description:

  Improve error handling in gitian builds:

  - Set fail-on-error and pipefail flag, this causes a command to fail when either of the pipe stages fails, not only when the last of the stages fails, so this improves error detection.
  - Also use `xargs` instead of `find -exec`, because `find` will not propagate errors in the executed command, but `xargs` will.

  This will avoid some issues like #15541 where non-determinism is silently introduced due to errors caused by environment conditions (such as lack of disk space in that case).

Tree-SHA512: d5d3f22ce2d04a75e5c25e935744327c3adc704c2d303133f2918113573a564dff3d3243d5569a2b93ee7eb0e97f8e1b1ba81767e966af9015ea711a14091035
 2019-03-09 06:45:19 +01:00
Wladimir J. van der Laan
32da92bdf6 gitian: Improve error handling 2019-03-07 17:21:46 +01:00
MarcoFalke
a74d588f21
Merge #14954: build: Require python 3.5 
fa2797808e test: Remove python3.4 workaround in feature_dbcrash (MarcoFalke)
dddd1d05d3 .python-version: Specify full version 3.5.6 (MarcoFalke)
faa7cdf764 scripted-diff: Update copyright in ./test (MarcoFalke)
fa0e65b772 scripted-diff: test: Remove brackets after assert (MarcoFalke)
fab5a1e0f4 build: Require python 3.5 (MarcoFalke)
fa6bf21f5e scripted-diff: test: Use py3.5 bytes::hex() method (MarcoFalke)

Pull request description:

  Python 3.4 is EOL after March 2019, so switch to 3.5. See https://devguide.python.org/#status-of-python-branches

  This pull does the following in a bunch of commits:
  * scripted diff to use the `bytes::hex()` method in place of previous wrappers (`b2x`, `bytes_to_hex_str`, `hexlify`, ...)
  * Update the build system (gitian and travis) to remove python2.7 and replace it with python3.5
  * Another scripted-diff to remove brackets after `assert`. This is unrelated to the python3.5 switch, but a stylistic commit, so probably not worth to split up. The motivation behind it is to avoid asserting on data structures (such as tuples of length one), which never fails:
  ```py
  >>> assert(False,)   # with brackets
  >>> assert False,    # without brackets
  SyntaxError: invalid syntax
  >>> assert False     # proper assertion
  AssertionError
  ```
  * And then a final scripted diff to update the copyright headers in the `test` subfolder, since I touched most of the files anyway and it wouldn't make sense to split this commit out into a separate pull.

  For reference (contributed by luke-jr):

  Ubuntu LTS (bionic): 3.6.5
  Debian stable (stretch): 3.5.3
  RHEL 8 (expected before v0.19): 3.6.x
  Gentoo stable: 3.6.5
  Arch: 3.7.1

Tree-SHA512: 643c28cd2d5b9543ce4bf8ad2a8b282bc79b37dc5b25c9c8358e6ce201e2a67a546463e5f3430b16652eb2489d7c3ed4b0772cd2e2bf790fe68a5e3cc8a25029
 2019-03-05 09:13:13 -05:00
MarcoFalke
45f434f44d
Merge #15288: Remove wallet -> node global function calls 
f7efd87c8f Change brace formatting (Russell Yanofsky)
a1df1b48a8 Remove use of IsInitialBlockDownload in wallet code (Russell Yanofsky)
1106a6fde4 Remove use of uiInterface.LoadWallet in wallet code (Russell Yanofsky)
318f41fb2c circular-dependencies: Avoid treating some .h/.cpp files as a unit (Russell Yanofsky)
d02b34c8a8 Remove use of AcceptToMemoryPool in wallet code (Russell Yanofsky)
e2c8ba9f6e Remove uses of InitMessage/Warning/Error in wallet code (Russell Yanofsky)
c5e59a96a8 Remove uses of GetAdjustedTime in wallet code (Russell Yanofsky)
6d6bcc77c0 Remove use of g_connman / PushInventory in wallet code (Russell Yanofsky)
00dfb2a440 Remove uses of g_connman in wallet code (Russell Yanofsky)
cc3836e8f9 Remove uses of fPruneMode in wallet code (Russell Yanofsky)
cc02c796d3 Remove uses of fee globals in wallet code (Russell Yanofsky)
1fb0a4a04e Remove use of CalculateMemPoolAncestors in wallet code (Russell Yanofsky)
cd32160af0 Remove use of GetTransactionAncestry in wallet code (Russell Yanofsky)
291276f7f4 Remove use of GetCountWithDescendants in wallet code (Russell Yanofsky)
bdc6628683 Remove use of IsRBFOptIn in wallet code (Russell Yanofsky)
80f52a2267 Remove uses of CheckFinalTx in wallet code (Russell Yanofsky)

Pull request description:

  This change removes wallet calls to node functions that access global chain and mempool state.

  This is the next step in the larger #10973 refactoring change, which removes all other accesses to node global variables from wallet code. Doing this is useful to provide a better defined interface between the wallet and node, and necessary to allow wallet and node code to run in separate processes in #10102.

Tree-SHA512: 40dbaf1f59fb22b32e70b054b30ba5638d638aa3240fa30e0f721d53c721cd6138a7ab4d423a24d7d2fda0b956e68d44c733abc2c9259c3d6c9fd6d4be89aa23
 2019-03-04 13:13:36 -05:00
MarcoFalke
fab5a1e0f4
build: Require python 3.5 2019-03-02 10:40:23 -05:00