Multiprocess tracking issue #28722
Comments
This was referenced Oct 24, 2023
fanquake
added a commit
to bitcoin-core/gui
that referenced
this issue
Nov 13, 2023
3b70f7b doc: fix broken doc/design/multiprocess.md links after #24352 (Ryan Ofsky) 6d43aad span: Make Span template deduction guides work in SFINAE context (Ryan Ofsky) 8062c3b util: Add ArgsManager SetConfigFilePath method (Ryan Ofsky) 441d00c interfaces: Rename CalculateBumpFees methods to be compatible with capn'proto (Ryan Ofsky) 156f49d interfaces: Change getUnspentOutput return type to avoid multiprocess segfault (Ryan Ofsky) 4978754 interfaces: Add schedulerMockForward method so mockscheduler RPC can work across processes (Ryan Ofsky) 924327e interfaces: Fix const virtual method that breaks multiprocess support (Ryan Ofsky) 82a379e streams: Add SpanReader ignore method (Russell Yanofsky) Pull request description: This is a collection of small changes to interfaces and code which were needed as part of multiprocess PR #10102, but have been moved here to make that PR smaller. All of these changes are refactoring changes which do not affect behavior of current code --- This PR is part of the [process separation project](bitcoin/bitcoin#28722). ACKs for top commit: achow101: ACK 3b70f7b naumenkogs: ACK 3b70f7b maflcko: re-ACK 3b70f7b 🎆 Tree-SHA512: 2368772b887056ad8a9f84c299cfde76ba45943770e3b5353130580900afa9611302195b899ced7b6e303b11f053ff204cae7c28ff4e12c55562fcc81119ba4c
This was referenced Nov 17, 2023
fanquake
added a commit
that referenced
this issue
Nov 22, 2023
…ion warnings 21bfee0 depends: bump libmultiprocess to fix capnproto deprecation warnings (Ryan Ofsky) Pull request description: This incorporates PR bitcoin-core/libmultiprocess#88 and reverts the NO_WERROR CI workaround added in #28735 Upstream diff: bitcoin-core/libmultiprocess@61d5a0e...414542f --- This PR is part of the [process separation project](#28722). ACKs for top commit: maflcko: lgtm ACK 21bfee0 hebasto: ACK 21bfee0, I have reviewed the code and it looks OK. I've also skimmed through the related changes in the https://github.com/chaincodelabs/libmultiprocess repository. Tree-SHA512: b5addb0deed694eeec62a0ae08b4715a811110201f39f3e6cadee8fc4e6231b0e66c844a98512072a1445bac122ab561dc1711e27fb4d7ac5c08ac46780a4acf
achow101
added a commit
to bitcoin-core/gui
that referenced
this issue
Jan 2, 2024
91dc48c doc: Add multiprocess design doc (Ryan Ofsky) Pull request description: Add multiprocess design doc and existing multiprocess documentation into design and usage sections. Links to rendered markdown: https://github.com/ryanofsky/bitcoin/blob/pr/ipcdoc/doc/design/multiprocess.md https://github.com/ryanofsky/bitcoin/blob/pr/ipcdoc/doc/multiprocess.md --- This PR is part of the [process separation project](bitcoin/bitcoin#28722). ACKs for top commit: fjahr: ACK 91dc48c achow101: ACK 91dc48c TheCharlatan: ACK 91dc48c stickies-v: ACK 91dc48c - left a couple of improvements but agreed that iterating in future PRs is better. Tree-SHA512: 8890abd85555eb3a64e75e3393f867839771a83aaba7667c19eccac2e959fb37b13c3bc1906ff06ff3d66609e2c72835b4b9a22d31e997af4489092418eeb001
achow101
added a commit
that referenced
this issue
Jan 23, 2024
6acec6b multiprocess: Add type conversion code for UniValue types (Ryan Ofsky) 0cc74fc multiprocess: Add type conversion code for serializable types (Ryan Ofsky) 4aaee23 test: add ipc test to test multiprocess type conversion code (Ryan Ofsky) Pull request description: Add type conversion hooks to allow `UniValue` objects, and objects that have `CDataStream` `Serialize` and `Unserialize` methods to be used as arguments and return values in Cap'nProto interface methods. Also add unit test to verify the hooks are working and data can be round-tripped correctly. The non-test code in this PR was previously part of #10102 and has been split off for easier review, but the test code is new. --- This PR is part of the [process separation project](#28722). ACKs for top commit: achow101: ACK 6acec6b dergoegge: reACK 6acec6b Tree-SHA512: 5d2cbc5215d488b876d34420adf91205dabf09b736183dcc85aa86255e3804c2bac5bab6792dacd585ef99a1d92cf29c8afb3eb65e4d953abc7ffe41994340c6
achow101
added a commit
that referenced
this issue
May 15, 2024
8d491ae serialization: Add ParamsStream GetStream() method (Ryan Ofsky) 951203b net: Simplify ParamsStream usage (Ryan Ofsky) e6794e4 serialization: Accept multiple parameters in ParamsStream constructor (Ryan Ofsky) cb28849 serialization: Reverse ParamsStream constructor order (Ryan Ofsky) 83436d1 serialization: Drop unnecessary ParamsStream references (Ryan Ofsky) 84502b7 serialization: Drop references to GetVersion/GetType (Ryan Ofsky) f3a2b52 serialization: Support for multiple parameters (Ryan Ofsky) Pull request description: Currently it is only possible to attach one serialization parameter to a stream at a time. For example, it is not possible to set a parameter controlling the transaction format and a parameter controlling the address format at the same time because one parameter will override the other. This limitation is inconvenient for multiprocess code since it is not possible to create just one type of stream and serialize any object to it. Instead it is necessary to create different streams for different object types, which requires extra boilerplate and makes using the new parameter fields a lot more awkward than the older version and type fields. Fix this problem by allowing an unlimited number of serialization stream parameters to be set, and allowing them to be requested by type. Later parameters will still override earlier parameters, but only if they have the same type. For an example of different ways multiple parameters can be set, see the new [`with_params_multi`](https://github.com/bitcoin/bitcoin/blob/40f505583f4edeb2859aeb70da20c6374d331a4f/src/test/serialize_tests.cpp#L394-L410) unit test. This change requires replacing the `stream.GetParams()` method with a `stream.GetParams<T>()` method in order for serialization code to retrieve the desired parameters. The change is more verbose, but probably a good thing for readability because previously it could be difficult to know what type the `GetParams()` method would return, and now it is more obvious. --- This PR is part of the [process separation project](#28722). ACKs for top commit: maflcko: ACK 8d491ae 🔵 sipa: utACK 8d491ae TheCharlatan: ACK 8d491ae Tree-SHA512: 40b7041ee01c0372b1f86f7fd6f3b6af56ef24a6383f91ffcedd04d388e63407006457bf7ed056b0e37b4dec9ffd5ca006cb8192e488ea2c64678567e38d4647
This was referenced Jul 23, 2024
achow101
added a commit
that referenced
this issue
Sep 9, 2024
30073e6 multiprocess: Add -ipcbind option to bitcoin-node (Russell Yanofsky) 73fe7d7 multiprocess: Add unit tests for connect, serve, and listen functions (Ryan Ofsky) 955d407 multiprocess: Add IPC connectAddress and listenAddress methods (Russell Yanofsky) 4da2043 depends: Update libmultiprocess library for CustomMessage function and ThreadContext bugfix (Ryan Ofsky) Pull request description: Add `-ipcbind` option to `bitcoin-node` to make it listen on a unix socket and accept connections from other processes. The default socket path is `<datadir>/node.sock`, but this can be customized. This option lets potential wallet, gui, index, and mining processes connect to the node and control it. See examples in #19460, #19461, and #30437. Motivation for this PR, in combination with #30510, is be able to release a bitcoin core node binary that can generate block templates for a separate Stratum v2 mining service, like the one being implemented in Sjors#48, that connects over IPC. Other things to know about this PR: - While the `-ipcbind` option lets other processes to connect to the `bitcoin-node` process, the only thing they can actually do after connecting is call methods on the [`Init`](https://github.com/bitcoin/bitcoin/blob/master/src/ipc/capnp/init.capnp#L17-L20) interface which is currently very limited and doesn't do much. But PRs [#30510](#30510), [#29409](#29409), and [#10102](#10102) expand the `Init` interface to expose mining, wallet, and gui functionality respectively. - This PR is not needed for [#10102](#10102), which runs GUI, node, and wallet code in different processes, because [#10102](#10102) does not use unix sockets or allow outside processes to connect to existing processes. [#10102](#10102) lets parent and child processes communicate over internal socketpairs, not externally accessible sockets. --- This PR is part of the [process separation project](#28722). ACKs for top commit: achow101: ACK 30073e6 TheCharlatan: Re-ACK 30073e6 itornaza: Code review ACK 30073e6 Tree-SHA512: 2b766e60535f57352e8afda9c3748a32acb5a57b2827371b48ba865fa9aa1df00f340732654f2e300c6823dbc6f3e14377fca87e4e959e613fe85a6d2312d9c8
achow101
added a commit
that referenced
this issue
Sep 25, 2024
1a33281 doc: multiprocess documentation improvements (Ryan Ofsky) d043950 multiprocess: Add serialization code for BlockValidationState (Ryan Ofsky) 33c2eee multiprocess: Add IPC wrapper for Mining interface (Ryan Ofsky) 06882f8 multiprocess: Add serialization code for vector<char> (Russell Yanofsky) 095286f multiprocess: Add serialization code for CTransaction (Russell Yanofsky) 69dfeb1 multiprocess: update common-types.h to use C++20 concepts (Ryan Ofsky) 206c6e7 build: Make bitcoin_ipc_test depend on bitcoin_ipc (Ryan Ofsky) 070e6a3 depends: Update libmultiprocess library for cmake headers target (Ryan Ofsky) Pull request description: Add Cap'n Proto wrapper for the Mining interface introduced in #30200, and its associated types. This PR combined with #30509 will allow a separate mining process, like the one being implemented in Sjors#48, to connect to the node over IPC, and create, manage, and submit block templates. (#30437 shows another simpler demo of a process using the Mining interface.) --- This PR is part of the [process separation project](#28722). ACKs for top commit: achow101: ACK 1a33281 TheCharlatan: ACK 1a33281 itornaza: ACK 1a33281 Tree-SHA512: 0791078dd6885dbd81e3d14c75fffff3da8d1277873af379ea6f9633e910c11485bb324e4cde3d936d50d343b16a10b0e8fc1e0fc6d7bdca7f522211da50c01e
PastaPastaPasta
pushed a commit
to PastaPastaPasta/dash
that referenced
this issue
Sep 27, 2024
…eprecation warnings 21bfee0 depends: bump libmultiprocess to fix capnproto deprecation warnings (Ryan Ofsky) Pull request description: This incorporates PR bitcoin-core/libmultiprocess#88 and reverts the NO_WERROR CI workaround added in bitcoin#28735 Upstream diff: bitcoin-core/libmultiprocess@61d5a0e...414542f --- This PR is part of the [process separation project](bitcoin#28722). ACKs for top commit: maflcko: lgtm ACK 21bfee0 hebasto: ACK 21bfee0, I have reviewed the code and it looks OK. I've also skimmed through the related changes in the https://github.com/chaincodelabs/libmultiprocess repository. Tree-SHA512: b5addb0deed694eeec62a0ae08b4715a811110201f39f3e6cadee8fc4e6231b0e66c844a98512072a1445bac122ab561dc1711e27fb4d7ac5c08ac46780a4acf
PastaPastaPasta
pushed a commit
to PastaPastaPasta/dash
that referenced
this issue
Sep 27, 2024
…eprecation warnings 21bfee0 depends: bump libmultiprocess to fix capnproto deprecation warnings (Ryan Ofsky) Pull request description: This incorporates PR bitcoin-core/libmultiprocess#88 and reverts the NO_WERROR CI workaround added in bitcoin#28735 Upstream diff: bitcoin-core/libmultiprocess@61d5a0e...414542f --- This PR is part of the [process separation project](bitcoin#28722). ACKs for top commit: maflcko: lgtm ACK 21bfee0 hebasto: ACK 21bfee0, I have reviewed the code and it looks OK. I've also skimmed through the related changes in the https://github.com/chaincodelabs/libmultiprocess repository. Tree-SHA512: b5addb0deed694eeec62a0ae08b4715a811110201f39f3e6cadee8fc4e6231b0e66c844a98512072a1445bac122ab561dc1711e27fb4d7ac5c08ac46780a4acf
PastaPastaPasta
pushed a commit
to PastaPastaPasta/dash
that referenced
this issue
Sep 27, 2024
…eprecation warnings 21bfee0 depends: bump libmultiprocess to fix capnproto deprecation warnings (Ryan Ofsky) Pull request description: This incorporates PR bitcoin-core/libmultiprocess#88 and reverts the NO_WERROR CI workaround added in bitcoin#28735 Upstream diff: bitcoin-core/libmultiprocess@61d5a0e...414542f --- This PR is part of the [process separation project](bitcoin#28722). ACKs for top commit: maflcko: lgtm ACK 21bfee0 hebasto: ACK 21bfee0, I have reviewed the code and it looks OK. I've also skimmed through the related changes in the https://github.com/chaincodelabs/libmultiprocess repository. Tree-SHA512: b5addb0deed694eeec62a0ae08b4715a811110201f39f3e6cadee8fc4e6231b0e66c844a98512072a1445bac122ab561dc1711e27fb4d7ac5c08ac46780a4acf
PastaPastaPasta
pushed a commit
to PastaPastaPasta/dash
that referenced
this issue
Sep 28, 2024
…eprecation warnings 21bfee0 depends: bump libmultiprocess to fix capnproto deprecation warnings (Ryan Ofsky) Pull request description: This incorporates PR bitcoin-core/libmultiprocess#88 and reverts the NO_WERROR CI workaround added in bitcoin#28735 Upstream diff: bitcoin-core/libmultiprocess@61d5a0e...414542f --- This PR is part of the [process separation project](bitcoin#28722). ACKs for top commit: maflcko: lgtm ACK 21bfee0 hebasto: ACK 21bfee0, I have reviewed the code and it looks OK. I've also skimmed through the related changes in the https://github.com/chaincodelabs/libmultiprocess repository. Tree-SHA512: b5addb0deed694eeec62a0ae08b4715a811110201f39f3e6cadee8fc4e6231b0e66c844a98512072a1445bac122ab561dc1711e27fb4d7ac5c08ac46780a4acf
PastaPastaPasta
pushed a commit
to PastaPastaPasta/dash
that referenced
this issue
Oct 1, 2024
…eprecation warnings 21bfee0 depends: bump libmultiprocess to fix capnproto deprecation warnings (Ryan Ofsky) Pull request description: This incorporates PR bitcoin-core/libmultiprocess#88 and reverts the NO_WERROR CI workaround added in bitcoin#28735 Upstream diff: bitcoin-core/libmultiprocess@61d5a0e...414542f --- This PR is part of the [process separation project](bitcoin#28722). ACKs for top commit: maflcko: lgtm ACK 21bfee0 hebasto: ACK 21bfee0, I have reviewed the code and it looks OK. I've also skimmed through the related changes in the https://github.com/chaincodelabs/libmultiprocess repository. Tree-SHA512: b5addb0deed694eeec62a0ae08b4715a811110201f39f3e6cadee8fc4e6231b0e66c844a98512072a1445bac122ab561dc1711e27fb4d7ac5c08ac46780a4acf
This was referenced Jan 30, 2025
This was referenced Feb 7, 2025
fanquake
added a commit
that referenced
this issue
Feb 25, 2025
01f7715 depends: Update libmultiprocess library to fix CI failure (Ryan Ofsky) Pull request description: Bump libmultiprocess library to include bugfix bitcoin-core/libmultiprocess#159 which should fix intermittent CI failures reported in #31921 This change is bumping the libmultiprocess version instead of cherry picking the bugfix. It could cherry-pick the bugfix instead, but there are reasons to prefer bumping the version: - Bugfix might interact with earlier PRs, and the latest version is better tested with testing done in many CI configurations in [#30975](#30975) and [#31802](#31802) - Even though we are in feature freeze for a release, the MULTIPROCESS=1 option is currently not enabled for release, so this PR only affect CI builds and local builds, not the release build. This update brings in the following changes: bitcoin-core/libmultiprocess#140 build: don't clobber user/superproject c++ version bitcoin-core/libmultiprocess#142 build: add option for external mpgen binary bitcoin-core/libmultiprocess#143 cleanup: initialize vars in the EventLoop constructor in the correct order bitcoin-core/libmultiprocess#146 cmake: Suppress compiler warnings from capnproto headers bitcoin-core/libmultiprocess#147 cmake: EXTERNAL_MPGEN cleanups bitcoin-core/libmultiprocess#148 util: fix -Wpessimizing-move warning bitcoin-core/libmultiprocess#145 CTest: Module must be included at the top level bitcoin-core/libmultiprocess#149 Avoid `-Wundef` compiler warnings bitcoin-core/libmultiprocess#152 refactor: Fix compiler and clang-tidy warnings bitcoin-core/libmultiprocess#155 scripted-diff: s/Libmultiprocess_EXTERNAL_MPGEN/MPGEN_EXECUTABLE/g bitcoin-core/libmultiprocess#156 refactor: Remove locale-dependent function calls bitcoin-core/libmultiprocess#157 refactor: Avoid using std::format bitcoin-core/libmultiprocess#159 bugfix: Do not lock EventLoop::mutex after EventLoop is done bitcoin-core/libmultiprocess#161 cmake: Avoid including CTest if not top level project bitcoin-core/libmultiprocess#164 Bump minimum required cmake to 3.12 --- This PR is part of the [process separation project](#28722). ACKs for top commit: fanquake: ACK 01f7715 Tree-SHA512: a6a795e4d4e13e9d35c9346f3c83d5da817f1452bdc4a9412aeadc4c652ad6e5047f4c77756570594a70ec9095cc786772a0469e306dc19bb5a508fd515c37f4
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
This issue will be updated to reflect the current state of bitcoin core multiprocess support.
PRs for review related to building & releasing:
PRs for review related to mining:
PRs for review related wallet & gui:
Issues:
PRs / issues closed:
[qt] Don't call method on null WalletModel object #10242[qt] Move some WalletModel functions into CWallet #10295Make feebumper class stateless #10600Refactor: separate gui from wallet and node #10244Add AssertLockHeld assertions in CWallet::ListCoins #10605Avoid using numeric_limits for sequence numbers and lock times #14636Refactor: Start to separate wallet from node #14437Remove uses of chainActive and mapBlockIndex in wallet code #14711Suggested wallet code cleanups from #14711 #15342Move ParseConfirmTarget from rpc/mining to rpc/util #15373Remove wallet -> node global function calls #15288Suggested interfaces::Chain cleanups from #15288 #15531Refactor: separate wallet from node #10973Replace Connman and BanMan globals with NodeContext local #16839gui: Set CConnman byte counters earlier to avoid uninitialized reads #17906interfaces: Describe and follow some code conventions #18278Multiprocess build support #16367refactor: Add ChainClient setMockTime, getWallets methods #17999depends: Add --sysroot option to mac os native compile flags #18743gui: Avoid wallet tryGetBalances calls in WalletModel::pollBalanceChanged #18587Multiprocess build support #18677refactor: Move node and wallet code out of src/interfaces #20494depends: Set CMAKE_INSTALL_RPATH for native packages #20046Remove pointer cast in CRPCTable::dumpArgMap #21035MOVEONLY: Move common init code to init/common #21732multiprocess: Add basic spawn and IPC support #19160refactor: Make SetupServerArgs callable without NodeContext #22216refactor: Rearrange fillPSBT arguments #22214multiprocess: Add ipc::Context and ipc::capnp::Context structs #22218Unregister wallet notifications before unloading wallets bitcoin-core/gui#360refactor: Add FoundBlock.found member #22215refactor: Avoid wallet code writing node settings file #22217refactor: remove ::vpwallets and related global variables #19101MOVEONLY: Expose BanMapToJson / BanMapFromJson #22848multiprocess: Start using init makeNode, makeChain, etc methods #22219multiprocess: Make interfaces::Chain::isTaprootActive non-const #23003multiprocess: Add new bitcoin-gui, bitcoin-qt, bitcoin-wallet init implementations #23006multiprocess: add interfaces::ExternalSigner class #23004multiprocess: Delay wallet client construction #23005multiprocess: Add interfaces::Node::broadCastTransaction method #23499multiprocess compatibility updates #28721depends: bump libmultiprocess to fix capnproto deprecation warnings #28907doc: Add multiprocess design doc #28978multiprocess: Add basic type conversion hooks #28921serialization: Support for multiple parameters #28929multiprocess: Add -ipcbind option to bitcoin-node #30509multiprocess: Add IPC wrapper for Mining interface #30510Prune mining interface #31196refactor: mining interface 30955 followups #31197depends: Fix build withMULTIPROCESS=1in Guix environment #30940Mining interface: getCoinbaseMerklePath() and submitSolution() #30955Update libmultiprocess library #31105Add waitFeesChanged() to Mining interface #31003Add destroy to BlockTemplate schema #31288refactor: Fix "modernize-use-starts-ends-with" clang-tidy warning #31480Drop script_pub_key arg from createNewBlock #31318Prune mining interface #31196depends: Update libmultiprocess library before converting to subtree #31740ci: intermittent issue in rpc_misc.py node0 stderr terminate called after throwing an instance of 'kj::ExceptionImpl' [15:12:14.943] what(): mp/proxy.cpp:242: disconnected: write(m_post_fd, &buffer, 1): Broken pipe #31151multiprocess: build failure on Alpine with depends &DEBUG=1#31455Pass custom DEP_OPTS and CONFIG_FLAGS to guix-build #31763multiprocess: Lock CapnpProtocol::m_loop with mutex #31815depends: Update libmultiprocess library to fix CI failures #31945Add waitNext() to BlockTemplate interface #31283Add checkblock RPC and checkBlock() to Mining interface #31564Links:
The text was updated successfully, but these errors were encountered: