Embed metacall with node loader inside an environment with v8 already in place #493
Comments
|
This issue can be solved by patching the NodeJS build system, which is something that we are already doing so I have no fear about doing this again for this special use case. Another option would be to figure out this: #231 With this solution it may be possible to relink v8 on the fly, but I feel it can be more problematic because C++ mangling, but it is still doable. |
|
Now this issue has solution? |
|
@kingleonide almost, I am working on it. Can you explain your use case? Or you are from @twelvee 's team? The work is being done here: #533 The same feature will allow to use V8 from existing executable, it is the exactly same use case as the described in the PR. The only thing we need here is to tweak a bit this file: https://github.com/metacall/core/blob/5b592ac0e9a8e498e3e706623d0a788276f566e0/cmake/FindNodeJS.cmake Specifically, we need to implement an option to compile NodeJS as shared library, passing V8 includes but without the V8 library, this will compile the library without V8 symbols resolved. If we compile node library with the same compiler as the one that compiled the executable (in the case of this issue, I already have a script for patching the NodeJS build system based on GYP: https://github.com/metacall/core/blob/develop/cmake/NodeJSGYPPatch.py The only thing we should patch is the following lines: That should do the job. If you help me with this we can do it faster. Otherwise I will try to do it once I finish the PR. |
|
Half of the job is done, it is working already for linux amd64, and it works well. It also works for MacOS.
The implementation avoids all the problems we had before, it can detect if the runtime is already loaded, then it searches for the existing functions and replaces all with the existing ones, hooking safely with the import tables, instead as for now which is through dissasembler and patching on the fly by inserting jumps. @twelvee @kingleonide as you can see it is working here: Here is the test: core/source/ports/node_port/CMakeLists.txt Line 227 in 064ac36 This basically executes a js code that is using metacall to call other languages, on startup it initializes node_loader pointing to the runtime inside the node executable. The next implementation I am preparing will be even more robust and you will avoid completely loading any library so avoiding conflicts by having two runtimes loaded into memory, even if you do not initialize both of them. So errors like this (which are produced by C++ constructors and similar) will be gone: Also now we support from node version 12 to 22 and following. |
🚀 Feature
The environment I'm trying to embed metacall into and use it with nodejs loader already has v8 libraries. The process itself is a Counter-Strike 2 server, which is closed source and unfortunately I can't just "turn off" the existing v8 version.
NodeJs (libnode) uses a slightly modified version of v8 internally and it is impossible to build nodejs with the --shared-v8 flag (nodejs/help#3302).
Describe the solution you'd like
It would be cool to be able to use metacall in my environment, but that requires building libnode and linking it to an existing version of v8.
Describe alternatives you've considered
Turning off the existing v8 leads to a server crash, apparently it is tied to many features of the source 2 engine (counter-strike engine).
Additional context
The problem is not metacall itself, rather libnode and a conflict between two different versions of v8 libraries. However, it may be possible to use metacall to build the necessary version of libnode for my environment.
When trying to load files (metacall_load_from_file) I get segfault when libnode is mounted:
I have tried to build nodejs itself and metacall differently, however I always run into this exact problem.
lddoutput of libnode:Here's all injected libraries (dl_walk):
Open
Here's everything related to already installed v8 libraries, libnode should be linked to this libraries. They are located right next to executable in the same folder.
v8-related.zip
NodeJS v20.11.1 uses v8 version: 11.3.244.8-node.17
NodeJS v19.X.X uses v8 version 10.7.X.X-node.X
NodeJS v18.19.1 uses v8 version: 10.2.154.26-node.28
Valve v8 version: 10.6.194.17
OS: Ubuntu 22
The text was updated successfully, but these errors were encountered: