Web Worker Status #148
Comments
|
I found out a way to "emulate" Web Workers using the setInterval function, but I'm still working on the design. |
|
If you are emulating them you don't need setInterval, since you already are
using a single events queue. To properly inplement them we need to use
threads on the C side, or better going fully down the road and implement
preemptive threads, that can be done using interrupts. Next step would be
processes with memory isolation, but they are out of runtime.js design.
El 23/6/2017 2:59 AM, "SpaceboyRoss01" <[email protected]> escribió:
… I found out a way to "emulate" Web Workers using the setInterval function,
but I'm still working on the design.
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
<#148 (comment)>,
or mute the thread
<https://github.com/notifications/unsubscribe-auth/AAgfvv7T5rd9Ptjy6LedDe00cqbFAjk6ks5sGw37gaJpZM4OCaiD>
.
|
|
I'm emulating Web Workers and Multitasking using the setIntervals |
|
Do you have the code somewhere? On the other hand, I'm not sure how much
important this would be, since Node.js already have support for having
several isolates using a single events queue, so we could use something
similar... The biggest problem would be if someone makes an infinite loop
that would block the full system, but having several isolates and using
interrupts to move to another one would do the trick...
El 23/6/2017 7:55, "SpaceboyRoss01" <[email protected]> escribió:
… I'm emulating Web Workers and Multitasking using the setIntervals
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
<#148 (comment)>,
or mute the thread
<https://github.com/notifications/unsubscribe-auth/AAgfvrIAwgIcj-ue8mZ9cjFBcew8Pp6yks5sG1NEgaJpZM4OCaiD>
.
|
|
This is the experimental code: function main() {
console.log("Hello, World");
}
var running = false;
var pid = setInterval(() => {
if(running) return;
running = true;
main();
while(running);
clearInterval(pid);
},1000) |
|
Can you explain what are you trying to achieve with that? It goes directly to a blocking infinite loop and do nothing more... |
|
It is multitasking. To kill the task, set |
|
As I told you, you got an infinite loop, so on a single thread you'll get blocked. Do you have the full source code of the proof of concept somewhere? |
|
No, I only have that piece of code |
|
It's going to be pretty hard and time consuming to implement, because requires working with multiple CPUs and/or allowing time sharing multitasking. This will dramatically increase complexity of the codebase. I think being single-threaded is fine for the OS that runs in the cloud on virtual CPUs, i.e you can run a separate runtime.js instance per physical CPU and get "multitasking" that way. |
That's why I'm talking about of concepts like
Single threaded is not ok, but it's fine. What we would need is to have several concurrent tasks, and I think "emulate" threads and processes (I think this is how greenlets works) on a single CPU/thread is good enough for now. Far for ideal, but cooperative multitasking would be a good first aproach before moving to preemptive threads/processes so far nobody does a blocking loop, and we could move later in the future to multi-CPU multitasking. Yes, I'm still willing to port NodeOS to runtime.js and I needs processes some way or another :-P |
|
Fair enough, also it makes sense for WASM, which needs multithreading. Definitely doable but requires some refactoring and code cleanup. I agree, cooperative multitasking is a good start. As WASM becomes more and more important, I'm thinking we can rename this project to |
Not yet, but it's planned...
Please no, |
|
Well, V8 is wasm VM too, the engine is the same :) |
|
the web assembly piece is really interesting, it provides a library OS that is language agnostic, maybe one of the few unikernels that can claim that! @iefserge are you talking about implementing your own language? and just for my own clarification, is it easier to all provide threading for wasm vs js, or is the complexity the same? edit: i should add that my need for Web Workers is not super important at the moment, but it would be a great "nice to have" down the road. |
|
If we change the name, it should be [something].js, of course :-) |
|
@aafrey yeah, though language has nothing to do with runtime itself, it would just compile to wasm and run alongside with js. Just playing with things, not sure if I'll get to the point of working language :) @piranna I like that name, but |
|
@iefserge you mentioned in some older issues (#75) that background tasks can be run effectively, provided they are non blocking. What would a good implementation look like for running background tasks on a single thread? can there be true background tasks in a single threaded environment or just asynchronous tasks ? |
|
I just want to say that having WASM integration will definitely help runtime.js (mainly because pointers). Maybe we could take something like TurboScript that has a similar syntax to JavaScript and add some runtime.js integration (direct memory access and custom malloc, free, etc.). |
|
@facekapow |
|
What would a good implementation look like for running background tasks on
a single thread? can there be true background tasks in a single threaded
environment or just asynchronous tasks ?
Being the system single thread and with an unified memory, they can just
call to setInterval and setTimeout to push events to the events queue and
exec themselves as usual in Javascript...
|
|
I've recently been reading about wasm because I don't fully understant what it is. Is wasm just low-level assembly access through JavaScript? |
|
@aafrey Well no, I just mean that it could help with managing pointers, since in an operating system you have to deal with memory directly. The fact that JavaScript doesn't have pointers makes this much more difficult. Btw, that was just a clarification of what I was saying; to answer your question: yeah, I'm pretty sure you can both copy values and pass pointers between WASM and JS. |
|
I found out how to use a custom memory heap in Runtime.js using the |
|
That can help with specific things. |
|
@piranna that's what I was thinking, so thanks for confirming. |
|
I found how to use a custom memory manager in Runtime.js by reading and writing at a specific section of memory to allocate and free memory for specific tasks, like multitasking. |
|
@SpaceboyRoss01 WASM is cross platform assembly via a JavaScript engine. It has no native system access, it was created to achieve near native speeds for web code. But, since there is a way to interface with WASM through JavaScript, we can provide access to runtime's native functions. |
|
@facekapow That is what I was asking about what WASM is. |
It's a bytecode specification, similar to the one of the Java VM or Python. In fact, I have been using its opcodes to define the instructions of a CPU of my own :-)
Pointers are not needed to manage memory data, you can use structs and MemoryViews. In fact, that's the way you do it with wasm, and it's pretty easy and secure since you work with arrays instead of pointers operations and you can access only the memory area of the MemoryView. |
|
@piranna Well yes, in plain JavaScript you can use Uint arrays with ArrayBuffers for specific memory areas backing them to deal with memory. But what I mean is, languages like C and C++, Rust, TurboScript, and others that compile to WebAssembly and use pointers can help make memory easier to work with. |
|
If you read the WebAssembly specification, you'll see that there're no
pointers arithmetic at all at low level, and memory regions need to be
defined in advance and are accesed and checked the same way UIntArrays are
done ;-) In fact, is dictated that compilers must take this in account when
compiling to WebAssembly as a target platform :-)
El 23/06/2017 18:40, "facekapow" <[email protected]> escribió:
… @piranna <https://github.com/piranna> Well yes, in plain JavaScript you
can use Uint arrays with ArrayBuffers for specific memory areas backing
them to deal with memory. But what I mean is, languages like C and C++,
Rust, TurboScript, and others that compile to WebAssembly and use pointers
can help make memory *easier* to work with.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<#148 (comment)>,
or mute the thread
<https://github.com/notifications/unsubscribe-auth/AAgfvnXmgIdell8mapMbhXdl6pAviJydks5sG-qAgaJpZM4OCaiD>
.
|
|
after doing some additional reading, it looks like threads are coming to wasm eventually (WebAssembly/design#1073) would there be any benefit in beginning that process now with runtime? I know a number of libraries handle web workers with a polyfill when they are not supported by a browser. Maybe we could do something similar. build out a worker class, |
|
@piranha Yeah, I just found that out for myself reading through some source code for TurboScript. Damn. It made my hopes of writing device drivers (e.g. USB host controller support) much harder (I know runtime is a server OS, but you can't blame me for wanting to have traditionally native OS things written in web technologies 😁). |
|
Why do you think makes it more difficult? Low level needs to be done with
C/C++, but over it everything can be done in high level languages... If you
talk about ports, just expose in() and out() functions. Take a look on the
Python portio module, with that I did several years ago an I2C driver to
override some Linux security measurements...
El 24/06/2017 00:10, "facekapow" <[email protected]> escribió:
@piranha <https://github.com/piranha> Yeah, I just found that out for
myself reading through some source code for TurboScript. Damn. It made my
hopes of writing device drivers (e.g. USB host controller support) much
harder (I know runtime is a server OS, but you can't blame me for wanting
to have traditionally native OS things written in web technologies).
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<#148 (comment)>,
or mute the thread
<https://github.com/notifications/unsubscribe-auth/AAgfvvCVvLsy3n_g_UHDxIZjuY-jFtUDks5sHDfxgaJpZM4OCaiD>
.
|
|
The problem is that EHCI does everything in memory, no ports. So the convenience of being able to overlay a struct onto an area of memory simplifies it so much. Oh well. |
|
@facekapow how is wasm enabled? in the |
|
I don't find any problem about exposing an UIntArray at the exact memory
address EHCI needs...
El 24/06/2017 00:39, "facekapow" <[email protected]> escribió:
… The problem is that EHCI does everything in memory, no ports. So the
convenience of being able to overlay a struct onto an area of memory
simplifies it so much. Oh well.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<#148 (comment)>,
or mute the thread
<https://github.com/notifications/unsubscribe-auth/AAgfvv-Bzkofpa85SevFmKa2y1SOrK1qks5sHD6MgaJpZM4OCaiD>
.
|
|
Take in account, Memory objects in wasm are expected to not be executable...
El 24/06/2017 02:37, "facekapow" <[email protected]> escribió:
… @aafrey <https://github.com/aafrey> No, in *runtime.json* put:
{
"v8": {
"flags": "--expose_wasm"
}
}
@piranna <https://github.com/piranna> Yeah, well I've been having some
trouble with code not executing if I try to modify memory via a Uint8Array.
Don't worry about it, it's my dumb hopes, I'll fix it somehow.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<#148 (comment)>,
or mute the thread
<https://github.com/notifications/unsubscribe-auth/AAgfvupQAVAQ8GHwY_LkAYCnVdPvtbizks5sHFo7gaJpZM4OCaiD>
.
|
|
closing since conversation died off. feel free to re-open if this comes up again :) |
There was some initial discussion awhile ago regarding threading and using web workers to accomplish that task. What's the current status? I have a use case where they would be helpful. There are a number of NodeJS implementations of web workers. Could some of those be adapted? Some clues as to where to start would be appreciated. Thanks!
The text was updated successfully, but these errors were encountered: