restrictions on chrome.tabCapture.captureOffscreenTab()

[Connum]

NWJS Version : 0.35.2
Operating System : Win64 (but supposedly applies to all systems)

Expected behavior

    function onTabStream(stream) {
        console.log(stream, chrome.runtime.lastError);
    }
    chrome.tabCapture.captureOffscreenTab('render.html', { audio: true, video: true }, onTabStream );

Should result in stream being a MediaStream object, regardless of whether the start_url (first argument of captureOffscreenTab) is an http(s)/data protocol URL or a relative URL in the nwjs app's scope (or using a chrome-extension protocol URL).

Actual behavior

stream is undefined and checking the message in chrome.runtime.lastError, it states "Invalid/Missing/Malformatted starting URL for off-screen tab." This is due to restrictions in Chromium's /chrome/browser/extensions/api/tab_capture/tab_capture_api.cc, which should not be applied to nwjs context.

How to reproduce

Try to capture a page ("render.html" in these code examples) in the nwjs app's context as a hidden tab - see code above.

Temporary workaround(s)

1. I managed to circumvent this issue by serving the file(s) I need in the hidden tab as http via the connect and serve-static packages (let's say on port 3000) and then loading 'http://localhost:3000/render.html'. But of course this leads to not being able to use node/nwjs API calls directly in the page's scripts, no direct communication between the tab and the rest of the app (or having to workaround that by using ajax to access an intermediary file) and other restrictions (mixed-content problems would have to be handled by serving as https and potentially handling certificate problems... etc,).

2. Another makeshift solutionis to open the page to render in a separate window and hiding this with a mixture of options and a minimize() call:

  nw.Window.open('render.html', {
      show_in_taskbar: false,
      resizable: false,
      focus: false,
      frame: false,
      width: 0,
      height: 0
  }, function(w) {
      win.width = 1920;
      win.height = 1080;
      w.minimize();
  });

This results in the popup flickering shortly when starting up the application, but interaction between the page and the rest of the app is possible without restrictions. However, there are small black bars on top and bottom of the rendered output that I can't get rid of so far, and (I fixed the black bars by setting the width and hight specificly before minimizing the window in the callback.) I think the performance is a bit worse than the captureOffscreenTab() solution.

[rogerwang]

Will look to fix this. Regarding the workaround, you can make a remote page trusted and node.js enabled with the node-remote field in manifest.

[Connum]

Will look to fix this. Regarding the workaround, you can make a remote page trusted and node.js enabled with the node-remote field in manifest.

Thanks! I will look into your proposal - meanwhile, I found a way of fixing the black bars with the "hidden" window approach and will update the workaround in the original post.

[rogerwang]

This is fixed in git and will be available in the next nightly build.

[Connum]

Thanks for the blazing fast fix! Looking forward to testing it!

[rogerwang]

https://dl.nwjs.io/live-build/nw35/12-20-2018/8e98fc542/v0.35.3/

[Connum]

One thing I noticed is that you have to explicitly get and use the chrome-extension:// URL - would it be possible to implement that passing a relative URL will automatically be handled?

[rogerwang]

Will look into it.

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[Connum]

Using the latest 0.51.0 (and at least with 0.50.3 also), captureOffscreenTab() seems to have stopped working completely:

chrome.tabCapture.captureOffscreenTab(`chrome-extension://${chrome.runtime.id}/foobar.html`, {
                audio: true,
                video: true
            }, (stream) => console.log(chrome.runtime.lastError.message, stream));

results in Error starting tab capture null

I tried remote addresses, local URLs as well...

[mcpiroman]

Now (0.57.0) it looks like there is no captureOffscreenTab function anymore. Do I miss sth?

[rogerwang]

@mcpiroman upstream removed this feature in recent releases: https://chromium-review.googlesource.com/c/chromium/src/+/3059531