Response object for HTTP Upgrade/CONNECT

[stephank]

I started a thread on nodejs-dev about this, but it looks like my posts are still in the moderator queue. Will link it here once it gets through.

Briefly:

• The first commit is a test showing that a pipelined Upgrade request to a Node server can cause an out-of-order response. This is because the upgrade handler gets immediate access to the socket, while other requests may still be processing async.
• The second commit fixes this by providing a response object even for Upgrade, that is also queued properly along with other responses.
• In addition to fixing the test, I find it makes sense to have a response here. Per the spec, CONNECT and Upgrade must be followed by a proper response, before handover.

Regarding make test, I have one failing test both before and after this patch set, namely simple/test-http-full-response. I consider this a platform irk. make test-debug plain crashes the test suite on simple/test-repl (also before and after), but a fair amount of HTTP tests do pass.

[stephank]

I have a branch of einaros/ws on top of this, with working tests and examples:

https://github.com/stephank/ws/tree/upgrade-response

[stephank]

More branches of websocket projects:

• faye-websocket: https://github.com/stephank/faye-websocket-node/tree/upgrade-response
• sockjs: https://github.com/stephank/sockjs-node/tree/upgrade-response

These are all examples of projects reimplementing basic ServerResponse functionality.

---

I noticed a recurring pattern, along the lines of:

socket.on('data', dataListener);
dataListener(upgradeHead);

Perhaps we can simply emit upgradeHead as the first data event on the socket, after the switchProtocols callback finishes? (Or do we have to do this in a nextTick? I'm not sure if that guarantees ordering of events when new data arrives quickly.)

[stephank]

I went ahead and rebased on current master, and implemented the earlier thought of simply emitting upgradeHead as a data event.

The branches of ws and faye-websocket have also been updated. (Sockjs didn't need any further updates.)

[piscisaureus]

@isaacs Could you flush the moderation queue for nodejs-dev? (or give me / @bnoordhuis admin rights)

[koichik]

I think that @stephank's the new API is really nice. However, it breaks backward-compatibility. Because 'upgrade' event is widely used (by Socket.IO, especially), I do not want to change it.

Instead, I think that Node should close the connection when pipelined Upgrade/CONNECT request was detected. Because:

• If Upgrade/CONNECT request is idempotent, client can resend the request.
• If Upgrade/CONNECT request is not idempotent, client should not pipeline the request.

[stephank]

Thanks for pitching in! That will, of course, solve the problem as well.

You're absolutely right when it comes to the API. I was still hoping we could move mountains by showing the API can be better, and demonstrating that I was willing to put work in projects currently using upgrade. The only reason I don't have a branch of Socket.IO is because I couldn't get its test suite running on Node 0.7.7.

Is there a minimum set of projects using upgrade that we want to ensure compatibility with? Would it help if we got their maintainers on board with this API change?

[koichik]

@stephank - What WebSocket's client does pipeline the Upgrade request?

[stephank]

There isn't any I'm aware of, that test is theoretical. Though it does actually work and triggers the bad HTTP server behavior on 0.6.14 and 0.7.7.

I'm in this for the response object, which can be very useful in combination with middleware. What led me to doing this was trying to add upgrade support to Connect middleware. See the related pull request for Connect, which implements it using the current Node API.

Briefly, that pull works, but a lot of middleware really wants to manipulate the response, e.g. add cookies, other headers, sometimes even abort the request (ie. basicAuth wants to reply with 401).

[koichik]

@stephank - Sorry, I'm -1 to the PR. Because there is no problem in the real world, and the API is marked Stable (i.e. Backwards-compatibility is guaranteed.).
Closing.

[stephank]

For reference, linking mailing list thread:
https://groups.google.com/d/topic/nodejs-dev/MSPF_3yYCSY/discussion

[koichik]

By the discussion in the ML, we will revisit in v0.9. Reopening.

[creationix]

I just found this issue. I've been wanting this feature for a long time! What can I do to help? Basically my goal is to be able to handle websockets 100% from a handler function without needing access to the server object. This makes composing layers (like connect middlewares) possible when using websockets.

[stephank]

• This needs to be rebased on master. Renewed interest helps; I'll see about doing that this week. :)
• On the mailing list, @theturtle32 raised the point of sending both the HTTP response and WebSocket handshake in a single frame. This pull request basically splits the write()s, which could be a problem. It looks like (in the tests) a single frame is sent because Nagle's is enabled by default, but it's likely that users will want to disable Nagle's as soon as they get the socket. We need figure out if this is a problem and, if so, how to fix it.
• Other than that, I'm just waiting for feedback. :)

[stephank]

Rebased. Test results still good.

[stephank]

I tried to create a test case for the issue with Nagle's. I basically disable Nagle's directly after switchProtocols, then write the response body to the HTTP Upgrade. A plain TCP client then counts data events to check for segmentation.

https://gist.github.com/3426025

The test currently passes. (ie. I only get one data event.)

Can someone with a bit more knowledge of net / libuv internals verify if this is a proper test case? I'm not sure if simply counting data events is enough verification.

[adammw]

Any progress on this proposal getting into one of 0.9.x unstable releases?

The lack of a response object is making it hard for frameworks to implement upgrade request routing easily, meaning that upgrades are typically handed manually with a lot more work required.

As to the issue of backwards-compatibility that everyone seems to be hanging on, even with this proposed API break, wouldn't the only change required be something like this or am I missing something?

server.on('upgrade', function(req, socket, head) {
  if(!(socket instanceof net.Socket))
    socket = req.connection;

  //... legacy code that uses socket directly ...
})

[bnoordhuis]

PR no longer applies. Is this still an issue with master and if so, why?

[stephank]

@adammw: Sure, but API compatibility means things have to stay working without code changes to the libraries/applications using the API.

@bnoordhuis: It is still an issue. This is a proposal to change the upgrade/connect event API to no longer receive plain sockets, but a Response object. The PR is an implementation.

I'll take a look at another rebase on current master.

[stephank]

Here we go, rebased against c6e958d. Some things don't sit quite right with me, though:

• Both master and this branch pass tests on a release build on my Mac. Tests on a debug build fail, however, even on plain master, so I have no comparison there. (test-debug-brk-file hangs, tried running some others manually and got assertion failures.)
• http.js still uses old-style streams and pretty much forces this on servers using upgrade requests. (I'm guessing this is well known.) Right now, I'm still pausing/resuming the socket and injecting the head buffer back in using socket._readableState.onread() after the switch callback had its chance to install handlers. I think eventually, the onread() should be done before the switch callback, so that the callback gets an immediately readable stream?

[stephank]

Rebased on current master (8e311d2). Now using stream.push(). I succeeded in running make test-debug on my Mac this time, with the exact same results on master and this branch. (Several seemingly unrelated failures.)

Remaining points:

• Waiting on new-style streams in http.js. (issue http parser should not switch socket into old-mode #4488)
• Would still appreciate comments on separate write()s, as per this earlier comment.

[stephank]

Right, with #4488 closed, I rebased on master (926c90b) again. The socket.pause() and socket.resume() calls are gone. I now also socket.push() the upgradeHead before the switchProtocols() callback. Test results are good.

Remaining:

• Would still appreciate comments on separate write()s, as per this earlier comment.

[creationix]

Should @stephank keep working on this? I'd hate to see him do all this work if there is no intention to accept it. This is a feature I would really like, but it is a change to the HTTP API which is stable now.

@stephank, is your change backwards incompatible with any existing node code? And if so, how significant is the change?

My approach of late has been to replace the http module entirely in userspace. (see web in npm).

[stephank]

@creationix Yes, it's an API change to the upgrade and connect event callbacks.

It's the difference between this server we have right now:

server.on('upgrade', function(req, sock, head) {
    sock.write('HTTP/1.1 101 Switching Protocol\r\n' +
               'Connection: Upgrade\r\n' +
               'Upgrade: Echo\r\n\r\n');

    sock.push(head);
    sock.on('readable', function() {
        var chunk = sock.read();
        sock.write(chunk);
    });
});

And this server on my branch:

server.on('upgrade', function(req, res) {
    res.writeHead(101, {
        'Connection': 'Upgrade',
        'Upgrade': 'Echo'
    });

    res.switchProtocols(function(sock) {
        sock.on('readable', function() {
            var chunk = sock.read();
            sock.write(chunk);
        });
    });
});

A server that handles both is slightly finicky, though:

server.on('upgrade', function(req, resOrSock, head) {
    var headers = {
        'Connection': 'Upgrade',
        'Upgrade': 'Echo'
    };

    if (resOrSock.switchProtocols) {
        resOrSock.writeHead(101, headers);
        resOrSock.switchProtocols(setupSock);
    }
    else {
        resOrSock.write('HTTP/1.1 101 Switching Protocol\r\n' +
                        Object.keys(headers).map(function(name) {
                            return name + ': ' + headers[name] + '\r\n';
                        }).join('') +
                        '\r\n');
        resOrSock.push(head);
        setupSock(resOrSock);
    }

    function setupSock(sock) {
        sock.on('readable', function() {
            var chunk = sock.read();
            sock.write(chunk);
        });
    }
});

[stephank]

There's now a package upgrade-ex in NPM that implements this stand-alone on top of Node.js 0.9.6.

@adammw suggested I look into backwards compatibility, but this is kind of the reverse. I can transform an 'old style' upgrade into a 'new-style' this way, but I've not yet figured out how to make 'old style' work on top of 'new style'.

[isaacs]

@stephank So, it seems like the main blocker is that now you do on('upgrade', function(req, socket, head) {}) and with the patch, you'd do on('upgrade', function(req, res) {}). Why not making the new signature on('upgrade', function(req, socket, head, res) {})? It's a bit ugly, but would keep backwards compatibility. Changing event signatures is very destabilizing.

[adammw]

Changing event signatures is very destabilizing.

That's worrying in of itself.

on('upgrade', function(req, socket, head, res) {}) is too ugly for my liking.

My suggestion is using different event name is used in place of upgrade (upgradeEx, upgrade-req, upgrade2, etc.) like is done in the update-ex npm package. Depending on which events have listeners depends on which behaviour is taken within the http module. And since they have different names, this also has the plus of being able to be detected which is being listened to, and allowing different signatures.
In the meantime, the upgrade event can still work as before but with a warning when a listener is attached telling the user/maintainer to upgrade their package to use the new event and event signature.

I really like this proposal, and think it works a lot better than the existing upgrade event behaviour, although I worry that now it is a npm module its inclusion into the nodejs core may be questioned, even though it is an improvement/replacement for existing functionality.

And considering the breakage and updates that will be needed from modifications of streams in 0.9/0.10, I think considering this sooner would be better than later.

[Nodejs-Jenkins]

Can one of the admins verify this patch?

[adammw]

Seems like there may now be some motivation to make some changes to upgrade, but in a rather different way at #4813.

[stephank]

The last push contains a rebase on master (46da8c2), and reworded commit messages.

I had to change test-http-upgrade-client2.js, because it was doing really strange things that I'm not sure are valid on current node versions any way.

[stephank]

Rebase against current master (b255f4c).

I used to have a commit that removed upgradeHead on both the HTTP server and client, but master has since deprecated it and replaced it with an empty buffer for compatibility.

So the commit was no longer necessary, and I'm no longer touching the HTTP client code. I do, however, remove the empty buffer on the HTTP server, considering this is an API change any way.

I also removed the commit that fixed test-http-upgrade-client2.js, which also seems fixed in master already.

[bnoordhuis]

@stephank What benefit does this change have over the current approach?

[stephank]

Can't see why jenkins is failing, but I believe these are not failures related to the pull.

@bnoordhuis In short: middleware. See also #4813.

[bnoordhuis]

Right. @isaacs, this one's for you.

[stephank]

Rebase against current master (b26d346). You guys are really good at breaking my stuff for years on end. :(

upgradeHead removal was apparently reverted again on master. I didn't add it back to the client.

[isaacs]

@stephank Sorry for the absurd delay on this. I will review this in more detail very soon (today or Monday).

The main blocker that I can see right off the bat is that we probably can't get away with changing the server.on('upgrade') signature in a backwards-incompatible way (hence the ugly compromise in #4813). That being said, the core pipelining-related bug here definitely needs to be fixed, and #4813 is important, so I think at least most of the work you've done will be relevant and useful (even just the test is really great).

Thanks again, and my apologies for letting it linger so long. This is next in my queue after the OutgoingMessage refactoring.

[stephank]

@isaacs Thanks for taking a look.

Note though, the test is more or less theoretical. It definitely triggers bad behaviour, but I'm not aware of this ever happening in practice, with real clients.

I'm curious how you'll end up tackling #4813. Middleware is the important issue for me here.

[stephank]

If my understanding is correct, there is no intention of merging this, and I have no intention of updating it any further.

Unless you guys want to keep this issue around for a hypothetical 2.0, you may as well close it.

Path ahead seems to be a separate module and getting people to use that. I might at some point in the future revive upgradeEx, but I'm currently otherwise occupied.

[jasnell]

Keeping this around and open for 2+ years after the original poster gave up on updating it isn't helpful. Looks like there's been recent albeit not particularly active discussion on it on the io.js side. Given that it's unlike this specific PR will ever land in v0.x, closing. Can reopen if necessary.