Streaming responses #1600
Comments
|
I'm not sure what my opinion can add, but I'll give it a shot. The approach seems straight forward to me. However, I also believe it is inherently different form Where this approach represent a TTFB optimization on responses that are server bound but slow (i.e., long database queries), the Does it improve performance? On a performance note, not all servers and not all clients would oblige the TTBF optimization suggested here. Not all servers start sending the response before it's complete - the I know iodine won't send the incomplete response (errors might occur and need to be reported) and this is one of the things I am likely to change for iodine 0.8.0. Even if the application servers stream the data, browsers might not show all the data until it's complete (or until a large enough buffer was filled). This is also true for XHR requests where the data might not become available until the whole (or a large enough part) of the data was received. Side-note: Another reason iodine doesn't support this model of TTFB optimization is because IMHO this is a design flaw / abuse, which meant I didn't feel the need to support for any of the projects I had at the time. IMHO, if the response is slow, an API redesign should be considered rather than optimizing the TTFB on slow responses. My issue with this use of approaches is that they don't offer client responsiveness and errors could be hard to detect. Also, if an error occurs mid-stream, the server might not be able to do anything to signal the error (the client might assume a disconnection meant an EOF rather than an error). Fragmenting the slow part into faster (yet smaller) request-response cycles is usually better than using streaming for a TTFB optimization, as it allows both more responsiveness and allows better error handling / indication. |
|
P.S., This was a very interesting use of the template engine internals, allowing the I'm not sure it would work for every template engine, but the same could be done today with any template engine. For example (using iodine's mustache template engine): require 'iodine'
class SlowDBQuery
# emulate a slow DB fragmented into a loop of faster chunks
# the `{number: i}` represents a data structure (i.e., Hash) returned by the DB
def each
10.times {|i| yield({number: i}); sleep 1; }
end
end
class DefferedBody2
TEMPLATE = Iodine::Mustache.new(template: "<li>Hello World {{ number }}</li>")
attr_accessor :head, :tail, :enum
def each(&block)
yield @head if @head
@enum.each {|i| yield TEMPLATE.render(i) } if @enum
yield @tail if @tail
end
end
APP = Proc.new do |env|
body = DefferedBody2.new
body.head = "<html><body><ul>"
body.enum = SlowDBQuery.new
body.tail = "</ul></body></html>"
[200, {}, body]
endYes, the code isn't fully optimizes and it is slightly naive, but it does prove that the current The optimizations at this point are mostly a question or application code rather than Rack design. EDIT: Works in the terminal using: APP.call()[2].each {|s| puts s}Didn't try it within a server. |
|
Nice work building and trying it out. You should try it in an actual server, it's fun. I think what it shows is that Full hijack is impossible for HTTP/2. The only use-case I'm familiar with is HTTP/1 WebSockets. Is there any other reason for full hijack? If there are no good use-cases for But, it begs the question, if there are no good use-cases for |
|
There's some historical context in the original PR thread, especially starting from #481 (comment), and very especially:
I think my mental model would suggest partial hijack if an application wanted to implement e.g. a custom That said, Rails doesn't use partial hijacking, nor can I find any evidence it ever has. We use full hijack for Action Cable, I wonder if anyone actually uses partial hijack. I think I recall that WEBrick supports partial and not full... but I can't remember whether I've come across an application/middleware that uses it.
I think you mean partial hijack here -- is that right? (Unclear because both full and partial use (different) things called |
|
@matthewd thanks for the links.
Both. Full hijack is a relic of HTTP/1 which cannot be supported in any logical way with HTTP/2. Partial hijack can already be done using the Welcome feedback on this conclusion - but I feel like if the same logic applies to Rack, wouldn't it be nice to remove |
On this point, why not use partial hijack? you can return the connection headers (upgrade and so on for HTTP/1) and then read from |
Another use case is SSE / long-polling, which won't work with If SSE was implemented using
SSE / long-polling could use either one. If one was to be removed, it could fall back to the other. In fact, WebSockets could use both as well (a partial hijack to send the upgrade headers). It's really a matter of developer choice.
By god, I wish! Removing However, this will break existing long polling and web socket implementations. In fact, I think Discourse uses |
It seems to me that all these use-cases collide with concurrency optimizations. They add unexpected threads / fibers to the settings selected by application developers. For example, running In addition they add network logic to the MVC framework - a framework that could be free of network logic. I believe each of these could be improved. How can we (as server developers) make this work better? ActionCable As server developers we could abstract away the whole network layer, simplifying the Rails code base, making testing easier and ultimately making performance and optimizations easier for application developers. I believe a WebSocket callback object (as previously suggested) would be the best approach, but the real question is what would the Rails team prefer? What would make things easier for you? ActionController::Live and streaming views I believe we could add server side streaming that isn't thread bound (it may break away from the CGI model, but could be made backwards compatible). As wonderful as For example, data streamed before the response was returned to the server (by the middleware) could be delayed - allowing headers added my the middleware to be sent before committing the response. I believe these use cases cause server-side concerns to mingle with the framework code and result in a lot of duplication of work. It would be much better if Rack 3.0 (or a separate specification, if you prefer) could help minimize the duplication of effort and code. |
|
I don't think we should remove full hijacking, as it has use cases in HTTP/1 that are not handled any other way. One example of this is messagebus, which uses full hijacking to take the socket and add it to the pool of sockets it manages. This cannot be done without hijacking of some form. For example, you can use hijack with messagebus and unicorn to serve a large number of long polling clients, even though unicorn itself is single threaded. The way SPEC is already written, hijacking is not required, and servers do not have to support it. It only specifies how it should work if it is supported. So you can remove hijacking from Falcon and still be completely compliant with the current rack SPEC. We could add a note to SPEC that full hijacking is not compliant with HTTP/2, but that's as far as I think we should go. |
|
@jeremyevans , I would like to offer a counter-opinion. I believe MessageBus is a prime example of why we should deprecate Reading through the MessageBus code shows how a wonderful idea is hampered by the flawed The IO is collected using This makes the code susceptible to slow clients attacks, buffer saturation concerns and quite failures that will pop up under heavy stress - issues that are never tested against and never reported / logged. The fact is that keeping I don't like breaking backwards compatibility, but I believe that we can offer MessageBus a superior and more secure interface. Breaking backwards compatibility, IMHO, will be better than ignoring the fact that |
|
@boazsegev I think if you can identify actual problems with MessageBus, you should probably report them upstream. All software has bugs, and it sounds like the bugs you are describing could happen to a webserver that implements the rack API. Hijacking is basically having the application operate as the webserver for the hijacked connection. In order to handle MessageBus's usage, it needs the equivalent of hijacking. It needs to be able to signal to the webserver that it will handle the connection and the server doesn't need to deal with it. Otherwise it can't work with existing single-threaded servers such as Unicorn. If you have an idea to replace hijacking while still allowing MessageBus to work with Unicorn, it's definitely something that we can consider. Hijacking is not inherently more error prone and insecure than the handling of connections that the webserver does. It could be an issue that you don't trust application/middleware developers as much as you trust webserver developers. However, I'm not in favor of removing the capability just because an application/middleware developer might not handle all edge cases properly. In terms of HTTP/2 and HTTP/3, those connections can just not set rack.hijack, marking hijacking as not available. That's within the rack SPEC. Servers do not have to support hijacking at all and they can still be compliant with the rack SPEC. That doesn't mean hijacking should be removed when using HTTP/1, which is by far the most common case for ruby application servers. I think we are certainly open to considering interfaces that are superior in all aspects. None such have been proposed yet. All proposed alternative interfaces thus far have had significant tradeoffs. |
|
@jeremyevans , I'm mostly writing just my personal opinion and counter arguments for us to think about both sides. I'm not even sure what I would decide if I had to make the call. It's obviously a difficult decision.
True. All I'm proposing is that we discuss a few options and their tradeoffs and see if we can come up with something better than what we've got - something that will offer a solution to all existing use-cases. I am also in the opinion that we should consider placing
Yes, you're totally right. In fact, this is the exact reason why I believe I would never expect a network programmer to make the mistakes I pointed out (such as not testing the value returned from a call to I don't believe we should expect application developers to know anything about writing network code and I believe the specification / interface we offer should abstract away network details and protect users from its pitfalls.
I can, and you're right, but I'm both busy and not in the mood. It's likely that every other gem / app that uses the I'm not about to start an education crusade when I can fix the issue at the source (offer a better API / interface). In fact, for me, this is what this discussion is all about - solving the issue at the source. This discussion can do more to solve the issues in MessageBus than a hundred PRs.
I beg to differ. The server code is written by people who spend a considerable amount of time writing network code. Asking the application developers to have the same experience and knowhow is, IMHO, unfair.
I think that if I removed The fact that officially the feature is optional doesn't mean that it isn't required in practice. I believe that unless we come up with a better (and common) interface, there's no option except somehow supporting By offering an updated interface to network events, we could easily make code that's truly transparent to the underlying protocol, improving development experience and ease of use. |
I agree that we should continue to discuss options and their tradeoffs. I also think we can consider deprecating
Note that in many cases, I would expect mistakes for any programmer, myself included. There are probably more of these types of bugs in rack applications using hijack compared to rack webservers, but there are also a lot more rack applications using hijack compared to rack webservers.
To the extent that we can do this in a backwards compatible way for existing hijack users, it seems like a good idea.
Seems unlikely that every single rack.hijack user is making exactly the same mistakes, but I agree that the problem isn't rare.
What your proposing to "fix the issue" (removing
It seems we are in violent agreement then. You say you beg to differ, but then make the same point I made, which is that the problem is exactly the same, your issue is which people are asked to handle it.
If everyone using iodine is using
I agree that
I think we are open to proposals in this area. |
|
@SamSaffron Wondering if you have time to give your input here? I am interested in looking at MessageBus as a use case for I was looking at the implementation and I see 4 (?) distinct paths:
This is a similar design to ActionCable - using the rack compatible web server as the front end and some kind of multi-threaded backend which you pass connections into. Do you think the |
|
I think rack.hijack works ok, the thin pattern is pretty much abandoned these days as almost nobody uses thin. It did have a minor advantage in that it provided a bit evented IO. I guess the big bit that is missing imo is some light weight abstraction on top of the sockets you are managing to handle epoll and timers. At the moment we do scheduling using https://github.com/discourse/message_bus/blob/master/lib/message_bus/timer_thread.rb this scales OK, but there is potential that buffers can fill up and scalability go down when you are distributing large events to thousands of clients. I do agree is can be some what confusing that sometimes we are making payloads by hand including HTTP headers and all that jazz and in other cases using rack shortcuts. Unifying codepaths could be interesting. I guess my biggest motivator though for change here would be an increase in scalability, if new patterns could reduce latency significantly for cases where we are publishing messages to 10-20k active client it would be interesting. Atm at Discourse we use short polling for anon users to avoid housekeeping costs that long polling entails, but that means anon get ever so slightly less live updates. |
|
@jeremyevans , I believe that we're mostly in agreement.
At this point I would suggest we focus on I would also suggest (assuming a viable alternative is found) leaving As major players migrate to the new scheme, servers might stop supporting I believe that in 3-5 years time all actively maintained gems will migrate to the new alternative (assuming it's viable and better for them than the current Does that sit well with everyone? (@ioquatix ?) |
|
Thank you for your time and for joining the discussion.
Do I understand correctly that you prefer an evened approach? If yes, which callbacks do you require? If not, what would be ideal for you?
Could you explain more about what you see as epoll handling and timer handling? Are you looking for simple connection timeout support or something more? Are you looking for
I believe this is possible to a point. This would depend on the network layer, the concurrency model and the pub/sub design. It's not dictated by the interface that Rack will offer. At this point I could make a sales pitch for my own server or for falcon... but at the end of they day, we're trying to find the best design (not the best code). I believe that once we do, blazing fast solutions will pop up. |
|
@ioquatix I assume that at this point we're looking for a better solution for If so, this might bring us back to some of the discussion started in #1272. I still keep some details in the iodine repo, though iodine extends these ideas to support raw TCP/IP as well (as indicated here and sadly undocumented for HTTP requests). I think you didn't like the idea of setting variables in the What would you suggest we do different this time so the discussion can produce something actionable? P.S. The one thing I would ask is that the specification we end up with doesn't require the Rack gem. I prefer if the servers could follow the specifications without requiring any of the code. |
I'm strongly in agreement with this. I think doing anything that bypasses normal request/response processing in Rack should be avoided. "There be the dragons". These issues have already been enumerated. I feel like we should be limiting users to the concurrency model exposed by Rack and the server itself - e.g. multi-process, multi-thread or multi-fiber is the safe approach, and I think we can support this a bit better by exposing the nature of the server from I wanted to see if this was possible, and it seems like it is. if defined?(Falcon::Server)
puts "Using Fiber implementation..."
class Bus
def initialize
@clients = []
end
def publish(message)
@clients.each do |queue|
queue.enqueue(message)
end
end
def subscribe
queue = Async::Queue.new
@clients << queue
queue.each do |item|
yield item
end
ensure
@clients.delete(queue)
end
end
else
puts "Using Thread implementation..."
class Bus
def initialize
@clients = []
@mutex = Thread::Mutex.new
end
def publish(message)
@mutex.synchronize do
@clients.each do |client|
client.push(message)
end
end
end
def subscribe
queue = Thread::Queue.new
@mutex.synchronize do
@clients << queue
end
while item = queue.pop
yield item
end
ensure
@mutex.synchronize do
@clients.delete(queue)
end
end
end
end
MESSAGES = Bus.new
class DeferredBody
def each(&block)
MESSAGES.publish("Hello World from #{Thread.current}!")
MESSAGES.subscribe do |message|
yield message
end
end
end
APP = Proc.new do |env|
[200, {}, DeferredBody.new]
end
run APPThe only difference in the above is the use of This ensures that the server is responsible for providing the required level of scalability and you tune it via the same set of parameters. |
I would say this is beyond the scope of Rack.
It also means that such code paths would never be compatible with HTTP/2, which the example I gave above, when running on Falcon, works just fine.
I think this is addressed as a server implementation issue. If you use Falcon, I believe you can see improved scalability, and it also seems like iodine could achieve that too.
Given my example above, is an alternative actually required? Because we can do everything via My current thinking/goal is to try and show we don't need I still believe concurrency models are outside the scope of Rack. It becomes server specific. I believe a gem like |
What the provided example doesn't cover is cases where hijack is used to take control of the connection so it is no longer handled by the webserver. @SamSaffron provides an example with MessageBus serving 15-20k active clients. You are not going to be able to do this using streaming responses (
It's always easy to make things simpler by removing more complex parts and breaking the related use cases. We need to keep supporting the related use cases, and if can be made simpler while doing that, that is something we should consider.
I think Rack's current API doesn't define concurrency models. Hijacking allows you to use a different concurrency model for specific connections compared to normal connections that are not hijacked. While more complex, it enables features that may not otherwise be possible without substantial changes (such as changing webservers). |
I think there will always be use-cases that will desirer to degrade (or upgrade) to raw TCP/IP. In the context of HTTP/2, this might degrade (or upgrade) to a tunneled stream. For example, to avoid
No vested interest here either. I don't even care if iodine is surpassed and becomes abandon-ware. I just have a lingering love for Ruby and want to see it flourish.
Agreed.
Maybe, but I'm not sure it's an actual requirement. If the Rack specification abstracts away the network layer, all that's left for message_bus to handle is the pub/sub side of things, which might not require concurrency management at all (except, possibly, a Mutex).
Honestly, iodine currently doesn't really support There are 2 reasons for this:
Currently,
I'm sorry, I can't follow the same design pattern with iodine, since However, I can use an alternative to I don't think this is what you wanted, but it would look like this: require 'iodine'
module MyCallbacks
GLOBAL_CHANNEL_NAME = :global
# called when the callback object is linked with a new client
def on_open client
# subscribes the client
private_channel = client.object_id.to_s(16)
client.subscribe GLOBAL_CHANNEL_NAME
client.subscribe private_channel
# global publish
client.publish GLOBAL_CHANNEL_NAME, "JOINED: client #{client.object_id.to_s(16)}"
# pushes a Hello World message every 1 second
Iodine.run_every(1000, 10) { client.publish private_channel, "Hello World" }
end
def on_message(client, message)
client.publish GLOBAL_CHANNEL_NAME, "#{client.object_id.to_s(16)}: #{message}"
end
# called when the client is closed (no longer available)
def on_close client
client.publish GLOBAL_CHANNEL_NAME, "DISCONNECTED: client #{client.object_id.to_s(16)}"
end
# Allows the module to be used as a static callback object (avoiding object allocation)
extend self
end
RACK_UPGRADE_Q = 'rack.upgrade?'.freeze
RACK_UPGRADE = 'rack.upgrade'.freeze
APP = Proc.new do |env|
env[RACK_UPGRADE] = MyCallbacks if(env[RACK_UPGRADE_Q])
[200, {}, ["Please connect using SSE / WebSockets"]]
end
## in config.ru
# run APP
## in terminal
Iodine.listen handler: APP, service: :http
Iodine.threads = Iodine.workers = 2
Iodine.start
## SSE testing from browser:
# var source = new EventSource("/"); source.addEventListener('message', (e) => { console.log(e.data); });
## WebSocket testing from browser (broadcasts a global message):
# ws = new WebSocket("ws://localhost:3000"); ws.onmessage = function(e) {console.log(e.data);}; ws.onclose = function(e) {console.log("closed")}; ws.onopen = function(e) {ws.send("hi");}; |
|
I think my best picture of a future hijack replacement would be that it unify HTTP/1.1 Upgrade and HTTP/2 CONNECT -- so yes, in a more partial-hijack style, providing the initial header response as usual, and then afterward handing off a socket. H1 hijack already doesn't provide the real raw socket (read: TLS), so an H2 equivalent IO that actually encapsulates its content for an H2 CONNECT tunnel seems consistent... almost to the point that the underlying "raw"-socket-wanting application/framework needn't care whether it's been established via H1 or H2, just as it can currently ignore whether TLS is involved. |
If I understand you correctly, you don't really expect a Ruby native IO from This abstraction might not play well with IMHO, I think we could ask for more from the future. So much potential.
Wouldn't you prefer that the server continued to "poll" the IO for incoming data (so you didn't have to)? Wouldn't it be nice if you got notified if the connection was lost / closed instead of polling the IO state? What about timeouts? Wouldn't it be nice it you didn't have to manage timeout logic...? IMHO, using a callback object would allow you to simplify your code and focus on what you want done instead of forcing you to maintain a whole network "backend" that simply duplicates what the server already implemented... Side note:
You're assuming... I think that if you tried that with iodine (using it's TLS support), you'd find that you got the raw IO and needed to initiate a new TLS session. Though that may be considered a bug 😅 |
It would be nice if it was as simple as shifting to a more scalable web server though. But because of the way we force the implementation, it's not possible. There are also valid reasons for using puma with thread per connection given the current eco-system of gems which expect to run per-thread, e.g. ActiveRecord. I've also had people tell me they prefer Unicorn because there should be in theory no chance of thread safety issues. So, I don't actually see this as a disadvantage, so much as exposing the concurrency model of the server. For many users, process per connection could be acceptable, even for long running connections (at the very least it's great from an isolation POV if you were, for example, dealing with sensitive data). Thread per connection in Ruby 2.7 also should be fairly scalable, e.g. I wouldn't recommend it but you should be able to run at least thousands of threads with Puma.. in 2.7 I reduced memory usage and per-thread allocation costs. Beyond that you probably want an event driven model which everything going well will be a solid part of Ruby 3. So, yes, right now with the situation we are in,
That's already what falcon does for a partial hijack. Here is how the server side of There is no equivalent for full hijack for HTTP/2 without changing the definition for full hijack.
Right, so:
So, I think that full |
|
By the way, here is the rack handler for This To me, this approach is superior to partial Full hijack, as outlined above, is not supported for HTTP/2, and works as expected for HTTP/1 in Falcon. I added it so that ActionCable would work, and it seemed like fun at the time. So, I don't recommend it because if you run your server in HTTPS mode (the default for Falcon) you will not have support for full hijack. |
|
Oh, I guess I should add, there IS one way to make HTTP/2 behave like a full hijack socket, and I also implemented it... but not for full hijack - but for proxying via HTTP/2 CONNECT. It turns out OpenSSL needs an actual socket. So if you want to proxy HTTP/2 or TLS connections over an existing HTTP/1 or HTTP/2 connection, you need to expose a socket. For HTTP/2, we make a stream with a |
I do get your approach, I really do. But I don't feel we can expose these interfaces without also implicitly including a specification for concurrency, which I think we all agree it outside the scope of Rack. Rack is really great because it works completely synchronously. There is no asynchronous behaviour in Rack itself. To me, this is really a great part of rack, maybe the best part. It's just a function of If |
Is the "Not thread safe" a consequence of the "Can't be used with IO.select", or is there some other issue I'm not immediately seeing? Seems like the IO.select part could possibly be solved upstream, in the long term. Given Ruby's open-ended duck-typing-preferred nature, and how important IO.select is to the use of IO objects, it seems unfortunate that an artificial IO object can't participate. (Though that'd only move the needle slightly for this particular case, because in practice Action Cable uses nio4r, not actual-IO.select.) |
Here is a high level overview of what HTTP/2 looks like.
All reads and writes must go via the main connection/framer. At least in For an individual stream, there is no object to provide to
|
|
Bearing in mind that if someone tried to do it, and writes HTTP/1 formatted data, they will probably have unexpected results :) |
|
@ioquatix , to answer your questions (and add some of my own),
Yes. Pretty much like any single threaded code flow, if you block the main thread - it stalls. If you call You can set up iodine to use multiple threads, but that just means that you will have multiple chances to block FYI: this is also true for node.js and your web browser, where they try solving it by preventing the JS API from allowing the developer to block... but add a
Yes, but this is part of the reason Some use cases require a more complex control-flow. They want to write and read and test for timeouts and wait for DB updates - in other words - they want to react to external events. WebSockets and long-polling are simple use-cases that fit this description, but they're not the only ones. There's server-2-server communication, video streaming, gossip protocols... All these could benefit from a better
I understand your desire for simplicity - I love simple. But at the end of the day, no one will give up the If users don't get the same features (or better solutions), they will keep using You might be able to emulate I don't think it would be fair for Rack 3.0 to require all servers to switch to a fiber-per-connection model. |
As I think you know, It's limited to This is why the move to ... actually, this may be considered a design requirement that would minimize code errors (preventing users from using IO.select). @matthewd, @ioquatix , @jeremyevans , @SamSaffron - quick question: Assuming that:
wouldn't it be better if polling was integrated into the alternative solution so users didn't have to do it? Also, can anyone see use cases that require hijacking (can't be performed with |
Well, my understanding of the callbacks you've proposed, require non-linear flow control. Assuming that's correct, then it requires the definition of a model for concurrency, e.g. "event driven callbacks". It's not a hard line I guess, but if you implement a single connection per process web-server, can you implement your event driven model in a purely sequential way?
To me, this is a model of concurrency, which I don't think we want to introduce. There is value in servers which are process-based (unicorn), thread-based (puma) or fiber-based (falcon). How would polling add value to a process-based server? How would it work in a fibre-based server that already has a more elaborate model for concurrency? It introduces a model which will create an impedance mismatch with existing models of parallelism & concurrency. |
Sure I can. A thread/fiber-per-connection design would work better though, since on a per-process web server you won't be able to share the In a sequential (non-event based) implementation, the client "thread" is in a read loop: # pseudo code
client.handler.on_open(client)
while(!client.closed?)
data = client.read_with_timeout()
if(data)
client.handler.on_message(client, data)
else
client.write_locked? || client.closed? || client.handler.on_timeout(client) || client.close
end
end
client.handler.on_close(client)The client The client # pseudo code
class Client
def write(data)
client.write_lock
start = 0
while(start < data.length && !client.closed?)
written = client.io.write(data[start..-1])
(written.to_i > 0) ? (start += written.to_i) : client.close() # assuming IO is blocking
end
client.write_unlock
client.handler.on_drained(client) # if implemented, I wouldn't implement this with a sequential design
end
endAll in all, there aren't many critical paths and they could probably be avoided by making some callbacks optional (such as requiring
I don't think it's correct. A non-evented server could give the illusion of calling these callbacks in an evened way, but it should be fairly easy to implement them as sequential code. |
|
I hope this input can provide useful: we do use the partial hijack, and I disagree that
From the user perspective I think this is somewhat misguided, because when I do use hijack and I know I am only operating in HTTP 1.1 context I do so deliberately, and thus handling slow clients is my problem (which I indicate to the webserver by opting for the facility in the first place). I believe the wording is also poorly chosen because you are implying that if I am using the facility I am doing so while being clueless to the possible side effects and "quiet failures". In our case they are not quiet because code that runs within the hijack is subject to APM error capturing just like any other code. Instead of letting sockets be handled through Yes, it is not HTTP/2-compatible because it does not observe the framing guarantees (I would rather leave my opinions on HTTP/2 out of the conversation) - but with HTTP1 it allows us to service multiple hundreds of very heavy downloads per box, without having the string churn (and overload of separate IO.select loops) which would be the case with |
|
I think the ideal solution is that you can write your code, and if you write it in a generic way, you get the scalability of the server you choose. Farming sockets off to a custom reactor in a separate thread seems like a lot of work. |
|
And it is 😭 but the other solution with Puma would be to persuade them to use nio4r for response writes - as at the moment it is only used for reading the request, writing the response still uses |
|
@julik ,
I apologize if my language seemed offensive. I didn't mean to imply that web application developers are clueless about network concerns. I think that any unsafe code is the result of a design failure on our part, since app developers should have been free to assume that the IO would just work. They should have been free to assume that the server will take care of all the gritty details of any network concerns.
I assume you'll be trying out falcon, iodine or both. I think falcon's green threads might solve the CPU pressure. On my part, I know that iodine polls for |
|
This is now resolved. |
@matthewd gave an example of one option by capturing the stream callback.
For reference, here is a fully working example:
It also works for HTTP/2
If we can do this already, is there any point to partial
rack.hijack?@boazsegev do you have any opinion about capturing the stream output this way?
The text was updated successfully, but these errors were encountered: