Web server proxies, as work queues with apps pulling requests

One thing I have thought about for a while is to reverse how load balancers work. Today to my understanding, they receive web requests and then distribute them across an array of application servers. The proxy can measure the response time and as such can decide which server to send the next request to, or it can use some kind of algorithm, such as round-robin.

But the component that knows best when it is ready to process a new request is in a way the app server itself. So why not let the proxy maintain a queue of incoming requests and then have the app servers pull requests from the queue?

Having such an architecture leads to another interesting feature, that app servers can be created and then simply start pulling work from the proxy, without the proxy needing to be reconfigured to accommodate for the new instance of an app server. If using SSL certificates, adding app servers could just be a question of firing up new instances and having them poll the queue while blocking until they get a job from the queue.

Furthermore the proxy can, if an app server is slow to process a job, serve out an intermediate response, that tells the client to continue polling (e.g. through JSON-RPC) until the result comes through. In this way you can work around fronting server timeouts.

So if a request becomes a "job", caching can be memoizing a job, and the proxy can decide whether to serve a cached version or wait for a job to finish. If a certain job with a memoized signature is slow to respond, its caching can be increased. If an app server serves jobs much slower than other ones, it can be put out of commission.

However an app server may benefit from serving several requests concurrently due to I/O waiting and in that case the setup gets a bit more complicated.

Update: I've got a reply on Reddit pointing out some problems with the above design