Coroutine seems not working for me ...

[petermp79]

I am load testing lwan using wrk load test. Sending requests on 100 connections.

However, the number of outstanding incoming requests is limited by the number of I/O threads and cant go over it (I kept a counter to track this info).

I thought coroutine support in lwan will allow for more requests to be sent than the number of I/O threads.

Wonder if I am missing something.

Thanks,
Peter

[lpereira]

This is indeed odd: your thought that Lwan is capable of handling more requests at the same time than the number of I/O threads is indeed correct. Unless you're writing your own handlers, and making blocking calls in them... in any case, there's not much Lwan can do.

I regularly test with 100, 1000, and 10000 connections, with only 4 threads handling them; everything seems fine. Can you please share details of your setup? Which Lwan version are you running, how many threads, what operating system, anything else that you might find pertinent?

[petermp79]

Thank very much for a super quick response.

Basically, I have a handler that invokes a rpc call to a local grpc server. The call locks up waiting for response. Does it have to be asynchronous for coroutine to kick in? In this case, do you have a sample code I can use as a template.

The web handler calls the SendRequest() in the below class:

class TriggerClient {
public:
TriggerClient(std::shared_ptr channel)
: stub_(lightfaas::NewStub(channel)) {
//faasLog->info("Connection created");
}

    // Assembles the client's payload, sends it and presents the response back
    // from the server.
    Event *SendRequest(Event& request, Event *response) {
        // Context for the client. It could be used to convey extra information to
        // the server and/or tweak certain RPC behaviors.
        ClientContext context;

        pthread_mutex_lock(&req_cnt_mutex);
        request_cnt++;
        if (request_cnt > request_max)
            request_max = request_cnt;
        pthread_mutex_unlock(&req_cnt_mutex);

        //faasLog->info("Send request: {}", request.userid());
        // The actual RPC.
        Status status = stub_->processEvent(&context, request, response);

        pthread_mutex_lock(&req_cnt_mutex);
        request_cnt--;
        pthread_mutex_unlock(&req_cnt_mutex);

        // Act upon its status.
        if (status.ok()) {
            return response;
        } else {
            faasLog->info("--- status.error_message(): {} ", status.error_message());
            return NULL;
        }
    }

private:
    std::unique_ptr<lightfaas::Stub> stub_;

};

Thanks,
Peter

[petermp79]

I forgot the details. I am running current version I just clone from github, on Linux 16.04, with 4 I/O threads.

[lpereira]

It just so happens that I'm working on making it possible for you to wait for a file descriptor; the idea is that you'd use non-blocking FDs, but tell Lwan to "wait until I can read from this fd" -- this would cooperatively yield from the coroutine, allowing others to be executed. Part of the infrastructure for this is in place, I just have to finish it. It's good to know there's another use case.

[petermp79]

I would love to give it a try and let you know the result on my side. I have implement the same in Go. Somehow, Go does it all transparently from programmer.

[lpereira]

That's because networking in Go is abstracted for you in a way that its user-mode scheduler is aware of. Even then, its scheduler is able to steal work from blocked threads, by either moving the work to a non-blocked thread, or spawning a new one if no threads are available at the moment.

Lwan does some of these things -- for instance, you'll see I/O wrappers around syscalls such as read() and writev(), but these are only useful if writing/reading from the connection coroutine would block. It can't do much if, for instance, sendfile() blocked while transmitting the contents of a file. It can't steal work from threads, too. There are hints you can give (e.g. there's a low-priority thread that does readahead() to minimize the possibility of sendfile() blocking, but you gotta tell the thread to do it yourself), but the burden goes to the programmer using the library and not the runtime, as is the case of the Go. Lwan is, afterall, just a toy project.

But there has been some desire to wait on a file descriptor other than the one handling the HTTP connection -- say, your gRPC client, or a connection to a Redis instance -- so I'm working on this.

Can't commit to any ETA, though, as Lwan is kind of in a low priority state for me these days.

[lpereira]

Waiting on file descriptors other than the client file descriptor has been possible for a while now. We even have a module that makes use of this (FastCGI).