Fragmentation in the asynchronous realm

[gotnone]

It seems like there are dozens or even hundreds of asynchronous implementations for C++. This means that a complex program may have dependencies creating a handful of completely independent thread pools.

For a particular application I have been working on I have library dependencies maintaining at least 4 completely independent thread pools. If you analyze the dependencies, you see that 1,2, and 3 are all networking based.

1. A vendor supplied Genicam library
2. cpr (based on libcurl) for accessing the client's API over HTTP
3. libplctag for looking up the RFID tag of the items being processed from a PLC
4. concurrencpp for organizing the tasks in the program and dispatching them to appropriate thread pools or completely independent threads as appropriate.

In my particular use case, there is no fear of running out of system resources, but it seems that this duplication of effort is a potential issue for C++ in general. It may also be limiting the usefulness of some libraries in high performance or tight resource environments.

Do you think that the introduction of co-routines into c++ without a standardized networking library will lead to further fragmentation in the ecosystem? Libraries like libfv by @fawdlstty duplicate some of the feature set of concurrencpp, but, from my understanding, don't share the same thread pools.

How can this fragmentation and duplication of effort be minimized?
Are there any strategies for sharing thread pools?
Is there a way for library authors and application developers to avoid coupling their code to a particular co-routine / concurrency library?
What additional performance or development costs would developers incur from trying to maintain this independence?

[fawdlstty]

The asynchronous implementation of libfv is based on the thread pool in boost.asio. The thread pool interface is open and supports reuse under the library permission mode. If you have a good idea, I would also like to improve it to support concurrencpp or implement easier thread pool reuse.

[David-Haim]

Hi there, and sorry for the very late response, I was busy with personal issues and didn't have much time to do concurrencpp related things.
For you questions:

Do you think that the introduction of co-routines into c++ without a standardized networking library will lead to further fragmentation in the ecosystem? Libraries like libfv by @fawdlstty duplicate some of the feature set of concurrencpp, but, from my understanding, don't share the same thread pools.

So, the problem here is the coroutine-executor integration and less about the coroutines themselves. different coroutines implementation can (theoretically) work with each other without causing any problem. for example, concurrencpp coroutines can call and await cppcoro coroutines, and vice-versa. it's when you want to mix different executor/scheduler types when things can turn ugly.
Also, it's unhealthy for an application to have many threadpool instances in the same process. usually, threadpools create threads as the number of the cores in the system for maximum performance, and having many threadpool implementations might create way more threads than the number of cores, causing perfomance penalty because all of the context-switching.

How can this fragmentation and duplication of effort be minimized?
Are there any strategies for sharing thread pools?
Is there a way for library authors and application developers to avoid coupling their code to a particular co-routine / concurrency library?

My suggestion is to pick one threadpool implementation, and to try to use that and only that as much as possible. this can be acheived by:

• using callbacks and then scheduling the task continuation in the chosen threadpool, schematically it looks like

lib_a::do_something_async().then([lib_b_threadpool](auto result){ 
   lib_b_threadpool.post([result]{
       ... 
   });  
});

• using resume_on if possible, for example:

auto result = co_await lib_a::do_something_async();
co_await resume_on(lib_b_threadpool);
//use result as originally intended

• with concurrencpp, there's a way to create user defined executor. if you choose a different executor to use then the concurrencpp ones, you can wrap the foreign executor with a concurrencpp executor and continue to use that wrapped executor with concurrencpp as usual. see creating user defined executor

What additional performance or development costs would developers incur from trying to maintain this independence?

As for performance, if you go with the suggestions I have given, the performance penalty is minimal. As for development, I really need to see the code base to judge but sharing one threadpool across multiple libraries is doable, if the different libraries provide those integration points (callbacks, resume_on, etc.)

I hope it helps!

[fawdlstty]

Actually this problem is solved when a coroutine library becomes part of the standard library ;)
I really hate the asynchronous ecosystem of c++ :(

[David-Haim]

If you think that std::execution will solve anything I have bad news for you..

[fawdlstty]

I think a design like c# would solve my problem