Draft: Add cpu affinity options to librist

This patch adds the possibility to tell librist which threads to use for its internal workings. We use something similar to this in our company for linux only. This patch also adds thread affinity management on windows and freebsd.

Let me know what you think

marked this merge request as draft

Thank you!

Do you have a specific use case that requires it in the library? From a perspective of maintainability I'd like to keep the API as lean as possible, this seems both like a very specific use case and a problem that could be solved at another level (tools, or calling application). Also I'm afraid for portability, how will this compile on say, an android or iOS device?

If you're fine with re-implementing limited to the tools, that would be great.

Hi!

Thanks for the quick response. Our use case is an application with tight real-time requirements that runs on a system where the kernel scheduler is more or less disabled because most cpus are isolated. Also this would be a great thing to have on large system with multiple NUMA nodes to optimize memory latency. I also see the issues with maintainability. Solving this at the level of the calling application would be fine for me. This would require access the the protocol-thread handle or even a list of threads through the libRIST API. But i guess the internal threading of a sender/receiver instances should stay a hidden implementation detail, right?

In an ideal world: yes ;)

I guess a callback that is issued whenever we create a new thread would work for me. The maintenance burden of wrapping pthread_create in a new function that creates threads, and if needed calls the callback to expose the handle is practically none. I'd need to lookup the specifics of copying thread handles though (on Linux this is just an int, so safe of course), not sure what the pitfalls might be. Ideally not Linux only, but that would also work I guess.

I guess copying should be fine, I guess using the callback to give a pointer to the pthread_t object would be the simplest, the burden of copying it is then on the calling application if it wants to keep it around.

Doing anything special with it, i.e: cleanup handlers would (for now) be unsupported.

That sounds very reasonable. Should this callback be attached to a rist_ctx with a separate API call or do we want an additional argument to rist_{sender/receiver}_create()?

Adding an argument is not backwards compatible ;)

And this is something that very few would use, so versioning to a new API and deprecating the old does is not warranted ;)

What I've been mentally toying with though is a generic rist_set_value API, perhaps something like this:

int rist_set_value(struct rist_ctx *ctx, enum value_type type, void *val1, void *val2)
{

}

That would be extendable in the future, keeping the API surface smaller.

I agree. I like the idea of a generic set_value api. Because this sounds like two different tasks i will open 2 issues, a merge request for the thread-callback stuff and close this one.

This check is incorrect, pthread_t is not guaranteed to be an int (posix does not specify it as such) and could be a struct for example.

Good catch!

closed

mentioned in issue #130 (closed)

mentioned in issue #131 (closed)