On Wednesday, August 27, 2014 10:51:03 PM UTC-5, Les Cargill wrote:
> John Bode wrote:
> > On Thursday, August 21, 2014 11:22:17 PM UTC-5, naobi wrote:
> >> please give me some tips how to start programming using threads in C....
> >> What what do i need to learn first...
> >
> > First, ask if the problem you're trying to solve really needs a multi-
> > threaded solution. You'd be surprised how often the answer is "no".
> >
> > Second, take time to learn about deadlocks and race conditions, how they
> > happen, and how to avoid them *before* learning any specific threading API
> > or toolkit. Too many of us just dive right in and start writing
> > code without really understanding the concepts first. This is bad enough
> > for single-threaded code, but for multi-threaded code it can be deadly.
> >
>
> +1
>
> > As for specific threading libraries or toolkits, that depends on your system.
> > My experience is limited to pthreads on Linux, and my experience isn't that
> > extensive. I'm currently working on a multithreaded program that periodically
> > requests status from several hundred running processes and sends the results to
> > a Xymon monitor.
> >
>
> I bet that can also be done with one thread...
>
Yeah, and given the issues I've had over the last couple of weeks, I'm
starting to lean that direction.
It's just going to be painful. Believe it or not, the threaded solution
was easier to visualize and implement, given the framework I'm using (I'm
trying to leverage our existing code as much as possible). The non-threaded
solution means I won't be able to use as much of the existing framework, and
will have to write my own non-blocking I/O.
The multi-threaded solution is finally stable, I'm just not happy with
the memory usage (specifically the resident size). It's not *bad*, but it could be a lot better.
[snip]
>
> Actually, I think you do. Are things not buffered to where you lose things?
>
It's more that these pings *have* to go out like clockwork; many of them
act as keep-alives so that the remote host doesn't drop a connection on us.
The threaded solution means I can always service a new request off the queue,
regardless of whether the previous request has completed or not.
On 8/25/14, 1:02 PM, James Kuyper wrote:
> I've no relevant experience, and only a limited amount of theoretical
> knowledge of this subject. However, my understanding of the reason why
> you "*should* use a single mutex for both" is that if you have one
> thread that does:
>
> lock m0 -> lock m1 -> unlock m1 -> unlock m0
>
> and a second one that does
>
> lock m1 -> lock m0 -> unlock m0 -> unlock m1
>
> they can get into a deadlock. When you have a small number of resources,
> it's probably straightforward to keep track of and avoid such problems,
> by never using both patterns in the same program. But doesn't that get
> too complicated when you have a large number of resources and many
> different combinations of them that you might want to lock?
>
The normal rule that I learned to prevent deadlocking with multiple
primitives is that if you can order your locking primitives so that
whenever you attempt to lock on a primitive, you are only holding on to
"lower priority" primitive, then you will be deadlock free.
This rule can even be inserted into your OS, so it will give you a run
time error when you attempt to violate it.
When you have a large number of things to lock, it can become a
challenge to find the proper "order" to place them in to met this
requirement, but failing to be able to find one is a good indication
that deadlocks are possible.
On Thu, 2014-08-28, Les Cargill wrote:
> Jorgen Grahn wrote:
>> On Wed, 2014-08-27, Les Cargill wrote:
....
>>> This is why you don't necessarily want to use APIs that block.
>>
>> What's especially frustrating is APIs which wouldn't /have to/ block,
>
> None actually ever have to...
>
>> but do unconditionally anyway because the author is lazy, or doesn't
>> understand the varying needs of the users of the API.
>>
>> Even if you're not against threads,
>
> They certainly have their place. All other things being equal, a
> non-threaded solution is usually superior. But as concessions to
> agonies like this go, threads ain't a bad one.
>
> > you may want to e.g. (Unix) select
>> on a file descriptor so you can have timeouts, or handle two related
>> file descriptors in one thread of execution.
>
> But you can't do that ( SFAIK ) unless you have something that 1) maps
> that to file descriptors, and 2) allows configuring said FDs as
> nonblocking.
Precisely. But /often/ an API can give you that freedom.
One case I see a lot is subscribing to messages from some other
process, where the actual implementation is
a) a TCP connection (or some kind of socket)
b) a way to choose the right endpoints for it (hide IP addresses
and port numbers from the user)
c) deserialization of the data
This is often hidden behind a blocking call
msg = wait_for_foo_message(foo_connection);
or a callback setup:
register_foo_callback(foo_connection, my_foo_callback);
But an API could instead choose to expose (b) and (c). Then I could
manage the I/O in any way I see fit, and they can control the rest.
> If nothing else, sacrifice a thread, make *it* block then use that with
> a nonblocking mechanism. Socket servers over localhost work just peachy.
> They are a boatload 'o code but them's breaks.
/Jorgen
--
// Jorgen Grahn O o .
John Bode wrote:
> On Thursday, August 21, 2014 11:22:17 PM UTC-5, naobi wrote:
>> please give me some tips how to start programming using threads in C....
>> What what do i need to learn first...
>
> First, ask if the problem you're trying to solve really needs a multi-
> threaded solution. You'd be surprised how often the answer is "no".
>
> Second, take time to learn about deadlocks and race conditions, how they
> happen, and how to avoid them *before* learning any specific threading API
> or toolkit. Too many of us just dive right in and start writing
> code without really understanding the concepts first. This is bad enough
> for single-threaded code, but for multi-threaded code it can be deadly.
>
+1
> As for specific threading libraries or toolkits, that depends on your system.
> My experience is limited to pthreads on Linux, and my experience isn't that
> extensive. I'm currently working on a multithreaded program that periodically
> requests status from several hundred running processes and sends the results to
> a Xymon monitor.
>
I bet that can also be done with one thread...
> The biggest mistake I make on a regular basis is to create overlapping critical
> sections. I want to synchronize access to a queue and to a log file, but I
> want them to be independent, so I create separate mutexes for each. This works
> great if the locks are completely disjoint:
>
> lock m0 -> unlock m0 -> lock m1 -> unlock m1
>
> or "scoped":
>
> lock m0 -> lock m1 -> unlock m1 -> unlock m0
>
> However, if the locks and unlocks are allowed to overlap, like this:
>
> lock m0 -> lock m1 -> unlock m0 -> unlock m1
>
> then you are *guaranteed* to create a deadlock at some point. I *should* use
> a single mutex for both, but I don't want thread A to block on grabbing a new
> job off the queue
Actually, I think you do. Are things not buffered to where you lose things?
If you're really that memory constrained, you may need to reconsider a
lot of things.
> while thread B is writing to the log, so I have to be careful
> that those locks don't overlap.
>
But if you unlock the job queue before grabbing the log mutex...
I really feel like having many resources locked at once ( really, more
than *one* ) is a real challenge to test, and to be avoided if at all
possible.
Just do one thing, then the other. Use private buffers if you can or
have to. Life is hard enough doing one thing at a time...
Also, I forget - do mutexes in the pthreads world have a nonblocking
version with an error code? Aha!
int pthread_mutex_trylock(pthread_mutex_t *mutex);
"pthread_mutex_trylock behaves identically to pthread_mutex_lock, except
that it does not block the calling thread if the mutex is already locked
by another thread (or by the calling thread in the case of a “normal” or
“errorcheck” mutex). Instead, pthread_mutex_trylock returns immediately
with the error code EBUSY. "
https://www.sourceware.org/pthreads-win32/manual/pthread_mutex_init.html
*If I read that right*, use of that alone should enable you to never
deadlock. You may need to consider fault conditions but that's better,
IMO.
> Another issue I'm fighting is that each thread has its own stack and takes
> up some amount of virtual memory, meaning my monitor program has a fairly
> heavy memory footprint (heavier than I'm comfortable with, anyway, as it
> can occasionally spawn as many as 40 or 50 threads at a time depending on
> how congested things get).
>
I have to think you can reduce that significantly. If n threads are
simply running the same entry point*, you can declare control blocks
as arrays and index them as-if they were a "thread". We used to
jokingly call this "writing your own OS" but it fixed some things in
some cases.
* int pthread_create(pthread_t *thread,
const pthread_attr_t *attr,
void *(*start_routine) (void *),
void *arg);
"start_routine" is the entry point.
Then again, this may be worse. It's nearly certainly less readable -
although there's always something you can do.
> Most of these issues stem from my inexperience, both with mutithreaded
> programming in general and with pthreads in particular. But if nothing else,
> it gives you an idea of what sort of pitfalls lay before you.
>
--
Les Cargill
Jorgen Grahn wrote:
> On Wed, 2014-08-27, Les Cargill wrote:
>> Nobody wrote:
>>> On Fri, 22 Aug 2014 08:49:18 +0100, Richard Heathfield wrote:
>>>
>>>> Threads are for people who can't get the hang of state machines.
>>>
>>> That's like saying that high-level languages are for people who
>>> "can't get the hang of" assembler.
>>>
>>> Apart from the (possibly substantial) effort involved in manually doing
>>> what's normally the compiler's job, state machines are typically not an
>>> option if the task requires the use of any third-party API functions which
>>> can block.
>>>
>>
>> This is why you don't necessarily want to use APIs that block.
>
> What's especially frustrating is APIs which wouldn't /have to/ block,
None actually ever have to...
> but do unconditionally anyway because the author is lazy, or doesn't
> understand the varying needs of the users of the API.
>
> Even if you're not against threads,
They certainly have their place. All other things being equal, a
non-threaded solution is usually superior. But as concessions to
agonies like this go, threads ain't a bad one.
> you may want to e.g. (Unix) select
> on a file descriptor so you can have timeouts, or handle two related
> file descriptors in one thread of execution.
But you can't do that ( SFAIK ) unless you have something that 1) maps
that to file descriptors, and 2) allows configuring said FDs as
nonblocking.
If nothing else, sacrifice a thread, make *it* block then use that with
a nonblocking mechanism. Socket servers over localhost work just peachy.
They are a boatload 'o code but them's breaks.
>
> /Jorgen
>
--
Les Cargill
--
Les Cargill
Mega Search
Sign Up 
