If I had OS X 10.4 Tiger, I could set up my kernel to do packet prioritization, but since I'm still using 10.3 Panther, I had to come up with something else.
Fortunately someone else had already done the hard part and created throttled. It is a user-space daemon that sucks all the packets in using the kernel firewall divert rule and sends them according to a few rules that make life a bit easier.
I can divide traffic to high and low priority and it also gives highest priority to packets containing the ACK bit, which has the benefit of keeping your downlink live and well even if your uplink is congested.
I also reviewed the code of throttled and having just spent the last 3 years working on a pretty tricky piece of threaded software, I discovered some thread-locking related problems in the code.
I created a patch to fix them and sent it to the authors of throttled.
Here's the email I sent to them:
Aihe: Patch for throttled
Päiväys: 1. helmikuuta 2007 22:37:12 GMT+02:00
Hello. I have been using throttled for a week now and it really works!
I was curious to see how it does its magic and took a look at the source code. I think I discovered a small problem that could lead to the sender thread going to sleep and never waking up, or at least until throttled gets a new outgoing packet.
The problem is that the size of PacketQueue is checked without holding queueMut mutex. This means that sender thread may check PacketQueue size at a moment when it is empty and decides to go to sleep. But then, just before sender calls pthread_cond_wait, scheduler runs one of the receiver threads, which places a packet into the queue _and_ calls pthread_cond_signal. This signal does nothing, because there is no-one sleeping on the condition variable. Then scheduler starts running the sender thread again, which calls pthread_cond_wait too late, it missed the signal to wake it up.
Sure, the sender gets woken up when the next outgoing packet causes receiver thread to call pthread_cond_signal, but sometimes that might take a long time.
I have made the attached patch to fix the problem. I hope you consider merging the patch to the next version of throttled.
-- Sami Tikka
I never heard back from them. Because I'd like others to also be able to enjoy this amazingly useful piece of software, I'm making the fixed version and the patch available.