Look on the Sheldon Brown website. http://sheldonbrown.com/cables.html
There are simple explanations on how to set cable length.
Unfortunately, there is no "formula" and the exact length is judgement-based, and two cabling jobs on the same frame might be slightly different, but both excellent in that they are within the parameters of "proper cable housing length". Also, aesthetics comes in to play in that some of us will try to minimize housing length while still adhering to the principles.
Two general principles that I think are emphasized in Sheldon's articles and in my own experience are: (like someone else has already said), make the curves as sweeping as possible, but no longer than necessary. Too tight = too much friction. Excessive length = too much friction. But there's some leeway of course in what functions well. This is where my personal aesthics happens to come into play. I will fudge on the side of shorter housing length - as much as I can while still maintaining good smooth functioning - just because I like the looks of smaller loops better. If I get a bike (either new or used) that looks to me like it's got excessve housing lengths, I'll take it apart and shorten them.
I think that many shops err on the side of excessive housing length for a couple of reasons - one, it's less likely to cause problems than too-short length and two, they just go with whatever cable "set" they have, if they happen to be using a set, and don't trim it. My shop/factory built tandem, and my wife's small-framed shop/factory built road bike both had ridiculously long cable housings. Some are several inches longer than they need to be and actually violated the second principle (read on!).
The other general principle is, wherever a cable ends a sweep and sets into a cable housing stop, it should be going straight in - should have finished its sweep so it's going directly in line with the cable stop before it actually enters it. If it's too long or too short, it will tend to insert into the stop at an angle causing excessive friction.
As an example of this, many people cut their rear derailleur housing too short so that it comes into the derailleur cable stop at an angle. Sram RD's , compared to Shimano, need quite a large loop to accomplish this and they look odd to people who are used to the fairly tight loop that can work with Shimano. This is fairly subtle to look at and see and doesn't smack you on the side of the head as being wrong unless you're aware of it. (look closely at the photos in Sheldon's articles focusing on the actual place the cable housing enters the stop).
Anyway, there's no straight formula, you need to understand the simple principles as described in Sheldon Brown's article (and others, of course) and then cut your cables as you judge - probably erring on the long side and then trimming if you're not happy with how it looks.