If someone wants to send me a check for a few thousand $$, we'll run both tests...
We're well aware of the limitations of various protocols. I'm not an aerodynamicist, I'm a journalist; that means I'm not an expert, but I'm good at finding experts and using their expertise to make sure we do things properly.
Frankly, we didn't go with a mannequin last time because it wasn't ready, and a bad mannequin is far worse than no mannequin. I'm debating going with one for the next test. The positive is that it (probably) provides a better look into what's really happening, the negative is that we can't compare to previous tests (though we were always reticent to do so anyway, as small conditional changes can have an impact).
For what it's worth, our non-mannequin test showed very similar results to both Giant's mannequin testing and Cervelo's mannequin testing — looking at the relative strength/weakness of each model. So I wouldn't call a non-mannequin test useless.
This is also why we don't use the Tour "numbers only" model. Every bike gets ridden, and ridden a lot. In our scoring, quantitative and qualitative are equal. It's not uncommon for the best bike in the lab to lose overall — as the S5 did last year. It won handily in the wind tunnel, but it rides like a block of wood. The Propel lost significant points for its stupid brakes this year. The segment of consumers that cares only about wind tunnel results is a very small one indeed. We try to balance it all, and ask the question "which one would we buy?"
Frankly the more "Real World" conditions for the test the better. That means testing with a pedaling mannequin , 2 bottles and cages. Would also be great to see a standard round tubed bike tested in the same conditions to see what the real benefit of an aero frame.