Dan... you're freaking me out dude. You're describing testing that I personally ran over the course of 5 years, 7 events, hundreds of dyno pulls, dozens of mods, dozens of cars, and dozens of 1/4 track runs. A "test" consisted of 3 dyno pulls, a mod swap, and three more pulls. The car wasn't removed from the dyno. The mod was made in place, as close in timing as was possible. We called these tests the "MFOs"... Modifications Face Off.
Hmm...not sure if me "freaking you out" is a good thing or a bad thing (tough to tell when it's all typed text).
Where did you do your dyno work? When I was with the Porsche Boxster group I co-founded it was up in they Bay Area. I'm trying like hell to remember the name of the dyno place (and can't now) but it was in Mountain View.
EDIT: OK, I found it. It was Dyno Spot Racing (see below).
We spent a lot of time and money there and a lot of time and money on mods and most showed gains that were so small they could've been due to testing error and/or changing conditions.
That's why I'm pretty unexcited about CAIs and other bolt-on mods. Once I consider the questionable returns, the costs, the headaches of removing perfectly good parts AND the dealership issues when going in for service and warranty work I'm not going to bother.
I don't want to sound critical of others who do these mods. I figure I went through it ONCE and I let everyone else do it so they can learn too
EDIT (continued): Dyno Spot Racing now in San Jose, CA
When we were spending time there some years back they were working with GIAC chips/tuning which was one of the mods we did.
From their website (just an FYI - not an endorsement of any kind):
Dynospot Racing, Inc. Your Service Repair Experts for European and Japanese Imports
Dynospot Racing, Inc. is a true'bricks and mortar' business located in the heart of Northern California's ultra high-tech Silicon Valley that specializes in tuning and service of Audi, BMW, VW, and Honda. We actually own a Dynojet 248H dynamometer (Click Here for Dyno Charts) and develop and produce performance parts ourselves. We, at Dynospot Racing, Inc., are extreme performance enthusiasts like you. We love and have a passion for what we do and would gladly like to help you meet your needs with what we know. We also specialize in automotive performance parts.
Our Mission
Dynospot Racing, Inc. was started with the sole purpose of providing automotive enthusiasts with the best possible service and repair available on the market today, for your European Import or your Japanese Import. Here, at Dynospot Racing, Inc, we continue to carefully choose each product that we carry using price, quality, and “true” performance gains as our main criteria. When we are not satisfied with the available items offered on the open market, Dynospot Racing, Inc. often develops its own products. Such is the case with our Dynospot “S” and “R” series Race Cams and Dynospot “G-FLOW” MAF (Mass Air Flow) conversion kits. Part of Dynospot Racing’s mission is to stress to the public the concept of tuning, making the parts that you have work together. Remember, there are no generic performance recipes in the car-tuning world. Don’t find this out the hard $$$$ way.