Obviously the IDE channel is going to operate at the speed of the slowest device on that channel.
Since you have HDDs on the primary and the NEC was added only to the secondary, this should not cause any speed degradation while booting up.
FWIW, you can try setting the jumper on all devices correctly instead of leaving them at Cable Select.
This may/may not improve your situation - but won't hurt to try. Hey if it gets worse, you can always go back
Are you sure the difference is significant ?
Did you try removing the NEC drive and try booting again - did that make it as fast as it was before ? If you have not, it would be interesting to try it.
My sense of how fast my system performs is something that changes rapidly over time