According to millward:
That would depend on your particular hardware, and your intended usage...
I don't know that to be true in any of the newer systems, but it may have been the case with some of the earlier BIOSes. Many of them didn't implement the CD-ROM booting feature in a fully standard way, so there could be such limitations with some systems.
I'd try it and see. In your case, it may be best to leave it as the primary slave, and make each of your two hard drives masters on each of the two IDE controllers.
Linux uses device names based on the (fixed) hardware configuration for IDE devices, as follows:
hda primary master hdb primary slave hdc secondary master hdd secondary slave ...
So, removing a slave device wouldn't affect anything. If you removed a master device, you might have to turn it's corresponding slave into a master, which would change names. However, there isn't much in Linux that depends on these hardcoded names anymore. You might have to fix /etc/fstab, and either /etc/lilo.conf or /boot/grub/grub.conf, depending on your loader.
But, if you make your CD-ROM drive a slave (primary or secondary), you should be able to remove it without changing anything. Spreading your two hard drives across the primary and secondary controllers (as masters) will give you a slight performance advantage over having them both on one controller (as master and slave), since the two controllers and drives can then overlap I/O operations.
The CD-ROM drive would slow down disk I/O on the controller it's tied to, but if you'll be removing it later, that would be fine.