Since you’re hearing the noise in CD mode, that means it’s likely not being radiated and picked up by the antenna. Probably related to +12V or ground connection.
Your radio likely has two +12V connections - a red accessory and a yellow battery. The amp has a battery connection, a ground and a trigger that connects to the radio (commonly a blue or blue/white wire at the radio).
The conditions you described as causing the noise may, or not be related.
What brand/model radio and amp are you using?
Easy things to check:
Radio and amplifier ground connection are secure and there is no paint or rust at the grounding bolt. Ideally, the radio and amplifier are grounded at the same point.
The amplifier +12 connection is directly to the battery - be sure to fuse this connection as close to the battery as possible.
I’m guessing you’re using a relay to power your horn...?
If so, one commonality between the horn and starting the engine is a relay/solenoid. Both use a coil of wire to create an electromagnet when powered. These devices create large voltage spikes (positive and negative) on the +12V circuits. These spikes commonly cause pop noises in audio systems. So first thing to check would be your connections as noted above.
Also, does the radio and amp turn off when the ignition key is turned to the Start position?
If no, this may be the reason for your engine crank pop noise.