Disconnect the two small wires (Red w/blue and a Brown one) from the solenoid. Re-connect the battery and test for voltage at either one. NEITHER one should have a reading with the key "OFF". With the key "ON", the brown wire should have a reading of anywhere from 6 to 12 volts (depending on the condition of the resistor wire, if applicable).
If the Red w/blue wire has any voltage whatsoever with the key "OFF", either your ignition switch is bad or you have a short in that circuit somewhere. There should only be a reading when the key is in the "START" position.
If no reading at the Red w/blue wire, then it's probably a stuck solenoid. You can try hitting it with a hammer as was mentioned, but even if you fix it I would go buy a new one since this one may do it again at any time. You can keep is as a spare if you want, but I'd just toss it and get another one to avoid the headaches down the road.
Just out of curiosity, have you done any other modifications to the electrical system? Such as an internally regulated alternator? If so, sometimes you can get feedback through the system that won't allow the engine to shut off, and it won't usually effect the starter circuit, but I was just wondering about any other mods you might have done in the meantime.
Paul