BigPrince, are you running a late-model PMGR ("mini" type) starter by any chance? If so, that's a common issue it seems.
Let me know and we can walk through the possible solution. You can do some searching too, but I'm not sure what the keywords might be for that. "Mini starter" maybe? "starter run-on"???
Paul