They were stuck. I flushed all the rules and set the policies to ACCEPT and still contrack was tracking connections and that IP was getting blocked somehow.
I suspect unloading the netfilter contrack modules would have resolved it but I wasn't willing to do that until off hours and since I had to wait until off hours I just rebooted to be on the safe side and the problem went away and has not returned.
I'll probably never know exactly what went wrong but I definitely will be more careful in the future when removing rules.
John