A naughty step/corner - make them stand without talking or being silly for 2 minutes. If they talk, then the 2minutes starts over. It can be tough sometimes, but don't give in, otherwise they won't learn!
Tbh it sounds to me as if you are dealing with your son in exactly the right way. The whole poing of punishment is that it is a consequence of bad behaviour and yes it is going to be upsetting for your child. I know Freddie is younger but we have started dealing with his misbehaving in exactly the same ways you do. And yes it does involve tantrums and screaming at times - I don't know that there is any way around those. I just ignore them and get on with things and let him work it out himself. I do explain to him why he is being punished etc but there really is nothing that will stop him venting his frustration.
In all fairness you shouldn't have to change the way you are rearing your child just because it's bugging your neighbour (she sounds like a right stroppy cow) especially when you are clearly doing a great job.
What is important is the well being of your son... SOD your neighbour!