Just to throw my hat in the ring, as has been stated, it must be a 'Precise Pepperbox Revolver' that you own and - to have him defeat you - strip completely naked - no weapons, no clothes, nothing - and change your stance to stand upright every round and also change your aim to either aim for his hands to do less damage, or aim for his head and have less chance of hitting him.
It is possible that you will still defeat him, (he is really weak), but just keep trying. It doesn't cost you anything and eventually he will win the battle and that part of the quest will be completed.