Assuming you have the best of everything, you then will also have the money to re-spec your skills the way you want later. So the best dueling clothes are those that give you the most dueling skill points, doesn't matter which skills they are. I don't count the extra health points as a serious dueling skill.
I would give 1 point for aim, dodging and either vigor or shooting in both offense and defense. Toughness and reflex are good for either 1/2 point or 1 point (depending on your opponent which you can't chose on defense) so give them each 3/4 of a point in both offense and defense. Appearance and Tactics get 1 point in either offense or defense but not both.
So for Melee dueler, fancy felt hat gives you 19 3/4 pts on offense and 27 3/4 for defense. The fancy leather hat gives you 28 3/4 for both offense and defense, clearly better. Even the trusty black felt hat is slightly better on offense than the fancy felt, at 20 pts.
The melee footwear has the fancy riding boots against the brown boots. The fancy riding gives 33 for offense and 25 for defense. The brown give you 20 for offense and 30 for defense, so you might want them both.
The melee neckbands have: brown shawl (at 23 offense, 10 defense) the Mexican bandana (at 17 for either, though as part of the Mexican set gives you between 18 3/4 and 20 1/2 for either) and the golden buffalo (at 16 1/2 for either offense or defense). I have the first two and don't see much reason for the buffalo, as it doesn't help me for any more jobs now either.
Melee clothing has the fancy dress coat hands down at 30 1/2 pts for both offense and defense.