Duellers attack workers anyways, even if they don't know their stats since ninja workers are very rare. Its not like not putting that signature will prevent any duels. Additionally, its the builder's choice if he wants to show his skills/stats/hours/whatever, there's no compulsion, so if the player is ready to face the duellers against it, he should be allowed to do so.