Yep no difference...both get awful character bonuses...
Even if you are going to be a builder...choose a different class...long term you'll regret being a builder class because long term the bonuses become redundant.
Consider going the solider route, you get a 50% HP bonus...which for a worker mounts up to a lot resulting in you being a more effective fort fighter
and you also get your weaponry somewhat earlier which will maybe help defend your honour from those worker hitting duelists...maybe