albatross around one's neck in French

(idiomatic, figurative) A characteristic, possession, associate, or previous deed that hinders one's ability to succeed.