The bedrock of our democracy is the rule of law and that means we have to have an independent judiciary, judges who can make decisions independent of the political winds that are blowing.
The two things that can hurt you are if you need money or if you need fame. Those are the things that can be your Achilles heel. But if you don't need money and you don't need fame, then you're free.