If you don't want to deal with them, fine. But don't hamper other people from dealing with them.
Directing remains very psychological, and it takes a lot of time and reflection. When you're an actor, it takes less time, and you can express yourself physically.