In a way, a certain amount of self-criticism is a good thing, because it keeps you humble. Realizing that no matter what success you've achieved, you can still make enemies makes you humble, too.
Unless you're trying to make a movie on the sly, there's no way to get around this. If you want to use public spaces, film on the streets, have the cooperation of the police, you have to have a permit.