Fame is a delicate and dangerous creature; I saw people who didn't honor it, who refused to take responsibility for it, get destroyed by it. I also saw that stardom in and of itself was empty.
Another factor is the decision, made in 1976, to sharply divide the FBI and the foreign intelligence agencies. The FBI would collect within the United States; the foreign intelligence agencies would collect overseas.