That's partially true; a secret isn't necessary something kept because the person keeping it did it. To jog your memory, many Black families have lots of generational secrets kept by family down through the years. Or folk keep secrets that others do, to keep from allowing children or spouses to be hurt by infidelity on the part of someone else. I concur that people who committed acts that are kept secret are more than likely things they don't want aired out.
As for the CIA, FBI, and et al... I think the President puts too, much trust in the agencies that are suppose to protect citizens; President Obama has a Utopian kind of view of the world whereas the folk who work for the CIA / FBI hide behind their authority and abuse our liberties under the guise of Terrorism protection.