Journalist Ronald Kessler's In the President’s Secret Service: Behind the Scenes with Agents in the Line of Fire and the Presidents they Protect seems like a real page turner. He interviewed about 100 members of the Secret Service going back to the Kennedy administration, asking them about their impressions of the Presidents they protected. Here is a summary:

Kennedy - serial adulterer
Johnson - serial adulterer and a lunatic
Nixon - strange and  unsociable
Ford - friendly but cheap
Carter - a phony
Reagan - friendly, concerned, apologetic for taking them on trips away from their families
Bush - decent and thoughtful
Clinton - liked but inconsiderate; "Hillary did not speak to us."
Bush 43 - in person not uncomfortable like he is in front of a microphone, down to earth
Obama - personally a decent man