What Business Is It Of Congress?

Ex-CIA Director General David Patraeus's affair has stirred up congress.  They want to know why they were not told.  They probably want to know the details and see any pictures taken too.

As I see it, that was General Patraeus's privae affair (literally) and is nobody's business but his, his mistress(es), and his immediate family.  Shouldn't congress get their mind out of the dirt and get back to more important matters, like balance the budget?

What if James Bond was not part of the British Secret Service but a CIA Agent instead.  Everybody knows Bond has plenty of affairs while working on a case.  Would congress want to know all about them too?