Color Is Hard: Don't Mix Khaki and Black

I grew up with the understanding that khaki went with just about everything.  As a young man unschooled in paying attention to these things, I accepted this as gospel truth.  Now, I've moderated my position a bit because... 

Khaki does not go with black.

I'm not quite sure what the problem is, but the combination of khaki slacks and black shoes and/or a black shirt just strikes my eye as dissonant.  Maybe it's the fact that khaki does not have any element of black in it (unlike black shoes' best friend, gray wool).  Maybe it's that black shoes suggest Serious Business, while khaki slacks suggest Business Casual.  I'm spinning my wheels trying to think of a situation (other than a dance club) in which I'd wear a black shirt.

I'm not ready to call "Faux Pas" on this combination because so many guys do it.  But we do better for ourselves when we wear black shoes with pants with some black in them and brown shoes with pants with some brown in them.