Perhaps this is unexpected, but I like a good t-shirt for casual wear.
TeeMagnet is all you really need. Six to eight new shirts per day. I probably buy 1 out of every 50. Some of what you pick is up to your personal aesthetic, but think about it like a tie in terms of coordination with your pants. A shirt with a hint of blue in it will link up to the color of your jeans; a navy shirt with jeans will tend to look like a leotard. On the other hand, a blue shirt with chinos will look good.
If you have a jacket or sweater that picks up another color in the shirt, so much the better. Bonus points if your shoes match the mix as well.
Some background colors that often look good with jeans: orange, red, green, tan.
Apart from color, you should pay attention to design. Small designs look funny--you want it to reach mostly across the chest. In the alternative, watch for asymmetrical designs. They often look fabulous.
Finally: no MMA themes allowed. If you are at a bar wearing an Affliction/TapOut/what-the-&%#-ever t-shirt covered with garish skulls and crosses and gothic lettering, it's your own damn fault.