The best way to gain a great reputation with search is to produce great results. So staying away from prospects that you can't generate positive results for is actually a boon to them, and to you. Eric Enge serves up several scenarios for when it would be best to turn a potential client down, starting with a business that's too small.
While Enge isn't saying that small businesses can't succeed with search, sometimes the amount they spend for a search firm or consultant will outweigh the potential gains--even if they snag top rankings for all chosen keywords and top placements for all of their PPC ads. If the ends won't justify the means, be straight up with the prospect and step back.
Other possible clients may not be able to meet the content demands (due to either lack of time, resources or desire), and still others may want results at any cost. Being forced to veer off into black-hat territory (and potentially get their site banned) is bad for them and for your reputation.