This is another one or those questions that never goes away and about which reams of rubbish have been written.
In Letter to a Christian Nation (Knopf, 2006), in the section Are Atheists Evil?, Sam Harris writes:
If you are right to believe that religious faith offers the only real basis for morality, then atheists should be less moral than believers. In fact, they should be utterly immoral. (pp. 38-39)
Harris then goes on to point out something that I don't doubt is true, namely, that atheists ". . . are at least as well behaved as the general population." (Ibid.) Harris' enthymeme can be spelled out as an instance of modus tollendo tollens, if you will forgive the pedantry:
1. If religious faith offers the only real basis for morality, then atheists should be less moral than believers.
2. Atheists are not less moral than believers.
Therefore
3. Religious faith does not offer the only real basis for morality.
The problem with this argument lies in its first premise. It simply doesn't follow that if religious faith offers the only real basis for morality, then atheists should be less moral than theists. This blatant non sequitur trades on a confusion of two questions which it is essential to distinguish.
Q1. Given some agreed-upon moral code, are people who profess some version of theism more 'moral,' i.e., more likely to live in accordance with the agreed-upon code, than those who profess some version of atheism?
The answer to this question is No. But even if the answer is Yes, I am willing to concede arguendo to Harris that it is No. In any case (Q1) is not philosophically interesting, except as part of the run-up to a genuine philosophical question, though (Q1) is of interest sociologically. Now contrast (Q1) with
Q2. Given some agreed-upon moral code, are atheists justified in adhering to the code?
The agreed-upon code is one that most or many atheists and theists would accept. Thus, don't we all object to child molestation, wanton killing of human beings, rape, theft, lying, and the swindling of investors by people like Bernard Madoff? And in objecting to these actions, we mean our objections to be more than merely subjectively valid. When our property is stolen or a neighbor murdered, we consider that an objective wrong has been done. And when the thief or murderer is apprehended, tried, and convicted we judge that something objectively right has been done. Let's not worry about the details or the special cases: killing in self-defense, abortion, etc. Just imagine some minimal objectively binding code that all or most of us, theists and atheists alike, accept.
What (Q2) asks about is the foundation or basis of the agreed-upon objectively binding moral code. This is not a sociological or any kind of empirical question. Nor is it a question in normative ethics. The question is not what we ought to do and leave undone, for we are assuming that we already have a rough answer to that. The question is meta-ethical: what does morality rest on, if on anything? For example, what justifies the shared judgment that the swindling of investors is morally wrong? Or rape, or wanton killing of people, or slavery?
There are different meta-ethical theories. Some will say that morality requires a supernatural foundation, others that a natural foundation suffices. Here you can read the transcript of a debate between Richard Taylor and William Lane Craig on this topic. I incline toward the side ably defended by Craig. Although I respect Taylor very much as a philosopher and have learned from his work, he seems to me to come across in this debate as something of a sophist and a smart-ass.
But the point of this post is not to take sides on the question of the basis of morality, but simply to point out that Sam Harris has confused two quite obviously distinct questions. For if he had kept them distinct, he would have seen that the question whether morality requires a basis in religion is logically independent of the question whether theists are more moral than atheists. He would have seen that invoking the platitude that atheists can be as morally decent as theists has no tendency to show that morality does not require a supernatural foundation. He would have seen that (1) is false.
I should add, however, that while the two questions are distinct, they are related. For if people come to believe that there is no justification for the agreed-upon moral code, then they will be less likely to adhere to it. Or suppose a person thinks that the justification for the prohibition against rape is merely prudential: it is imprudent to rape women because you may get caught and go to prison. If that were the justification, then a man without a conscience who encounters a defenceless solitary woman in an isolated place would have no reason not to have his way with her. But if the man had the belief that the moral wrongness of rape is grounded in the holy will of an omniscient deity, then the man would have a reason to resist his inclination.