It isn't that being "religiously unaffiliated" or not decides directly how you vote. It is the way the people who ask you to vote for them act in regards to religion. Unfortunately in the US we are forced to decide from only two options. One option has a long and recorded history of not just acting sympathetic to the strongly religious, which wouldn't be offensive, but of making their policy based on religion.
Even that wouldn't be much of a problem if they didn't always seem to choose religious tenants that have nothing to do with compassion or empathy. There is a strong focus on silly rules the breaking of which cause no harm to anybody. Take the "issue" of homosexuality. How does two people you will never meet having sex affect your life? It doesn't. There is no possible way it can. However there are religious people who insist on reducing homosexuals to 2nd class citizens because they are doing one thing the religious disagree with.
Now if you say cheating others of their money is wrong I am all with you. I don't care if your justification is that you believe it is harmful to people and shouldn't happen or if your God says its wrong.
As to the topic of a ban of religion in government. Those who would support such a thing are referring to my first example of the discrimination against homosexuals. If the only justification someone can make for a law is that god says it is wrong, then it has no place in public policy.