Nobody can deny the fact that Christianity has played a huge role in our history. From the first Thanksgiving to the ideas of Jesus Christ that are embroidered in our culture today, Christianity and the Bible is responsible a big part of our heritage.
However, many conservatives will take this fact way out of context. They'll think that you have to be a Christian to be patriotic, which is simply not true. Following the more secular teachings of Jesus Christ (being charitable, loving one another, treating strangers with kindness) is what the men who founded this country were for.
I don't want to waste my time listing all these obscurant far-right arguments, so instead I'll list the facts straight from our forefathers.
“If I could conceive that the general government might ever be so administered as to render the liberty of conscience insecure, I beg you will be persuaded, that no one would be more zealous than myself to establish effectual barriers against the horrors of spiritual tyranny, and every species of religious persecution.”- George Washington, letter to the United Baptist Chamber of Virginia (1789)
“Question with boldness even the existence of a God; because, if there be one, he must more approve of the homage of reason, then that of blindfolded fear.”- Thomas Jefferson, letter to Peter Carr (1787)
"In regard to religion, mutual toleration in the different professions thereof is what all good and candid minds in all ages have ever practiced, and both by precept and example inculcated on mankind.”- Samuel Adams, The Rights of the Colonists (1771)
“Persecution is not an original feature in any religion; but it is always the strongly marked feature of all religions established by law. Take away the law-establishment, and every religion re-assumes its original benignity.”- Thomas Paine, The Rights of Man (1791)
“Congress has no power to make any religious establishments.”- Roger Sherman, Congress (1789)
"The way to see by faith is to shut the eye of reason."- Benjamin Franklin, Poor Richard's Almanack (1758)
"I contemplate with sovereign reverence that act of the whole American people build a wall of separation between Church & State."- Thomas Jefferson, letter to the Danbury Baptists (1802)
"To argue with a man who has renounced the use of reason is like administering medicine to the dead."- Thomas Paine, The American Crisis No. V (1776)
Note: You can read Paine's whole pamphlet, where he expresses his atheistic beliefs, here.
“Our civil rights have no dependence on our religious opinions, any more than our opinions in physics or geometry.”- Thomas Jefferson, A Bill for Establishing Religious Freedom (1779)
"Christian establishments tend to great ignorance and corruption, all of which facilitate the execution of mischievous projects."- James Madison, letter to William Bradford, Jr. (1774)
"There is nothing which can better deserve our patronage than the promotion of science and literature. Knowledge is in every country the surest basis of public happiness."- George Washington, address to Congress (1790)
"During almost fifteen centuries has the legal establishment of Christianity been on trial. What has been its fruits? More or less, in all places, pride and indolence in the clergy; ignorance and servility in the laity; in both, superstition, bigotry and persecution."- James Madison, General Assembly of the Commonwealth of Virginia (1785)