My 10th Great Grandfather, Samuel Wilbore, who along with William Blackstene bought what is now "Boston Commons" and gave it to the town, was banished from Boston in 1637, because of his association with a religious group lead by Anne Hutchinson John Wheelwright and possibly Roger Williams, Anne Hutchinson was the unauthorized Puritan leader of a dissident church discussion group.
So much for religious tolerance in the collonies.
They fled to to Portsmouth, R.I.

Puritans were one of many groups which has separated from the Church of England in the 16th - 18th century. Most American Puritans sought the liberty to practice their own faith but refused to grant others the same freedom. To create a holy commonwealth and a godly society, the founders of Massachusetts Bay and Connecticut instituted religious establishments-arrangements by which the civil government favored one church and penalized anyone who dissented from its teachings.

No surprise, then, that the founders of most colonies in mainland British North America moved quickly to set up religious establishments. Throughout New England (with the notable exception of Rhode Island), Congregationalism (a.k.a., Puritanism) was the state-supported church, while in New York and all colonies to the south (with the notable exception of Pennsylvania), the Church of England (a.k.a., Anglicanism, known today as Episcopalianism) enjoyed that favored position by the end of the seventeenth century.

Elsewhere in British North America (outside of New England and Pennsylvania), colonial Anglicans enjoyed less official support, despite the benefits of establishment. In the Carolinas as well as in New York, New Jersey, and Delaware, Anglicans never made up a majority, and thus competed with (and were obliged to tolerate, even if grudgingly) ethnically diverse groups of Presbyterians, Baptists, Quakers, Dutch Reformed, and a variety of German Pietists.
Source: Church and State in British North America, Divining America, TeacherServe at the National Humanities Center

Books:
Founding Faith: How Our Founding Fathers Forged a Radical New Approach to Religious Liberty: Steven Waldman:

Links:
Religion and the Founding of the American Republic (Library of Congress Exhibition)
The Antinomian Controversy in the Massachusetts Bay Colony 1636-38
Religion in Eighteenth-Century America - Religion and the Founding of the American Republic | Exhibitions - Library of Congress