The beginnings of the Church in this area of coastal Georgia date back some four hundred years, long before Oglethorpe established himself at Frederica. From 1566-1742 Spanish missionaries labored, suffered, and were martyred in this locality for the cause of the faith. In 1791 the island of Jekyll was purchased by the du Bignon family. From time to time the head of this family arranged to have a visiting priest celebrate Mass for his family and his slaves. The assumption is that these visiting priests were missionaries attached to the Diocese of Baltimore, the Diocese of Charleston, or they were visiting priests from France. The Catholic Directory for 1840 shows that the Diocese of Charleston had seven priests in charge of the missions in Georgia; and it was the same year that a new district was formed, the principal stations of which were St Mary's, Jekyll Island, Brunswick and Darien. Sometime after 1844 the du Bignon family moved from Jekyll Island to Brunswick and Mass was said at their home on Union Street for the few Catholic families of Brunswick who were mostly of French and Irish descent.

In 1868, a small frame structure was erected with the chapel facing Egmont Street and living quarters for the priest adjoining it, facing on Mansfield Street. This site was donated by a non-Catholic family, the Darts. The chapel was dedicated to Saint Francis Xavier. The parish began preparing for a new and more suitable and larger church in 1878. It was on Sunday, April 27, 1884, that Bishop Gross dedicated the new and present church.

Between 1885-1888 Father W.A. McCarthy was recorded outlying districts and celebrating baptisms for Willacoochee, Alapaha, St. Mary's, Darien and Doboy. During the yellow fever epidemic of 1893, Father Hennessey refused to leave town, although everyone who possibly could was doing so. This courageous priest went from house to house, regardless of creed, aiding the victims of this dread disease. With them and their descendants the name of Father Hennessey is held in benediction.

In 1897 Saint Francis Xavier and the Southeast Georgia missions were entrusted to the care of the Marist Fathers. The War with Spain in 1889 arrested the commerce of the city and in October of that year a combined cyclone and tidal wave destroyed the larger part of the business section of Brunswick. The flood entered the church and destroyed almost all of the windows. When the Marist first came to Brunswick, they rented a home on Union Street as a residence, but during the Spanish-American War when money was scarce, the priest moved to the Shannon house and lived there until a rectory was built in 1899. This rectory was originally designed to house only two priests and cost $2,100.

Shortly after the Marist Fathers arrived in Brunswick, in addition to their own mission field, the Bishop requested them to take over the missions of the Albany parish until 1901-1902 when Bishop Kiley was able to assign a priest to Albany; the missions of Albany were Willacoochee, Alapaha, Douglas and McGovern Settlement.

The pastor from 1908 until 1913 had erected the Chapel of the Immaculate Conception at Ludowici. The succeeding pastor was responsible for the building of the new church & the Plant Building in Waycross. Xavier Hall was acquired by the Knights of Columbus which served as a meeting place for parish organizations.

The pastors between 1924-1953 built and dedication St Williams on St. Simons Island, the Church of the Nativity of Our Lady in Darien, St. Joseph's Church at Jesup (replacing the Chapel of the Immaculate Conception in Ludowici), built a school in Waycross, and began building the the convent.

The Saint Francis Xavier Parochial School was built and dedicated February 9, 1958.

Under Father Paul Burkort, S.M. the church, rectory, and Hall were razed; the new church built; Richmond Street was acquired from the city; and Reynolds was finally paved.