KELLYSVILLE: Long Gone But Not Forgotten

KELLYSVILLE: Long Gone But Not Forgotten

By John Toler

Union Army sketch artist Pvt. Robert K. Sneden drew a panorama of Kellysville as it appeared in November, 1863, during the Second Battle of Rappahannock Station.

For generations, Kelly’s Ford on the Rappahannock River below the town of Remington has been the go-to place for low-key recreation, including fishing, swimming and canoeing. The river, rushing through scattered boulders, and the quiet woods reveal no evidence of the once-booming commerce and industrial village of Kellysville that once existed there.

Sneden’s close-up drawing of Kelly’s grist mill, “showing the effect of artillery,” the damaged cotton factory in the background, and front porch of Kelly’s house at far left.

John P. Kelly (1789-1866), a native of Westmoreland, Virginia, came to the area in 1829, eventually owning 1,000 acres along the Culpeper side of the Rappahannock. An astute businessman, Kelly built a water-powered grain mill with a half-mile millrace, and in 1837 opened a toll bridge over the river.

With the building of the Rappahannock Canal linking farms and businesses in northwestern Fauquier and Culpeper counties with the port city of Fredericksburg, Kelly’s business ventures grew to become Kellysville. Additional facilities included a sawmill, shops producing shoes and dyes, a community store, and a clothing factory run by Kelly’s son, Granville.

Smaller enterprises included a wheelwright, cooper (barrel maker), a brick kiln and a creamery. At its height, 100 workers, both free and enslaved, were based in Kellysville, with many living in cabins and houses on the property.

Demand for the flour shipped downriver grew to the point that the mill ran continuously until midnight Saturday, resuming at midnight Sunday. Two boats each carrying 100 barrels of flour regularly made the trip down the river and back, according to Kelly’s grandson, D. W. Kelly, in the March, 1941 edition of the Virginia Chronicle.

Described as a heavy-set, smooth-shaven Irishman, John P. Kelly’s success as a businessman was eclipsed by his reputation as a mean-spirited, conniving self-dealer. While his Kellysville complex was the largest non-farm operation in the region, he was “…the most feared and hated man in Culpeper County,” wrote Sarah K. Bierle on the Emerging Civil War website.

Kelly’s frequent confrontations with his neighbors, notably George Wheatley, who had two mills and a tannery upriver from Kellysville, often came to blows over issues like wandering livestock or diverted river water. Any stranger crossing Kelly’s property without invitation—or paying a toll—was likely to be attacked and beaten.

Suffering the worst treatment were Kelly’s slaves, numbering 71 according to the 1860 slave census. Whippings were meted out for minor infractions or for no reason at all.

In an article published in The Fauquier Democrat in 1936, Works Progress Administration field worker M. D. Gore recounted one ugly incident. Returning home one day, Kelly was not met at a stile by the young boy who was to take charge of his horse. Angrily, Kelly rode the animal into the house, where he confronted the cook in the kitchen, who simply asked him what he wanted.

Further enraged, Kelly struck her in the mouth with his fist, knocking her down and breaking off a front tooth. “But the tooth cut the joint at (Kelly’s) index finger,” wrote Gore. “The housekeeper put sugar of lead on the wound, but it poisoned him from the start, and grew steadily worse.” Although Kelly later sought medical treatment for blood poisoning, gangrene eventually set in and his right arm had to be amputated below the shoulder.

Detail of Pvt. Sneden’s map of the battle lines on Nov. 7, 1863 shows Kellysville and the Wheatley property along the Rappahannock River.

Kelly’s business success continued through the 1850s, but significant challenges were on the horizon. The Rappahannock River Canal, which had struggled from the beginning with financial problems and natural disasters, was reorganized several times but was effectively closed by 1857.

No longer able to transport flour and goods by water, Kelly put his wheelwrights to work building wagons to make the runs to Fredericksburg. But there would be serious challenges as the nation moved toward Civil War.

Even before Union army units moved into the area in early 1862, the nature of the coming war was felt in Kellysville. Confederate troops passing through “requisitioned” Kelly’s chickens and hogs and other property.

Cloth for Confederate uniforms was being made in Kellysville, but by June, Granville Kelly was so concerned that advancing Union forces would capture the machinery from the mill and clothing factory that he had it dismantled and sent by wagon to Lynchburg and Danville.

The first big hit on Kellysville happened a few months later on March 17, 1863, when 2,100 Union cavalrymen under Brig. Gen. William W. Averell crossed the Rappahannock from the Fauquier side, attacking Confederates led by Gen. Fitzhugh Lee. The skirmish is remembered as the Battle of Kelly’s Ford.

Union troops took Kellysville, but soon learned that the milling and machinery had been removed. Meanwhile, Lee’s men counter-attacked, forcing Averell’s troops to withdraw. The result was inconclusive, but the effect on the village was severe. Few of Kelly’s businesses could still operate, and many of the former slaves – freed earlier by Lincoln’s Emancipation Proclamation – fled behind the Union lines.

The next major military action involving Kellysville was on Nov. 7, 1863, when Union troops attacked Confederates under Gen. Robert E. Lee positioned along the river during the Second Battle of Rappahannock Station. Maj. Gen. William H. French attacked the Rebels at Kelly’s Ford, and a force under Maj. Gen. John Sedgwick struck the bridgehead further upstream.

The Confederates were overrun, and retreated into Orange County. Union troops then occupied the area, including what remained of Kellysville. Pvt. Robert K. Sneden, a Union mapmaker and sketch artist, came to the village after the fighting.

“We met ‘Old Man Kelly’ at the mill. He looked gloomy and sullen, and being a rabid Rebel denounced our men in loud and futile oaths for stealing his cattle and hogs,” Sneden wrote. He described the damage done by Union artillery, and made sketches of the shattered village.

John Kelly remained at Kellysville through the rest of the war, and it was noted that he “found religion” and helped his neighbors rebuild, including Mt. Holly Church, which had been destroyed by the Union army.

Kelly died in 1866, and was buried in the family cemetery off Edwards Shop Road. Over the years, what was left of Kellysville was either dismantled and hauled away, or retaken by nature. The last survivor was the old Kelly home, which burned in the 1930s.

“Now there is nothing left of this thriving center,” wrote D. W. Kelly in the Virginia Chronicle. “It was destroyed during the War Between the States, and never rebuilt.” In later years, vandals stole John Kelly’s gravestone, leaving only memories of the man and the village he founded.