The Elk Hill Plantation was one of the earliest in Nelson County. Originally built by the Reid family, the farm was bought by the Colemans in 1805 and remained in that family for five generations, until the 1970s. Today a portion of Elk Hill is under the operation of Rockfish Valley Farm LLC. In addition to 173 acres of farmland and a plantation house (built around 1790), the farm also contained several outbuildings: a late 18th Century smokehouse, an 1860s office, a tobacco barn, a chicken house, an ice house, a double crib barn, and a two seater outhouse. The antebellum labor on this farm was primarily provided by enslaved labor, which numbered over 100 individuals in the 1850s. Sometime during this period a separate cemetery was created for use by the enslaved population. Located about a half mile from the main house, the graveyard contains about 125 burials. Only 37 of the graves contain standardized markers (carved from granite or marble, or cast from metal), 25 of the graves contain uninscribed quartz markers, while 33 are carved from locally available fieldstones (a handful in this group are unmodified). Two of the stones are particularly unusual, tablet style markers, made from concrete, with two-dimensional crosses carved into them (see above photo).

The earliest inscribed marker dates to 1925, but there are dozens of uninscribed fieldstones that probably date to the antebellum period. After decades of 19th century use, the African American residents of Elk Hill continued burying their dead here but created a new cemetery, several hundred yards away (and lower) than the original. This cemetery (seen in the photo below) is still in use by the Elk Hill Baptist Church.