Less than 20% of the Southern population owned African Americans. There are many misconceptions as to what caused and fueled the American Civil War. Slavery was indeed one of them. Increased tariffs on products shipped to northern ports was a significant one. Really significant was the South's secession from the union and voting its own President which angered Northerners greatly. It was a major investment to own an African American especially if they possessed valuable skills such as blacksmithing, carpentry, or farming skills. Certainly owning a black man or woman, was a prestigious classification, but very few Southerners and Northerners, fit in this category of wealth. Thus, the stories that have been perpetuated throughout history of all slave owners beating their "slaves" is quite inaccurate. It is a myth that the media of that time stimulated to build a case to send the North and South to war. In addition, politicians, many of them wealthy and slave-owners, were used as the example or model for the Southern population, when in fact the average Southerner lived day to day and could not afford to purchase another human being for $500-$1,500 .
In addition, while many people have been taught that Southerners were the only ones who owned African Americans in the United States, Northern sentiment was possibly even more harsh toward African Americans. Blacks were beaten and even killed in city streets of New York City, New York, toward dissolvement in the mid to late 1800's, but it was the American Civil War which sped up the abolition.
It depends heavily on the region. South Carolina had the largest percentage of the population who owned slaves. Areas of the coastal plain, which are areas within about 100-200 miles inland, were flat, fertile, sunny, and had the subtropical climate where cotton and especially rice thrive. Cotton and rice were sold and shipped to various ports, including Boston. Combined with the labor intensity of growing cotton and rice, farms growing them usually had slaves. Note that Africans had previously grown rice in Africa, so they were accustomed to the process. Areas in the mountains, as well as wooded, undeveloped areas (then called back woods), on the other hand, had few, if any, slaves.