Is displaying the Confederate flag racist?

I've been told by southerners that displaying the flag of the Confederate States amounts to a display of "southern pride." I think it amounts to a display of racism, given the history of the south. That flag was used in a time when the agricultural economy of the southern states relied on slave labor. Many southern states seceded from the Union, largely because of their nefarious interests in preserving slavery. The Confederate flag represents these states and their ideology. Hence, I think it's morally questionable (at least) to display it. I don't think the south should take pride in or honor the Confederacy. Am I right or wrong in my thinking? What should I think of people who choose to display the Confederate flag?