https://www.bustle.com/articles/120208-5-popular-phrases-with-shockingly-r****t-meanings-and-origins

"Call a spade a spade" has been in use for nearly half a millennium (and variations on it date at least as far back as A.D. 120). But it didn't start to take on the r****t connotation it has now until the early 20th century.

Evidently, in the 1920s, "spade" started being used as a slang term, and then a derogatory slur, for a black person. This r****t shift in the phrase's meaning also lead to the equally offensive phrase "black as the ace of spades."