Why do some people call me racist?

I used to work for a phone company that had certain prices to call certain countries. The way these prices were determined depended on the cooperation of said country. Needless to say Africa, and the Caribbean never seemed to cooperate so black customers would always pull the racist card on me. Then I would just tell them to speak to the government of that country if they want change. I'm sure they didn't bother.