Contemporary critics of Darwinian evolution (e.g. over at Uncommon Descent) are often also interested in criticizing Darwin himself, as well as making connections between Darwinism and racism, eugenics, and the view of social progress known as Social Darwinism. There are indeed connections between these ideas, but neither those who would demonize Darwin nor those who hold him up as a sort of saint have got the full picture. The truth is, of course, more complicated.
The idea of evolution undoubtedly had a strong effect on Victorian beliefs and attitudes. An evolutionary idea of progress permeated Victorian culture; one might say it was the zeitgeist of the 19th century and even the beginning of the twentieth. Europeans saw themselves as the pinnacle of progress, while nonwhite peoples were seen as inferior and less intelligent, with the primates just below them, and so on. But, this cultural bias towards other races existed before Darwin and his theory of evolution. As early as the 17th century (Darwin published The Origin of Species in 1859) naturalists were studying human skulls from various cultures and concluding that whites were the superior race. Louis Agassiz, Samuel Morton, and Robert Knox were prominent 19th century writers/scientists who, before Darwin, concluded that blacks were an inferior race, even the “lowest grade of humanity.”
While we certainly cannot blame Charles Darwin for these prejudices, his theory was seen to provide scientific support for them. His specific theory of evolution, suggested that all life forms compete in the struggle for existence. The more fit will survive while the less fit die out. It was clear, then, that European culture, having survived and progressed far beyond any others, was superior to other cultures. In other words, Darwin’s mechanism could explain, scientifically, how some races could come to be superior to others. It was also used to validate slavery. For example, it was argued that if blacks were set free that they would inevitably go extinct. Blacks, being inferior to other races, would lose in the struggle for existence; it was therefore charitable to keep them as slaves and preserve their race (besides, it was also well-known that blacks became vicious when given freedom and an education). We must remind ourselves again, though, that the idea of evolution did not originate with Darwin and was already quite well-known by the time Darwin published his book. The idea of a struggle for existence, both biological and social, had also already been made popular by Thomas Malthus and Herbert Spencer. In other words, Darwin’s theory was built upon, and was in some sense a product of, preexisting Victorian values.
Victorian anthropologists, before Darwin, were ethnocentric: Europeans were seen as the highest, most advanced race, while other races were viewed as lower and inferior. However, these anthropologists were largely monogenist, meaning they viewed all races as being part of the same human species. In the late 19th century, after Darwin published The Origin of Species, many physical anthropologists became polygenist, viewing other races as separate species from Europeans. Eventually a developmental view of society emerged. In short, this developmental viewpoint saw native peoples as relics of our evolutionary past; people who had not evolved much higher than the apes, and whose adults had intelligences similar to that of European children (a side note: many Victorians also saw European women as having intelligences similar to a child’s). This evolutionary, developmental view of culture did not exactly encourage an equitable view of races and cultures.
It is safe to say that Darwin’s theory was used to support the ethnocentric and racist Victorian view of other races. It also perhaps tilted physical anthropology into a slightly more racist mode (it also eliminated previously strong ties with missionary work). But we cannot place too much blame on Darwin. Racism was a general prejudice of the time, and cannot be linked solely to Darwinism (even the general idea of evolution cannot take all the blame: Louis Agassiz, mentioned above, was a creationist who viewed blacks as having been separately created and inferior to whites). Darwin's theory may be responsible for adding scientific credibility to racism, but not its genesis.
Darwin himself, on the other hand, was an abolitionist and was convinced that the differences between races was one of education and upbringing, not inherent natures. This was made clear to him through his experiences with "civilized savages," men from primitive cultures who had been raised in England and whose behavior and intellect were indistinguishable from Europeans.