Is it unfair to judge the past for things we supposedly know better now? Possibly, but that it not the question, really. Part of the problem is that we are not better than the past in the way we think we are. So really, what we need to do is judge ourselves.
In other words, it is the same ideology that persists in us that must be examined.
Let's say, for example, the idea of making Junipero Serra a saint. We who had fourth grade in California knew of his missions up the coast, from San Diego to San Francisco. He was a Franciscan, and the current Pope chose the name of Francis for himself.
It is unfair to blame him for mistreating the Indians, we are told, because nobody knew any better at the time. Now, of course, we know it is wrong to enslave people and destroy their culture, impose your religion on them, kill them with epidemics. Easy for us to say!
But to excuse him from this on the grounds that he brought Christianity to them (supposedly a good thing) and that the Europeans were going to colonize anyway, so it was good to have Christianity soften the blow, well, that is said by someone in the present who should know better. Really, Serra is dead so he won't care if he is judged harshly. When we claim to be sparing him we are really sparing people in the present.