He just isn't the wholesome character that everyone thinks he is.
No, he definitely isn't the nicest guy around. It's clear by the 7th book, and perhaps even by the 6th (my memory is a little fuzzy on the books and the movies, which I'm a bit more familiar with at this point, leave so much out), that's he's clearly got some demons in his past, and has done straight-up evil shit from time to time.
I don't mind Gambon at all in the role; it was always gonna be hard changing mid-series like that, and he did very well over all. I don't know anything about Richard Harris' acting career, but I think he might have been able to pull it off, if he actually had the physical fitness for it (hard to tell how much is CGI and how much is an actual set they have to move over). I can picture Harris with a harsher expression, a certain hardness in his voice, delivering some of the harsher lines in later movies. It would've been neat to see how it was handled.
I have seen people complain about this particular scene and Gambon's Dumbledore a few times now.
Having Richard Harris play Dumbledore as he did in the first two films ruined the character. It's not that he was bad in the role, it's that the books turned him into a completely different character by the end. I really don't know how Richard Harris would have done Dumbledore in the 6th film, for example, and the backstory you learn about in the last book just doesn't fit with the portrayal in the first two films.
I think people got the character wrong, not Gambon. In the scene where you see Snape's memory when he learns about Harry having a part of Voldemort's soul inside him, he accuses Dumbledore of raising him like a lamb to the slaughter, which is completely true. Plus there's a load of other shit in the last book you learn about, like the way he treated his sister. He just isn't the wholesome character that everyone thinks he is.
This scene tho.