The 26-year-old Vettel has been the target of hecklers throughout the season, with his domination of the sport and an incident in Malaysia when he ignored team orders not to pass team-mate Mark Webber cited as the most common reason.
"It is difficult for him to handle. Whilst he's not betrayed it, he's felt it and not understood it," Horner told Sir Clive Woodward during a special interview with BBC Radio 5 live.
At the Malaysian Grand Prix, Webber was leading the race from Vettel with 13 laps to go and the drivers were told to hold their position until the end of the race, but the German ignored the orders and passed the Australian.
Since then, Vettel has apologised for the incident.
"The perception comes from what happened at Malaysia and that he stole a win from Mark Webber. He was then portrayed as the bad guy," Horner continued.
"It has been convenient for some of his rivals to treat him like that and they have encouraged it, whether that's Fernando Alonso taking off his cap and throwing it in the crowd as soon as Sebastian talks on the podium to get a reaction.
"The great thing about Sebastian is that he has not risen to it, but he has felt it and is hurt by it. Of course, there is this pantomime villain scenario with an X-Factor mentality.
"The biggest thing about him is his strength of character.
"He doesn't look at the internet or read newspapers about motorsport. So often he seems so uniformed, but he feels what is going on.
"At 26 years of age, he's hugely impressive with the way he carries himself and the way he has developed as a young man.
"He is always under pressure, always under scrutiny. Of course, the more successful you are, the bigger the fall, and people are looking for that fall. But he has dealt with that outstandingly well."
With two races to go this season, Vettel has won the last seven grands prix and 11 of 17 this season.
