Bottom-placed Rangers are now seven points adrift of the Barclays Premier League safety zone after Christian Benteke's 17th goal of the season finally settled a see-saw game in Villa's favour.
Redknapp said: "If we had come in 4-0 up at half-time it would have been about right, wouldn't it? That's the best I've seen us play during that first half. It was so one-sided, it was unbelievable for an away team to dominate like that. But they got a lift from the equaliser after being absolutely awful first half and probably couldn't believe their luck."
He added: "That goal knocked us for six. I'm upset for the lads. I thought they played so well and we should have taken a point at 2-2 but then we gave a way a bad goal."
Rangers dominated the opening 45 minutes and deservedly led through Jermaine Jenas with only a string of fine saves from Villa keeper Brad Guzan keeping the home side in contention.
But Paul Lambert's side equalised on the stroke of half-time through Gabriel Agbonlahor and went ahead through Andreas Weimann and, after Andros Townsend made it 2-2, Benteke struck the decider.
Redknapp is refusing to throw in the towel in the battle for survival, and he added: "We've got to keep going. We can't go home and feel sorry. It's going to be hard but there are still eight games to go. If we keep playing well, we can win four or five games.
"We're probably looking at four wins and two draws. It's going to be hard but it's not impossible. I'm not giving up. We've still got to keep going."
Villa boss Lambert was elated after the midlands club achieved back-to-back Premier League wins for the first time in nearly two years, and he said: "Guzan has made three world-class saves and I will reiterate what I said before, that I wouldn't swap him for anyone.
"But I thought we passed the ball well and the goal rocked us. It will rock an experienced team when you lose a goal, not just a young team. But credit the team, we scored right at the death of the first half and after that we were exceptional."
Source: PA
Source: PA