Left-back Delaney blasted home from 12 yards in the 55th minute to give his side victory in the Carling Cup First Round.
In a lively opening to the game Dexter Blackstock went close with a shot from 15 yards that was saved by Petr Brezovan in the Swindon goal.
Swindon striker Billy Paynter had his side's first chance of the game in the 14th minute but could only fire his shot straight at Lee Camp in the Rangers goal.
Rangers took the lead in the 32nd minute when Angelo Balanta bundled the ball home over the line after the Swindon defence had made a hash of clearing a corner.
Their lead lasted barely a minute as tricky winger Anthony McNamee crossed for striker Simon Cox to smash the ball home after a quick turn from ten yards.
Swindon then took a shock lead just before the break and it was McNamee who again did the damage.
The left winger crossed from the left hand side for Paynter to score with a diving header from ten yards out.
Midfielder Daniel Parejo almost got a goal back for QPR two minutes before the break when he was put clean through after beating a Swindon offside trap. The midfielder, who is on a season long loan from Italian club Genoa, tried to take the ball round the goalkeeper but Brezovan bravely dived at his feet and nicked the ball off his toes.
The second half started well for Rangers and they grabbed an equaliser barely 30 seconds into the half. Former Fulham midfielder Lee Cook crossed for Blackstock to head home from six yards.
Rangers took the lead nine minutes later through Delaney and then seemed content to settle for the narrow victory.
Swindon piled on the pressure but could not find a way past a determined defence as Rangers scraped through to the second round of the Carling Cup.