Harlow started the brighter of the two sides as their pace in attack threatened to over run the Ducks defence. Adam Wheeler was forced to make a save from John Coley early in the game before Ollie Stanbridge kept matters equal by heroically scrambling Martin Young's shot off the line.
The Ducks took the lead on 13 minutes when Dwain Clarke's free kick fell into the path of Roni Joe who slotted home from close range.
Shortly thereafter Aylesbury made it two as another Dwain Clarke free kick was punched clear by Harlow keeper James Hassell only for the ball to fall to Stuart Maynard who lobbed the ball into the bottom corner of the net from about 25 yards out.
As they had done in the first half Harlow started the second the brighter side and pulled a goal back in the 48th minute when Martin Young netted after Wheeler had parried a Neil Moore shot.
Danny Cowley and Neil Moore both had chances to pull Harlow level before returning sub Chris Bangura secured the points for the Ducks four minutes into injury time.