The result sees Town remain in third spot in the Evo-Stik East table. They missed the chance to move into second place ahead of Markse United who were not in action.
Jack Normanton put Brighouse in front after he converted Ousman Cham's cross at the back post just before the 20-minute mark.
It remained 1-0 until the fourth minute of injury time when the hosts were awarded a penalty for a foul in the area.
Vernon stepped up and lashed the ball into the left side of the goal, beating Jordan Porter, who had guessed the right direction, in the Town goal.
Brighouse can move into second spot on Saturday afternoon if they better Marske United's result.
Town travel to AFC Mansfield while Markse host Jamie Vardy's former club Stocksbridge Park Steels.