How do you replace the voltage regulator on a 1986 Chevy Celebrity?

It is in the alternator.

The voltage regulator will be inside the alternator. Replace the alternator, or take it to a place that rebuilds/fixes them.

It requires metric sockets, internal torques sockets, a soldering iron, a small flat screwdriver and some thin wires or drill bits.

There is a plastic cover on the alternator over the connections for the voltage regulator, these need to be taken apart with the soldering iron and flat screwdriver to be able to take the alternator apart. Then remove the housing bolts (pay attention to how the two halves line up with each other) using the internal torques sockets. After separating the two halves of the alternator look at the brushes to see if they are too worn, also check the bushings for play. If the bushings are bad replace the alternator, or have it rebuilt. If the brushes are worn, they can be replaced, but try not to lose the springs. If I remember correctly the voltage regulator is held in with 2 small screws, removed with a metric socket.

Putting it back together, bolt in the new regulator. push brushes into the slots and hold with wire or drill bits. Put two halves together exactly as they were before, re-solder it, remove wires or drill bits and replace the plastic cover.

Also, when you have it apart, is a good time to tighten the hot wire stud if it is loose.

I did this on one of mine and the $5 voltage regulator gave me another year of use out of the alternator before I had to replace it.