There are many buildings on Wall Street. Which one are you referring to? You can take the Downtown 4 or 5 trains (of the 4-5-6, the dark green line) from 42nd Street-Grand Central Station to the Wall Street Station (on Broadway, between Rector and Wall Streets).