The rates are posted on the USPS web site. Square envelopes are charged a "non machinable surcharge" in addition to regular postage. That surcharge is 20 cents for any piece of first class mail as of August 2012. The non machinable criteria are outlined in the "Domestic Mailing Manual" in the "Physical standards" section. Letters need to have an aspect ratio between 1.3 and 2.5 to be considered machinable. The aspect ratio is the width divided by the height; square envelopes have an aspect ratio of 1, which is less than minimum 1.3 required. There are other conditions as well, but even if multiple criteria are met, only a single surcharge is charged.
Other than that, normal first class postage applies: 45 cents for the first ounce and 20 cents for each additional ounce up to 3.5 ounces. (as of August 2012)
If the dimensions cause it to be classified as a "large envelope" ot the weight exceeds 3.5 ounces, (smallest side > 6 1/8"), the "Large Envelope" rates apply.
I recently had to go to the post office to pick up two postage due items. One was 5x5 and one was 6 1/2 x 6 1/2. Each one had a normal 45 cent stamp on it. I had to pay 20 cents more for the 5x5 (because it exceeded the aspect ratio) and I had to pay 45 cents more for the larger one (to make up the difference in postage for a "Large Envelope"). Technically, the postage due should have been $0.65 for the 6 1/2 x 6 1/2, but since it was both large and nonmachinable, but I guess they let the surcharge slide.