Were the atomic bombs dropped on Hiroshima and Nagasaki absolutely necessary to ensure the capitulation of Japan?
Well, let's first think of Japan's demands for surrender right before the atomic bombings to get a look at Japan's view.
Imperial Japan demanded the following concessions for their surrender.
- They wanted to keep their emperor, and they wanted to keep the home Islands (Honshu, Hokkaido, Kyushu, and Shikoku)
- They wanted the Imperial Headquarters to oversee the disarmament and demobilization of their military.
- They wanted to have the Japanese government be responsible for punishing war criminals
- They wanted no American occupation of Formosa (Taiwan) and Korea.
Contrary to what most people would think, in terms of standard morality and human decency, most of these demands were unreasonable. Keeping the emperor and the home islands were basic points that the American government had long since anticipated and privately accepted before the bombing, but the last 3 points were outright immoral. First off, Japan's high command was literally split between 2 factions: the Emperor and his pro-surrender faction versus the Imperial military (the no-surrender faction). Leaving the disarmament of the military in the hands of people who did NOT want to surrender even at the risk of death was obviously not something the Americans would have liked.
Furthermore, Imperial Japan had committed as many war crimes as the Nazis such as human massacres, human experimentation and torture under Unit 731, and forced conscription of people in their colonies as soldiers. The entire Imperial high command was aware of these atrocities (the emperor himself later acknowledged these), but did nothing to stop it until Japan surrendered. Thus, it can be assumed that allowing the Japanese government to try its own generals and officers for war crimes is in essence a ludicrous idea. It's like asking a unrepentant criminal to be in charge of punishing himself for his crimes: it's just not going to happen.
As for the last concession, allowing Japan to keep Formosa and Korea would not have been a good idea. Just a few months before the atomic bombings, the Imperial Military of Japan massacred over a 100,000 civilians in Manila as a last ditch payback for losing battles. Statistics vary, but given that Japan had killed several hundred thousand Koreans via forced labor during their occupation, allowing Japan to keep the two countries would not be conducive for the well being of human lives. Given the prevalent policies of forced labor, Japan wold have likely used the two colonies for forced labor to recover from its losses in the war, killing hundreds of thousands if not millions more.
Strictly speaking, if the question is to be read as "Would the Japanese have surrendered if the US did not use the Atomic Bomb?" the answer is that Japan would have surrendered, but at a cost of millions of lives all across the pacific, even when you exclude USA casualties. Given that war is inherently immoral, the only determining factor is whether the atomic bombing killed fewer people than the alternative. The atomic bombing worked in this logic: kill 200,000 people to save millions. It was cruel, but also the lesser of the two evils.
As a benchmark, we know the Japanese effectively stopped fighting by August 14th, after the use of the Atomic Bombs.
Note that the Japanese Imperial Council was currently split almost evenly into pro-surrender, and no-surrender-at-any-cost factions. However, the military was almost entirely on the no-surrender side, and it is highly unlikely that they would have obeyed the civilian faction to surrender, even if that faction was the majority. As such, the US faced a situation where the government of Japan would surrender, but the armed forces would continue to fight.
So, what forces the military faction into recognizing defeat? In real life, the use of the atomic bombs allow the Emperor to intervene in the Council, and use the "supernatural" nature of these weapons as a face-saving excuse to allow the military to surrender without suicidal effort. The problem of surrender was exacerbated by poor diplomatic communication between the US and Japan, over what conditions a surrender would be allowed.
The question would be then, what would change the military commander's minds, since the Emperor would need a significant event in order to intercede and break a decision deadlock.
Factually, August and September 1945 were to be rather sparse in terms of scheduled fighting - the IJN was dead, so the would be no naval combat. Japan was saving its forces to fight the expected invasion, scheduled for Oct 1945 by Allied planners. However, there was still plenty of combat in China and the Indochina theater (Burma, Thailand, etc.). Additionally, there are estimates that Allied POW and civilians in occupied areas were dying at a rate of 1,000 per day due to neglect and abuse by Japanese forces. So, each month causes about 30,000 additional deaths, merely by continuing the war, even with no additional combat.
As there is no reason to surrender beforehand, Operation Olympic starts (the invasion of southern Japan). Conservative estimates are for a 3-month campaign, costing 100-200,000 US dead, and 5 times that Japanese. This would be followed by Operation Coronet, the occupation of the rest of Japan, at about the same timeframe, and double the cost.
So, for every month of combat in the Japanese Home Islands, you cost 40,000+ US dead and 250,000+ Japanese dead. There are very likely probabilities that the use of chemical (i.e. poison gas) would have been authorized for use by US forces if the Japanese insisted on fighting as they had at Okinawa or Iwo Jima.
In addition to all this, the continued bombardment of Japanese cities by B-29s was systematically burning Japan to the ground. By Dec 1945, the estimate was that no city of more than 100,000 people would have been less than 50% totally destroyed. This puts over half the Japanese population homeless, and 5-10% of the whole civilian population killed by firebombing (or its aftereffects).
In a similar manner, the total destruction of the Japanese coastal marine and the Japanese railroads meant that food distribution was almost impossible. Widespread famine almost occurred in early 1946 even after the surrender and with the occupying Allied forces there to provide food distribution. With Japan continuing to fight past January 1946, the country would experience a massive famine. A good estimate is 2% of the total population dies EACH MONTH starting in Jan 1946 just from famine.
So, how likely was it that the Council would allow a surrender? It is one of the great imponderables. In my opinion, there is no possibility of a surrender before Olympic finishes. That means, in the best case, a Surrender around Jan 1946. Which means 50,000 KIA and 250,000 WIA on the Allied side, 1,000,000 KIA+WIA on the Japanese side, plus 100,000+ POWs and friendly civilians dead. I would hope that this slaughter would have been enough to stop the war. But, there is a distinct possibility that the Japanese would have effectively chosen ritual suicide for the entire nation. Which means, no surrender until Summer of 1946, at which time, the butchers' bill likely stands at 1,000,000+ Allied (including POWs and civilians in Japanese-occupied nations), and up to 20% of all Japanese dead (i.e. 10 million plus).
So, best case scenario is a balance between the 200,000 casualties of the atomic bomb (all of which are Japanese), vs 1.5 million casualties Operation Olympic would have caused (one-third Allied, two-thirds Japanese). Worst case is at least 15 million casualties after Operation Coronet (15% allied, 85% Japanese).
I would dismiss the notion that President Truman merely wanted to test the destructive power of the atomic bomb. The initial explosion tests on US soil confirmed the destructive force of the bomb. It wasn't a case of the USA having developed just one bomb that could not be wasted on a "test" in the desert. There were several bombs at the ready. The real reason for dropping the first bomb on Hiroshima was to expedite the end of the war with as few casualties as possible. If Truman truly wanted to "punish " the Japanese, he could have ordered the whole arsenal of A-bombs to be dropped on several Japanese cities. He didn't do that but allowed only one to be dropped; then since the Japanese didn't surrender, he ordered a 2nd to go down. Finally the Japanese realized there were possibly many more bombs with the same destructive power and so were brought to their knees.
This wise approach by the 33rd President achieved victory at a lower casualty cost than what might have been had conventional bombing raids/invasion landings taken place
Had the war not ended when it did, the US had production plans, the industrial infrastructure, and the bombers to make and deliver a total of 23 atomic bombs on Japan before the end of 1945. The primary thing limiting production was the Hanford plutonium production reactors, each of the 3 reactors could only produce enough plutonium in a month to make one fatman core.
- august: 3 (only 2 were used, third had been built before the surrender and arrived in San Fransisco on august 18, it might have been dropped on Japan around august 25 had the surrender not happened)
- september: 3
- october: 3
- november: 7 (increase in production due to a planned switch to composite plutonium/uranium cores)
- december: 7
Note: Truman could not "have ordered the whole arsenal of A-bombs to be dropped" as no such arsenal existed at the time. The atomic bombs were being dropped on Japan exactly as fast as the factories could make the materials and Los Alamos could prepare the atomic bomb kits for shipment.