His decision is a setback for Chancellor Angela Merkel's plans for a fourth government shortly before before it is due to have been sworn in.
The rare public broadside at his comrades came after SPD and Merkel's conservative CDU/CSU bloc announced a hard-fought coalition deal on Wednesday that could see Schulz claim the Foreign Ministry.
Schulz has turned down the post of the foreign minister under increasing pressure from his own party, because of his earlier pledge that he will not become part of the cabinet headed by Angela Merkel.
This tougher stance was already on display in the rhetoric voiced during the recent visit to Israel by Germany's current foreign minister, Sigmar Gabriel, also an SPD politician.
Mr Schulz, facing criticism inside the party, said he did not want disagreements about his role to damage the chances of forming a new coalition.More news: Boy climbs into claw machine to get stuffed animal
More news: LA Times to be sold to billionaire doctor turned entrepreneur
More news: Twitter shares surge by a quarter on first ever profits
"I hereby forgo joining the federal government", Schulz announced, adding, "at the same time, I sincerely hope that this will end the personnel debates within the SPD".
With the coalition deal subject to a yes-or-no referendum by all 460,000 of the SPD rank and file, the sound of activists' teeth grinding at Schulz's apparent move to secure himself a cosy job had by Friday become impossible to ignore.
The smallest of the parties in Germany's prospective new government has signed off on this week's coalition deal, but bigger hurdles lie ahead.
Schulz's choice to step aside "deserves the highest respect and recognition", said Andrea Nahles, his likely successor as party chair.
Many grassroots members in the center-left party are sceptical about another tie-up with Merkel after the SPD suffered it worst election result of the postwar era in September's election. Many of Merkel's deals have infuriated her Christian Democrat party.