The central committee of Algeria's ruling party relieved its chief Abdelaziz Belkhadem of his duties on Thursday, in the FLN's biggest internal crisis for a decade, an AFP journalist reported.
Members of the National Liberation Front held a vote of confidence, with 160 voting in favour of and 156 against the resignation of the secretary general, with seven spoilt ballot papers.
Belkhadem admitted defeat, saying: "I have triumphed because my party has triumphed. I quit with my head held high and hope that whoever succeeds me will do better."
His adversaries had feared the influential former premier may force the party to re-elect him.
The FLN, to which President Abdelaziz Bouteflika also belongs, and which holds almost half of the seats in parliament, had been riven between supporters of Belkhadem, 67, and high-profile critics who accused him of nepotism.
"Belkhadem is playing with his political destiny at this meeting (for the vote of confidence). We fear a repeat of what happened at the last central committee meeting" in June, said former minister Boudjemaa Haichour, one of the leading party rebels, ahead of the vote.
Belkhadem, who insisted that he won the central committee's vote of confidence then, was accused of orchestrating an "organic coup," and the two factions nearly came to blows.
Eight ministers, all on the central committee, subsequently called for his resignation and accused him of using "state institutions to realise personal ambitions."
On Tuesday, Belkhadem denied outright the existence of a crisis and accused his critics of trying to stir ill-feeling between him and Bouteflika.
If there were a crisis, "it must stem from ideas, differences in orientation or management among its members, which absolutely do not exist," he said in an interview with the private Nessma TV channel.
He also said he was being targeted personally.
The call for him to step down came after another former premier, Ahmed Ouyahia, quit as secretary general of FLN ally the Rally for National Democracy (RND) after coming under pressure from within his party.
The FLN, which was the main organ of the long and bloody struggle for independence from France that Algeria finally won in 1962, has not experienced such a severe crisis since 2003.
At the time, FLN secretary general Ali Benflis emerged as a rival to Bouteflika, who sacked him as premier in a move that split the party and the government.
Under Belkhadem's leadership, the FLN won back-to-back legislative and local elections, in 2007 and 2012, despite claims of widespread fraud in last year's parliamentary polls.