That followed an announcement late on Monday that Richard Ferrand, an aide to President Emmanuel Macron and minister for territorial planning, facing a separate judicial probe, would leave government for a job as leader of his party in parliament.
Macron is preparing a cabinet reshuffle before embarking on far-reaching economic and social reforms after his centrist Republic On the Move (LREM) party won a commanding majority in a parliamentary election on Sunday.
The ministers' departure gives Macron, who promised to clean up French politics, a chance to reshape his government without including names who could prove awkward.
Conservative party The Republicans, seeking to put pressure on Macron, said the investigation into Modem should force the departure of its chief, Francois Bayrou, who is justice minister, and European Affairs Minister Marielle de Sarnez.
De Sarnez said all options were open regarding her future.
That could mean that a reshuffle initially meant to be a formality after LREM's win on Sunday will be more far-reaching than expected. It is set to be announced Wednesday, with conservative Prime Minister Edouard Philippe staying on.
LREM's solid majority means Macron does not need Modem's votes to get legislation through parliament, giving him more flexibility when deciding whether to keep them in a government that includes members of several parties.
However, if he does purge the Modem ministers from his cabinet he may have to replace them with politicians from the Republicans to maintain his left-right balance. That could give ammunition to leftist opponents who call him a right-winger in disguise.
In a sign of how Macron's election victories have dynamited France's political landscape, Republicans lawmaker Laure de la Raudiere said she and a group of about 40 MPs would form their own grouping within the party that would support Macron on a case-by-case basis.
Others in the Republicans staunchly oppose Macron.
Goulard is a member of the Modem party that allied itself with Republic on the Move in the presidential and legislative elections. Since June 9, Modem has been the subject of an investigation into allegations of misuse of public funds.
"Defense is a demanding portfolio. The honor of our armies, that of the men and women who serve and who put their lives in danger, should not be mixed up with controversies that have nothing to do with them," Goulard said in a statement.
Being a target of a preliminary investigation in France is not an indication of guilt. Prosecutors can decide to either drop it or proceed to a full-fledged investigation.