After rare talks with his Chinese counterpart Wang Yi, Taro Kono said the ministers had "forward-looking" and "fulfilling" discussions about rebuilding ties between the Asian powers.
They agreed that Japan would host a trilateral summit between Prime Minister Shinzo Abe, Chinese Premier Li Keqiang and South Korean President Moon Jae-in, without setting a date.
After such a meeting, they hoped to organise reciprocal visits by Abe and Chinese President Xi Jinping, Kono said.
"We agreed to fully improve the bilateral relations through exchanges of the leaders ... with Premier Li's visit at the time of the trilateral talks as the starting step," Kono told reporters.
However, Kono warned there would be "no true improvement of Japan-China relations without stability in the East China Sea".
"We agreed the importance of making it a sea of peace, cooperation and friendship," he said.
The world's second and third largest economies have a fraught relationship, complicated by longstanding maritime disputes and Japan's wartime legacy.
The dispute over islands in the East China Sea -- known as the "Senkakus" in Japanese and the "Diaoyu" by the Chinese -- remains a source of tension.
But Tokyo is eager to get the relationship back on a firmer footing, especially as it fears being shut out of negotiations on North Korea's nuclear programme in which Beijing is likely to be a major player.
China demonstrated its significant influence over its reclusive ally when Xi hosted the North's leader Kim Jong-un and his wife in Beijing last month.
With Moon and US President Donald Trump preparing for separate direct talks with Kim, Japan fears being left on the sidelines.
Kono, who visited Beijing in January, stressed that the two countries share the same goal on North Korea.
"We had direct and in-depth discussions on North Korea in light of the current situation," he told reporters after the meeting, without revealing details.
"We confirmed that we shall continue our close coordination while also completely implementing related UN Security Council resolutions to realise the irreversible, verifiable and complete denuclearisation of North Korea and its missiles."
Wang, a former ambassador to Japan, said his visit was China's response to "positive" messages and policies from Tokyo but acknowledged "some complex and sensitive elements".
"But together with Japan's efforts... we would like to bring China-Japan relations back on a path of sustainable and normal development."
Abe, who is due to meet Trump in Florida on Tuesday to discuss North Korea, will meet Wang on Monday.
The time is ripe for the two nations to improve relations to mark the 40th anniversary of their 1978 Treaty of Peace and Friendship, both ministers said.
Wang, who will be in Tokyo until Tuesday, will also take part in the fourth China-Japan High-level Economic Dialogue.