The Cologne administrative court said the western city must ban the dirtiest diesels from its centre and other streets from April 2019 to tackle dangerously high levels of nitrogen dioxide (NO2) emissions.
In the smaller city of Bonn the restrictions would apply to two streets.
The ruling, which the regional government intends to appeal against, is the latest victory for German environmental group DUH which has launched a raft of court cases to force local authorities to boost air quality.
Major urban areas including Stuttgart, Frankfurt and the capital Berlin have already been slapped with legal orders to cut emissions, while Hamburg decided of its own accord to expel the worst polluters from some zones.
Faced with mounting public anger, Chancellor Angela Merkel's government is scrambling to ward off the unpopular driving bans that promise to not only cause transport upheaval but further hammer the resale value of diesel vehicles.
Diesel engines have come under intense scrutiny since Volkswagen admitted in 2015 to installing cheating software in millions of cars to dupe emissions tests.
The scam made the cars seem far less polluting in labs than they were on the road, and suspicions of similar trickery have since spread to other carmakers as well.