A witness estimated a dozen had been hurt in the collision - which he called "intentional" - saying one girl got "tore up" after the car "backed up and they hit again."
He said the dark sedan "raced down here, jumped over the speed bumps and it backed up and it hit everyone again."
"There was a girl that was on the ground; she was trying to get up," he added.
Emergency personnel in ambulances quickly arrived at the scene in the normally tranquil city of Charlottesville. An AFP journalist witnessed injured people on the ground and others in tears.
The crash occurred as hundreds had descended on the city either to take part in or voice condemnation for a far-right rally that quickly erupted in violence.
Virginia Governor Terry McAuliffe had earlier declared a state of emergency as Charlottesville descended into violence.