Well, this question is quite easy to answer: It depends on your location. Your car may be running largely on electricity and there won't be as many emissions coming directly from the exhausst of your car, but that doesn't mean that there are none. If you live in a region where most of your eletricity is produced by coal or nuclear power plants then your HEV or electric car might pollute the environment even more than a car fitted with a internal combustion engine would.
Therefore, people in heavily indutrialized regions with little or no solar, wind or water power plants should avoid using regular "plug-in" HEV's or electric cars, if they really want to drive "green".
Right now, most countries try to shift their electricity production from coal/oil/nuclear power to solar/wind/water etc. In theory this is a great idea, but its relisation will that decades and all nations will have conserve energy to shorten the time. If our demand for electricity rises as fast as new electricity can be produced by using renewable energy sources, we won't be able to close down any plant at all. If you think about the enormous amount of energy needed for personal and public transport you will realise that there is a giant flaw in "buying many HEV's and eletric cars to become green". Our demand for energy, especially electricity for homes (heating, cooling etc.) is rising, even without counting the car industry. Normally, the government would build a more "green" plants and a few storage power stations to deal with the demand. However, the increasing number of electric cars might soon be seen as a problem for this system.

An example:
Let's think about the United States of America. A huge country with more than 300 million people. What would happen if you were to give each one of them an electric car with unlimited range?
A normal electric car needs around 30 kWh per 100 miles. The average American drives 15000 mile per year. That means, each American would use 4500 kWh just for transportation. Multiply this by the number of US-citizens and you will end with a demand for 1350 billion kWh per year. As a comparison, an average US-power plant generates 11,8 kWh billion per year. That would amount to 115 new nuclear power plants. If you are not a fan of nuclear power, let me translate this for you: The hydropower plant at Hoover Dam is one of the country's largest hydroelectric installations and it produces a "mere" 4 billion kWh per year. To satisfy the demand for eletricity, the government would have to build more than 300 installations the size of Hoover Dam.
http://www.eia.gov/tools/faqs/faq.cfm?id=104&t=3
http://www.usbr.gov/lc/hooverdam/faqs/powerfaq.html
http://wiki.answers.com/Q/What_is_national_average_of_miles_each_American_driver_drives_per_year?#slide=2
http://activeemobility.blogspot.co.at/2012/05/how-much-electric-does-ev-use.html