Yes, If being purley "green" is your concern, one should consider emissions used to make the panel, or whatever engery and other resources went to construct your energy source of choice. It applies to the enviromental impact of dams or mines, concrete and materials haulage, and other factors, and disposal of spent fuel (especially nuclear), and disposal of the facilities and/or equipment at their end of life.

All that said, as an investment to not pay the power companies, solar might seem that much of a better deal when those panels reach the market.