You will not get any punishment. If anyone says something, tell them you have a lawyer. I will be your lawyer if anything happens.
Many Pundits create misconceptions among people that something will happen, or women should not chant Gayatri Mantra. All this is wrong, there is nothing like that. Chant with love and not out of fear.