During the recycling process, the plastic windows are easily removed once the paper is soaked in water. Paper recycling can cope with small impurities like plastic windows, staples, and tape. 

So recycle these paper envelopes like any other recyclable paper product.