First and foremost, ask for a back-up or alternative e-mail address. Not all consumers want to provide another address, so make this optional. Explain the purpose of the second address in simple language: "If your primary e-mail address becomes undeliverable for any reason, we will send future e-mails (or a notice) to this address." For those consumers who provide a back-up address, you will have a permission-granted means to stay in touch.
Using an e-mail change of address service is also an option, but the permission aspect is troubling. If I give permission to send Brand X e-mails to my account, and then close that account, do I still want to receive e-mails from Brand X? Maybe. A better practice is to send e-mail to customers who definitely request it.
By the same token, don't assume consumers are definitely not interested in your e-mail -- unless they unsubscribe. Since you cannot be certain whether desire is there, regardless of how many e-mails a consumer has ignored, let the consumer decide. I have seen instances where the 13th e-mail was the one that turned lapsed opt-ins into recent responders.
You should, however, take steps to reactivate your non-responders. First determine what the average consumer does with your e-mails -- for example, clicks on one to three e-mails within the first six months after opting in. (Gauge response by clicks, not opens, as opens are not trackable for text and image-blocked e-mails.) Then look at those who do not follow the typical pattern and send a reactivation e-mail with a special offer (discount, coupon, gift with purchase, free shipping) or digital freebie (screensaver, wallpaper, ringtone, white paper) and a we-want-you-back message.
Direct mail can also be productive, but may only be cost-effective for commerce-driven e-mailers.
Perhaps most important, find out why your opt-ins are opting out. Put a brief survey on your unsubscribe page and include a free-form textbox for comments. You can't please all of the people all of the time, but you may identify problems you can fix.
Understanding consumer complaints, developing a reactivation program and collecting a secondary e-mail address will help patch the hole in your leaky bucket.