I noticed when I change banners on my profile page, the old ones are saved. Is there a limit to how many you can save and can I delete banners once they're saved?

So far I don't find any of limit how many banners you can save. And you can't delete any of banners you save here.

The reason you can't delete banners (or backgrounds) is probably because you've spent gold buying them in the first place. If you deleted them, you wouldn't have what you spent the gold for any more. They might make it possible to delete these things in exchange for returning the gold in future.