You have to keep them clean. Filter media traps detritus and crap, and that stuff rots in the filter. But the water keeps flowing through it, so a lot of that ends up getting washed back out into your tank.
A protein skimmer completely removes that waste from your system (into a cup) so it's not getting washed back into the tank.
If you have a filter, you need to thoroughly clean it frequently, like once a week. Or else you are just leaving all the bad stuff in your system.