You retain access to the researched abilities and bonuses, but you can not build structures or ships that require that number of research bases to build. For example, weapon / shield advances or planetary colonization techs would stay, but if you dropped down to 4 military bases, you can't build Heavy Cruisers anymore.
You probably Don't want to scuttle your research bases....unless someone is about to blow them up anyhow.