The VIMOS Public Extragalactic Redshift Survey (VIPERS): Environmental effects shaping the galaxy stellar mass function