This is caused by a Saiyan looking from the moon to the planet (the planet acts like a moon). The Oozaru is incredibly strong and only a saiyan who has completed this transformation can achieve Super Saiyan 4. For the record, this is only done in Dragonball GT, a sequel of Dragonball Z.
The ones who achieved this goal are: Goku, Vegeta