Games and movies don't really scare me. "Scary" just isn't scary. Blood, guts, monsters, zombies... none of that is scary. Yes, most good horror games have that kind of stuff in them, but that's not what makes them scary.
Sudden jump scares and loud noises also aren't scary. They're merely startling. They're cheap tools to cause a moment of adrenaline. Which is fine, if you use them sparingly. If used too much, or if the timing is wrong, it can be really cheesy.
Go for creepy instead. Don't reveal your entire story at once. Do it with little clues here and there. Lead the player one way with your clues, then give them a twist later on, BUT DON'T LIE TO YOUR PLAYERS. Red herrings are fine, but outright false information is bad.
Use a "hook." Examples are the children's rhyme in the Nightmare on Elm Street movies. Or the "Ch-ch-ch ah-ah-ah" noise in the Friday the 13th series for when Jason is near. These sorts of things, and other music or sound cues, can be pretty effective when used right. The Ratman murals and crazy writing on the walls are another example of a hook. Those little hidden areas really lend a creepy vibe to the game.
Silent Hill uses a hook as an actual game mechanic with the radio static noise. Whenever you hear it, you know something is nearby, and if you can't see right away then it then it's all the creepier.
If you're especially good you can make your hook a clue in itself. The game Deadly Premonition does this all throughout. The main character (York) seems to be speaking to an alternate ego or imaginary friend (Zach) throughout the whole game. And every time he asks "right, Zach?" he taps the side of his head with two fingers. He does this repeatedly throughout the game. I don't want to spoil the meaning for anyone who hasn't played the game so I'll make it real small:
[size=30:8jqiz264]Tapping on his temple with two fingers isn't just a gesture to indicate that he's talking to an imaginary friend. It turns out that the main character, York, is actually the alternate ego and "imaginary friend," and Zach is the real person who's mind is trapped inside. When he was a kid, Zach saw his father shoot himself in the head and was traumatized. The two fingers tapping the temple is a pantomime of pointing a gun at your head, and the words "right, Zach?" were his father's last words right before he pulled the trigger. When you realize that York has been mimicking Zach's father's suicide through the whole game, it's a really friggin creepy moment.[/size:8jqiz264]
Additionally, you should show the consequence of failure before attempting a task. For instance, the zombie munching on the corpse at the beginning of Resident Evil. Or something lurking at the surface of the water for a moment and disappearing before having to cross a lake. Or something falling down a chasm you have to traverse, like you accidentally kick a rock over the edge with your shoe or something. Whether they notice it or not it gives the player a subtle "I don't want that to happen to me" feeling while they're playing, which wouldn't necessarily be there if the consequences weren't made apparent. Again, this is a trick that should be used sparingly or else it gets old.