Look for loose rockers   If ur keen to go further, unbolt/remove the intake rockers, then (firmly) tap each valve stem with a nylon/soft blow hammer. (using a normal steel hammer will damage/mushroom the valve stem tip)   Tap hard enough, to slightly compress the spring/bounce the valve off it's seat and back. Listen to the note.   You'll hear any stuck valves.

I'll pull the cover tomorrow. What specifically should I look for?

That's 100% a stuck intake valve/valve not sealing. Just need to confirm which cylinder.   Have u pulled the rocker cover yet?   After sitting for yrs, my XF wagon had x3 stuck intake valves. With alot of patience, I attempted to free them using penetrene down the valve stem, and a nylon hammer. That partially worked.   In the end, I removed the spark plug/s, and using a drinking straw as a feed/guide, fed a whole heap of thin nylon rope (from bunnings) into each problem cylinder, leaving the rope end sticking out the spark plug hole. Next I rotated the engine, so the piston sandwiched the rope against the underside of the valves.   With that done, I removed the valve spring, wrapped the valve stem in some tape as protection, then fitted a cordless drill to the exposed valve stem and ran the drill back and forth with more penetrene down the valve stem/guide.   Doing the above, eventually got them loose.       Ur carburettor is ok. RPM's will always increase with a vacuum leak - ur introducing extra air into the engine, same as opening the throttle.

I just checked for crossfire but didn't see any however I noticed that the engine revs go up significantly when I pull a vacuum line. Could a fucked up carby cause any of this?

The dipping in vacuum is in sync with the popping so I think a sticky valve is possible   

Yeah I'll get a vid of the vaccum gauge but it does seem to be jumping at idle and cylinder 1 seems to do nothing. It is making a rattle at the front of the engine aswell. For the crossfire test do I just spray water at the dizzy?

They clatter like nobody's business. Could be a sticky intake valve, but I'd expect to hear valve-train ticking?   A vacuum gauge, would show a sticking intake valve as momentary positive pressure. (needle would flicker to low vacuum)  

The engine sounds healthy, it's just missing on one cylinder.   @deankxf, I'd check for spark leakage in the dark, with a spray bottle of water.   And also do a cylinder drop test - with engine running, remove/refit each plug lead in turn at the spark plugs, and listen for engine note change/rpm drop (advise to pre-loosen all leads before starting engine, then use gloves/pliers to avoid zappage)

I think deans on it, only other thing it could be is a valve not seating

@Thom reckon this sounds like  cross firing (crap dizzy cap or leads) or a sticky valve..  seems to run smooth at some point so it's surely not a pushrod?  what's a sticky lifter do when they go bad?

definitely odd, my first thought would be to look at the dissy cap for cross firing (you can do this in the dark possibly, as Dad's 351 was like one of those millennium balls at night when it was sprayed with rusty water from the radiator)