i reckon the pushrod falling out would have been from a sticking valve or lazy lifter (sitting ages issue)   that "lethargic" Intake Cylinder 1 valve, if it is moving less (appears so) if you were comfortable undoing the rocker, pulling the push rod and then finding a way to pull that lifter out (i find a sharpened bit of coat hanger wire can hook them out, might need to lift them up and down several/20+ times to get the carbon off them) you can see if the lifter is wiped out (if so, the cam will be wiped out)    if it's not causing issues (other than running a bit crap) i'd just drive it and see how much you like the car etc. 
i had a crossflow with a wiped cam, chewing lifters once, it would wear down one lifter and get all clacky and i'd pull the worn out lifter and fit another second hand one in, get another 10,000km out of it and repeat. (find another engine in the meantime)    if you end up fixing everything that's worn, you'll be clocking up a decent chunk of money (price a cam, lifters, timing chain for eg i doubt you'd be under $500) and then you still have a worn out rings/bearings engine and a full reco would be up around $4000 by the time you did hoses, belts and coolant/oil/ plugs/cap/leads and all the rest.   fair chance someone with a crossflow fitting a V8 will give a good running one up for $250 odd if you keep your eyes peeled 

I noticed that the cylinder 1 intake valve looked lethargic so I recorded it. What do you guys think?

Still runs like crap but the knocking is gone. Still popping tho so I'll check the valves. I took this video of it for your viewing pleasure. Let me know if you see something off

@SPArKy_Dave What's the torque spec for the rocker arm bolt?

Something seems off

Look for loose rockers   If ur keen to go further, unbolt/remove the intake rockers, then (firmly) tap each valve stem with a nylon/soft blow hammer. (using a normal steel hammer will damage/mushroom the valve stem tip)   Tap hard enough, to slightly compress the spring/bounce the valve off it's seat and back. Listen to the note.   You'll hear any stuck valves.

I'll pull the cover tomorrow. What specifically should I look for?

That's 100% a stuck intake valve/valve not sealing. Just need to confirm which cylinder.   Have u pulled the rocker cover yet?   After sitting for yrs, my XF wagon had x3 stuck intake valves. With alot of patience, I attempted to free them using penetrene down the valve stem, and a nylon hammer. That partially worked.   In the end, I removed the spark plug/s, and using a drinking straw as a feed/guide, fed a whole heap of thin nylon rope (from bunnings) into each problem cylinder, leaving the rope end sticking out the spark plug hole. Next I rotated the engine, so the piston sandwiched the rope against the underside of the valves.   With that done, I removed the valve spring, wrapped the valve stem in some tape as protection, then fitted a cordless drill to the exposed valve stem and ran the drill back and forth with more penetrene down the valve stem/guide.   Doing the above, eventually got them loose.       Ur carburettor is ok. RPM's will always increase with a vacuum leak - ur introducing extra air into the engine, same as opening the throttle.

I just checked for crossfire but didn't see any however I noticed that the engine revs go up significantly when I pull a vacuum line. Could a fucked up carby cause any of this?

The dipping in vacuum is in sync with the popping so I think a sticky valve is possible