You are using "--save webcam.yuyv", but what are you saving the data to? SD Card? RAM (/tmp)?
It looks like for your experiment that you are only capturing 1 frame ( delay 1 second, skip the next 2 frames, save the 3rd then exit). But is that your final goal, or do you eventually want to continuously stream in data at some frame rate?
YUYV ( YUV422 ) would be 640x480x2 = 600kBytes
I wonder if you increase the speed of the kernel jiffies timer if it would reduce the latency of passing of packets from the USB driver level to the V4L2 level and effectively increase your overall bandwidth.