Installing VirtualBox Guest Additions on Fedora 16 and 17 fails because it looks like the kernel headers are not installed by default. After you’ve installed Fedora to VirtualBox, you can manually install the required kernel headers with these steps (summarized from this post)
sudo yum -y update kernel sudo yum -y install kernel-devel kernel-headers dkms gcc gcc-c++
Reboot then install the VirtualBox Guest Additions from the Devices menu.