The easiest solution is a 4" hole saw. One hole at the top for exhaust, two at the bottom for intake.
I just have to say I bought the 110 cfm model of this fan a few months ago and it was awful, a complete waste of money. After about 4 days it started getting louder (and it wasn't so quiet to begin with) ... My 6" vortex is much quieter.
Did you buy the Panasonic Inline? Or a Panasonic Ceiling fan like the one in the picture, 1. post.?