Put some soft cloth where that fan is at. Also line the exaust duct with cloth. This will absorb sound waves. Make sure any contact point that the actual fan itself is contacting is sturdy. That alone will help to reduce vibration noise. Vibration will cause the most noise opposed to sound waves
Just today I installed this small fan inside a closet underneath the stairwell that, through the sheetrock, will vent to the crawlspace under the house. The fan is too heavy to mount to sheetrock directly, so the exhaust hole was bored right next to a stud so I could at least use one lag screw to anchor this thing on. But after spending an hour mounting it and turning on the fan (both at full speed and with a Active Air speed controller) the vibration in the sheetrock and the wall was enough to rattle out ur fillings. Well, not quite that bad, but I wanted the vibration gone, and damned if I couldn't find a way to dampen it with rubber/felt/foam/washers/etc. Every possible mounting option was explored and attempted and I finally had to settle on hanging the damn thing from string off a lag bolt, and to stop the swaying using a single metal screw with a hand-fashioned foam "collar" to hold the flange to the sheetrock. It's clear -- I could not allow the metal screw to contact both the fan flange and sheetrock/stud, or else immediate and annyoying vibrations result. I absolutely could not find a way to do this when mounting the flange directly to the sheetrock/stud. As you may see in the photo, the flange is not tightly coupled to the sheetrock lest vibrations begin...and I would argue that it's quite likely the sheetrock itself is not well fastened to the studs leading to the original issue. Anyway, a hanging mount will work, even if it's not alltogether rigidly assembled.
You can put a towel, buy a muffler, build one, or increase the size of your fan. I foolishly bought a muffer when it came all it was was sheet metal with egg foam on the inside...I could of built that with a box and egg foam from home depot for 5 bucks. I have a 4 inch fan and have it on max and its hella loud why I got a muffer, but soon going up to a 6 inch fan. A 6 inch fan at half speed pulls the same air that a 4 inch one does at max and is considerably quieter from larger blades. Larger blades equals less noise....why on quite PCs they use large diameter fans vice small ones.