You'll need to fill in the gaps around the fan blades. I did this with pink foam insulation. After that modification,place the filter on the back of the fan. If the filter suctions onto the fan that's a good sign you're pulling air through the filter and not from the front of the fan.
There's going to be some reverse flow around the corners of the exhaust-side, but net net, so long as you've got sufficient flow and a sufficiently-rated filter, you'll be moving air through the filter and trapping particles within it.
A circular exit mask might help slightly, but you're talking a few percent efficiency improvements, not a quantum leap in efficacy.
Simple, cheap, available, and effective are the goals here. Diminishing returns set in early.
