Yes, it is the external libraries and toolchain that you need to link against - most of them need to be specially installed alongside their 64-bit counterparts. PAN has a lot of external dependencies :)
This is why I would (personally) install a separate Mingw32 environment so there can be no accidental complications or crossover between the two.
Even once you've solved that issue, some of the Mingw toolchain is a bit different to Linux, or simply has things missing because Windows cannot do it, and so does not support some of the assumptions made by PAN and there are a few little "hacks" necessary to make PAN build and run. I've collected these hacks over time and probably don't remember half of them, so I would have to start from the beginning again for 32-bit.