To take advantage of a Kalman filter you need to know the "uncertainty" of your measurement, so basically know when the measurement is getting bad.
Also good sensor/measurement models that can at least approximately deal with, e.g. thermopiles near buildings, vibrations on IMU, ...
These are hard to model.
Indeed. However, the thermopiles with 6 sensors can easily know whether or not there is contrast. And the accelerometers can detect when the noise band becomes larger than half of gravity. So switching not impossible.
Yes, you can detect if contrast get's bad, add some simple heuristics to convert that to measurement noise. You still have a very simplified measurement model and you have to choose your parameters conservatively to account for the modelling errors. The bigger problems are that you can't detect some bias because you are flying close to a mountain/structure or stuff like reflections..
Also it's hard to detect when you are really starting to get problems with accel measurements if you have high vibrations from the start, e.g. problems might only start if you get aliasing or other weird effects you only see at certain vibration frequencies...
It's definitely possible to combine them... just saying that is not quite straight forward if you want to get good performance and robustness in cases where one sensor gives you "bad" measurements. You either need to model these errors (if possible, e.g.a systematic bias) or be very conservative and just artificially increase the measurement noise a LOT.
Only switching to a fallback sensor if one get's really bad is a bit easier...