Thinking about this... What orientation were you holding your Cosmo in when you tried this? Was it in keyboard level, typing on flat surface position? Was it in screen level, doing common Android things position?
...
If the orientation sensor and magnetometer are in different portions and the sensor handling code doesn't correctly compensate for this things would go wrong. Potential bug?
Sorry for derailing the thread. We'll be done really soon, I hope... Anyway, in case you were asking me in particular, as stated, I held the screen - not the keyboard - parallel to the ground.
My reasoning is that the main PCB (the actual "phone") sits in the lid. As this reasonably is based on a MediaTek reference design, reasonably intended for a plain no-keyboard smartphone, most electronics, including all sensors, are very likely to sit on that PCB. Prototyping photos from when the Cosmo had become a proper smartphone PCB with a bunch of wires for screen, keyboard and power seems to agree.
As holding a "normal" smartphone in a "normal" way, results in its main PCB being somewhat parallel to the ground, that is reasonably what the sensors are optimized for by default. The bottom part, unique to the Cosmo, should mostly just contain the battery, keyboard, a few connectors and the side button. Those are components that, in regular smartphones, tends to be found on various little daughterboards and flex cables. Thus, I doubt that any of the sensors are in the keyboard part (except the battery temperature sensor, which, likely sits inside the battery).
One thing I didn't mention was that, before I calibrated the pitch sensing (the rotation axis parallel to the hinge), it didn't zero when the screen was actually parallel to the ground, but if I had my Cosmo in the opened position on a table and tilted the entire device backwards until the top of the screen touched the table (at which point the screen is slanting slightly backwards) the pitch would read zero. This makes me believe that someone calibrating the sensors was a bit lazy, and didn't care that the screen isn't quite level in that position.
To calibrate mine "properly", I put my device on a flat surface with the display glass down, with enough of the screen off the table so I could poke at the on-screen calibrate button. That seems to get me to within one degree.
Today I tried holding the keyboard parallel to the ground instead and got much "worse" results. When pointing the device North, it really slowly settled on a heading of 270 degrees, and it was off in any other direction I tried too. Recalibrating the pitch to zero when the keyboard is parallel to the ground made NO DIFFERENCE at all, so it seems the pitch and roll are NOT INVOLVED in any compensation at all - not in the GPS Status and toolbox app anyway - but I think it's true for other apps too, as I guess they can't reach the hardware directly and thus has to use what the hardware drivers are saying. Another phone I had seemed to DO such compensation, as the compass would be accurate even with the screen vertical, but the same app on the Cosmo only finds North (and only North) when the screen is flat to within a few degrees.
Thus it seems really important to keep the screen flat to get ANY kind of useful results (I can at least find North that way), so I'd strongly recommend calibrating pitch to zero when the screen is level, to help with that. Then we need to keep telling Planet that compass works consistently, which means it IS reacting to earths magnetic field, just not in the right way.