Had the same thought, Nick, but then I started looking in the parts manual (ETK) and found that there are two sizes of console: short console (without armrest) and long console (with armrest.) Check out
www.circlebmw.com parts dept/E46 accessories/armrest and take a look at the install instructions; I think there's a reference to cutting carpet or something. I have not actually disassembled the thing but my sense is that this mod might be a little awkward. More power to you, though. Larry