Do you have another vehicle that wife/partner/baby mama drives (covering most bases not being insulting)? The X3 is the smaller/midsize suv/crossover and I imagine that interior space is smallish. I have an 8yo and a 3yo in carseat behind the drivers seat. I have a MDX and a prius (I know, shoot me) and my '13 raptor. The 3yo kicks my seat in the MDX and the prius, but not in the raptor, either out of respect (lol) or his legs just doesn't reach. Sure, I could switch the seats around but for various reasons, we keep the configuration this way.
You'll most likely have the car seat rear facing and in the middle, so less of an issue for you but something to think about. Hell, you may very well be in a different vehicle by the time it becomes an issue.
Do what you feel is best for you and your family. I think you'd be able to get 35-38k in a private sale. I've considered downsizing myself since we work from home and don't drive much nowadays but the MDX is the wife's and its nice to have a smaller car, much more nimble and quick. I still haven't gotten used to how big the raptor is and thus am not zipping in and out of traffic.