The current X4’s USB-A ports are 10 Gbps, which is very nice. That makes them compatible with 10 Gbps USC-A to USB-C adapters. I think they’re more flexible as-is than USB-C would be, and probably a bit sturdier. IMO, of course. 
An N150 or N355 configuration would be great, but what I’d really love is something like a PLCC slot to do RAM upgrades.
I’d like the GPIO and PoE/12V power pins to be longer. With the current setup, once a hat is attached, there’s no room to mount a decent heatsink on the NVME or any sort of NVMe to whatever adapter with any thickness. I’ve got a 40mm Noctua taped to the side of my X4 with PoE hat to cool the NVME, and it works great, but it’s spectacularly unwieldy.
Honestly, I’m probably in the minority, but I think the X5 should be bigger. A Raspberry Pi-like form factor is super-useful, but since the X-line is not compatible with Pi cases, I don’t think it has to be quite that small. Maybe 1.5-2x the size of the Pi. There’d be more room for components like a proper PWM fan header. Maybe even room for a RAM slot.