Ubuntu isn't the desktop, it's the whole system. And if you kill the desktop, the desktop manager (probably KDM or GDM) will start a new one. It sounds like you want to run in console mode with no graphical desktop. If it's embedded, are you sure you're running a desktop?
It might be as simple as the command (as root): telinit 3
That would switch you from graphical mode (5) to console mode with network (3). It would log you out and you'll have to log in again, but directly to a non-graphical bash shell.
Otherwise, you can configure the "runlevel" to boot into. That might be as simple as editing /etc/inittab and changing "5" to "3". I say "might" because everything has changed in recent years, plus I'm not currently running Ubuntu, plus you're running embedded ARM, which might be different than Intel desktops.
Newer systems change run levels something like this (but investigate before doing this):
cd /lib/systemd/system
ln -sf runlevel5.target /etc/systemd/system/default.target