Its more like "An OS on a stick". Calling an OS a computer is a bit misleading.
You put the USB stick into the computer (see, you need an additional REAL computer) and boot an entire OS (like Linux) from the USB stick.
So this way you can carry an OS and your data files etc on a USB stick - and use your preferred OS from any computer you have access to. Some even put application on it (after all you need application to work with the OS right)... Linux already comes with most needed apps with the distribution itself.