They remain until the game ends, but you don't have to fight them if you don't want to. Of course that, sometimes, it can be a problem if they are inside a city you want to capture, but I wouldn't recommend you to fight the neutral troops outside cities, unless you have a lot of extra troops.
----
"Whenever death may surprise us, let it be welcome if our battle cry has reached even one receptive ear and another hand reaches out to take up our arms".