Life is not a linear progression. Most people think it's school-> job -> marriage -> house -> kids -> grandkids -> retirement, but in my opinion, that isn't living, it's existing. I have had houses come and go, went through a divorce, have great kids, a great fiancé, and one constant is that you never know what's happening next. You prepare and plan best you can but always adjusting on the way.
So, if it ever comes down to feeding or caring for the kids or the car, then SELL IT! But if it's a choice between ticking off a check box on the "life list" and enjoying what you already have, then keep it.
Finally, more of a general statement than anything else, home ownership can no longer be viewed as the "must have" investment. Ask anyone who lost half the value of their home and is underwater. I'm not in that situation, I've done fine, but just saying what USED to be the sure fire thing, isn't anymore.
Good luck