TE: Multiply and Fill the Earth

"Be fruitful and multiply". Teachings exist indicating that this is a command to us; that if we don't multiply then we are somehow sinning. But how does the idea of this being a command hold up when applied to everyone? Is it a sin to not have children or to remain single?