Yes, but the Taxpayer's Bill of Rights discourages the IRS from seizing primary residences. Also, the IRS doesn't like the negative publicity generated when it takes a home. Furthermore, IRS collectors cannot decide on their own to seize your home. The IRS must first get a court order, which you can contest.
Nevertheless, if the IRS collection division has tried -- and failed -- to get any cooperation from you (for example, if you have not answered correspondence or returned phone calls, lied about your income, or hidden your assets), the IRS may go after your residence as a last resort.