Not very. Fewer than 2% of us are ever investigated for tax fraud. And if you are, the likelihood of a civil fine or criminal charge is under 20%. The unofficial minimum amount of taxes owed before the IRS will file criminal fraud charges is over $70,000, in cases involving at least three years of fraud. To some extent, whether you are charged depends on your line of work. Most of those prosecuted for tax fraud work in organized crime or are public figures. For more information, see Nolo's article Negligence Versus Tax Fraud: How Can the IRS Tell the Difference?
For detailed information on all aspects of IRS audits, see Surviving an IRS Tax Audit, by attorney Frederick W. Daily (Nolo).