My neighbor built a shed that is technically about 20 feet into my property. While I don't care so much about the shed itself, which is small, it has occurred to me that he could claim adverse possession over the whole path leading up to the shed, as well as the land under the shed. Can he? What should I do to avoid this?
You have good reason to be concerned. In claims for adverse possession, minor infractions can give way to larger ones.
To establish a legal claim of adverse possession, your neighbor would need, under the law of most states, need to use the property in a manner that is hostile, actual, open and notorious, and exclusive for a continuous period as prescribed by your state's statute. Although the statutory period varies from one jurisdiction to another, it is usually at least seven years. Read more about adverse possession claims.
This means that if your neighbor built and uses the shed for the statutory period, and you don't say anything to stop him or assert your ownership of the land, he might be able to establish his own ownership through adverse possession.
More concerning, perhaps, than his "ownership" of the shed and the land underneath it, would be his claim to the 20-feet pathway he's made for himself between his land and the shed. He could claim that he's also used that pathway without any intervention on your part. This trims the size of your property significantly, and could impact the value of your land.
There are a number of steps you might take.
First, having become aware of the issue, you should put your neighbor on written notice. Send a letter or, if you feel more comfortable, have your attorney do so. That letter should make reference to specific property boundaries as described in publicly filed deeds. It should clearly state that you do not consent to the use of your property, and that you revoke any perceived license that the neighbor believes he was given.
Second, you might ask the neighbor to sign a written agreement stating that he acknowledges that the land is yours, and that your permission for him to use the land is freely revocable at any time. Or, you might present him with a rental agreement. Rental agreements are a useful method of preventing a trespasser from establishing adverse possession, because they establish a different understanding of the ownership and usage arrangement.
If your neighbor is friendly, and didn't realize the mistake, he would probably be apologetic. He would also understand your perspective: he built a structure on your land! You might offer that he can maintain the structure, but in exchange for a nominal rent and a written acknowledgement that the property is yours. This way, he can have his shed, and you can have peace of mind, knowing that it won't impact your property rights.