Exchange involving adjacent lot

My primary residence is waterfront. The lot to my immediate east is for sale. Both lots are approximately 40,000 square feet apiece. I want to buy the lot and have new surveys for each property and take approximately 5,000 square feet from the vacant lot and add it to mine. Zoning is R-15 so there is no issue there. Without going into all the details, by removing several large trees and dozens of large bushes my view will be enhanced tremendously. Also, due to the unusual shape of my lot, the 5,000 additional SF will make my yard far more appealing, while having a minimal impact on the vacant lot. I intend to sell another water lot I own in another town and do an exchange to buy the vacant lot next to mine. My question is this, if I buy the vacant lot while doing a reverse exchange, will changing the size of the lot and adding it to my primary residence lot be legal? The legal descriptions of both lots will stay the same, but the sizes and values will certainly change. Something tells me the IRS might consider it mingling investment property with primary residence and void it. I may have to buy it without doing an exchange.

The legal descriptions of both lots will stay the same

uh…no it won’t. So many feet East to such and such marker thence so many feet North Northeast to next marker…all that changes.

Having done it once (to avoid problems with a sale we deeded a 1-foot strip to a neighbor that he had a permanent carport built on) I can tell you that changes to a plat have to go through city officials who record such things. Surveys, partial deeds, lots of paperwork.

I’m not sure how the replat would affect the taxability of the exchange.

Due to a garage encroachment at the time of my original purchase I already went through that with the property and there was no change other than new surveys for both. No metes and bounds descriptions to contend with. They added about 50 SF and there was no change. There is no one size fits all for setbacks and encroachments; however, I will have to face city zoning and eventually the assessors when I make changes which involve the shoreline and that setback.