The problem may lie with your fireplace. It is more than likely that your neighbor's smoke and cooking odors are drifting above the town house and entering your house through the fireplace. Even if you keep the damper closed, there may still be a backdraft that is taking place. This might occur when the exhaust fan is operating or the clothes dryer is on. Cover the front of the fireplace with a plastic seal, making sure that it is airtight, and then see if the problem persists. If it does, you will at least know that the odors are not entering your house through the fireplace. If the fireplace is the problem, you might think about sealing it permanently if you do not use it. The problem may also lie with your furnace chimney. Check for odors in the furnace room while exhaust fans are operating and the furnace is off. If the furnace chimney is backdrafting, you will need a combustion air supply to the furnace room.