If this occurs only when you turn on the hot-water faucet, it may be due to an accumulation of sludge in your hot water tank. You can fix this by flushing the tank with water. Add a hose to the spigot at the bottom of the tank and run it to the floor drain. Run water from the tank until the water turns clear. Homeowners are recommended to perform this task annually. If the rusty water is coming out of the cold-water faucet, it is coming from galvanized steel pipes. The pipes have rusted on the inside and the rust is being carried through the pipes every time you turn on the faucet. There are two possible solutions to this problem. The first is to replace the pipes. If the galvanized steel pipes are part of the piping system within your house, you can replace them with copper pipes. If the affected pipes run from the house to the street, however, then a major digging operation will be necessary. The second, easier, solution is to add phosphate crystals to a filter unit that is installed just after the main water valve. Over time the crystals form a coating on the inside of the pipes that hold the rust in place.