Refinancing at the right time will provide a number of benefits to a homeowner, but it can also cause a disruption in your normal costs or lower your home equity. Some of the drawbacks of refinancing include paying closing costs, longer loan terms, less equity in the home, and a temporary credit score hit. For homeowners who have paid off a large chunk of their mortgage, refinancing may not make financial sense even if the interest rate is dropping.