You should consider refinancing when you can get a lower rate on your existing mortgage. This can happen when market rates move down. Or when you increase your income or credit rating. Or perhaps when you simply found a better deal in the market.
However, when you refinance your house, you need to pay fees. Therefore, before you go ahead with refinance you need to make sure you gain more in lower rates than you spend in fees. Read the exact terms offered to you, and calculate how much you will pay on your current loan over the next 3-5 years. Then compare this to the amount you pay on a new proposed loan, including all fees. If the new loan wins, go for it.
Sometimes, the new loan will only be a better deal if you plan to stay with the mortgage for a longer duration, say 10 years. In that case, the choice is harder. Who knows, maybe rates fall down further, and you decide to refinance again after 3 years. Or you might want to move to a new house. Nobody can give you the perfect answer, so such decision will always be a gamble. Perhaps, it is better to wait for a better opportunity, since the hassle of refinancing is not worth it if it takes you 10 years before you offset upfront fees with the better interest rate.
While you always have to be careful about trusting other people with your money, this kind of calculation is something a financial advisor could help you with. Just follow his or her calculations very carefully to make sure you understand what is going on.
You may also need to refinance if you need additional cash, although this cash-out refinancing is quite dangerous. Of course, you should try to time your cash-out refinancing to the period of lower market rates, if at all possible.
