Smart machining technology is now under intensive research worldwide. As a kernel technique of smart machining, on-machine measurement (OMM) technology can automatically measure tool diameter and length with laser tool setters on machine so that the machine controller compensates for the tool wear while machining, which can improve part accuracy without manual tool measurement on tool pre-setters. However, there is a dilemma that the OMM technique cannot predict tool failure so that the operator can replace tools before it fails. To address this problem, this research proposes a new approach to predict failure of round-insert face mill in rough and finish machining to automatically change tools right before their failure. First, the geometric equation of flank wear land width of round-insert face mill is formulated; second, after a tool diameter is measured, the flank wear width is calculated. Third, an experimental method is proposed to determine the tool radius reduction threshold and the tool location of measurement, and then the tool failure can be predicted in rough machining. Then, a new method is established to determine the criteria of tool radius reduction in finish machining according to the machined surface roughness. Finally, several experiments are conducted to verify this approach, and it is applied to a practical example. This approach can be directly applied in industry, and it can advance the smart machining technology.