One of the major impacts on energy conservation in industrial processes is the integration of heat exchanger network (HEN). Thus, the design of an optimal-cost HEN is a challenging research topic in recently decade. A mixed-integer nonlinear programming (MINLP) stage-wise model by commercial optimization software; GAMS, is developed and allow any split stream flow through multiple exchangers in series as well as bypass stage before mixing non-isothermally for simultaneous synthesis. In the MINLP model where multiple local optima may present, the initialization strategy is therefore developed to systematically find feasible starting point resulting in better HEN design compared to published results in the literatures. In addition, the retrofit of HENs is obtained by adding retrofit constraints to HENS model.