machine learning achieves tremendous success in image classification, speech recognition, and medical diagnosis. in the meantime, machine learning also brings some interesting mathematical questions: how to solve the resulting optimization more efficiently and robustly? how to apply the machine learning technique to tackle challenging problems in mathematics. in this talk, i will view the neural networks model from a nonlinear scientific computing point of view and present some recent work on developing a homotopy training algorithm to train neural networks ``layer-by-layer'' and ``node-by-node.'' i will also review the neural network from numerical algebraic geometry point view and provide a novel initialization for relu networks.