Optimization with Unavailable Gradient Information

In this chapter, we consider an optimization problem with unknown gradients of the objective function. Since the target function itself is often non-concave and non-convex, one usually needs to implement iterative derivative-free optimization methods. We consider another perspective of leveraging a DNN to approximate the objective function, and use available gradient information from the trained DNN for optimization. The solution is fine-tuned by an existing derivative-free optimization method to correct potential approximation errors. We apply this DNN -assisted optimization method to a case study to optimize the graphical approach for multiplicity control in Phase 3 clinical trials. This is an impactful application to increase the probability of achieving multiple study objectives with a strongly controlled family-wise error rate (FWER). Simulation results show that the DNN -based method has a good balance of robustness, time efficiency, and the performance in identifying an optimal value as compared to some existing derivative-free optimization methods.