Combining simulation and optimization: Multipurpose modelling of camera-based optical metrology systems

When trying to build the best possible optical 3D sensor, there are a lot of different problems to tackle: What is the best sensor design? Where should the specimen under test be located? How should system parameters like f-number be set? What are the best calibration parameters to describe the system? As it turns out, these seemingly different optimization tasks are closely related and can be addressed using the same model of system behavior by only varying the parameters to be optimized as well as the specific loss function. In this contribution we will show how especially driven by the current development of gradient-based optimization methods in the field of artificial intelligence, the same differentiable model can be used for multiple purposes in a generalized framework. It is presented how particularly camera-based optical metrology and imaging systems can be calibrated, simulated, and optimized with this approach, and how these distinct modelling purposes can benefit from each other.