Heterogeneous GPU allocation, training/serving placement, workload packing, parallelism, and dynamic replanning