12_Vertex AI

Question 1

CPU vs GPU vs TPU

CPUs

• Quick prototyping that requires maximum flexibility
• Simple models that do not take long to train
• Small models with small, effective batch sizes
• Models that contain many custom TensorFlow/PyTorch/JAX operations written in C++
• Models that are limited by available I/O or the networking bandwidth of the host system

GPUs

• Models with a significant number of custom TensorFlow/PyTorch/JAX operations that must run at least partially on CPUs
• Models with TensorFlow/PyTorch ops that are not available on Cloud TPU
• Medium-to-large models with larger effective batch sizes

TPUs

• Models dominated by matrix computations
• Models with no custom TensorFlow/PyTorch/JAX operations inside the main training loop
• Models that train for weeks or months
• Large models with large effective batch sizes