In this chapter, we discussed ways to improve system efficiency in model parallelism training.
After reading through this chapter, you should understand why vanilla model parallelism is very inefficient. You should also have learned two techniques to improve system efficiency in model parallelism training. One is pipeline parallelism; the other is intra-layer split methods.
In the next chapter, we will discuss how to implement a model parallelism training and serving pipeline.