Hi all,

Please see the report from SIG MindSpore Compiler - Auto Parallel

 

#MindSpore Compiler Special Interest Group (SIG)

This is the working repo for the Compiler special interest group (SIG). This repo contains all the artifacts, materials, meeting notes and proposals regarding ANF IR, auto differentiation, auto parallel, graph optimizer, VM and any other programs for high level graph compilation in MindSpore. Feedbacks and contributions are welcome.

Auto Parallel:  Auto Parallel is a novel approach to achieve automatic parallel training and inference, including data parallel, model parallel and hybrid parallel. Auto Parallel is part of MindSpore graph compiler, and aims to improve the usability and performance of model training and inference. 

 

#Current Progress：

Support resnet50 auto parallel training.

#Development plan:

We plan to support ReID, recommendation, and NLP models, such as resnet, wid&deep and bert, in next few months. At present, we have support resnet which used in ReID. Some key feature  plan to do: 

1. Support GPU/CPU backend. At present, only support Ascend.

2. Support auto parallel inference, using multi device to speed up inference, and run bigger model.

3. Improve cost model to search a better performance parallel strategy.

4. Develop more parallel operators.

 

#SIG Leads

#Logistics

#Discussion

Meeting notes