TrAC Parallelism in Deep Learning
The Parallelism in Deep Learning digital badge focuses on theory and application of how to use different methods of parallelism in deep learning, leveraging data parallelism and model parallelism workflows for AI models on HPC infrastructures. The Parallelism in Deep Learning course is offered by Iowa State University's Translational AI Center (TrAC) and is part of a larger Advanced AI Techniques pathway program and intended for audiences within the spectrum of the software and technology industry, including software engineers, data scientists, data engineers, data analysts, research scientists, and software developers.
Skills / Knowledge
- Artificial Intelligence
- Deep Learning
- High Performance Computing
Earning Criteria
Required
The Parallelism in Deep Learning Badge is earned after successful completion of a 4-week, asynchronous, self-paced online course consisting of 4 modules. The 4 modules cover essential topics in theory and application of how to use different methods of parallelism in deep learning, leveraging data parallelism and model parallelism workflows for AI models on HPC infrastructures.
This course offers a blend of hands-on activities, assignments, video lectures and tutorials.
Learning Outcomes:
Explain the need for parallelism in deep learning and its impact on scalability
Construct workflows leveraging data, model, and hybrid parallelism in distributed environments
Evaluate optimization dynamics, including advanced techniques and optimization methods
Refine resource utilization strategies by maximizing GPU/CPU efficiency during distributed training
Assessment:
Participants will be assessed on:
Engagement with each module
One exercise in setting up a compute environment for parallelism in deep learning
One coding assignment centered on training a model using data parallelism and model parallelism
2 quizzes demonstrating basic knowledge of parallelism in deep learning
About TrAC
The Translational AI Center will break down disciplinary silos to bring together core Iowa State artificial intelligence researchers and subject matter experts interested in applying new technologies to their work. The center will initially focus on conducting core artificial intelligence research, as well as pursuing five application areas of artificial intelligence:
Materials design and manufacturing
Biology, healthcare, and quality of life
Autonomy, intelligent transportation, and smart infrastructure
Food, energy, and water
Ethics, fairness, and adoption.
In addition to serving as a scientific hub for translational artificial intelligence, the center will organize research seminars, host workshops, training, and onboarding programs, offer seed funding for research projects, and serve as an intermediary between private industry partners seeking research services and appropriate university faculty.