The course will cover hardware approaches for implementing neural-inspired algorithms. In recent years, machine learning and AI have re-emerged as effective solutions to a number of difficult and economically relevant problems. They will likely enable autonomous vehicles, healthcare solutions, assistive technologies, etc. These solutions will be deployed in datacenters, mobile phones, self-driving cars, and sensors. The course will start with a brief primer on why machine learning has made significant strides in the past decade. We will then move to discussing specialized processors (accelerators) that can efficiently execute a large family of machine learning algorithms (for both inference and training). We will focus our discussions on accelerators for artificial/spiking neural networks, and convolutional neural networks -- areas that have dominated recent architecture conferences. We will end the course by discussing how the learned concepts can apply to other relevant application domains, e.g., genomic analysis.
The course does not have any formal pre-requisites, but is intended
primarily for graduate students with some familiarity in architecture
and/or machine learning. The lectures will be self-contained, i.e., I will
provide sufficient background in architecture and machine learning to
make the material accessible. Most class lectures will be based on
recent research papers (see tentative schedule below). Students will
also work in groups on semester-long projects -- the projects will
compare the implementations of various cognitive tasks with different
algorithms and hardware approaches.
College of Engineering Policies (Disability, Add, Drop, Appeals, Safety, etc.): Guidelines from the college.
Class rosters are provided to the instructor with the student's legal name as well as "Preferred first name" (if previously entered by you in the Student Profile section of your CIS account). While CIS refers to this as merely a preference, I will honor you by referring to you with the name and pronoun that feels best for you in class, on papers, exams, group projects, etc. Please advise me of any name or pronoun changes (and please update CIS) so I can help create a learning environment in which you, your name, and your pronoun will be respected.
The University of Utah values the safety of all campus community members. To report suspicious activity or to request a courtesy escort, call campus police at 801-585-COPS (801-585-2677). You will receive important emergency alerts and safety messages regarding campus safety via text message. For more information regarding safety and to view available training resources, including helpful videos, visit SAFEU.
Dates | Lecture Topic | Slides | Annotated Slides |
---|---|---|---|
Tue Aug 20 | Overview, landscape, history of neural-based hardware | ppt pdf | ppt pdf |
Thu Aug 22 | Intro to Deep Learning Algorithms | ppt pdf | ppt pdf |
Tue Aug 27 | Custom SIMD Architectures: DianNao | ppt pdf | ppt pdf |
Thu Aug 29 | The DaDianNao Architecture | ppt pdf | ppt pdf |
Tue Sep 3 | Deep Compression | ppt pdf | ppt pdf |
Thu Sep 5 | Deep Compression Architectures | ppt pdf | ppt pdf |
Tue Sep 10 | Systolic architectures: Eyeriss | ppt pdf | ppt pdf |
Thu Sep 12 | Commercial architectures: Google TPU, Tesla FSD | ppt pdf | ppt pdf |
Tue Sep 17 | Commercial architectures: NVIDIA Volta, Graphcore, Intel NNP. Training intro. | ppt pdf | ppt pdf |
Thu Sep 19 | Training Innovations: vDNN, ScaleDeep. | ppt pdf | ppt pdf |
Tue Sep 24 | No Class | ||
Thu Sep 26 | Analog Accelerators: ISAAC | ppt pdf | ppt pdf YouTube videos: Part 1 and Part 2 |
Tue Oct 1 | Spiking Neuron Intro | ppt pdf | ppt pdf |
Thu Oct 3 | TrueNorth, Projects discussion | ppt pdf | ppt pdf |
Tue/Thu Oct 8/10 | Fall Break, Take-Home Midterm Exam | ||
Tue Oct 15 | Comparing SNNs and ANNs | ppt pdf | ppt pdf |
Thu Oct 17 | Self Driving Car Pipeline | ppt pdf | ppt pdf |
Tue Oct 22 | Exploiting Variable Precision | ppt pdf | ppt pdf |
Thu Oct 24 | Simba | ppt pdf | |
Tue Oct 29 | Projects Discussion | Notes | |
Thu Oct 31 | Ineffectuals | ppt pdf | |
Tue Nov 5 | In-Memory Processing and Projects Discussion | ppt pdf | |
Thu Nov 7 | Gradient Overheads, 3D CNN Architectures | ppt pdf | |
Tue Nov 12 | Sequence Alignment Basics | ppt pdf | |
Thu Nov 14 | Molecular Dynamics Accelerators | ppt pdf | |
Tue Nov 19 | Sequence Alignment Accelerators | ppt pdf | |
Thu Nov 21 | Systolic Arrays I -- sort, matrix mult | ppt pdf | |
Tue Nov 26 | Systolic Arrays II -- add, mult, eqn solver, graphs | ppt pdf | |
Thu Nov 28 | Thanksgiving | ||
Tue Dec 3 | Project presentations | ||
Thu Dec 5 | Project presentations | ||
due Dec 15 | Take-Home Final Exam |