This course covers general introductory concepts in the design and implementation of distributed systems, covering all the major branches such as Cloud Computing, Grid Computing, Cluster Computing, Supercomputing, and Many-core Computing. The specific topics that this course will cover are: scheduling in multiprocessors, memory hierarchies, synchronization, concurrency control, fault tolerance, data parallel programming models, scalability studies, distributed memory message passing systems, shared memory programming models, tasks, dependence graphs and program transformations, parallel I/O, applications, tools (Cuda, Swift, Globus, Condor, Amazon AWS, OpenStack, Cilk, gdb, threads, MPICH, OpenMP, Hadoop, FUSE), SIMD, MIMD, fundamental parallel algorithms, parallel programming exercises, parallel algorithm design techniques, interconnection topologies, heterogeneity, load balancing, memory consistency model, asynchronous computation, partitioning, determinacy, Amdahl's Law, scalability and performance studies, vectorization and parallelization, parallel programming languages, and power.
