Date

Topic

Readings, other comments on the topic

Thought questions

Thu Aug 29

1. Overview of our topic: bringing machine learning to the edge.

 Just a get-to-know you meeting.  Ken will probably show some Microsoft slides from a recent MSR faculty summit where they told us about Azure IoT Edge and "Intelligent Edge".

Chris, Bharath and Ken wrote a white paper on the topic we discussed today: 

Cloud-Hosted Intelligence for Real-time IoT Applications. Ken Birman, Bharath Hariharan, Christopher De Sa. 2019. SIGOPS Oper. Syst. Rev. 53, 1 (July 2019), 7�13. DOI:https://doi.org/10.1145/3352020.3352023

If interested in topics we touched on and keen to read more, good places to start are these Wikipedia pages: 

* The Price of Anarchy

* The Tragedy of the Commons

* Causal ordering model.  The classic paper on this topic is here.

* Consistent cuts.  The classic paper on this topic is here.

Tue Sept 3

2. Coordination in an edge environment 

We touched on the idea of consistency in one specific situation, but not in a very systematic way. n my lecture I'll pose the problem in a more careful way, and then we will look at variations that have developed for different settings: data centers, wireless swarms.  

A classic practical question in distributed computing involves the best ways to "package" mechanisms that can guarantee consistency as well as fault-tolerance.  For example, our little robot might need to agree with some other robots on which will look into the possible insect problem involving the flower.  Here we have a consistent agreement question, but because connectivity is erratic and hence participants might briefly "vanish", it has fault tolerance dimensions too.

The core ideas we'll focus on underly many tools that often look superficially different, and yet could really all reside on a single basic substrate, were the field to reach agreement inside itself about how that substrate should look.  So in fact, the things we will discuss are key to the way that mechanisms such as locking, data replication, notification mechanisms and many other "tools" work.  Believe it or not, this list really could even include blockchain!  

We probably will read some papers on this topic for Thursday but none are required for today.

At this point, it would be good to read these papers, which were mentioned above:

On the topic of vector clocks, the wikipedia link actually is quite a good reference.

Leslie Lamport. 1978. Time, clocks, and the ordering of events in a distributed system. Commun. ACM 21, 7 (July 1978), 558-565. DOI=http://dx.doi.org/10.1145/359545.359563

K. Mani Chandy and Leslie Lamport. 1985. Distributed snapshots: determining global states of distributed systems. ACM Trans. Comput. Syst. 3, 1 (February 1985), 63-75. DOI=http://dx.doi.org/10.1145/214451.214456 

Thu Sept 5

3.  Packaging coordination primitives

We'll wrap up our discussion of how to implement a coordination primitive and shift to how to "package" it for use by programmers.

This leads to a few different abstractions: atomic multicast, append-once logs (also known as Paxos), 2-phase commit, distributed barriers, publish-subscribe.  We'll have a look at the options and will think about the pros and cons of using them in our edge IoT setting.

This isn't a one-size fits all story but more of a question of thinking about the patterns of computing we will need to support and how each matches, or fails to match, with the likely use cases.

Just the same, we will discuss:

The Freeze-Frame File System. Weijia Song, Theo Gkountouvas, Qi Chen, Zhen Xiao, Ken Birman. ACM Symposium on Operating Systems Principles (SOCC 2016). Santa Clara, CA, October 05 - 07, 2016.

Tue Sept 10

4.RDMA and the FaRM KVS

This will be a two-part lecture.  First we will be learning about a powerful hardware-accelerated communication option called RDMA.  The technology can mimic TCP or UDP, but can also support distributed shared memory that is as fast or perhaps even faster than your NUMA laptop (which has more than one core, and more than one memory "unit" -- it is in fact a miniature distributed computer on a single board).

Next, we will talk about the "thought problem" (see the panel to the right).  Please come to class with your best answer already prepared, and email a short description of your proposed solution to Ken, with a short explanation of why you think it is the best possible.  Short means: "no longer than necessary to make your points."  This said, Ken is hoping to see 1 or 2 page answers, not 10 or 20!  By preparing a solution before class and emailing it before class, you will be ready for this discussion.  Sending it after class would be like waiting to know the answer, and only then writing it up: easier, but less useful to you in terms of learning by doing.  So try and get this to Ken before class.

Finally, we will study a use-case for RDMA: a "key-value" storage system called FaRM.  The concept of a key-value system is obvious: these are just hash tables (you hash the key to find the value).  The novelty is that a KVS system in a distributed setting spreads the data over the nodes in the machine, replicated ("sharded") for fault-tolerance.  Usually shards have 2 or 3 members (replicas).  FaRM goes one step further and offers a full transactional API: atomic updates to multiple (key,value) pairs and atomic multi-shard reads, implemented on RDMA.

One wonderful classic paper we will eventually be thinking about as we discuss FaRM and Derecho is this:

Hints for computer system design. Butler W. Lampson. In Proceedings of the 9th ACM symposium on Operating systems principles (SOSP '83). ACM, New York, NY, USA, 33-48.

Even if you don't have time to read the "hints" paper, please do read this paper before class:

FaRM: Fast Remote Memory. Aleksandar Dragojević, Dushyanth Narayanan, Orion Hodson, and Miguel Castro. . In Proceedings of the 11th USENIX Conference on Networked Systems Design and Implementation (NSDI'14). USENIX Association, Berkeley, CA, USA, 401-414. 2014

Thought question: consider a system doing a series of simple 2-phase vote operations.  It has a leader P, and the membership {P, Q, ... } is fixed and known to every member.  No failures occur.

Each interaction is as follows: P issues a request ("Please prepare to do X").  Every member gets ready and then responds ("I am ready to perform operation X").  When P has received all responses, it authorizes the action ("Now you should perform X").  Later members report completion ("Finished with X") and then P finalizes ("X is finished").  This allows garbage collection of any temporary data.  Assume that X itself is a uniquely identified update operation and that 100 bytes of space are needed to represent it.

P is in a loop and will be doing 1 billion of these updates.  Our goal is to do them as fast as possible.

Now, assume we are given RDMA and that it has a fixed low one-way latency of 1us (1/1,000,000 second) and comes in a series of speeds: 5 gigabytes per second (written 5GB or 40G, the latter representing bits/second), 12.5GB, 25GB, and with speeds like 1.6T (200GB) anticipated.  Yet the latency won't change: it would be stuck at 1us for all of these data rates.  Moreover, assume that each RDMA device has a smallest packet size, proportional to its speed.  So for example if you send 100 bytes and the minimum packet size is 100 bytes, you could get perfect efficiency from the RDMA hardware.  But if you only sent 50 bytes on that device, it would automatically pad your request to 100, and 50% of the potential bandwidth would be wasted.

You can also assume that the network can perform at most 100M RDMA operations per second, in total, no matter what the individual RDMA operation actually does, who issues it or to whom, and no matter the rated total speed.  So a "faster" RDMA device moves more data per second using bigger packets, but actually has fixed latency (delays) from node to node, and in fact would max out at 100M RDMA operations if you somehow tried to ask it to do more.  Of course if you don't issue 100M of them per second for whatever reason, this limit wouldn't matter. 

What is the best implementation possible for P's series of updates, given this particular hardware setup?

Thu Sept 12

5.Ken's Derecho system.

Ken will be using this as an opportunity to test-drive a slide set for a tutorial he is going to teach on Derecho at the 2019 SOSP in a few weeks.  Like FaRM, Derecho is an RDMA technology, although it also works with TCP if RDMA is not available.   But while Derecho does support a key-value object store, the actual core of the system focuses on a group communication model.  We'll discuss the motivation, the model, the implementation and the performance.  Then we'll see some future opportunities related to aspects of Derecho that might be theoretically optimal, but that in fact in a practical sense are probably not remotely as fast as they could be.

Derecho: Fast State Machine Replication for Cloud Services. Sagar Jha, Jonathan Behrens, Theo Gkountouvas, Matthew Milano, Weijia Song, Edward Tremel, Robbert Van Renesse, Sydney Zink, and Kenneth P. Birman. 2019.ACM Trans. Comput. Syst. 36, 2, Article 4 (April 2019), 49 pages. DOI: ttps://doi.org/10.1145/3302258

Tue Sept 17

6.  FarmBeats

(guest lecture)

Thu Sept 19

7.  Derecho object store

(guest lecture)

Tue Sept 24

8.  Zookeeper:  A deeper dive.  ZAB protocols.  Zookeeper API.

A simple totally ordered broadcast protocol. Benjamin Reed and Flavio P. Junqueira. 2008. In Proceedings of the 2nd Workshop on Large-Scale Distributed Systems and Middleware (LADIS '08). ACM, New York, NY, USA, , Article 2 , 6 pages. DOI=http://dx.doi.org/10.1145/1529974.1529978

The life and times of a zookeeper. Flavio P. Junqueira and Benjamin C. Reed. 2009. In Proceedings of the 28th ACM symposium on Principles of distributed computing (PODC '09). ACM, New York, NY, USA, 4-4. DOI: https://doi.org/10.1145/1582716.1582721 

ZooKeeper: Distributed Process Coordination. Flavio Junqueira and Benjamin Reed. 2017, O'Reilly. ISBN-13: 978-1449361303. ISBN-10: 1449361307 Apache Zookeeper Site: https://zookeeper.apache.org

Question: Is ZAB described in a clear and convincing way in this LADIS paper, or the PODC paper?

ZAB is a multicast protocol, not a durable storage protocol, but Zookeeper uses periodic checkpoints once every five seconds to provide persisted storage.  Does this policy actually solve Paxos?  If not, what are some of the ways that an application might notice the difference?

Thu Sept 26

9.  The two results we will talk about are: the FLP impossibility result and the weakest failure detector for guaranteeing progress.   

Impossibility of distributed consensus with one faulty process. Michael J. Fischer, Nancy A. Lynch, and Michael S. Paterson. In Proceedings of the 2nd ACM SIGACT-SIGMOD symposium on Principles of database systems(PODS '83). ACM, New York, NY, USA, 1-7. 

The weakest failure detector for solving consensus. Tushar Deepak Chandra, Vassos Hadzilacos, and Sam Toueg. 1996. J. ACM 43, 4 (July 1996), 685-722. 

Slides are here.

This lecture is about some very difficult, yet important, mathematics.  Think of it as a one-day theoretical "side tour" into the mathematics of the distributed computing area.   

Some CS6465 students lack the background to read these kinds of papers.  Don't panic!  Not everyone is prepared to read this sort of very difficult theory.  Even so, it can be useful to know about it.  If you can't follow the math, just try and understand the basic idea of what they are saying.

A thought question: what does "impossibility" mean in the FLP paper?

Tue Oct 1

10.  What abstractions will edge programmers actually want? 

Remote regions: A simple abstraction for remote memory. Marcos K. Aguilera, Nadav Amit, Irina Calciu, Xavier Deguillard, Jayneel Gandhi, Stanko Novakovic, Arun Ramanathan, Pratap Subrahmanyam, Lalith Suresh, Kiran Tati, Rajesh Venkatasubramanian, and Michael Wei.  To appear, NSDI 2018.

Slides from this talk.

We'll be looking at an ultra-simple remote memory idea, built on RDMA at VMWare.

Thu Oct 3

11.  Continued discussion about roles a technology like the ones we've discussed in class up to now.

We will situate our discussion in the context of an edge infrastructure using modern function computing models.

Cloud functions" are a hot new model that seems to be the next big thing for programming cloud applications.  You can read about Azure Function Server, Function PaaS models, or Amazon AWS Lambda to see examples. 

A cloud function is really just a short-running program triggered by some kind of event (think of a remote method invocation), that does anything you like, and then terminates.  These functions don't retain any local data (they are "stateless") but they can definitely write to the file system or to a database, etc.  They just don't create local data structures that would be used on the next event -- each event sees a "clean" initial state.

Functions can normally be coded in languages like Python, although Microsoft prefers C# .NET or F# .NET.   Functions run as programs inside container environments, and really are no different from other programs in private virtual machines.  But the model is intended to be very lightweight, with millisecond startup delays, and very elastic: "pay for cycles you actually use."

Functions run on very basic VMs, and hence don't have direct access to things like local access to GPU accelerators (one can definitely access a GPU accelerator from a container VM if the system is set up to allow that, but a function "server" wouldn't be configured that way).  Instead, think of a function as the director of an orchestra: it sends various tasks on their merry way, but does little direct work of its own.  So for GPU tasks, a function would typically hand objects like images off to GPU servers that have accelerators attached to the server nodes.  This is a source of delay, but common in today's solutions.

There isn't any special reading for this lecture.  The goal is simply to get a better sense of what functions can do, and what they can't do, and how this informs the broader issue of programming the cloud IoT edge and the reactive outer tier of the cloud itself.

Tue Oct 8

12.  Spark RDDs and file system caching performance.

Spark: Cluster Computing with Working Sets. Matei Zaharia, Mosharaf Chowdhury, Michael J. Franklin, Scott Shenker, Ion Stoica. HotCloud 2010.

Improving MapReduce Performance in Heterogeneous Environments, M. Zaharia, A. Konwinski, A.D. Joseph, R. Katz and I. Stoica, OSDI 2008, December 2008. 

Spark is the world champion for "big data" computing, but normally runs in a batch style and is viewed as a kind of back end layer of the cloud, doing big computations offline that you'll later draw on through massive files that represent the output (machine-learned models, precomputed indices, etc).  The RDD model is a cool and widely popular example of a different kind of function PaaS, even though they don't really pitch it that way.  In fact Spark RDDs could be of real interest near the edge, even without MapReduce (RDDs can be used from SciPy, GraphLab, MatLab, Mathmatica...)

Questions to think about: RDDs give Spark a big benefit for Hadoop jobs, but those are used mostly in the back-end of the data center for analytics.  Could there also be an edge opportunity?  What would reuse of RDDs at the edge require?

Thu Oct 10

13.  Temporal queries.  The concept of adequate caching

Same papers

This topic is really based on work Theo Gkountouvas has been doing on caching for systems that deal with queries to temporal data:  Data with a time index, like a machine learning system fitting data to an autoregressive integrative moving average (ARIMA) model.

(Oct 12 - Oct 15)

Fall break, no class

Let's hope for amazingly colorful leaves!  (It somewhat depends on the timing of the first real frost: we need one or two nights of really cold weather to trigger a "flash" from green to bright colors, and that doesn't happen some years.)

Thu Oct 17

14. The amazing power of GPUs and GPU clusters.  CUDA.  Dandelion: Programming tool for GPU management.

Dandelion: a compiler and runtime for heterogeneous systems. Christopher J. Rossbach, Yuan Yu, Jon Currey, Jean-Philippe Martin, and Dennis Fetterly. 2013. In Proceedings of the Twenty-Fourth ACM Symposium on Operating Systems Principles (SOSP '13). ACM, New York, NY, USA, 49-68. DOI: https://doi.org/10.1145/2517349.2522715

slides

This topic is kind of a pivot for us.  We'll start to look at hardware accelerators we might want to attach to our edge computing infrastructure.  GPUs are normally programmed using a language called CUDA, but there is a perception that CUDA is a barrier to widespread exploitation of the technology.  Dandelion is one example of a response (not super successful, but very well explained).

Tue Oct 22

15.  Challenges of integrating GPUs with other parts of the O/S stack, RDMA, etc.

GPUfs: the case for operating system services on GPUs. Mark Silberstein, Bryan Ford, and Emmett Witchel. 2014. Commun. ACM 57, 12 (November 2014), 68-79. DOI: https://doi.org/10.1145/2656206 

slides

Once you are building the GPU service itself, you need ways to get to data, and to the network.  Here are a few additional papers on other aspects of those topics.

SPIN: Seamless Operating System Integration of Peer-to-Peer DMA Between SSDs and GPUs Shai Bergman, Tanya Brokhman, Tzachi Cohen, Mark Silberstein. USENIX ATC, 2013. 

GPUnet: Networking Abstractions for GPU Programs.
Sangman Kim, Seonggu Huh, Yige Hu, Xinya Zhang, and Emmett Witchel, Amir Wated and Mark Silberstein. OSDI 2014.

Thu Oct 24

16.  Tensor Flow, both as a tool to control GPU or TPU computations, and as a more general programming language for distributed computing.

TensorFlow: A System for Large-Scale Machine Learning
Martin Abadi, Paul Barham, Jianmin Chen, Zhifeng Chen, Andy Davis, Jeffrey Dean, Matthieu Devin, Sanjay Ghemawat, Geoffrey Irving, Michael Isard, Manjunath Kudlur, Josh Levenberg, Rajat Monga, Sherry Moore, Derek G. Murray, Benoit Steiner, Paul Tucker, Vijay Vasudevan, Pete Warden, Martin Wicke, Yuan Yu, and Xiaoqiang Zheng. 2016. In Proceedings of the 12th USENIX conference on Operating Systems Design and Implementation (OSDI'16). USENIX Association, Berkeley, CA, USA, 265-283.

Tensor flow is portrayed as a distributed systems tool, and it certainly supports useful forms of compositionality.  Nonetheless, it turns out to be primarily used just on a single machine at a time to manage applications that talk to local hardware like GPU clusters or TPU accelerators.  Beyond looking at the work as they present it in this paper and in slides, in this lecture we will discuss the (mixed) experiences people have had with tensor flow  as a true distributed programming tool.

slides

TensorFlow emerged in part because Google has some skepticism about the CUDA + GPUfs + GPUnet concept.  

Mostly, TensorFlow is used on some single computer to control a single attached GPU or TPU cluster.  But it can also support fault tolerant distributed computing, in its own unique style.  Nobody really knows how effective it is in that fancier style of use.

Tue Oct 29

17.  Google paper on large-scale distributed learning.

Weijia Song will be teaching on this date because Ken will be up at SOSP 2019

Large Scale Distributed Deep Networks. Jeffrey Dean, Greg S. Corrado, Rajat Monga, Kai Chen, Matthieu Devin, Quoc V. Le, Mark Z. Mao, Marc'aurelio Ranzato, Andrew Senior, Paul Tucker, Ke Yang, Andrew Y. Ng.  Neural Information Processing Systems (NIPS) 2012.

This paper predates the Tensor Flow papers by a few years, so  it isn't necessarily an example of a system that could be implemented using Tensor Flow.  But it does illustrate what Google has been doing in this setting.

Thu Oct 31

18. Berkeley's recent work on "Ray", a system for optimizing distributed machine learning tasks.

Ray: A Distributed Framework for Emerging AI Applications. Philipp Moritz, Robert Nishihara, Stephanie Wang, Alexey Tumanov, Richard Liaw, Eric Liang, Melih Elibol, Zongheng Yang, William Paul, Michael I. Jordan, and Ion Stoica, UC Berkeley.  OSDI 2018.

The U.C. Berkeley group wasn't convinced that Tensor Flow and RDDs solve every edge computing need.  Ray is their recent proposal for an edge processing language oriented towards AI applications.

Tue Nov 5

19. Routing data through an FPGA: the Catapult model.

A reconfigurable fabric for accelerating large-scale datacenter services.  Andrew Putnam, Adrian M. Caulfield, Eric S. Chung, Derek Chiou, Kypros Constantinides, John Demme, Hadi Esmaeilzadeh, Jeremy Fowers, Gopi Prashanth Gopal, Jan Gray, Michael Haselman, Scott Hauck, Stephen Heil, Amir Hormati, Joo-Young Kim, Sitaram Lanka, James Larus, Eric Peterson, Simon Pope, Aaron Smith, Jason Thong, Phillip Yi Xiao, and Doug Burger. ISCA 2014. Published as SIGARCH Comput. Archit. News 42, 3 (June 2014), 13-24. 

Version of the slides.

GPU and TPU clusters have the "advantage" of being basically similar to general purpose computers, except for supporting highly parallel operations in hardware (ones matched to the needs of graphics programming, or tensor transformations).  But there are other interesting accelerators, too.

We'll look at FPGA, which is a kind of hardware "filter" and "transformation" unit you can place right on the wire.

Also interesting: Serving DNNs in Real Time at Datacenter Scale with Project Brainwave. Eric S. Chung, et. al.  IEEE Micro 2018.  DOI:10.1109/mm.2018.022071131

Thu Nov 7

20.  Clusters of FPGAs and their relevance to ML/AI.  Microsoft's datacenter of FPGAs model.

A cloud-scale acceleration architecture Adrian M. Caulfield; Eric S. Chung; Andrew Putnam; Hari Angepat; Jeremy Fowers; Michael Haselman; Stephen Heil; Matt Humphrey; Puneet Kaur; Joo-Young Kim; Daniel Lo; Todd Massengill; Kalin Ovtcharov; Michael Papamichael; Lisa Woods; Sitaram Lanka; Derek Chiou; Doug Burger 2016 49th Annual IEEE/ACM International Symposium on Microarchitecture (MICRO) 

Here is what we get with "grown up" FPGAs, but the topic is fairly complex.  The key idea is that if you have enough FPGAs you can create big clusters that function as powerful hardware supercomputers for certain tasks, like audio (speech) and image (vision).  People have been figuring out how to map deep neural networks into FPGA clusters.

The work is quite technical and we'll sort of skim it, with the goal of just being able to think about what an edge needs to look like if it will use tricks like this for "amazing performance."

Tue Nov 12 

21. Clusters of FPGAs and their relevance to ML/AI.  Microsoft's datacenter of FPGAs model.

Sirius: An Open End-to-End Voice and Vision Personal Assistant and Its Implications for Future Warehouse Scale Computers. SIGARCH Comput. Archit. News 43, 1 (March 2015), 223-238. DOI: https://doi.org/10.1145/2786763.2694347 

Journal version of the same paper: here. This has a little more detail and includes some additional experiments, but the shorter conference version is probably fine unless you find something puzzling or incomplete and want to read a little more.

Thu Nov 14

22.In-class brainstorming session!

Instead of discussing a paper, we will discuss your ideas for the homework problem Ken assigned on Piazza (also summarized to the right)

We have seen that applications might want to configure some mixture of data paths (for RDMA, perhaps), GPUs with their own onboard memory (GMEM), FPGAs as bumps in the wire, FPGAs as chips with architectures that might be like GPU or even like a general purpose processor, etc.  Each needs its own software installed before it can do whatever we might want to ask from it -- a GPU can't invert a matrix unless you preload the matrix inversion code, hit the proper reset sequence, load the data, and then press "run" (all done in software, of course, from the host machine).  There are even ways to load code right into switches and routers and NICs.

But we haven't see a nice programming language to control the setup.

So: imagine that we have a cloud data center hosting multiple applications that share the hardware.  Each application has access to this special-purpose "programming language" for telling the system what setup it wants in terms of the "wiring diagram" of network and devices, the code that needs to be loaded, the parameters to supply.  We might also need elasticity: "for each instance of my X role, I require 1GPU and 2 FPGA chips configured as follows..."

Either show how this could be done with the standard JSON-style configuration files we have all run into many times, or invent a new specialized programmng language we can use for this task.

Email your proposal to Ken, and plan to discuss it in class on Thursday.  There is no correct single answer.  We are just looking for awesome ideas!

Tue Nov 19

23.   Software as an out-of-band control plane for data flows.  Barrelfish and Arrakis.  iX.

The multikernel: a new OS architecture for scalable multicore systems.  Andrew Baumann, Paul Barham, Pierre-Evariste Dagand, Tim Harris, Rebecca Isaacs, Simon Peter, Timothy Roscoe, Adrian Schupbach, and Akhilesh Singhania. 2009.  In Proceedings of the ACM SIGOPS 22nd symposium on Operating systems principles (SOSP '09). ACM, New York, NY, USA, 29-44.  

Arrakis: The Operating System Is the Control Plane. Simon Peter, Jialin Li, Irene Zhang, Dan R. K. Ports, Doug Woos, Arvind Krishnamurthy, Thomas Anderson, and Timothy Roscoe. 2015. ACM Trans. Comput. Syst. 33, 4, Article 11 (November 2015), 30 pages.

Slides.

These are two famous operating systems papers that argue for new OS designs aimed at better management of modern hardware.  We've been discussing this question in terms of how code on a general purpose CPU might control subsystems on FPGA or GPU on the same node.  BarrelFish (the first paper) is about the resulting O/S structure.  The second paper looks at how to seamlessly integrate RDMA into the mix.

Thu Nov 21

25.  Machine Learning for Systems and Systems for Machine Learning.

 The Case for Learned Index Structures.  Tim Kraska, Alex Beutel, Ed Chi, Jeff Dean, Neoklis Polyzotis.  Google Brain.

More Google Slides on this whole ML everywhere concept.

Last week we talked about the puzzle of optimally mapping tasks to hardware and how we would need to estimate things like job size and object sizes and so forth.  Once you have a system like Barrelfish, those would seem like the next steps in using it.  The slide set is from Jeff's keynote talk at NIPS 2017, while the paper (referenced in the slides) is the deep dive on one concrete topic he mentions.  Seems like a good fit for us!

Tue Nov 26

26.Papers from SOSP 2019

This is a paper on matching the graph representations used for training convolutional neural networks to the GPU hardware in order to maximize the utilization level of the GPU platforms.  It led me to think about the way that we map applications to GPU and the sense in which this can be a bit like compiler optimizations for a programming language.

(Nov 27-Dec 1)

Thanksgiving holiday

Tue Dec 3

27.Papers from SOSP 2019

This is a paper on optimizing applications that mix code from more than one language and in fact, more than one kind of computing system.

Thu Dec 5

28.Papers from SOSP 2019

This is a paper that looks at accelerated video analysis using GPU clusters.

Tue Dec 10

29.Papers from SOSP 2019

This paper looks at quick ways to recover missing chunks of results from complicated computations where rolling back to the start and redoing everything would be way too expensive.