Skip to content

Systems We Make

Curating Complex Systems

Category: Distributed Programming

Apache Hadoop YARN: Yet Another Resource Negotiator

Detailed post to follow… Link to paper

Phoenix, an implementation of MapReduce for shared-memory systems

Abstract : Phoenix, an implementation of MapReduce for shared-memory systems that includes a programming API and an efficient runtime system. … More

map reduce

SCOPE: Easy and Efficient Parallel Processing of Massive Data Sets

Companies providing cloud-scale services have an increasing need to store and analyze massive data sets such as search logs and … More

data-parallel programming

Iterative Map Reduce – Prior Art

There have been several attempts in the recent past at extending Hadoop to support efficient iterative data processing on clusters. … More

data-parallel programming, distributed programming

In Search of an Understandable Consensus Algorithm

Abstract – Raft is a consensus algorithm for managing a replicated log. It produces a result equivalent to Paxos, and … More

consensus, paxos

HAIL – Only Aggressive Elephants are Fast Elephants

Typically we store data based on any one of the different physical layouts (such as row, column, vertical, PAX etc). … More

distributed programming, hadoop, map reduce

Spark: Cluster Computing with Working Sets

One of the aspects you can’t miss even as you just begin reading this paper is the strong scent of … More

data-parallel programming

A Note on Distributed Computing

This is a seminal paper published in 1994 but its ideas still go unheeded. Abstract: We argue that objects that … More

distributed programming

Generative Communication in Linda

Generative communication is the basis of a new distributed programming langauge that is intended for systems programming in distributed settings … More

distributed programming languages

Dryad: distributed data-parallel programs from sequential building blocks

Dryad is a general-purpose distributed execution engine for coarse-grain data-parallel applications. A Dryad application combines computational “vertices” with communication “channels” … More

data-parallel programming
Blog at WordPress.com.