We propose a new paradigm for building scalable distributed systems. Our approach does not require dealing with message-passing protocolsâ€”a major complication in existing distributed systems. Instead, developers just design and manipulate data structures within our service called Sinfonia. Sinfonia keeps data for applications on a set of memory nodes, each exporting a linear address space. At the core of Sinfonia is a novel minitransaction primitive that enables efï¬cient and consistent access to data, while hiding the complexities that arise from concurrency and failures. Using Sinfonia, we implemented two very different and complex applications in a few months: a cluster ï¬le system and a group communication service. Our implementations perform well and scale to hundreds of machines.