5 The Myth of "Quality of Service"

One could take the position that the way an object deals with latency, memory access, partial failure, and concurrency control is really an aspect of the implementation of that object, and is best described as part of the “quality of service” provided by that implementation. Different implementations of an interface may provide different levels of reliability, scalability, or performance. If one wants to build a more reliable system, one merely needs to choose more reliable implementations of the interfaces making up the system.
可以认为，对象处理延迟、内存访问、故障和并发控制的方式实际上是该对象实现的一个方面，最好将其描述为该实现提供的“服务质量”的一部分。接口的不同实现可能提供不同级别的可靠性、可伸缩性或性能。如果想要构建一个更可靠的系统，只需要选择构成系统的更可靠实现的接口。

On the surface, this seems quite reasonable. If I want a more robust system, I go to my catalog of component vendors. I quiz them about their test methods. I see if they have ISO9000 certification, and I buy my components from the one I trust the most. The components all comply with the defined interfaces, so I can plug them right in; my system is robust and reliable, and I’m happy.
从表面上看，这似乎相当合理。如果我想要一个更健壮的系统，我可以访问我的组件供应商目录。我考察他们的测试方法。看他们是否有ISO9000认证，然后从最信任的人那里购买组件。这些组件都有符合定义的接口，所以我可以直接将它们插入系统并且保证系统的健壮可靠。

Let us imagine that I build an application that uses the (mythical) queue interface to enqueue work for some component. My application dutifully enqueues records that represent work to be done. Another application dutifully dequeues them and performs the work. After a while, I notice that my application crashes due to time-outs. I find this extremely annoying, but realize that it’s my fault. My application just isn’t robust enough. It gives up too easily on a time-out. So I change my application to retry the operation until it succeeds. Now I’m happy. I almost never see a time-out. Unfortunately, I now have another problem. Some of the requests seem to get processed two, three, four, or more times. How can this be? The component I bought which implements the queue has allegedly been rigorously tested. It shouldn’t be doing this. I’m angry. I call the vendor and yell at him. After much finger-pointing and research, the culprit is found. The problem turns out to be the way I’m using the queue. Because of my handling of partial failures (which in my naiveté, I had thought to be total), I have been enqueuing work requests multiple times.
让我们假设我构建了一个应用程序，该应用程序使用(虚构的)队列接口为某个组件排队工作。我的应用程序尽职尽责地对表示要完成的工作的记录进行入队操作。另一个应用程序尽职尽责地将它们出队并执行工作。过了一段时间，我注意到我的应用程序由于超时而崩溃。我觉得这非常烦人，但我知道这是我的错。我的应用程序不够健壮。它太容易在超时时放弃。因此，我更改应用程序以重试操作，直到操作成功。现在我很高兴。我几乎从未见过超时。不幸的是，我现在有另一个问题。有些请求似乎被处理了2次、3次、4次或更多次。怎么会这样呢?据称，我购买的实现队列的组件经过了严格的测试。它不应该这样做。我很生气。我给供货商打电话，冲他大喊大叫。经过多方指责和研究，终于找到了罪魁祸首。问题出在我使用队列的方式上。由于我对部分故障的处理（我原本认为是全局故障），我已多次将工作请求排入队列。

Well, I yell at the vendor that it is still their fault. Their queue should be detecting the duplicate entry and removing it. I’m not going to continue using this software unless this is fixed. But, since the entities being enqueued are just values, there is no way to do duplicate elimination. The only way to fix this is to change the protocol to add request IDs. But since this is a standardized interface, there is no way to do this.
好吧，我仍然认为还是他们的错。他们的队列应该检测重复条目并删除它。除非这个问题得到解决，否则我不会继续使用这个软件。但是，由于排队的实体只是值，所以没有办法进行重复消除。解决这个问题的唯一方法是更改协议来添加请求id。但是，由于这是一个标准化的接口，所以无法做到这一点。

The moral of this tale is that robustness is not simply a function of the implementations of the interfaces that make up the system. While robustness of the individual components has some effect on the robustness of the overall systems, it is not the sole factor determining system robustness. Many aspects of robustness can be reflected only at the protocol/interface level.
这个故事的寓意是，健壮性不仅仅是构成系统的接口实现的功能。虽然单个组件的鲁棒性对整个系统的鲁棒性有一定影响，但它并不是决定系统鲁棒性的唯一因素。健壮性的许多方面只能在协议/接口级别反映出来。

Similar situations can be found throughout the standard set of interfaces. Suppose I want to reliably remove a name from a context. I would be tempted to write code that looks like:
类似的情况可以在标准接口集中找到。假设我想从上下文中可靠地删除一个名称。我会写这样的代码:

while (true) {
  try {
    context -> remove (name);
    break;
  }
  catch (NotFoundInContext) {
    break;
  }
  catch (NetworkServerFaliure) {
    continue;
  }
}

That is, I keep trying the operation until it succeeds (or until I crash). The problem is that my connection to the name server may have gone down, but another client’s may have stayed up. I may have, in fact, successfully removed the name but not discovered it because of a network disconnection. The other client then adds the same name, which I then remove. Unless the naming interface includes an operation to lock a naming context, there is no way that I can make this operation completely robust. Again, we see that robustness/reliability needs to be expressed at the interface level. In the design of any operation, the question has to be asked: what happens if the client chooses to repeat this operation with the exact same parameters as previously? What mechanisms are needed to ensure that they get the desired semantics? These are things that can be expressed only at the interface level. These are issues that can’t be answered by supplying a “more robust implementation” because the lack of robustness is inherent in the interface and not something that can be changed by altering the implementation.
也就是说，我一直尝试操作，直到成功(或者崩溃)。问题是，我与name服务器的连接可能已经停止，但是另一个客户机的连接可能保持正常。事实上，我可能已经成功删除了名称，但由于网络断开连接而没有发现它。然后另一个客户机添加相同的名称，然后我又删除了该名称。除非命名接口包含锁定命名上下文的操作，否则无法使该操作完全健壮。我们再次看到，需要在接口级别表达健壮性/可靠性。在任何操作的设计中，都必须问这样一个问题:如果客户选择使用与以前完全相同的参数重复此操作，将会发生什么?需要什么机制来确保它们获得所需的语义?这些东西只能在接口级别上表示。这些问题不能通过提供“更健壮的实现”来解决，因为缺乏健壮性是接口固有的，不能通过更改实现来更改。

Similar arguments can be made about performance. Suppose an interface describes an object which maintains sets of other objects. A defining property of sets is that there are no duplicates. Thus, the implementation of this object needs to do duplicate elimination. If the interfaces in the system do not provide a way of testing equality of reference, the objects in the set must be queried to determine equality. Thus, duplicate elimination can be done only by interacting with the objects in the set. It doesn’t matter how fast the objects in the set implement the equality operation. The overall performance of eliminating duplicates is going to be governed by the latency in communicating over the slowest communications link involved. There is no change in the set implementations that can overcome this. An interface design issue has put an upper bound on the performance of this operation.
关于性能也可以提出类似的论点。假设一个接口描述一个对象，该对象维护其他对象的集合。集合的一个性质是没有重复。因此，该对象的实现需要将重复消除。如果系统中的接口未提供测试引用相等性的方法，则必须查询集合中的对象以确定相等性。因此，重复消除只能通过与集合中的对象交互来完成。集合中的对象实现相等操作的速度有多快并不重要。消除重复对象的整体性能将由所涉及的最慢通信链路上的通信延迟来决定。集合实现中没有能够克服这一点的更改。一个接口设计问题为这个操作的性能设置了一个上限。

6 Lessons from NFS

We do not need to look far to see the consequences of ignoring the distinction between local and distributed computing at the interface level. NFS®, Sun’s distributed computing file system [14], [15] is an example of a non-distributed application programer interface (API) (open, read, write, close, etc.) re-implemented in a distributed way.
我们不需要深入研究，就可以看到在接口级别上忽略本地计算和分布式计算之间的区别的后果。NFS®，Sun的分布式计算文件系统[14]，[15]是以分布式方式重新实现的非分布式应用程序编程接口（API）（打开，读取，写入，关闭等）的示例。

Before NFS and other network file systems, an error status returned from one of these calls indicated something rare: a full disk, or a catastrophe such as a disk crash. Most failures simply crashed the application along with the file system. Further, these errors generally reflected a situation that was either catastrophic for the program receiving the error or one that the user running the program could do something about.
在NFS和其他网络文件系统之前，从这些调用之一返回的错误状态指示一些罕见的情况：一个完整磁盘或灾难，例如磁盘崩溃。大多数故障只是使应用程序和文件系统崩溃。此外，这些错误通常反映了这样一种情况，即对于接收到错误的程序来说是灾难性的，或者运行程序的用户可以对此做些什么。

NFS opened the door to partial failure within a file system. It has essentially two modes for dealing with an inaccessible file server: soft mounting and hard mounting. But since the designers of NFS were unwilling (for easily understandable reasons) to change the interface to the file system to reflect the new, distributed nature of file access, neither option is particularly robust.
NFS为文件系统中的部分故障打开了大门。对于无法访问的文件服务器，它基本上有两种模式:软挂载和硬挂载。但是，由于NFS的设计者不愿意（出于容易理解的原因）更改文件系统的接口以反映文件访问的新的分布式特性，所以这两个选项都不是特别健壮的。

Soft mounts expose network or server failure to the client program. Read and write operations return a failure status much more often than in the single-system case, and programs written with no allowance for these failures can easily corrupt the files used by the program. In the early days of NFS, system administrators tried to tune various parameters (time-out length, number of retries) to avoid these problems. These efforts failed. Today, soft mounts are seldom used, and when they are used, their use is generally restricted to read-only file systems or special applications.
软挂载将网络或服务器故障暴露给客户机程序。读和写操作返回故障状态的情况比在单系统情况下要频繁得多，而且在不考虑这些故障的情况下编写的程序很容易损坏程序使用的文件。在NFS的早期，系统管理员试图优化各种参数(超时长度、重试次数)以避免这些问题。这些努力都失败了。今天，很少使用软挂载，当使用它们时，通常仅限于只读文件系统或特殊应用程序。

Hard mounts mean that the application hangs until the server comes back up. This generally prevents a client program from seeing partial failure, but it leads to a malady familiar to users of workstation networks: one server crashes, and many workstations—even those apparently having nothing to do with that server—freeze. Figuring out the chain of causality is very difficult, and even when the cause of the failure can be determined, the individual user can rarely do anything about it but wait. This kind of brittleness can be reduced only with strong policies and network administration aimed at reducing interdependencies. Nonetheless, hard mounts are now almost universal.
硬挂载意味着应用程序将挂起，直到服务器重新启动。这通常可以防止客户机程序出现部分故障，但它会导致工作站网络用户熟悉的一种弊端:一台服务器崩溃，许多工作站——甚至那些显然与服务器无关的工作站——也会冻结。找出因果关系链是非常困难的，即使可以确定故障的原因，单个用户也很少能够对此做任何事情，只能等待。只有通过强有力的策略和旨在减少相互依赖的网络管理，才能减少这种脆弱性。尽管如此，硬挂载现在几乎是通用的。

Note that because the NFS protocol is stateless, it assumes clients contain no state of interest with respect to the protocol; in other words, the server doesn’t care what happens to the client. NFS is also a “pure” client-server protocol, which means that failure can be limited to three parties: the client, the server, or the network. This combination of features means that failure modes are simpler than in the more general case of peer-to-peer distributed object-oriented applications where no such limitation on shared state can be made and where servers are themselves clients of other servers. Such peer-to-peer distributed applications can and will fail in far more intricate ways than are currently possible with NFS.
注意，因为NFS协议是无状态的，所以它假定客户机不包含与协议相关的任何状态;换句话说，服务器并不关心客户机发生了什么。NFS也是一种“纯”客户机-服务器协议，这意味着故障可以限制在三个方面:客户机、服务器或网络。这种特性的组合意味着故障模式比对等分布式面向对象应用程序更简单，在这种应用程序中，对共享状态没有这样的限制，而且服务器本身就是其他服务器的客户端。这种对等分布式应用程序可以而且将以比NFS当前更复杂的方式失败。

The limitations on the reliability and robustness of NFS have nothing to do with the implementation of the parts of that system. There is no “quality of service” that can be improved to eliminate the need for hard mounting NFS volumes. The problem can be traced to the interface upon which NFS is built, an interface that was designed for non-distributed computing where partial failure was not possible. The reliability of NFS cannot be changed without a change to that interface, a change that will reflect the distributed nature of the application.
NFS对可靠性和健壮性的限制与该系统各个部分的实现无关。没有可以改进的“服务质量”来消除对硬挂载NFS卷的需求。问题可以追溯到构建NFS的接口，这个接口是为不可能出现部分故障的非分布式计算设计的。NFS的可靠性不能在不更改该接口的情况下更改，这种更改将反映应用程序的分布式特性。

This is not to say that NFS has not been successful. In fact, NFS is arguably the most successful distributed application that has been produced. But the limitations on the robustness have set a limitation on the scalability of NFS. Because of the intrinsic unreliability of the NFS protocol, use of NFS is limited to fairly small numbers of machines, geographically co-located and centrally administered. The way NFS has dealt with partial failure has been to informally require a centralized resource manager (a system administrator) who can detect system failure, initiate resource reclamation and insure system consistency. But by introducing this central resource manager, one could argue that NFS is no longer a genuinely distributed application.
这并不是说NFS没有成功。事实上，NFS可以说是已经产生的最成功的分布式应用程序。但是对健壮性的限制限制了NFS的可伸缩性。由于NFS协议本身的不可靠性，NFS的使用仅限于数量相当少的机器，这些机器位于地理上的同一位置，并且是集中管理的。NFS处理部分故障的方式非正式地需要一个集中式资源管理器(系统管理员)，他可以检测系统故障、启动资源回收并确保系统一致性。但是通过引入这个中央资源管理器，人们可能会认为NFS不再是一个真正的分布式应用程序。

7 Taking the Difference Seriously

Differences in latency, memory access, partial failure, and concurrency make merging of the computational models of local and distributed computing both unwise to attempt and unable to succeed. Merging the models by making local computing follow the model of distributed computing would require major changes in implementation languages (or in how those languages are used) and make local computing far more complex than is otherwise necessary. Merging the models by attempting to make distributed computing follow the model of local computing requires ignoring the different failure modes and basic indeterminacy inherent in distributed computing, leading to systems that are unreliable and incapable of scaling beyond small groups of machines that are geographically co-located and centrally administered.
延迟、内存访问、部分故障和并发性方面的差异使得合并本地和分布式计算的计算模型既不明智，也无法成功。通过使本地计算遵循分布式计算模型来合并模型，将需要对实现语言(或如何使用这些语言)进行重大更改，并使本地计算比其他需要复杂得多。通过试图使分布式计算遵循本地计算的模型来合并模型，需要忽略分布式计算中固有的不同故障模式和基本不确定性，从而导致系统不可靠，并且无法扩展到地理位置和集中管理的小型机器组之外。

A better approach is to accept that there are irreconcilable differences between local and distributed computing, and to be conscious of those differences at all stages of the design and implementation of distributed applications. Rather than trying to merge local and remote objects, engineers need to be constantly reminded of the differences between the two, and know when it is appropriate to use each kind of object.
更好的方法是接受本地计算和分布式计算之间存在不可调和的差异，并在分布式应用程序的设计和实现的所有阶段意识到这些差异。与其尝试合并本地对象和远程对象，还不如不断提醒工程师两者之间的区别，并知道什么时候适合使用每种对象。

Accepting the fundamental difference between local and remote objects does not mean that either sort of object will require its interface to be defined differently. An interface definition language such as IDL can still be used to specify the set of interfaces that define objects. However, an additional part of the definition of a class of objects will be the specification of whether those objects are meant to be used locally or remotely. This decision will need to consider what the anticipated message frequency is for the object, and whether clients of the object can accept the indeterminacy implied by remote access. The decision will be reflected in the interface to the object indirectly, in that the interface for objects that are meant to be accessed remotely will contain operations that allow reliability in the face of partial failure.
接受本地对象和远程对象之间的基本区别并不意味着这两种对象都需要以不同的方式定义其接口。接口定义语言(如IDL)仍然可以用来指定定义对象的接口集。然而，类对象定义的另一部分将是规范这些对象是用于本地还是远程。这个决定需要考虑对象预期的消息频率是多少，以及对象的客户端是否能够接受远程访问所隐含的不确定性。该决策将间接反映在对象的接口中，因为用于远程访问的对象的接口将包含允许在部分故障时保持可靠性的操作。

It is entirely possible that a given object will often need to be accessed by some objects in ways that cannot allow indeterminacy, and by other objects relatively rarely and in a way that does allow indeterminacy. Such cases should be split into two objects (which might share an implementation) with one having an interface that is best for local access and the other having an interface that is best for remote access.
完全有可能，一个给定的对象经常需要被某些对象以不允许不确定性的方式访问，而被其他对象很少需要以允许不确定性的方式访问。这种情况应该分为两个对象(可能共享一个实现)，一个对象具有最适合本地访问的接口，另一个对象具有最适合远程访问的接口。

A compiler for the interface definition language used to specify classes of objects will need to alter its output based on whether the class definition being compiled is for a class to be used locally or a class being used remotely. For interfaces meant for distributed objects, the code produced might be very much like that generated by RPC stub compilers today. Code for a local interface, however, could be much simpler, probably requiring little more than a class definition in the target language.
用于指定对象类的接口定义语言的编译器需要根据正在编译的类定义是用于本地使用的类还是用于远程使用的类来更改其输出。对于面向分布式对象的接口，生成的代码可能非常类似于今天RPC存根编译器生成的代码。然而，本地接口的代码可能要简单得多，可能只需要目标语言中的类定义。

While writing code, engineers will have to know whether they are sending messages to local or remote objects, and access those objects differently. While this might seem to add to the programming difficulty, it will in fact aid the programmer by providing a framework under which he or she can learn what to expect from the different kinds of calls. To program completely in the local environment, according to this model, will not require any changes from the programmer’s point of view. The discipline of defining classes of objects using an interface definition language will insure the desired separation of interface from implementation, but the actual process of implementing an interface will be no different than what is done today in an object-oriented language.
在编写代码时，工程师必须知道是向本地对象还是远程对象发送消息，并以不同的方式访问这些对象。虽然这似乎增加了编程的难度，但它实际上是通过提供一个框架来帮助程序员，在这个框架下，可以知道从不同类型的调用中可以获得什么。要完全在本地环境下编程，根据这个模型，从程序员的角度来说不需要做任何更改。使用接口定义语言定义对象类的规程将确保接口与实现之间所需的分离，但是实现接口的实际过程与今天在面向对象语言中所做的没有什么不同。

Programming a distributed application will require the use of different techniques than those used for non-distributed applications. Programming a distributed application will require thinking about the problem in a different way than before it was thought about when the solution was a non-distributed application. But that is only to be expected. Distributed objects are different from local objects, and keeping that difference visible will keep the programmer from forgetting the difference and making mistakes. Knowing that an object is outside of the local address space, and perhaps on a different machine, will remind the programmer that he or she needs to program in a way that reflects the kinds of failures, indeterminacy, and concurrency constraints inherent in the use of such objects. Making the difference visible will aid in making the difference part of the design of the system.
编写分布式应用程序需要使用与非分布式应用程序不同的技术。编写分布式应用程序需要一个与以前编写非分布式应用程序不同的思考方式。但这只是预期。分布式对象不同于本地对象，保持这种差异是可见的，可以防止程序员忘记差异和犯错误。知道一个对象在本地地址空间之外，并且可能在另一台机器上，将提醒程序员需要以一种反映此类对象使用中固有的故障、不确定性和并发性约束的方式进行编程。使差异可见将有助于使差异成为系统设计的一部分。

Accepting that local and distributed computing are different in an irreconcilable way will also allow an organization to allocate its research and engineering resources more wisely. Rather than using those resources in attempts to paper over the differences between the two kinds of computing, resources can be directed at improving the performance and reliability of each.
接受本地和分布式计算以不可调和的方式存在差异，还将允许组织更明智地分配其研究和工程资源。与其使用这些资源试图掩盖这两种计算之间的差异，不如将这些资源用于提高两种计算的性能和可靠性。

One consequence of the view espoused here is that it is a mistake to attempt to construct a system that is “objects all the way down” if one understands the goal as a distributed system constructed of the same kind of objects all the way down. There will be a line where the object model changes; on one side of the line will be distributed objects, and on the other side of the line there will (perhaps) be local objects. On either side of the line, entities on the other side of the line will be opaque; thus one distributed object will not know (or care) if the implementation of another distributed object with which it communicates is made up of objects or is implemented in some other way. Objects on different sides of the line will differ in kind and not just in degree; in particular, the objects will differ in the kinds of failure modes with which they must deal.
这里支持的观点的一个结果是，如果一个人把目标理解为一个由相同类型的对象构造的分布式系统，那么试图构造一个“objects all the way down”是错误的。会有一条线表示对象模型发生变化;在这条线的一边是分布式对象，在这条线的另一边(可能)是本地对象。在这条线的任意一边，另一边的实体都是不透明的;因此，一个分布式对象不会知道(或关心)它与之通信的另一个分布式对象的实现是由对象组成的，还是以其他方式实现的。线两边的对象不仅在程度上不同，而且在类别上也不同;特别是，这些对象必须处理的故障模式的类型不同。

8 A Middle Ground

As noted in Section 2, the distinction between local and distributed objects as we are using the terms is not exhaustive. In particular, there is a third category of objects made up of those that are in different address spaces but are guaranteed to be on the same machine. These are the sorts of objects, for example, that appear to be the basis of systems such as Spring [16] or Clouds [4]. These objects have some of the characteristics of distributed objects, such as increased latency in comparison to local objects and the need for a different model of memory access. However, these objects also share characteristics of local objects, including sharing underlying resource management and failure modes that are more nearly deterministic.
如第2节所述，我们所使用的术语本地和分布式对象之间的区别并非详尽无遗。特别是，还有第三类对象，它们位于同一台机器上的不同地址空间。例如，这些类型的对象似乎是Spring[16]或Clouds[4]等系统的基础。这些对象具有分布式对象的一些特征，例如与本地对象相比延迟增加以及需要不同的内存访问模型。但是，这些对象也与本地对象有一些共同特征，包括共享底层资源管理和更具确定性的故障模式。

It is possible to make the programming model for such “local-remote” objects more similar to the programming model for local objects than can be done for the general case of distributed objects. Even though the objects are in different address spaces, they are managed by a single resource manager. Because of this, partial failure and the indeterminacy that it brings can be avoided. The programming model for such objects will still differ from that used for objects in the same address space with respect to latency, but the added latency can be reduced to generally acceptable levels. The programming models will still necessarily differ on methods of memory access and concurrency, but these do not have as great an effect on the construction of interfaces as additional failure modes.
与分布式对象的一般情况相比，可以使这种“本地-远程”对象的编程模型更类似于本地对象的编程模型。即使对象位于不同的地址空间，它们也由单个资源管理器管理。因此，局部故障及其带来的不确定性是可以避免的。关于延迟，此类对象的编程模型仍然与相同地址空间中的对象的编程模型不同，但是增加的延迟可以减少到一般可接受的级别。虽然编程模型在内存访问和并发性上仍然存在差异，但是这些对接口构建的影响没有故障模式大。

The other reason for treating this class of objects separately from either local objects or generally distributed objects is that a compiler for an interface definition language can be significantly optimized for such cases. Parameter and result passing can be done via shared memory if it is known that the objects communicating are on the same machine. At the very least, marshalling of parameters and the unmarshalling of results can be avoided.
将此类对象与本地对象或一般分布式对象分开处理的另一个原因是，接口定义语言的编译器可以针对此类情况进行显著优化。如果知道通信的对象在同一台机器上，则可以通过共享内存进行参数和结果传递。至少，可以避免对参数进行编组和对结果进行解组。

The class of locally distributed objects also forms a group that can lead to significant gains in software modularity. Applications made up of collections of such objects would have the advantage of forced and guaranteed separation between the interface to an object and the implementation of that object, and would allow the replacement of one implementation with another without affecting other parts of the system. Because of this, it might be advantageous to investigate the uses of such a system. However, this activity should not be confused with the unification of local objects with the kinds of distributed objects we have been discussing.
本地分布式对象类还形成了一个组，可以在软件模块化方面取得显著的进展。由这些对象集合组成的应用程序将具有强制和保证对象接口与该对象的实现之间的分离的优点，并且将允许在不影响系统其他部分的情况下用一个实现替换另一个实现。因此，研究这种系统的用途可能是有利的。但是，这个活动不应该与本地对象与我们所讨论的分布式对象的统一相混淆。