DE3606211A1 - MULTIPROCESSOR COMPUTER SYSTEM - Google Patents

Description

HOEGER, STELL-REGHTa -PARTNER 360621

PATENTANWÄLTE UHLANDSTRASSE 14 c ■ D 7000 STUTTGART 1

- 10 -

A 46 929 b Applicant: ENCORE COMPUTER CORPORATION k - 176 257 Cedar Hill Street

February 24, 1986 Marlboro, Massachusetts 01752

UNITED STATES.

Multiprocessor computer system

The invention relates to a multiprocessor computer system, i.e. a computer system with a plurality of, narrowly interconnected processors.

The current developments in the computer industry show an ever increasing trend towards to larger and more sophisticated computer systems. These developments were in many cases by the allows higher speed and the lower cost of the electronic circuits. Furthermore, was a greater efficiency of the systems achieved through an improved organization of the same. Regarding the improved organization are particularly worth mentioning the multiprocessor computer systems in which several autonomous processor units or modules due to the division of labor together with considerable computing power provide.

There have been many different types over the years Developed from multiprocessor configurations. Did-

-1 1-

A 46 929 b

k - 176 - 11 -

February 24, 1986

Today, numerous providers of large computing systems and some manufacturers of minicomputers offer this Systems with 2 'to 4 processors. Currently these are Structures are still expensive to manufacture because of the high cost of typical processors. For this Multiprocessor computer systems have found their way mainly where high availability is required the computing capacity goes, for example in communication systems, in banking and for reservations with airlines.

Another reason for providing multiprocessor arrangements is the computing power and to increase the speed through the use of several processor units which work in parallel, see above that a data throughput is achieved which is greater than with any single processor, even if this works at high speed. Many algorithms and arithmetic operations typically performed on digital computers can be carried out in parallel operation. Since the costs. with increasing processor speed rise steeply above a predetermined mark, it can also be shown that a data throughput that is above a corresponding level is more economical by using a larger one Number of relatively slow processors can be achieved as by increasing the speed of one single processor. In view of the high speed of development with microprocessors

-12-

A 46 929 b

k - 176 - 12 -

February 24, 1986

also the number of problematic applications in which to process the data in a single Power is required that exceeds the capabilities of a single processor, already quite small and will continue to shrink.

Some of the advantages of using multi-processors are achieved through considerable Disadvantages with regard to the flexibility of the system and at the expense of programming difficulties. These disadvantages are typically due to the hierarchical organization of the processors. One characteristic weakness, which often leads to problems in terms of reliability, is the common organizational arrangement according to which each communication line and each input / output device is assigned to a very specific processor. A failure of this one processor leads to In this case, the whole system is no longer able to carry out all of the tasks assigned to it to meet.

In multiprocessor systems there are currently two typical ways of coupling the individual processors. In a multiprocessor system with closely coupled processors, each processor runs in a closed one Computing area consisting of a processor, a "private" memory, an input / output interface

-13-

A 46 929 b

k - 176 - 13 -

February 24, 1986

and a separate operating system. Flexibility and efficiency are in such a system limited because each processor works in isolation as if it were an independent system in a fast network. Also, there is no way to use more than one processor effectively to be used for the same task, if one does not want to accept that large ones with every switchover Quantities of data and accompanying information have to be transferred. So there are restrictions on the possibility of dynamic use of multiple processors if the processing is specific Data should be changed quickly.

In a multiprocessor system with closely coupled processors, the latter is a common bus, a common one Memory and a common operating system are assigned, and also the input / output devices are with all of them Processors coupled. Such an architecture only requires one copy of the operating system for Requires hundreds of processes running on large numbers of individual microprocessors. All Processors - and processes - share access to the entire main memory, to the whole Network value, to all input / output interfaces and to the entire mass storage device. This time-sharing operation allows maximum use of the available processors with a minimal amount of unused memory space and unused bus bandwidth because im

-14-

A 46 929 b

k - 176. - 14 -

February 24, 1986

With regard to the temporally staggered, shared access, multiple storage only to a minimal extent of data and a restoration of accompanying information is required. In such a system any processor can be used for any process at any time. The enormous flexibility of this design pays for itself in a better availability of the computing power, in greater expansion possibilities. and in a greatly expanded range of possible applications.

When designing a multiprocessor system you should various considerations must be taken into account in order to achieve the maximum level of performance. One of these One of the factors is that a particular vendor will have a reasonable variety of multiprocessor systems should have ready. This diversity should take into account both performance and costs. If one has to choose from a limited number of family members in a computer family, there is however, it is often not a satisfactory solution because it is expensive to design different computer family members and develop.

Another important consideration in designing a multiprocessor computer system is that the Structure of the system from a number of different module types, such as processors, input / output

-15-

A 46 929 b

k - 176 - 15 -

February 24, 1986

devices and memory modules the failure of a these modules do not cause a failure of the multiprocessor computer system should lead. Ideally, suitable programs should offer the possibility of buggy To simulate modules or their function and to put the modules themselves out of operation, so that a continuation operation with minimal downtime.

About the development costs for a multiprocessor system It is also important that the multiprocessor computer system not consist of a large numbers of unique cards such as those found in a typical mini-computer. Instead, the multiprocessor computer system should consist of numerous copies of a small number of Be built up modules, which allows faster and cheaper development, with the individual module types can be manufactured in large numbers, which is compared to older technologies with regard to advances which involves manufacturing costs.

In any multiprocessor system where performance and flexibility are of the utmost importance, the bus that connects the various modules of the system must have a very high data transmission speed. Such a system must It is also ensured that the decision on access to the bus is made by an arbitrator functioning, so-called arbiter module, so fair

-16-

A 46 929 b

k - 176 - 16 -

February 24, 1986

It is regulated that every module has a reasonable chance get access to the bus. To achieve a high data transfer rate, it is generally also advantageous to if the bus is a so-called "pended" bus, i.e. a bus with a structure, which allows the retrieval of information is separated in time from the answers given. Such a way of working, in which a " Keeping individual parts of certain operations "in the balance" enables communication of a number of relatively slow devices, such as processors, with other slow modules, such as the main data memories, without the bandwidth of a bus being poorly used for higher data transmission speeds is suitable as any single connected unit. But if you have certain Queries in the balance, they are supplemented by the identifier of the requesting unit and then at to the intended recipient as soon as the first opportunity arises. If the recipient then delivers the answer some time later, this is again provided with the identifier of the requesting unit. None of the participants "noticed" this Exchange of information that between the request or request and its answer or completion numerous other operations between other transmitting units and receiving units were carried out.

Any computer system that has more than one

-17-

A 46 929 b

k - 176 - 17 -

February 24, 1986

Processor contains, there is also a need that each processor is able to use memory automatically Perform test and adjustment operations. An obvious option is this automatic Ensuring operations consists in the path to the memory, i.e. the bus for an entire Provide read-modify-write cycle of operation. In a power sensitive system this is clearly undesirable. In the case of a "suspended" bus or a suspended bus that interleaves the read cycles with other bus operations such a path to the memory cannot remain fixed / switched. So the result is an external procedure is needed to create a memory location to lock. Since the memories in a system according to the invention are nested in the manner of a bank system, however, it is possible to lock a memory bank in the bank system. Given the size of the memory bank is however, locking the memory for an area of 4 Mbytes is undesirable.

Based on the prior art and the problems outlined above, the object of the invention is based on a flexible and economical multiprocessor computer system indicate which can contain a large number of individual processors. Included At the same time, the aim is to ensure that the individual processors are closely linked to one another.

-18-

A 46 929 b

k - 176 - 18 -

February 24, 1986

This object is achieved in a multiprocessor computer system according to the invention according to the invention by the features of the characterizing part of claim 1 solved.

It is an advantage of the system of the invention that the user can select the right level of performance or price for his needs without having to being limited to a choice among a limited number of computer family members.

It is a further advantage of the system according to the invention, that it is due to the use of a small number of module types which are taken out of service without affecting the rest of the system, a high level of reliability from the outset owns.

It is also an advantage of the system according to the invention that vector interrupt signals are not used for a long time can be transmitted by data and / or address buses.

There is another advantage of the system according to the invention in the fact that there are numerous copies of a small number of modules.

Another advantage of the system according to the invention is that it can be connected to a

-19-

A 46 929 b

k - 176 - 19th

February 24, 1986

another, similarly constructed multiprocessor computer system can be expanded.

There is another advantage of the system according to the invention in that a system bus with very high data transmission possibilities can be provided.

It is also an advantage of the system of the present invention that it has a storage system that does not block automatic test and setting operations can be carried out on the system bus.

Generally speaking, comprises a multiprocessor computer system according to the invention, one or more copies of a number of selected modules. A processor module comprises at least one processor that is independent with the other components of the system can work together. A system can contain a number of these modules. The system copes with failure one of these modules or one of the processors of one of these modules by "logically" hiding the module or processor from the system. Each of these processor modules also includes a cache memory, which saves frequently used commands and data. The cache memory significantly reduces the data access time, while at the same time as a result of the fewer number of requests that are transmitted over the bus the bus occupancy is reduced considerably.

-20-

A 46 929 b

k - 176 - 20 -

February 24, 1986

The cache memory can either be a so-called write-through cache memory or a non-write-through cache memory be.

Memory modules which are all in time-sharing Processors of the processor modules available hold at least one independent bank of RAM memory chips (Random access memory) ready. Everyone

Module supports at least quadruple nested
teleposition between the modules so that the memory modules can run at the maximum possible bus speed.

The key element of the multiprocessor computer system according to the invention is the system bus, which all Interconnects modules of the system. In practice this bus comprises four separate buses, viz a data bus, an address bus, a vector bus and a control bus. Connections via these buses are "suspended" connections, and the parallel pipelined connections on these buses Information or data allow this bus system to ensure high data transmission data.

In a preferred embodiment, the system bus has a transmission speed of 100 Mbyte / s.

The system control module works as a communication *) or link -21-

A 46 929 b

k - 176 - 21 -

February 24, 1986

Clearing house, as a bus coordinator and as a diagnostic center for the multiprocessor computer system according to the invention. The system control module includes the so-called bus arbiters, i.e. the arbitration and decision-making bodies for access to the vector data and address bus. The system control module further comprises the Clock unit that supplies all modules of the system with clock signals. The address bus arbiter supports reducing bus performance problems by running automatically ensures that every write or read request, which was not finally processed because the target device called did not accept the request could be repeated in a RETRY cycle. With the help of an Unjam cycle it is also prevented that data returned from the memory are kept away from the shared read-write data bus, when a large number of write operations are to be performed consecutively.

A multiprocessor system according to the invention can furthermore include network (Ethernet) / mass storage modules as well an adapter for an industrial standard bus such as the VME bus.

The multiprocessor computer system according to the invention can communicate with other multiprocessor computer systems A large field multiprocessor (LAmP) interface module can be linked. This LAmP interface

-22-

A 46 929 b

k - 176 - 22 -

February 24, 1986

The module can contain a cache memory, which can work in a similar way to the cache memory, which are assigned to the individual processor modules. The bus arbiters handle all of the Bus incoming requests via the LAmP interface in a similar way to any other request that comes in from another module that is on the bus connected is.

Further details and advantages of the invention are explained in more detail below with reference to drawings and / or are the subject of subclaims. Show it:

1 shows a block diagram of the individual components of a preferred embodiment a multiprocessor computer system according to the invention;

FIG. 2 is a block diagram of the components of a processor module of the system according to FIG. 1;

Figure 3 is a block diagram of the various components of the memory module of the system according to FIG. 1;

Figure 4 is a block diagram of the various components of the system control module the system of FIG. 1;

-23-

A46 929 b

k - 176 - 23 -

February 24, 1986

Figure 5 is a block diagram of the various

Components of the Ethernet / mass storage module the system of FIG. 1;

6 is a block diagram of a computer system having multiple multiprocessor computer systems of the type shown in Figure 1;

FIG. 7 is a block diagram of the components of FIG

Interface modules for connecting the various multiprocessor systems according to Fig. 6;

Figure 8 is a timing diagram of the read and write operations on the bus of the multiprocessor system according to FIG. 1;

9a shows a schematic representation of various modules of the system according to the invention to illustrate the mode of operation of the address bus arbiter devices of the multiprocessor system according to FIG. 1;

9b shows a schematic representation of various modules of the multiprocessor system according to the invention to clarify the decision-making process in the data bus arbiter scheme of the multiprocessor system according to FIG. 1;

-24-

A 46 929 b

k - 176. - 24 -

February 24, 1986

Fig. 10 is a schematic representation of the backplane wiring or that as a printed one Circuit formed bus board for a preferred embodiment of the system according to FIG. 1;

11 is a timing diagram for an idle cycle the system of FIG. 1;

Fig. 12a is a timing diagram for a directional Vector interrupt command issued by a module of the system according to FIG. 1 is transferred;

12b shows a timing diagram of a class vector interrupt command; which is transmitted by a module of the system according to FIG. 1;

13 shows a schematic representation of the data words, which are sent with a vector transmitted by a module of the system of Figure 1;

14 is a schematic representation of the data lines used in the vector bus arbiter scheme the system of Figure 1 can be used;

A 46 929 b

k - 176 - 25 -

February 24, 1986

15 shows diagrams of various examples until the bus is used in the system according to FIG. 16; FIG. 1;

Fig. 17 is a schematic representation of the for

the generation of delay signals, as shown in the timing diagram according to FIG required circuitry is shown; and

Figure 18 is a schematic representation of the circuitry of the memory module of the system according to FIG. 1.

The multiprocessor computer system according to the invention combines modular distributed computing capacity with fast, shared storage facilities and powerful, configured input / output devices, whereby these components are combined into a single computer which has capabilities ranging from the microcomputer field to the mainframe computer field can be enough. As FIG. 1 shows, the multiprocessor system 10 shown as an example comprises a total of four Module types as basic building blocks, namely processor modules 20, shared memory modules 40, a system control module 60 and network architecture / mass storage modules 90, an Ethernet network architecture being provided in the exemplary embodiment.

-26-

A 46 929 b

k - 176 - 26 -

February 24, 1986

The processor module 20 shown in Fig. 2 preferably comprises two independent processors 21, which with a clock frequency of 10 MHz and can be, for example, ICs of the NS 32032 type, as well as a Cache memory 22 shared (by both processors). Each processor module 20 is also with a memory management unit 24 / (which the generation of physical 32-bit addresses is permitted. The two processors also share an internal 32-bit data bus (IDB) 23 and an internal 32-bit address bus (IAB) 25. The buses IDB 23 and IAB 25 are opposite the processors 21 by the central processing unit (CPU) belonging data and address send / receive units with registers - block 26 and buffered with respect to the system bus 100 by means of bus data send / receive units 27 and address registers 28.

The cache memory 22 is provided to reduce the access time to the memory by frequent Required commands and data in a large bank (32K byte) of a fast, static RAM memory. The data from the processor memory are usually always written into the cache memory 22 when the memory locations of the main memory of one of the two processors of a processor module 2 can be read out with two processors or if new Data can be written into these memory locations.

-27-

A 46 929 b

k - 176 - 27 -

February 24, 1986

An index of the addresses of those memory locations, the contents of which are stored in this way, is in an additional memory arrangement 30 of the respective processor module saved. Subsequently, each attempt leads to the relevant memory locations of the system main memory 40 to achieve that the corresponding data in the cache memory 22 is accessed. However, accesses to the cache memory do not lead to the wait states of the processor, which are found in a Access to the processor main memory resulted because the processor from which the request originated, not competes with processors in other processor modules 20 when it has access to main memory would like to have. Instead, the data are simply transported via the internal bus 23 of the processor module 20. In the currently preferred exemplary embodiment, the "hit rate" averages over 90%.

The cache memory 22 for any processor module 20 is made using bus overhead logic with respect to the relevant changes in the main memory (caused by write operations generated by other system devices). The bus addition logic scans the system bus 30 for the memory activity of other system modules, which local Concerning cache memory addresses. When such writes are detected, the valid bit becomes (valid bit) 34 for the relevant cache memory address to the status "invalid (invalid)"

-28-

A 46 929 b

k - 176 - 28 -

February 24, 1986

switched, indicating that the cache memory data at the relevant memory location no longer transfers the data to the assigned memory location of the main memory correspond. When an external processor needs data from the appropriate cache memory address, consequently, he will recognize that the data stored in the relevant cache memory location is now invalid are. The processor then accesses the main memory in order to take the relevant data from there instead from cache memory 22. This operation automatically updates the data in cache memory 22 - the data is thus brought up to date. Since the additional bus memory 32 is independent of the CPU auxiliary memory 30 and "copies" this data, it is maintained valid data in the cache memory with the help of bus monitoring without affecting the speed access to the cache memory by the central processing units.

The cache memory 22 can be used as either a write-through cache memory or as non-write-through cache memories. If a processor module 20 requests a write operation with a write-through cache memory, the data is stored in both the Cache memory of the requesting processor module 20 as well as in the relevant memory location of the main memory 40 enrolled. Using a

-29-

A 46 929 b

k - 176 - 29 -

February 24, 1986

Write through cache is the data match maintained in the cache memories and in the main memory.

If a processor module 20 with a non-write-through cache memory requests a write operation, then this module fetches the data and the data are only written into the cache memory 22. The auxiliary bus memory 32 is updated to indicate that the main memory location that corresponds to the newly written cache memory location, no longer contains valid data. Any attempt by a processor module to access the relevant main memory space, except for the module with the cache memory, which has the valid Contains data is directed to the cache memory with the valid data. The use of a write-through cache reduces the "traffic" on the system bus 100, since fewer write operations are performed by the processor modules go out.

Each CPU 21 transmits and receives vectors over the system bus 100 - this is described below. Therefore, there is an interrupt FIFO queue with each processor 36, which stores the received vectors until they are processed, whereupon -'the data during the CPU interrupt acknowledgment cycles on the internal Data bus 23 are pushed out. Vectors from the system bus 100 are neither written into the FIFO queues

*) FIFO = first in - first out __ "_

= receive first - output first

**) Queue = queue

A 46 929 b

k - 176 - 30 -

February 24, 1986

still acknowledged when these FIFO queues are full.

As FIG. 3 shows, each shared memory module 40 preferably comprises two independent memory banks 41. The banks can be RAM memory chips of the MOS type with a storage capacity of 25 6 K bytes, and the total storage capacity can be up to about 4 Mbytes. Each module enables one Quadruple staggering between memory boards with the same memory size.

All data that is stored in the shared memory modules 40 are provided with an error correction code (ECC), with single-bit errors in each long word (32 bits) with each access can be corrected using the ECC code. Double-bit errors are recorded and reported. Farther each of the shared memory modules 40 will have all of its memory during the refresh cycles overlined, with all identified single-bit errors being corrected. Since with a complete Refresh cycle one complete data pass takes place every 8 s (with 256 k-RAMs), working with the correction code reduces the probability that an uncorrectable double bit error occurs at all. Because of the correction code two memory chips of the shared memory module 40 may fail (1 chip in each Bank) without having to interrupt the operation of the system.

-31-

A 46 929 b

k - 176 - 31 -

February 24, 1986

Each shared memory card 40 also carries a diagnostic microprocessor 46 that controls all memory banks when switching on and always checks if it has received a corresponding command from the system diagnostic processor of the system control module 60 receives. The shared memory card 40 also includes a control and status register, via which single-bit and double-bit errors as well as bus parity errors reported to the requesting processor.

The system control module (SCM) 60 works as a communication clearing house, as a bus coordinator and as a diagnostic center for the multiprocessor computer system according to the invention 10. The various components of this system control module 60 are sketched in FIG.

The module 60 initially comprises a diagnostic processor 62, which is based on a microprocessor of the type NS 32016 is set up and has access to a dynamic 128K byte RAM located on the board as well as a battery-backed (non-volatile) static 4K byte RAM. The diagnostic processor 62 performs After switching on the mains voltage, carry out the system diagnosis and set the start conditions. Farther the processor 62 provides a time of year and monitors the system control panel and the control panel connector of the system and two local user connections. The diagnostic processor 62 also takes over the control of the system bus 100 and all associated

-32-

A 46 929 b

k - 176 - 32 -

February 24, 1986

Module when a fatal system failure occurs. If the fault is due to the failure of a component one of the system modules is caused, the system control module 60 can use this module in the next deny access to system bus 100 for a new start. When a new start occurs, the system control module can inform the operating system that the module in question is inactive (out of order) treat is.

The system control module 60 further comprises a system bus interface 64, which enables the diagnostic processor 62 to access other modules that are equipped with the system bus 100 are connected, while at the same time allowing the other system modules manages to read out information from the common command / response memory of the system control module or to write information in this memory, which together with various clocks the block 66 forms.

The memory / clock block 66 is "visible" (accessible) to all active modules on the system bus 100. Of the However, block 66 does not initiate any active requests itself. It contains clocks that are used for this can be used to generate process identifications and clocked interrupts. Block 66 also includes a static 32K byte RAM memory, which is used to store commands and responses between the system

*) the interfaces are labeled "interface" in the drawing -33-

A 46 929 b

k - 176 - 33 -

February 24, 1986

To transmit control module 60 and other modules connected to the system bus 100.

As will be explained in more detail below, the system bus 100 actually consists of several independent ones Buses (address, data and vector bus), via which in the course of each bus cycle do not interrelate linked information can be transmitted. It is therefore necessary to have access to each bus separately according to the given rules of the game with the help of a referee or arbiter group decide.

To this end, there are a vector bus arbiter 68, a data bus arbiter 70 and an address bus arbiter provided, which belong to the system control module 6 0 in the preferred embodiment.

The system control module 60 further comprises the system or Master clock 74 for the multiprocessor computer system 10. The master clock is distributed by the system control module 60, and all bus clock lines on the system bus 100 are controlled by the master clock 74,

The Ethernet / mass storage module 9 0 (EMS module) comprises Interfaces 94, 98 for coupling to a local Ethernet area network and to a small computer system interface bus 91. In a preferred embodiment of the multiprocessor system according to FIG

-34-

A 46 929 b

k - 176 - 34

February 24, 1986

In accordance with the invention, the bus interface 98 of the EMS module 90 supports disk storage and one-half inch tape storage. Additional EMS modules can be installed, each of which has a data path for additional Can form disk storage.

As FIG. 5 shows, the EMS module 90 consists of four basic elements, namely the system bus interface 92, the interfaces 94 and 98, which were discussed above, and an EMS central processing unit 96 (EMS-CPU). the System bus interface 92 is shared by the other three circles. The interface 94 comprises an Ethernet controller, a direct memory access (DMA) machine, and local storage. This Memory is used as a sending / receiving unit for commands and status information, statistical network management data and diagnostic information is used. Any part of this memory can be made with the help of the DMA machine with data from the main memory of the multiprocessor system filled or reloaded into this.

The EMS-CPU 96 is preferably a microprocessor of the type NS 32032 with a local ROM memory for the program control, a local RAM memory for program and data storage, a local one Control / status register, with vector interrupt units and with two windows to the main memory of the multiprocessor system is equipped.

-35-

A 46 929 b

k - 176 - 35 -

February 24, 1986

The interface 98 comprises a bus controller for the bus 91, a data FIFO unit, a microprocessor and a DMA machine. The bus controller transfers data between the bus 91 and the data FIFO unit under the control of a central processing unit assigned to bus 91. The DMA machine can handle data between the main memory of the multiprocessor system and the data FIFO unit in each direction.

A VAME bus adapter module 99 can also be provided which can accommodate a wide variety of VME bus cards. This adapter complies the well-documented VME bus standards so that users of the multiprocessor system can implement new functions can directly access the system bus via an interface without the need 100 to take. With the adapter 99, the system can have real-time input / output interfaces as well as according to customer requirements Include specially designed interfaces.

Each multiprocessor computer system 10 of the above described type can also by a large field multiprocessor interface module 200 (LAmP module). As shown in Fig. 6, in this Case any of the systems 10 supported by a number of requirement modules and shared memory modules is symbolized, with a further corresponding system via the LAmP module 20 0 and a LAmP communication bus 202 connected. At the present

-36-

A 46 929 b

k - 176 - 36 -

February 24, 1986

A system with such LAmP modules can be configured up to 16 multiprocessor systems 10 comprise.

As FIG. 7 shows, each LAmP module 200 includes one System bus interface 204 and a LAmP bus interface 206. The LAmP interface 200 includes an LAmP cache memory 208 which reduces the access time to the memories of all systems except that System to which the requirements module belongs. The LAmP cache memory consequently reduces the number of requests which are transmitted over the LAmP communication bus 20 2, as the cache memory 208 the requests handles for the most frequently accessed memory locations. A LAmP cache overhead 210 Contains an index of the memory addresses of each cache memory location as well as the system number of the system to which the main memory in question belongs.

A tested LAmP group storage option 212 stores an index of all memory locations in the memory modules 40 of the multiprocessor system 10, their Contents of other multiprocessor systems connected via LAmP units 10. Requirements on the system bus 100, which do not influence the request and response modules that are shared with others via LAmP units Connected system buses 100 are connected by the tested LAmP group storage option 212 filtered out.

-37-

A 46 929 b

k - 176 - 37 -

February 24, 1986

The system bus

The system bus 100 is the primary system connection for interconnecting the various modules of the multiprocessor computer system 10 according to the invention. The system bus 100 connects processors, memories, peripheral devices with direct memory access and subordinate peripherals .

The system bus 100 is a "suspended" bus with a throughput of 100 Mbytes / s. The system bus 100 comprises a data bus 102 and a separate system bus 104, these two buses being independent of one another work. The system bus 100 is also a synchronously operating bus on which all data transfers synchronously with the clock of the bus clock.

As FIG. 1 shows, the system bus 100 consists of four separate buses: the data bus 102, the address bus 104, the vector bus 106 and the control bus 108. As shown in the drawing, the data bus 102 can transmit 64 bits of information plus the parity bits while over the address bus 104 32 information bits plus parity bits are transmitted. The benefit of using parallel data and address paths consists in dispensing with a time-consuming multiplex operation can be. As a result, the bus bandwidth is greatly increased. The access decisions for the address bus

-38-

A 46 929 b

k - 176 - 38 -

February 24, 1986

102 and the data bus 104 are only partially divided. The decision as to which facility is to occupy the bus with the currently highest priority requests, takes place with the help of a central arbiter and becomes the selected one requesting the bus Circle transmitted. The decision as to whether a particular module is different from the current one Should exclude the set of permissible requirement units, however, each module takes it itself. If a module has been granted access to the address bus, it supplies its addresses and, if necessary, its data on the bus. If a module has been granted access to the data bus, it delivers its Data on the data bus.

Before discussing the various decision criteria or schemes used for the system bus the different data transfer cycles are described. As mentioned above, takes place the transmission in the system of the invention as "suspended" transmission, i.e. the The address is sent to the recipient during data transmission in which data is to be read out, whereupon the system bus is inserting other operations while the data to be returned is being prepared. The bus does not wait for the data to be returned. If registered addresses are transmitted, then however, the data to be written always follow them in the next cycle.

-39-

A 46 929 b

k - 176 - 39 -

February 24, 1986

There are various modules that can request the use of the address bus 104. These modules include the processor modules 20, the input / output modules 90 and the LAmP interface modules 200. A module that wants to use the address bus sends out a request which uses the Control lines 100 is transmitted to the system control module 60 to which the arbiter circuits belong. As soon the address bus arbiter 72 then grants access to the address bus 104, in the next clock time the The address of the authorized request group is given on the address bus 104. When the desired operation is a write operation, then the data to be written is transferred to the data bus 102 in the clock period which follows the clock period in which the address was placed on the address bus 104.

There are also various modules that can request the use of the data bus 102. A module, who wants to use the data bus 102 sends a corresponding request via the control lines 108 to the system control module 60, to which the data bus arbiter 7 0 belongs. A module that provides access to the data bus 102, must also monitor the address bus 104 to see if the output of the Data on the data bus 102 is imminent. In these cases, the requesting module postpones its Request and requests the use of the data bus again at a later point in time.

-40-

A 46 929 b

k - 176 - 40 -

February 24, 1986

An example of the clock and transfer cycles of which As discussed above, it is shown in Fig. 8 where there are two control lines, namely one Control line connected to the data bus arbiter and_ a second control line connected to the address bus arbiter connected is. If a request from a requirements module (requirement module No. 1) arrives, the address bus arbiter 7 2 grants the request module no. 1 access during one clock period to address bus 104. During the next clock period, the desired address of request module no. given to the address bus. Since the request of the request module No. 1 concerns a read operation, the request module no. 1 must wait several bus clock periods before the data are output to it.

While the request module no. 1 is the desired address on the address bus 104, a request module generates No. 2 a request for a write operation and gains access to the address bus. The desired The address of the request module 2 is on the address lines during the immediately following clock period given (in the embodiment in period No. 3), and a write operation is required the data is output on the data bus 102 during the clock period following the period in which the address was put on the address bus, namely in period No. 4. During a write operation will not make a decision about access

-41-

A 46 929 b

k - 176 - 41 -

February 24, 1986

for data bus 102 is required. The operations for request module no. 2 are now complete. A few clock periods later, the data for request module no. 1 is sent back. At this time
the data bus arbiter 7 0 must grant the sending module access to the data bus 102. In the exemplary embodiment
this access is granted immediately, and during the
In the next clock period, the data is sent to the data bus 102, with which the operations of the request module # 1 are also dealt with. On the other hand, if the data bus 102 had been busy, the module sending the data would have had to wait until the bus had been allocated to it.

If, in the above example, for whatever reason, access to the buses had been granted for a large number of write requests, the requests after the identification of data would be blocked and thus the transmission of data to the relevant request modules would be prevented. To alleviate such problems, which now slow down data processing, a special control signal · UNJAM L is applied to the bus
located around the data bus 102 after a certain waiting time, for example after 4 bus cycles in which
the release of the data bus has been waited for. When the control signal · UNJAM is generated, the acceptance of requests by the address bus is interrupted so as to enable free access to the data bus within a few clock periods.

-42-

A 46 929 b

k - 176 - 42 -

February 24, 1986

Furthermore, no new requests for the data bus are accepted. The control signal UNJAM is deleted, as soon as the data bus has been released for the pending data.

The system bus 100 allows it to be a "pending." held "bus is the ability to have multiple requests for a single bank of memory in the To be kept levitated. The stores themselves have no way of accommodating more than two requests in the Pipeline operation to process (namely the request that is being processed and another that is is held in suspension) so that it is possible that a memory bank 40 is busy and not in the Would be able to receive the address (or write data, if a write cycle is required). If those Situation arises, the memory bank refuses to accept the address and generates a signal RETRY, which causes the transmission of the address to be attempted again four bus cycles later.

If any requirements module determines that a Address has been "naked", for example as a result of an occupied memory bank, it does not send any new requests until the "bare" address is done. This procedure ensures that there is no set of requirements is "locked out" for a longer time interval, since other modules between its retries send and their requests are passed up to the point that the relevant request unit also tried to achieve.

-43-

A 46 929 b

k - 176 - 43 -

February 24, 1986

As discussed above, the address bus 104 and data bus 102 each have independent arbiters, which monitor the requirements for data transfers. When a request is made, the uses Arbiter an algorithm that fair treatment guaranteed and ultimately enables access to the requirement group. The arbiter does this by he gives an admission for the selected module. The selected module then gives the address or the data on the bus. The address bus arbiter ensures a fair allocation of access to the bus by implementing the following Algorithm:

1. All non-processor modules, for example the system control module or the bus adapter, can Have priority over the processor modules. These modules form priority group A. If if any of these devices requests address bus 104, it will have access regardless of the presence a request from a module of priority group B, which is given below is yet to be defined.

2. Priority group B includes all processor modules. Access is then given to a module in group B for an address transfer if it is within its group, i.e. among the modules his group, one of which is a requirement

-44-

A 46 929 b

k - 176 - 44 -

February 24, 1986

pending, the module with the highest priority is and if there is no request from one Group A module is pending.

3. If a device is granted access to the bus, then you get it according to the algorithm following device then has the highest priority.

The address bus arbiter 72 consists of a centralized arbitration mechanism and a distributed one Control mechanism. The centralized decision-making mechanism receives the requirements and delivers the approvals based on a rotating priority scheme using module slot numbers. In the preferred embodiment, the central arbiter is part of the system control unit 60. An example of an address bus allocation scheme is discussed with reference to Figure 9a. The system shown in Fig. 9a contains 8 request modules. Assuming that module no. 1 Access to the bus was granted, then the Module No. 2 has the highest priority. If modules # and # 5 both request access to the bus, then will Access is granted to module no.5, as it is the module with the highest number after module no.2, from which a request from the bus originates. Following the granting of access to module no.5 The module with the highest priority is now module no.6.

-45-

A 56 929 b

k - 176 - 45

February 24, 1986

The distributed control mechanism consists of a state sequencer to each module, which determines whether a module has made a request for the address bus may submit. The address bus requests are modified by the request modules, if any the following conditions are met:

1. When a module of priority group A has a requirement to the central address arbiter, can this give the signal PRIORITY L on the control bus 108 to all modules of the priority group B to force them to withdraw their requirements, thereby reducing the number of modules in the group A access can be made with a higher priority.

2. All requirement groups suspend the requirement for writing data and for read-modify-write cycles, as long as a signal STALL CYCLE L (block cycle L) is on the control bus 108.

3. If a memory module cannot access the data bus, the requested data can be stored within To send back a specified number of bus clock cycles, it outputs the UNJAM L signal the control bus 108. Suspend the request circuits then write requests until the UNJAM L signal is no longer present.

A 46 929 b

k - 176 - 46 -

February 24, 1986

4. If a memory bank to which a data transfer command is being sent is busy, supply the memory does not put the MEMORY ACCEPTED L signal on control bus 108, thereby notifying the request circuit that it has received the request should try again. Those request modules for which there is no bus request at this point in time is not allowed to submit a request until the module, which is to be renewed Attempt was prompted to be served by the memory bank, causing a freeze on the currently existing set of requirement groups takes place. The requirements within the frozen The sentence will continue to be given rotating priority according to the decision criteria Bus, with the exception that the module that was asked to try again every time if he submits his request again, it receives the highest priority. The discontinuation of the However, the request by the retry module does not affect it the rotating priority for the remaining requests of the frozen set of modules. The requests are released when the module concerned is retried through the memory bank was served.

Access to the data bus is assigned based on the following algorithm:

-47-

A 46 929 b

k - 176 - 47 -

February 24, 1986

1. When a write cycle is in progress on address bus 104, data bus 102 always transfers the data to be written during the next data bus cycle , regardless of any other pending data bus request.

2. If there are no write _{data transfers pending /} the device with the highest priority which requests the data bus for reading data receives the permission for the data transfer. The logical order of priority of those devices that can submit a request for data transmission is as follows: system control module, bus adapter, LAmP module and memory modules.

3. The priority on the data bus is a strictly numerical priority.

An example of the data bus arbitration scheme is discussed below in connection with Figure 9b. In the embodiment according to fig. 9b the system comprises a bus adapter, an LAmP module, a system control module and three memory modules. In the example, the last module granted access was became the memory module # 2.

If no write data transfers are pending and if the LAmP module · and the memory module · No.3

-48-

A 46 929 b

k - 176 - 48 -

February 24, 1986

Request access to the data bus, then the access is due to the strict numerical priority of the data bus arbitration scheme granted to the LAmP module. If there is a request instead of the request by the LAmP module through memory module no. 1 and through memory module no. 3, then the Access granted to memory module no. 1 even if it last had access to the data bus not long ago has as the memory module no.3.

As discussed above, is in the multiprocessor computer system according to the invention the system of distributed cache memories is implemented. To guarantee the data correspondence between cache memories 22 and main memory 40 monitor all cache memories 22 certain bus operations on "hits" in their additional bus memories (BTAG) 32. If a "hit" is determined, then the local CPU additional memory 30 is updated by the CPU logic. the However, system requirements dictate that pipeline processing of such "hits" be minimal, so that it is possible that multiple "hits" in the additional memory 3TAG 32 fill the "line". In this case the C? U must output the STALL CYCLE L signal for the address bus arbiter to allow write operations and Block read-modify-write operations until there is room on the "line".

-49-

A 46 929 b

k - 176 - 49 -

February 24, 1986

All data which are transferred on the system bus 100 are accompanied by an addition, which the Circuit from which the information is requested is precisely defined, so that the data is sent to this circuit can be transferred back. This information is contained on the internal address and data lines ID. The information that is transmitted consists of a physical 4-bit slot number. and two reserved bits. These bits are generated by the request circuit and the written to Facility to be returned with the requested data. This information is used for two purposes. The physical slot number identifies all modules of a local one System with which the data is sent back (local here means that the system is a single Multiprocessor system connected to a single bus without additional systems that are connected via a LAmP module.) The reserved bits are not used by any Memory module and are simply sent back to the request circuit unchanged. This enables the request circuit to read the To correctly identify and assign data that come back from the memory. (For example a LAmP module may need this data to determine which data has just been returned will. A module with multiple processors

-50-

A 46 929 b

k - 176 - 50 -

February 24, 1986

could use the relevant bits to identify the processor that made the request submitted.)

Read cycles on the system bus are started by sending the ABUSREQ η signal. The address bus arbiter 74 recognizes the request and selects the relevant module if the priority is correct. the Address can be accepted by the addressed unit. The written unit can also show that it is busy and request that a new contact attempt be made after four bus cycles will. If the address is not available, the system can discard the address. In addition, the address can be if a LAmP circuit is available, taken over by this and forwarded to another local system.

Write cycles are also started by sending the ABUSREQ η signal. The address bus arbiter recognizes the requirement and, if the priority is correct, selects the appropriate module. The ones to write Data are always transmitted in the bus cycle that immediately follows the address, and with proceed with a hint as to which bytes of one of the long words are to be written. As with the read cycle, the Address can be accepted by the written circle or this can indicate that he is busy and

-51-

A 46 929 b

k - 176 - 51 -

February 24, 1986

that the renewed attempt for an access should take place in four bus cycles. The address can also be such as are discarded by the system during the read cycle because the written unit or location is not is available. In contrast to the read cycle, the data to be written always follows the address independently whether the address is accepted or not. In addition, the address and the write data can then if a LAmP module is connected to the system, from accepted and forwarded to another local system.

In the present embodiment of the multiprocessor computer system according to the invention, the bus cycles have a duration of 80 ns. This time is sufficient to get the required information for one Transfer bus cycle. However, there is not enough time for any module to retrieve the information to be processed within such a bus cycle. Therefore, the system bus is pipelined operated for a level. In other words, the data are transmitted in one cycle and in a second cycle time is available to decide what to do with the data.

The process of transmitting two long words (64 bits) in a data cycle is called a double pump cycle designated. Such a cycle is made by a

-52-

A 46 929 b

k - 176 - 52 -

February 24, 1986

Request circuit requested when the address is transmitted by the double pump cycle request line of the control bus 108 is called with the command REQDPMP L. The address for a double pump cycle must be present at a double longword boundary; i.e. that at least 3 bits of the address are 0 have to. A double pump cycle requirement that doesn't is linked to a long word in a defined way, leads to unpredictable results. The data, resulting from a double pump read request are shared in the same data bus cycle transmitted via the 64-bit wide data bus. However, there is a possibility that not every called device corresponds to a double pump cycle request; she can possibly guarantee only a single transfer. This fact becomes the request unit for the to be read Data communicated when this data is transmitted back to the request unit. At this time a signal NDPMP L is valid and indicates to the request unit whether the double pump cycle request has been met is or whether only a long word of a lower order is returned. A requirement unit, which only wants to perform double-pumping operations, can access the double-pumping enforcement line of the control bus 108 received. A transmission of the relevant signal forces a 64-bit transmission as well as the Subsequent return of the data, whereby the NDPMP L signal is not set.

-53-

A 46 929 b

k - 176 - 53 -

February 24, 1986

The bus signals and the various functions that are assigned to them (some bus signals have already been are discussed in more detail below. In the following description it must be taken into account that the signals in connection with the use of a multiprocessor computer system 10 as shown in FIG explained. The requirement modules can either be processor modules 20 or ETHERNET / mass storage modules Be 90. Furthermore, when referring to PARITY bit generation, the following definition is used intended: the generated PARITY bit generates the specified parity, i.e. an odd PARITY bit is set if the number of ones in its protected field is even, so that parity is odd or an odd number of ones is obtained.

ADD 02 - ADD 31: These lines carry the address of the point addressed by the request unit will. These operations are always carried out with long words, so that the bits ZERO and ONE are not needed. The signals BYTE η L and WORDSEL replace the bits ZERO and ONE during the write cycles. The address on these lines is output as "true", i.e. the bus is not inverted.

ADDP O - ADDP 3: These are the parity bits for the address lines. It uses the following parity algorithm used:

• 54-

A 46 929 b

k - 176 - 54 -

February 24, 1986

ADDP O has odd parity with CYCTYPE 0-1, ADD 02-07

ADDO 1 has odd parity with ADD 08-15

ADDP 2 has odd parity with ADD 16-23

ADDP 3 has odd parity on ADD 24-31

DATA 00 - DATE 63: These lines carry the data transmitted between the devices on the bus from all Longword parts are transferred. DATA 00 - DATA 31 carry 32 data bits. The bit with the lowest value is DATA 00 and the bit with the highest value is DATE 31. DATA 32 to DATE 63 carry 32 data bits from all odd longword positions. The bit with the least significant is DATA 32 and the bit with the highest significance is DATA 63.

DATAP 0 - 7: These bits take care of the parity the data lines. DATAP 0 is the even parity bit for the byte on DATA 00 - 07, while DATAP 7 is the even parity bit for the byte in DATA 56 - 6 3 is. The byte parity is only supplied for a long word, if only one is transmitted. Therefore, the memory module does not have to check the parity of a longword, which one is not written.

ABUSREQ η L: This signal is issued by the request unit η (n lies between 0 and the number N of the available query units, this number being η = 10 in the exemplary embodiment) if these

-55-

A 46 929 b

k - 176 - 55 -

February 24, 1986

requests an address transfer. The ten request unit slots output ABUSREQ 0 - 9 L. The bus adapter 99 outputs the ABUSREQ 10 L signal. Of the System control module 60 does not have to output a bus signal because it contains the bus arbiter. This signal must are output synchronously with the bus cycle.

DBUSREQ η L: This signal is output by the unit η (n is between 0 and 9 in the currently preferred exemplary embodiment) , which wants to send data back to the data bus. The memory modules 0 to 7 each emit the signals DBUSREQ 0 - 7 L. The bus adapter 99 emits the signal SBUSREQ 8 L. The LAmP interface module 200 emits the signal DBUSREQ 9 L. The system control module 60 does not have to emit a bus signal since it itself contains the bus arbiter. The signal under consideration must be emitted synchronously with the bus clock.

CYCTYPE 0-3: These signals are sent together with the address from the request unit to the system bus 100 driven, and they define the type of cycle, which address bus 104 is currently performing. The following DYCTYPE codes are used:

-56-

A 46 929 b

k - 176

February 24, 1986

- 56 -

CYCTYPE 0 12 3
O 0 XX

0 10 1

0 1X0

0 111

10 11

10 0 1

10 10

10 0 0

1 1 X X

Figure 11 shows reading on an automatic read-modify-write cycle at

indicates a private read access cycle

reserved for future assignment

indicates the reading of a public read access cycle

indicates a write invalidate cycle

modified write cycle modified write cycle reserved for future allocation

indicates that there is currently no valid address on the bus.

X = don't worry about it.

It should be noted that it is the responsibility of each device to determine what data is on the bus can return to monitor the CYCTYPE lines for an indication that a write address is on the Address bus is transmitted. When this event occurs, the request unit is during the next Data bus slot to send data to be written. Therefore, every device that plans, during the time slot, which follows a write address transfer to return data by one additional data bus cycle delay.

-57-

Λ * S, * Λ Λ

A 46 929 b

k - 176 - 57 -

February 24, 1986

BYTE η L: These signals are output in all bus cycles (write and read cycles) to indicate which of the bytes are valid. The commands BYTE 0 - 3L apply to the signals O to 3 of the long word, which was selected by the WORDSEL command.

WORDSEL H: This signal determines which of the long word signals BYTE η L must be applied. If negated, this signal indicates that the selected longword is the longword on data lines DATA 00-31 is. When the WORDSEL signal is sent out, the long word on the data lines DATA 32-63 selected.

BYTEP: This signal represents the odd parity of the signals BYTE η L, WORDSEL, FDPMP L, REQDPMP L and CYCTYPE 2 represents.

NOCACHE L: This signal is output by all units that return data that are not in the cache memory should be saved. Such data can come from any place which without visible bus activity for BTAG monitoring can be manipulated. Examples of such locations are two-port memories, registers on one Module or locations that are actually on another bus and only through a bus adapter be seen.

-58-

A 46 929 b

k - 176 - 58 -

February 24, 1986

PRIORITY L: This signal is supplied by a request unit that is not a class B unit, i.e. any request unit with the exception of a CPU. This allows these request units,
such as the EMS modules, the system control module 60
and the bus adapter 99 provides quick access to the bus 100.

MEMORY ACCEPTED L: This signal is output by a memory unit to indicate that it has the
Address on ADD 02 - ADD 31 has successfully decoded and that the request for a data transfer has been accepted.

MEMORY BYPASS L: This signal is asserted from a non-write-through cache memory which is based on
the current address on address bus 104 wants to respond. When this signal is present, it overflows the MEMORY ACCEPTED L signal
Refrain from transferring him in response to this
Address had planned.

CACHE ACCEPTED L: This signal is output only when the MEMORY BYPASS L signal is output. if it is issued, it indicates that a write-through cache is not in use accepted the address.

WRITE DATE PARITY L: This signal is output one bus cycle after the point in time at which the

-5 9-

A 46 929 b

k - 176 - 59 -

February 24, 1986

The MEMORY ACCEPTED L command would take effect (for a write data transfer) . This signal shows whether the write data was transferred successfully or not.

REQDPMP L: This signal is issued by the requesting unit to indicate that the requested bus cycle is a double pump cycle. It should be noted that the address of all double-pumping cycle requirements must be a double long word that must be paid if the results are not to be unpredictable.

DPMP L: This signal can be sent by the target unit of a Double pump cycle request can be issued. It is released when the double-pumping cycle, the demanded was done by a double pump cycle will.

FORCE DPMP REQ L: This signal is issued by the request unit to the called memory module forcing to do the request through a double pump cycle. If the bank in question the memory module is busy, the request is repeated.

STALL CYCLE L: This signal can be used by any request unit at any time due to a BTAG FIFO overflow or a determined bus address

-60-

A 46 929 b

k - 176 - 60 -

February 24, 1986

Parity errors are output. If the signal is present, the memory module must respond to the request discard, which caused the blocking operation, and the request units must write and Suspend read-modify-write requests.

UNJAM L: This signal is sent / if a memory has not been able to / during a given Number of bus cycles to get access to the data bus 102. If it is present, this prevents Signal new requests for the data bus 102 and suspend the assignment of the address bus 104 through the address bus arbiter 72 of the system control module 60.

ADDID 0-5: These signals are output by selected request units together with the addresses. They indicate to the called module which request unit is carrying out the transfer. That ADDID signal consists of two fields: the field with the physical slot number and two reserved bits. The following scheme is used / to define the physical slot field / ADDID 0-3:

Oxxx - 1001 requirement module 0-9 according to

Identification by the slot no.

1100 system control module

1101 bus adapter.

CpPY

A 46 929 b

k - 176 - 61 -

February 24, 1986

ADDID 4 and 5 are for use by the requester reserved. The memory returns these bits together with the requested data, and these bits are not modified. This enables the query unit to transfer the data in this way add that they are returned with any two information bits.

ADDIDP: This bit ensures even parity for the ADDID 0-5 signal.

DESTSEL 0 - 5: These signals are from a unit supplied, which supplies previously requested read data on the data bus. They are simply a copy of the Signals ADDID 0 - 5 as they are during transmission the address of this unit were used. They indicate to which request unit the requested Data are returned on the bus and consist of two fields: the field with the physical Schiitζ number and another field with two reserved bits. The following scheme is used to define the field with the physical class number, DESTSEL 0-3:

Oxxx - 1001 requirement module 0-9 according to

Identification by slot no.

1100 system control module

1101 bus adapter

1111 no valid read data on the Bus (valid write data can be present on the bus).

-62-

COPY

360621ϊ

A 46 929 b

k - 176 - 62 -

February 24, 1986

DESTSEL 4 and 5 are reserved for use by the request units. The memory delivers return these bits along with the requested data, and these bits remain unchanged. this makes possible to provide the request units / the data that are sent back to them with any two bits of information. When the data to be transferred write data then the requester sets the DESTSEL lines to ones if the cache is of the request unit is a write-through cache. When a requesting unit requests data, the are stored in a write-through cache memory, the DESTSEL signal contains the ADDID information the unit of request for the data. The finished In this case, the status of all ones will indicate to all other modules that they are write data, and these should be ignored by all requirement modules.

DESTSELP: This signal supplies odd parity for DESTSEL L-bits 0-5, for DPMP L and MOCACHE L.

SLOTID 0-3: These four signals are specially encoded in each slot on bus 100. this allows a module to read these lines and determine which slot they are in.

BCLOCK η L: This signal supplies the basic bus cycle. In the preferred embodiment, the

-63-

A 46 929 b

k - 176 - 63 -

February 24, 1986

Period duration of the clock pulses can be short and up to
be only .80 ns, with the pulse duration between 22 ns and 33 ns. The BCLOCK η L signal is distributed over three lines that are connected as follows:

BCLOCK 1 L memory modules 0-2
BCLOCK 2 L storage modules 3-5

BCLOCK 3 L memory modules 6, 7, request unit - slot 0, system control module

BCLOCK 4 L request unit - slot 1-4 BCLOCK 4 L request unit - slot 5-7

BCLOCK 6 L request unit - slot 8, 9; Bus adapter.

It should be noted that all lines are three modules
drive, except for the signals BCLOCK 3 and 4. At the
Under consideration, these two signals drive shorter stretches of the back wiring so that they
have an additional module load.

DCOK H: This signal is evaluated by the system control module · 60 and the power supply · in order to ensure the entire Reset the system hardware except for yourself. With the signal DCOK H, all system states go lost. This usually happens after the power is switched on, after an unrecoverable one System failure or if the DC power supply drifts out of tolerance.

-64-

A 46 929 b

k - 176 - 64 -

February 24, 1986

POWERFAIL L: This signal is sent by the power supply. It shows / that the AC voltage fails and that at least three ms with sufficient DC voltage remain before the DCOK H signal fails. The following matrix defines the possible combinations of the signals POWERFAIL L and DCOK H:

DCOK H POWERFAIL L INDICATION

set set power failure

set not set normal operation

not set set reset at

turn on

not set not set reset the

software

SCMNMI L: This signal is set by a module in the system that sends a non-maskable interrupt command to the diagnostic processor of the system control module 60 would like to achieve. When the signal is set, freeze all modules use their bus interfaces and trigger local, non-maskable interrupt commands. This signal is synchronous with the system signal BCLOCK. It must be set by each module on the leading edge of the ENDATA signal and held for at least one major cycle. All modules key the signal SCMNMI L when it occurs the leading edge of the CKDATA signal.

-65-

Λ fe β -

A 46 929 b

k - 176 - 65 -

February 24, 1986

UNFREEZE η L: This signal is set by the diagnostic processor 62 of the system control module in order to determine the Enable the bus interface of a request module. Although setting this signal the bus interface of the selected module, it leaves the data cache memory, the bus parity check and the interlocking logic are blocked. This logic is released again, when the system control module 60 cancels the UNFREEZE L signal. This event can be from a Requirement module can be monitored by making the UNFREEZE L-line readable in a local CSR. There are a total of 11 UNFREEZE L signal lines - one for each request unit, except for the system control module itself.

TESTRACK L: This signal is not related to the multiprocessor system tied together. Each module must have a pull-up resistor on this line. If the module is inserted into the burn-in test frame, then the connector that is plugged into it becomes this Ground pin. This enables the module to decide how to conduct its self-examination should.

ADDSEL η L: These signals (n is between 0 and 10) are set by the address bus arbiter to select the next module that will be selected to be its To put the address on the address bus. It should be noted that the system control module 60 does not have any access

-66-

A 46 929 b

k - 176 - 66 -

February 24, 1986

has permission line because it contains the arbiter. These Signals are not coded - there is an access permit line per requirement unit.

DATASEL η L: One of these signals is sent by the data bus arbiter set to select the next module that will be selected to add its data to the To give data bus. Each of the DATASEL L lines is connected to one of the modules. The DATASEL lines 0-7 are connected to memory modules 0 to 7, respectively. The DATASEL 8 L cable is connected to the bus adapter 99 connected, and the line DATASEL 9 is connected to the LAmP module 200.

In addition to the data bus 102 and the address bus 104, the system bus 100 includes a vector bus 106. This vector bus is responsible for the transmission of vector interrupt signals between the modules in each local system 10. All bus request units can cause interrupts for the other request units, and therefore all request units must Have access to the vector bus 106. The vector bus 106 does not only allow transmission of vectors between the request units, but it also supports the decision between the Requirement units for class interrupts. When a request unit gives an interrupt command wishes to transmit, this becomes a class of

-67-

A 46 929 b k - 176

February 24, 1986

- 67 -

Transfer orders. A directed interrupt designation allows the specification of a special requirements module. In this case it takes place the vector transfer direct, i.e. the vector goes straight to the specially selected request module. If, on the other hand, a class interrupt command is given all requirement units within this special class must make a decision, and only the requirement unit with the lowest priority can receive the interrupt command. The latter function will achieved by a parallel arbitration scheme, which will be described below.

The vector bus 106 also allows the transfer of vectors through the LAmP interface module 200 through to another system 10. This can be done in the Ways happen that one targets an entire class in another system, or by targeting a particular one Requirement unit specified in another system.

The vector bus 106 is a bus that has twelve open collector signal lines and two TTL signal lines includes. Ten of the open collector signal lines are used as vector data lines; an open one The collector signal line is called the LAmP selection / bus adapter request line used, and the other open collector signal line is used as an acknowledgment line.

-68-

A 46 929 b

k - 176 - 68 -

February 24, 1986

turns. The two TTL signal lines are vector bus phase lines. The vector bus is controlled by an arbiter controller, which is a component of the system control module 60 is. This controller also determines the phase position on the bus.

The bus 106 goes through three types of clock cycles: one Idle cycle, a directed vector cycle, and a class vector cycle. If the vector bus 106 idles, i.e. if no request units request access to this bus, the following specified operations performed repeatedly until a request unit issues a request during the request phase (as shown in Fig. 11):

1. Request of the bus 1 bus cycle

2. Allocation of the bus to 1 bus cycle

3. Idle 1 bus cycle.

In performing the directional vector transfer, the following operations are performed (as shown in Fig. 12a):

1. Request 1 bus cycle

2. Allocation of the bus to 1 bus cycle

3. Transmission of the vector 2 bus cycles

4. Acknowledgment of receipt

of the vector 1 bus cycle.

-69-

36062ΐΊ

A 46 929 b

k - 176 - 69 -

February 24, 1986

If the vector transmission is the transmission of a class interrupt vector, the following operations are performed (like this shown in Fig. 12b):

1. Request of the bus 1 bus cycle

2. Allocation of the bus to 1 bus cycle

3. Transmission of the vector 2 bus cycles

4. Decision of the processor, which units the vector

must accept 4 bus cycles (maximum)

5. Acknowledgment of receipt of the

Vector 1 bus cycle.

All transmissions via the LAmP interface module 200 to a different system 10 than the one in which the request unit is, regardless of its type, going through a vector bus cycle, as whether they were directed transmissions. This is because a transmission can only be made by the requesting unit to the LAmP module 200 takes place, and that the LAmP module 200 performs the correct vector bus cycle in the local system 10 specified as the target. One Transmission to a non-local system is identified by the fact that the LAmP select signal during two Vector bus transfers is set. One vector bus cycle corresponds to two system bus cycles.

Ten requirement units can be requirement lines

-70-

A 46 929 b

k - 176 - 70 -

February 24, 1986

to get a decision about access to the vector bus. The bus adapter 99 can also a request line for the LAmP selection / bus adapter request set order management. Since the system control module 60 contains the vector bus arbiter, must he didn't put a bus signal line to the vector bus to request. Access to the modules is granted on the same line on which they are requested became.

During a vector data transfer cycle, a number of pieces of information are passed on the vector bus on vector data lines 0 to 9, by the request circuit to which the bus is assigned became. This information is contained in two data words, as shown in FIG. The LAmP identification number 110, which is used when the vector is sent from a local system or is received which is a different system than the system used by the Bus 100 being served, which carries the vector, is a four-bit number. Two of these bits 110a belong to the first data word 114, and the next two of these bits 110b belong to the second data word 112. The In addition to the first two bits 110a of the LAmP identification number, the first data word 114 contains a Type identification number 122, which indicates how the SLOT ID / CLASS bits 118 are to be interpreted. When the type identification number 122 is 0, i.e. if the transfer is directed,

-71-

A 46 929 b

k - 176 - 71 -

February 24, 1986

then the SLOT ID / CLASS bits 118 contain a number, which indicates the slot of the back plane which contains the module which is to receive the vector. If the type is 1, the SLOT ID / CLASS bits contain the class number of the vector. The first data word 114 also contains a 3 bit CPU identification number 120, which identifies a processor of a module in the selected slot. In addition to the two second bits 110a of the LAmP identification number the second data word also contains a vector identification number 116.

After the data transfer phase occurs, one of two sets of events occurs, and depending on the type of vector that was transmitted. When a directed vector or a nonlocal system vector of some kind has been transmitted the called module simply transfers the previously received vector to its local processor and sets the VECTOR TAKEN signal. If it was a class vector and if the LAmP selection signal while the vector data transfer was not set, the request modules have to make a decision hit across the vector. The intent of the decision-making scheme is to have all modules in the same Enable class such as the transmitted vector to determine which of the modules has the lowest priority Has.

-72-

A 46 929 b

k - 176 - 72 -

February 24, 1986

A module priority is determined by the following information / which is given to the 8 vector data lines is / are shown in the diagram of FIG. The silo depth or FIFO queue counting bits 124 show that one or more vectors have been fed into the processor in succession. Allow the silo depth bits an even distribution of the vectors among the requirement units within a class. The slot identification number 128 indicates in which slot the module is placed. The decision-making scheme is a parallel decision-making process, in which a module all of the above information on a Bus with open collector and compares the information actually received from the bus with the information / which he spends. This comparison is allowed with the highest order bit and down to the lowest bit Order processed. When there is a mismatch between what is being spent and what is found on the bus, the module masks the transmission of all bits of lesser significance the end. Because in the decision information a module slot number is included, it is guaranteed that only one module wins the decision-making process at a time, even if two modules have the same priority and operational setting. This comparison and elimination process takes place asynchronously, which is why four bus cycles are provided for the vector reception decision corresponding to the maximum time required to complete the decision lines .

-73-

A 46 929 b

k - 176 - 73 -

February 24, 1986

The system bus 100 carries address and data information during a major cycle of the bus. The data transfers through bus 100 require a timing pulse and a clock edge. The timing pulse is used to block the current bus driver and to select a new bus driver and enable the new bus driver again. The clock edge is used to pass the data at all clock in the remaining modules of the system.

The accuracy of the distribution of these timing signals is critical to performance on bus 100. It is not possible to distribute these edges with the required accuracy because of the capacitive delay on the bus, due to the shift between the gates on different modules, due to threshold changes on the Gates, etc. Therefore, the scheme which is chosen for the bus 100 according to the invention is, to distribute only one timing signal edge over the system. This edge is used to create a delay line / pulse generator unit to trigger, which allows a precisely controlled pulse generation. The diagram of Fig. 15 shows the theory of the timing pulse and the timing edge generated by the delay line. That The ENDATA L signal is the signal that triggers the bus driver lock pulse 132 is generated, which clocks for the next driver selection. The pulse s 132 must be a pulse

A 46 929 b

k - 176 - 74 -

February 24, 1986

width that is large enough to hold everyone at his Switch off the running driver before the next driver is switched on. The trailing edge of pulse 132 is used to re-enable the next set of drivers. The qualification takes place on each module to generate the EN BUF TO BUS L signal. A signal CKDATA H is used as a data clock used.

The analysis of the timing of the bus according to the invention, which is explained with reference to FIG results from the use of a delay line as well as from an electrical analysis of the worst case with regard to bus distortion and threshold value changes. At the Reading the diagram of the analysis of the timing, it is noted that the signal CKDATA H for the previous Cycle, based on the ENDATA L signal, is applicable. In addition, two impulses are BUS TRANSFER ENABLE 1H and TRANSFER ENABLE 2H available. These pulses are required to ensure that the modules run reliably to enable internal transmission of the data received from system bus 100.

The circuit for generating the delayed pulses shown in Figs. 15 and 16 is in Fig. 17 shown. The delay circuit is clocked with the bus BCLOCK η L clocked, which is applied to the delay line via a NAND gate 136, whose

-75-

A 46 929 b

k - 176 - 75 -

February 24, 1986

other input fed by delay line 140 will. The output of NAND gate 136 becomes delay line 140 and the input of NOR gates 138 supplied. The second input signals for the NOR gates 138 are fed via the LOCAL ENABLE L connection. The outputs of the five NOR gates 138 provide the CKDATA Η signals. Another output signal the delay line 140 is fed to a NAND gate 142. The second input signal for the NAND gate 142 is supplied by a D flip-flop / its data line (D- <input) with the connection LOCAL SELECT H is connected. The NAND gate 142 provides the enable signal for the writing of Data to the bus signal. Another output signal the delay line 140 is fed to a NAND gate / its second input signal from a D flip-flop 148 is supplied, which as a data signal receives the ADDSEL signal. The NAND gate supplies the signal EN BUF TO BUS L. The last output signal the delay line 140 is fed to NAND gates 150, whose respective other inputs are connected to a 5 V voltage source. The NAND gates 150 provide the BUS TRANSFER signals ENABLE 1H and BUS TRANSFER ENABLE 2H.

System memory

The system memory of the multiprocessor computer system according to the invention comprises two independent storage

-76-

A 46 929 b

k - 176 - 76 -

February 24, 1986

banks 41, a control and status register (CSR) 48 and an associated diagnostic processor 46 for a Self-examination. The addressing structure supports nesting in four ways between storage modules

40 of equivalent size. The module nesting is made possible by. CSR bits 12 and 13 are controlled and carried out automatically between the banks. The memory bus interface is an integral part of the bus system 100 and as such can be viewed as an extension of the bus. The internal bus 154 of the system memory 40 ₇ BUF DATA operates at twice the data rate on the bus 100. Read data is transferred from a memory bank 41 or the CSR register 48 to bus interface buffers when the ENDATA signal is negated. Write data are transmitted by the bus interface when the CLKDATA signal is negated. This architecture enables the memory system to transfer both read and write data between the bus interface 165 and the memory banks (or the CSR register) within a single larger bus cycle.

As shown in the simplified block diagram of the memory system in Figure 3, the memory banks are

41 and the register 48 are separate subsystems, which are connected to an internal bus 154 via interfaces are. The internal bus 154 is established by a bus interface controller 156 or by the integrated Diagnostic processor 46 driven. If the

-77-

A 46 929 b

k - 176 - 77 -

February 24, 1986

Diagnostic processor 46 is connected to the internal bus 154, the memory for the bus interface controller 156 is invisible. The system diagnosis processor 46 must therefore implement a time-out mechanism whenever the self-test is initiated by writing to CSR bit 15. Since the CSR register 48 is not available in this operating mode, the data bus request logic implements a priority mechanism by means of which data transfers on the internal data bus 156 are synchronized with regard to the data availability of the individual banks. The parity of the addresses from the B \ s interface controller 156 is checked at the receivers and, if there is an error, the address is considered nonexistent. The parity of the write data is checked in the write data parity check logic 49. If an error is detected, the write data is discarded and the signal WRITE DATA PARITY ERROR L is set. The parity of the read data is generated locally in each subsystem. The parity of the ADDID field is not checked; the ADDIDP signal is only copied to the DESTSELP signal for CSR reference variables or the complement of DESTSELP is formed for memory reference variables. This is necessary because the DESTSELP signal has parity with the NOCACHE signal.

As Fig. 11 shows, each memory bank 41 comprises two

-78-

"36062 Ϊ1

A 46 929 b

k - 176 - 78 -

February 24, 1986

Rows of RAM memories / those with a single bidirectional Data bus 160 are connected. The interface connection with this internal bus is via an error detection and correction chip (EDAC) along with parity logic 164 for generating and Check parity for system bus 100. There are no error correction code bits on bus 100. The read data parity is generated in the parity logic 164 and transmitted to the bus interface 165 via BUF DATA. The write data parity is checked by the logic 49, and in the event of errors the signal WRITE DATA PARITY ERROR L set and the write cycle becomes modified to a refresh cycle. The CSR register 48 is recorded with the signal ADDID of the transmission. animals. The diagnostic processor 46 uses DATAP lines to check bits and / or syndromes between to transfer the memory banks and its data interface. The timing control of each of the memory banks 120 is carried out by separate controllers 166 which operate independently. With the currently preferred The exemplary embodiment is the controls or controllers as status sequencers of the type 82S105A educated. Two devices are operated in parallel per bank in order to provide sufficient control output signals produce. Seven basic types of cycles are defined for each controller, as follows:

-79-

A 46 929 b

k - 176 - 79 -

February 24, 1986

Cycle explanation

0 refresh

1 bus interface - read only

2 Bus interface - read lock

3 Bus interface - long word

to write

4 bus interface - write

5 Diagnostic circuit blocked -

read correct

6 Diagnostic circuit blocked -

correct writing.

When cycle 0 (refresh) is activated, bank 41 is selected when the refresh timer requires a refresher. A refresh cycle always has priority over anyone pending Request from the bus interface 165 or the Diagnostic Processor 48. When the INITIATE ECC signal is asserted, the controller locks the output register of the error detection and correction chip (EDAC) 162 and enforces an operating mode in which check bits be generated. In this way the random contents of the EDAC chip output register are put together written into memory with correct check bits. Then the RAM timer is triggered, so that the writing into the RAM memory 168 takes place. If the initialization of ECC does not take effect is the contents of the updated addresses

-80-

A 46 929 b

k - 176 - 80 -

February 24, 1986

read and stored on the EDAC chip 162. if error scanning is enabled, the data is checked for errors. Single-bit errors will be corrected and reported back to the memory. Multiple bit errors are ignored.

A cycle 1 (read only) is initiated when the relevant bank 41 loads its address memory and if there is no refresh cycle pending. If the bank isn't busy right now, the RAM timer is triggered by the address load logic. In parallel, the bank sequencer 166 started. If the bank 41 is currently busy, the RAM timer from the sequencer 166 will be on triggered a busy-to-idle transition state. "Read only" occurs during a cycle the access to the selected row of RAM memories, and the read data stored on the EDAC chip 162 and in the bank-to-buffer data registers 170 are stored. The sequencer 166 or the controller generates a request the data read via the bus interface 165 in parallel with the checking of the data for errors transferred to. If there are no errors, the previously loaded RAM data are transferred to the bus interface 165 transferred, otherwise data protection deleted when granted, and the corrected data (if it is a single-bit error) will be from the EDAC chip 162 again into the bank-to-buffer

-81-

A 46 929 b

k - 176 - 81 -

February 24, 1986

Register 170 loaded. Incorrectable data errors are recorded using the CSR register 48.

A cycle 2 (read interlock) is basically a read process that is automatically followed by a registered letter connects to the same storage location. In other words, this operation is one internal read-modify-write cycle on the Bank level. During this cycle, memory bank 41 performs a read operation identical to cycle 1 (read only) explained above and sends the data to the request unit. While During the write phase, the bank controller 166 causes all ones to be written to the byte storage location that has just been read out. The advantage of such a read-modify-write cycle, which is performed by the memory module 40 is automatic bit setting operations can be performed without moving the bus and prevent further access to the bus.

In cycle 3 (longword writing), longwords are written by taking 32 bits of data from writes the buffer-to-bank register 170 into memory along with the correct check bits.

A cycle 4 (write byte) is the read-interlock cycle 2 is similar in that both cases have a

-82-

A 46 929 b

k - 176 - 82 -

February 24, 1986

internal read-modify-write operation performed will. The content of the memory location to which access is made is read from the RAM memory, stored on the EDAC chip 162 and checked for errors. Single-bit errors, if any, are corrected automatically. Errors that cannot be corrected are signaled and in the CSR register recorded. In the case of an uncorrectable error, the write operation is dispensed with, and the cycle is ended. If there are no uncorrectable errors, the byte to be written from the buffer-to-bank registers along with the unchanged bytes, corrected if necessary, enabled by the EDAC chip 162 and placed on the RAM data bus 160. The resulting word will come together written into the memory with the new test bits.

A cycle 5 (lock diagnostic unit - correct read) is used to set the diagnostic processor to allow the RAM data to be read without intervention by the EDAC chip 162. In a corresponding way the operations of cycle 6 (block diagnostic circuit - correct write) are used to write Data and check bits that are in the buffer-to-bank registers are stored into memory bypassing the EDAC chip 162. This mechanism allows the diagnostic processor 46 all data or check bit patterns for checking

-83-

A 46 929 b

k - 176 - 83 -

February 24, 1986

of EDAC chip 162 into memory.

While the multiprocessor computer system according to the Invention has been described above with reference to preferred embodiments, it is understood that to the person skilled in the art, based on the exemplary embodiment taking into account the detailed description and the associated drawings, there are numerous possibilities for Changes and / or additions are available without departing from the basic idea of the invention would have to. In particular, the various modules, especially the memory and processor modules, modified to include other configurations of assemblies, e.g. more as two processors as elements of a processor module.

-SV-

- blank page -

Claims (1)

HOEGER, STELLRECHT & PARTNER 3βQ62 1 PATENTANWÄLTE UHLANDSTRASSE 14 c D 7OOO STUTTGART 1 A 46 929 b Applicant: ENCORE COMPUTER CORPORATION k- 176 257 Cedar Hill Street February 24, 1986 Marlboro, Massachusetts 01752 UNITED STATES. Claims 1. Multiprocessor computer system, characterized by the following features: Several processor modules are provided, each of which has at least one processor and the coupling serving interface devices includes, over which data, addresses and program interruptions (interrupts) are transmittable, wherein the processor also cache memory devices to save the contents of frequently. has controlled memory locations; system memory devices are provided, which interface devices are used for coupling have, over which data can be transmitted and to which each of the processors has access Has; they are operational control devices for assigning and controlling the flow of a number of processes provided for at least one processor; there are timing devices for generating Timing signals are provided which define successive operating intervals; there are system bus devices provided that -2- A 46 929 b k - 176 - 2 - February 24, 1986 with the interface devices of the processor modules used for coupling and the system memory devices are connected and include: a) Address bus devices for transferring memory addresses from one of the processor modules to the system storage facilities; b) data bus devices for transferring data from the system memory devices to the individual processors and from these to the system memory devices; c) vector bus devices for transmitting vector interrupt signals between each one of the processor modules and another processor module; d) control line devices as paths for signals from the operational control devices to the Processor modules and the system memory devices and from the processor modules and the system storage devices to and from the operational control devices are; and they are arbiters for arbitrating access to the address bus, data bus and vector bus devices intended. 2. System according to claim 1, characterized in that each of the multiprocessor modules the following elements includes: -3- A 46 929 b k - 176 - 3 - February 24, 1986 Writing devices for writing the contents of frequently accessed memory locations into the Cache storage devices; Processor add-ons for storing system memory addresses of the frequently accessed Storage locations, the contents of which are stored in the cache storage devices; Additional bus devices for monitoring the system bus devices for write operations that involve a System memory space relate to its content in the cache memory devices of the processor module is stored; Update devices, which are based on the bus add-on address devices to indicate to the processor options that the contents of a Cache memory space is different from the content of the main memory space corresponding to this. 3. System according to claim 1, characterized marked, that the system bus devices are designed for the interleaved execution of several operations are. 4. System according to claim 1, characterized in that the operation control devices address bus arbiter devices for assigning access to the address bus facilities, which include determining the priority of access for processor modules A 46 929 b k - 176 - 4 - February 24, 1986 and non-processor modules decide such that the priority is given to non-processor modules is granted to the processor modules, and that the access for the processor modules according to Art one revolution is chosen. 5. System according to claim 4, characterized in that that the address bus arbiter means reservation means for effecting a reservation on the address bus devices for the duration of one data read cycle. 6. System according to claim 1, characterized in that the vector bus devices are decision devices that determine which of several processors to receive a transmitted Vector is released. 7. System according to claim 1, characterized in that the vector bus devices are dividing devices to divide the processors into groups and for an even distribution of the vectors regarding the groups. 8. System according to claim 4, characterized in that the operational control devices are data bus arbiter devices for allocating access to the data bus devices, wherein the -5- A 46 929 b k - 176 - 5 - February 24, 1986 Data bus arbiter devices are designed such that they provide access to the data bus for one Release request module that requests a write operation and an address on the address bus facilities and where the data bus arbiter also decision-making bodies for those modules that request a read / reply operation via decide the priority of access to the data bus devices. 9. System according to claim 1, characterized in that the data bus arbiter devices are reservation devices to reserve the data bus devices before the time at which the data bus devices are required by a request module. 10. System according to claim 1, characterized in that that the operational control devices comprise repeaters with the aid of which the experiment performing a requested operation, which is requested by the target circuitry of the requested operation was not accepted, is repeatable. 11. System according to claim 4 and 10, characterized in that that the address bus arbiter means comprise additional means with the help of which -6- A 46 929 b k - 176 - 6 - February 24, 1986 a restoration of the priority of a repeated Request can be brought about without disturbing the circulation priority scheme. 12. System according to claim 1, characterized in that the operating control devices release devices include, with the help of which access for a data request can be granted, which Access to the data bus devices was denied for a predetermined time interval, with the Release devices include locking devices to prevent new requests from accessing to the system bus devices. 13. System according to claim 1, characterized in that interface devices are provided, which are connected to the system bus devices and an exchange of data, address and enable interrupt signals with at least one other multiprocessor computer system, such that the one multiprocessor computer system and the at least one further multiprocessor computer system operate as a single, bus-connected system of tightly coupled processors. 14. System according to claim 1, characterized in that the cache memory devices of the individual - 7 - A 46 929 b k - 176 - 7 - February 24, 1986 Processor modules include writing devices, with the help of which the same data in the cache memory devices and are writable to the system storage devices. 15. System according to claim 1, characterized in that the cache memory devices of the individual Processor modules comprise switching devices which, when actuated, send a single letter into the cache memory devices can be brought about and that the switching devices display devices include, through which can be displayed for the system storage devices that the at a Storage space of the cache memory devices from the data stored on a corresponding one Storage space of the system storage devices stored data are different. 16. System according to claim 1, characterized in that the system storage devices are storage devices for locking a memory location during a read-modify-write cycle and that the locking devices are designed such that they can transmit data and Allow addresses by the system when the memory space is blocked. -8th- A 46 929 b k - 176 - 8 - February 24, 1986 17. System according to claim 1, characterized in that interface devices are provided, which are connected to the system bus devices and an exchange of data and address interrupt signals with at least one other multiprocessor computer system and that the interface means cache memory means for storing the contents of frequently have controlled memory locations of the at least one further microprocessor computer system. 18. System according to claim 1, characterized in that the cache memory devices as write-through -Cache storage devices are designed in which data is stored in cache storage locations are to be written "can also be written into memory locations of the system memory devices. 19. System according to claim 1, characterized in that the cache memory devices as non-major Write memory devices are designed in which data stored in cache memory locations be enrolled, only be enrolled in this. 20. System according to claim 19, characterized in that that display devices are provided by which the request devices can be displayed -9- A 46 929 b k - 176 - 9 - February 24, 1986 is that data stored in a storage location of the system storage devices is in of the valid form are only stored in the write-through cache storage devices. -10-