CA1247249A

CA1247249A - System bus means for inter-processor communication

Info

Publication number: CA1247249A
Application number: CA000503727A
Authority: CA
Inventors: David L. Whipple; Tom R. Kibler
Original assignee: Wang Laboratories Inc
Current assignee: LG Electronics Inc
Priority date: 1985-03-15
Filing date: 1986-03-11
Publication date: 1988-12-20
Also published as: AU577427B2; US4719622A; EP0194462A2; EP0194462A3; AU5114785A; DE3689198D1; DE3689198T2; EP0194462B1; JPH0772889B2; JPS61211756A

Abstract

ABSTRACT OF THE DISCLOSURE
A system bus structure and operation for interprocessor com-munication in an information processing system incorporating a plurality of processing elements, each of which may be capable of independently executing certain information processing or handling operations. The system bus structure comprises an access control bus for conducting a bus access signal indicating the occurrence of an interprocessor communication and a communications bus which includes a first bus for conducting signals indicating that the interprocessor communication is a specific one of a first class of communications or is a member of a second class of communications and second bus for communicating the information content of the communication. The first class of communications includes memory related communications and the information content of a communication of the first class includes a source or destination memory address field and, if the field contains a destination address, a data field containing the information to be communica-ted. The second class of communications includes non-memory rela-ted communications and the information content of a communication of the second class includes a first field indicating the destination element, a second field indicating the type of communication, and a third field containing a message.

Description

t7'2~3 SYSTEM BUS MEANS FOR INTER-PROCESSOR COMMUNICATION

BACKGROUND OF THE INVENTION

1. Field of the Invention The present invention relates to a bus structure for use in an information processing system and, more particularly, to a system bus for interprocessor communication among processing elements connected from the system bus.

Z. Description of the Prior Art A basic architectural feature of many information processing systems, that is, computer systems, is the use of a common system bus to interconnect a plurality of processing elements, such as CPUs, memories and input/output controllers, each of which is capable of the independent execution of data processing and handling operations. The system bus provides the means by which the processing elements communicate, that is, coordinate their operations and exchange information, such as data.

As such, the structure and operation of the system bus is central to the operation and performance of the system and several recurring problems are seen in the design of such system busses. For example, it may be necessary to expand or reconfigure the system bus to meet differing functional requirements of the system. Additionally, it is desirable to be able to reconfigure the system bus without requiring ~Z~'72~3 changes to the processing elements and should be as simple to accomplish as possible in terms of the bus itself. Moreover, changes in the configuration of the-system bus should not in themselves alter the operation of the system or the processing elements.

Another probleln is the manner in which the processing elements determine priority of access to the system bus. In many systems, the relative priorities of access to the system bus of the processing t elements is either hard-wired or otherwise determined by the physical location of the processing elements in the system. The simple addition of a processing element, for example, another input/output controller, may require substantial physical rearrangements to the system. Physical bus access determination methods are undesirable also in multi-processor systems wherein many, if not all, of the processing elements essentially require equal access to the system bus.

In an alternate common method of determining system bus access, the processing elements are provided with a means for contending for priority of access to the bus. This approach reduces the possible speed of operation of the system bus due to the overhead required by the contention process, and may result in some processing elements being locked out of access to the bus for too long periods.

Yet a further concern is the manner in which interprocessor communications are actually performed. In many systems, all communications are treated in the same manner, so that frequent types of communications cannot be executed in an accelerated manner, thereby again limiting the speed of the system bus.

In addition, in many systems the types of communications between the processing elements are fixed. As a result, it is difflcult to provide for new types of communications or to expand those origlnally provided, so that any addition or expansion of communications must be fitted into the existing communications types. This in turn limits the flexibility of the system in adapting to new or expanded functions and may result in a degradation of system performance.

The present invention provides a system bus structure and operation having improvements and features which address the above described problems and limitations.

SUMMARY OF THE INVENTION

The present invention relates to a system bus structure and operation for interprocessor communication in an information processing system incorporating a plurality of processing elements, each of which may be capable of independently executing certain information processing or handling operations.

The system bus structure presented herein comprises an access control bus for conducting a bus access signal indicating the occurrence of an interprocessor communication and a communications bus which includes a first bus for conducting signals indicating ~'72~ 3 ~, that the interprocessor communication is a speciFic onP oF a fir~;t class of communications or is a member of a second class of c~m-munications an~ a second bus for communicating the.information content of the communication.

The first class of communications includes memory related communications and the inFormation content of a communication of the first class includes a source or destination memory address field and, if the field contains a destination address, a data field containing the information to be communicated. The second class of communications includes non-memory related communications and the information content of a communication of the second class includes a first field indicating the destination element, a second field indicating the type of communication, and a third field containing a message. The information content of a communication of the second class may further comprise a data field containing information to be communicated.

In the present implementation, the second bus comprises an address bus means for communicating the address fields of the first class of communications and the first, second the third fields of the second class of communications and a data bus for communicating the data fields of the first and second classes of communications.

The access control bus comprises a loop, ~ith the processor elements connected in series along the access control bus means loop, and the processor elements are connected in parallel to the communications bus.
.

708fi(~-59 72~

The bus access signal is conducted th:rouyh the aeeess control bus loop to each processor element in turn along the access eontrol bus loop, so that the relative priorities of aceess of the proeessor elements to the communications bus rotate around the access control bus loop as the bus access signal is propagated around the access control bus loop.
The normal state oE the bus access signal indicates that the communications bus is available for access by the processor elements. A proeessor element takes aeeess to the communications bus by blocking propagation of the bus aeeess signal, thereby foreing the bus aeeess signal propagated to all other proeessor elements eonneeted in the aeeess control bus loop to a state indicating that the eommunieations bus means is not available for aeeess.
It is thereby an object of the present invention to provide an improved system bus means for eommunieating between elements of the information proeessing system wherein there is a first bus means for conducting signals indicating whether an interprocessor communication is a specific one of a first class of communications or is a member of a seeond elass of communications and a seeond bus means for conducting the information content of the eommunieation.
The invention may be summarized aeeording to a first broad aspeet, as a system bus for eoupling eomponents of a data proeessing system whieh ineludes a memory eomponent and other components and carrying a communieation between at least two of the eomponents, the system bus eomprising: a first plurality of lines for carrying ~irst codes including a plurality of me~ory 7~
- 5a - 70840-59 operation codes speci:Eying a plurality oE types oE memory eom munications having an other component as a source and the memory component as the recipient and a single interprocessor communi-cation code specifying a non-memory communication having one of the other components as a source and another thereof as a recipient;
a second plurality oE lines which, when the first plurality of lines speci:Eies one of the memory communication types, carries an address in the memory eomponent and, when the first plurality oE
lines specifies a non-memory communication, carry one of a plurality of type codes specifying a plurality of types of non-memory communications and one of a plurality of recipient address codes specifying an other component as recipient; a third plurality of lines for carrying si~nals specifying the status of a current memory communication; and a fourth plurality of lines for carrying signals specifying the status of current non-memory communication.
Other objects, advantages and features of the present invention will be understood by those of ordinary skill in the art after referring to the following detailed description of the preferred embodiment and drawings, wherein:

72~g i BRIEF DESCRIPTION OF THE DRA~INGS

Fig. 1 is a block diagram of a system incorporating the present .
invention;

Figs. lA and lB are alternate system bus topologies;

Fig. 2 is a diagram of the bus structure of the present invention;

Fig. 2A is a diagrammic representation of interprocessor communications transmitted through the bus structure of the present invention;

Figs. 3A and 3B are schematic representations of the system bus interfaces incorporated into the elements of the present system; and, Fig. 4 are flow charts illustrating the operation of the present invention.

DESCRIPTION OF A PREFERRED EMBODIMENT

The following description will first present the overall structure of a system incorporating a presently preferred embodiment of the present invention, followed by a description of the bus structure of the present invention. The operation of the bus structure will then be described with the aid of flow charts, and the interface logic by which the elements of the system are interconnected to the system bus will be described.
. ~

~4!'7~g It should be noted that reference numbers appearing ln the drawings and in the following descriptions are comprised of three digits. The two least significant (rightmost) digits identify a~ particular element appearing in a particular drawing and the most significant (leftmost) dig~t identifies the figure in which that element first appears. For e~ample, element 124 is the 24th element appearing in Fig. 1 and first appears in Fig. 1. A reference numbers is assigned the first time the reference element appears in the descriptions and is used to refer to that element throughout the following descriptions and drawings.

A. Elements of System lOZ and General Operation (Fig.l) Referring to Fig. 1, therein is presented a block diagram of an exemplary System 102 incorporating the inter-processor bus structure of the present invention. As shown therein, the two primary elements of the system bus structure that are visible at this level are System Bus 104 and System Bus Priority ~SBP) Bus 106. System 3us 104, the detailed structure of which is described further below, is the means by which the elements of System 102 communicate with one another. SBP Bus 106, also described in detail below, is the link through which the elements connected from System Bus 104 determine access to System Bus 104.

As shown in Fig. 1, the elements comprising System 102 fall into two classes, those which are connected directly to System Bus 104 and SBP Bus 106 and those which are connected indirectly to System Bus 104, that is, through another element which in turn is connected ., ~Z47;2~9 directly to System Bus 104 and S~P Bus 1~. As described furtherbelow, elements directly connect~d t~ Sy~tem Bus 104 and SBP Bus 106 are fundamentally regarded as a peers with regard to access to System Bus 104, that is, each such element has equal priority of access to System Bus 104 with respect to all other such elements.
The access priorities of indirectly connected elements are determined by the access priorities of the associated elements through which they are connected to System Bus 104.

Considering first the elements connected directly to System Bus 104, each such element includes sufficient internal intelligence, for example, in the form of microcode control, to perform at least specialized functions independently of the other elements of System 102. Examples of such elements, as illustrated in Fig. 1, include Memory Units (MEMs) (1 to n+2) 108, Central Processing Units (CPUs) (1 to n~l) 110, Local System Controllers (LSC) 112, Remote System Controllers (RSC) 114, and System Bus Interfaces (SBIs~ (1 to n+l) 116.

The design of and functions performed by elements such as MEMs 108 and CPUs 110 are well known in the art and require no further description. LSC 112 and RSC 114 may, for example, be small computers of the personal or professional class adapted to perform certain system control functions, such as providing a user control interface, that is, a "soft control panel". In this respect, RSC 114 may differ from LSC 112 in being connected to a remote user/controller, for example, for diagnostic purposes, through a Telecommunication Link (TC).
"

2~9 g SBIs 116 are the elements, described above, through which indlrectly connected elements such as Satellite Processing Units (SPUs) 118 are provided with access to System Bus 104. As illustrated in Fig. 1, SPUs 118 are interconnected with each other and to an associated SBI
116 through an SPU Bus 120. The link between the associated SPU Bus 120, and thus the SPUs 118 connected therefrom, and System Bus 104 is in turn provided through the associated SBI 116. It should be noted that SPUs 118 and SPU Busses 120 may use the same structure and operation as described below with reference to System Bus 104 and SBP Bus 106, that is, may be arranged as local system buses identical to the main system bus. SPUs 118 and SPU Busses 120 may alternately be of any other suitable design for the system operation.

SPUs 118 essentially include all devices or system elements which, for example, due to data rates or functions, do not require direct access to System Bus 104 to perform thelr functions. Examples of SPUs 118 are input/output devices such as disc drives, displays, printers, telecommunications links, tape streamers and user terminals. SPUs 118 may further includes independent or associated processing units, such as other general purpose computers or specialized processing devices, such as scanners and specialized arithmetic or signal processors.

B. System Bus 104, General Structure and Operation (Figs. 1, lA and lB) Returning to System Bus 104, as described above System Bus 104 is the means through which the elements of System 102 communicate. In ~'72~9 the present em~odiment of System 102, and as shown in Fig. 1, System Bus 104 is a linear bus with each of the peer elements of System 102 connected therefrom, the connections to System Bus 104 being unidirectional or bidirectional as required by the function of the element. System Bus 104 may be extended as required by the particular configuration of a System 102, that is, to add or subtract system elements or to connect two or more System 102's into a single system.

It should be noted that, as described below, the logical configuration of System Bus 104 is defined by SBP Bus 106 and may assume any topological structure required by the function of System 102. For example, System Bus 104 may be physically arranged in the loop and star configurations illustrated respectively in Figs. lA
and lB. In the loop configuration, the ends of System Bus 104 are tied together to form a closed loop from wh;ch System Elements ~SEs) 122 are connected. In the star configuration, System Bus 104 is comprised of a number of bus segments radiating from a common junction and SEs 122 are connected from the radiating segments as required by the system configuration.

C. System Bus Priority Bus 106 (Fiq. 1) Referring again to Fig. 1, as described in detail further below SBP
Bus 106 is the means through which the System 102 elements connected to System Bus 104 determine access to System Bus 104. As shown in Fig. 1, SBP Bus 106 forms a loop with all of the elements connected from System Bus 104 being serially connected in the SBP Bus 106 loop.

~7~

--1 1 ,~.

It is assumed, 7n the exemplary System 102 presented hereln7 that all processing elements connected from System Bus 104 may have the capability to independently initiate interprocessor communications;
thus all elements connected from System Bus 104 are sho~n as connected in the SBP Bus 106 loop. In certain cases, for example, memory elements, the processing elements may be such that they do not initiate interprocessor communications but will only receive and respond to such communications. Such elements will require access to System Bus 104 to receive such communications and to respond to such communications, for example, by reading data from a memory element to a CPU element, but ~ill not be required to claim access to System Bus 104, that is, access to System Bus 104 will be provided by the element sending the communication being responded to. In such cases, these "response only" elements need not be connected in the SBP Bus 106 loop but will be connected to System Bus 104.

As described below, priority of access to System Bus 104 is passed from one element of System 102 to the next element in the SBP Bus 106 loop in a `'rotating daisy chain". That is, if a given element currently has access to System Bus 104, the next element along the SBP Bus 106 loop follo~ing the current element has the highest priority for next access to System Bus 104, followed by the next element along SBP Bus 106, and so on around the SBP Bus 106 loop until the current element is reached again. When the element currently having access releases System Bus 104, the opportunity to gain next access is passed through S8P Bus 106 to the next element along SBP Bus 106. That next element may take access to System Bus 104 or, if it does not do so, passes the opportunity for access to ,: , 7~49 its next element along SBP Bus 106, and so on until the element originally having access is reached again or some element along SBP
8us 106 takes access to System Bus 104. h The order of priority of access to System Bus 104 thereby rotates around SBP Bus 106 wlth each element in turn having an opportunity to gain access to System Bus 104. Thus the average priorities of access to System Bus 104 of all elements connected thereto will be equal, with the relative priorities of the elements at particular points in time being determined by their positlons along SBP Bus 106 relative to the element currently having either actual access to or the right to access System Bus 104.

Because of the rotating shifting of access priority to System Bus 104 among the elements of System 102 connected from SBP Bus 106, the elements connected to System Bus 104 do not contend fo~ access to System Bus 104. As a result, the access determination loglc, described further below, is simpli~ied and, by eliminating System Bus 104 overhead which would otherwise be used in resolving access priorities, the speed of communication between the elements connected to System Bus 104 is increased. In addition~ and because each element connected to System Bus 106 and SBP Bus 106 has an equal opportunity to gain access to System Bus 104, no element can be locked out of access to System Bus 104 for an extended period.

Moreover, and again because of the rotating shifting of access priority to System Bus 104 among the elements of System 102, the position of a System 102 element along either SBP Bus 106 or System ~ 7 2 Bus 104 has no bearing on the average priority of that element to access System Bus 104. That is, and as described above, all elements connected to System Bus 104 and in the SBP Bus 106 loop are peers having, on the average, equal access rights to System Bus 104. As such, an element may be added to System 102, or moved from one point along System Bus 104 and SBP Bus 106 to another, without effecting the average relative priorities of access to System Bus 104 of that element or any of the other elements connected to System Bus 104.

In this regard, SBP Bus 106 is represented in Fig. 1 as comprising a simple, clockwise loop with each element of System 102 being connected in series around the loop. It should be noted, however, that this representation is selected only for clarity of presentation. The elements of System 102 connected from System Bus 104 may, in fact, be connected in series along SBP Bus 106 in any desired order.

The order of elements along SBP Bus 106 may be effected, for example7 by the above described temporary priorities of access having effect whenever a certain element has access to System Bus 104. That is, whenever a given element has access to System Bus 104, the next element along SBP Bus 106 has highest priority of next access, and so on around the SBP Bus 106 loop. If, for example, it were known that a particular operation involving access to System Bus 104 by a first element were frequently followed by a related operation again involving access to System Bus 104 by a second element, the second element may be connected in SBP Bus 106 next after the first element. Thus, whenever the first element executed , 7Z~9 its operation, it would be known that the second element would have the highest priority of next accessing System Bus 104.

To illustrate with reference to Fig. 1, assume that SBIl 116 is primarily engaged in input/output operations of information be~ng operated upon by CPUl 110, for example, text processing. In such a case, and in order to enhance speed of response to user operations, it m~y be desirable to allow CPUl 110 next access to System Bus 104 each time SBIl 116 transfers information from a user, that is, a SPU
118, to one of MEMl-n 108. In this case, then, CPUl 110 could be connected in the SBP Bus 106 loop next after SBIl 116.

Finally, a second element of SBP Bus 106 is illustrated in Fig. 1 and referred to as Local Priority Link ~LPL) 124. LPL 124 is essentially a means by which the relative priorities of elements interconnected through LPL 124 may be fixed, as opposed to the rotating priorities determined by SBP Bus 106. As will be described in detail in a following description of the SBP Bus 106 element residing in each element connected therefrom, LPL 124 allo~s the fact of a pending requirement for access to System Bus 104 by one element to be passed to another element connected along a LPL 124 to inhibit any pending accesses to System Bus 104 in the second element.

D. System 102 Bus Structure (Fiq. 2) Referring to Fig. 2, therein is presented a diagrammic representation of System 102's bus structure. As described above and .iZ~'7;~L9 shown in Fig. 1, this structure includes System Bus 104, SBP Bus 106 and, in certain cases, an associated LPL 124.

D.l Memory Control Bus 202: Memory Operatlons and Interprocessor Communications As shown in Fig. 2, System Bus 104 includes a plurality of multiple and single line sub-busses. The first of these sub-busses is Memory Control (MC) Bus 202 which, upon the occurrence of a System 102 element obtaining access of System Bus 104, is used to communicate the type of System Bus 104 operation to be performed.

That is, when an element takes control of System Bus 104 that element signals this access by driving SBP Bus 106 to a state indicative of this fact and places on MC Bus 202 a code indicating the type of System Bus 104 operation to be performed. The elements of System 102 connected to System Bus lD~ detect the occurrence of a System Bus 104 access by monitoring the state of SBP Bus 106 and, when an access is indicated, determine the type of System Bus 104 operation to be performed by reading the code placed on MC Bus 202 by the element having access to System Bus 104.

Most System Bus 104 operations are memory related, that is, are reads from or writes to MEMs 108. As such, and as will be seen below with reference LO the MC Bus 202 codes, the entire class of non-memory related operations are indicated by a single code indicating that an "interprocesser" communication is to be executed, that is, a communication between two non-memory elements, such as an 72~9 SBI 116 band and a CPU 110. As described below, the elements connected to System Bus 104 must in such cases refer to other of the System Bus 104 sub-busses to determine and execute interprocessor communications.

The MC 8us 202 codes provided in the present lmplementation of System 102 include:
CODE TYPE OF OPERATION
O No operation;

3 Read the contents of an MM 108 control register;

4 Read a quad word (16 bytes) of information from a specified MM 108 address location;
Read an octal word ~32 bytes) of information from a specified MM 108 address location;
6 Read a double word t8 bytes~ of information from a specified MM 108 address location;
7 Read a word (4 bytes) o~ information from a specified MM 108 address location;
8 Perform an inter-processor communication;
B Write to an MM 108 control register;
C Write a byte into a specified MM 108 address location;
D ~rite a half word (2 bytes) into a specified MM 108 address location;
E ~rite a double word into a specified MM 108 address location; and, F Write a word into a specified MM 108 address location.

~7Z~9 -17 .~

It should be nated that the above codes are presented in hexidecimal form and that codes 1, 2, 9 and A are reserved for future use.
.
Interprocessor communications are thereby executed as a default case from memory related operations That is, a short "decision branch", reference to a code on MC Bus 202, is provided to identify and initiate memory related operations while a longer "decision branch", reference to further information on other sub-busses of System Bus 104 is required for non-memory related operations. This method thereby effectively increases the speed with which the majority of System Bus 104 operations, that is, memory related operations, may be initiated and executed by providing a shorter decision path for such operations while retaining flexibility in defining and executing all types of System Bus 104 operations.

D.Z System Address (SA) Bus 204 and System Data (SD) Bus 206 The next major sub-busses of System Bus 104 are System Address (SA) Bus 204 and System Data (SD) Bus 206. Considering first memory related operations, SA Bus 204 is the means by ~hich read and write addresses are communicated between elements requesting memory operations and the MEMs 108 executing the operations while SD Bus 206 is the means by which information is communicated between the MEMs 108 and the other elements of System 102.

7;~

D.2.a Memory Operations In a memory operation, as described above the System 102 requesting ~!
a memory operation first gains access to System Bus 104 through the operation of SBP Bus 106, described in further detail below, and places an appropriate MC Bus 202 code on MC Bus 202 to indicate the type of operation to be performed. The requesting element then places the read or write address onto SA Bus 204 and, if the operation is a write, places the data to be written onto SD Bus 206.
The addressed MEM 108 then writes the data into the corresponding storage location therein. If the operation is a read, the addressed MEM 108 reads the information from the addressed storage location and places the information on SD Bus 206, from which the information is read by the requesting element. In the present implementation of System 102, for example, SA Bus is 24 bits wide, expandable to 31 bits, while SD Bus 206 is 6~ bits, or a double word> wide.

Associated with SA Bus 204 and SD Bus 206 are three further single line sub-busses whose primary functions relate to memory operations.
The first of these is WAIT 208. This signal is asserted by an addressed MEM 108 during a memory read operation if the requested information is not available and is monitored by the requesting element, which may accordingly go into a wait mode until the information becomes available.

The second memory operation control is BUSY 210, which is asserted by an addressed MEM 108 during a memory operation and before a System Bus 104 transmission is initiated. BUS 210 indicates that .

'72~ :

-19~

System Bus 104 is not available and ls monitored by the elements of System 102.
, The third memory operatlon control is Valid Memory Access (VMA) 212, which is asserted by an addressed MEM 108 to indicate that a requested memory operation is valid, that is, that the address or data are valid. VMA is monitored by the element requesting the memory operation to determine whether the request was successful, that is, valid.

D.2.b Interprocessor Communications (Fig. 2A~

Now considering non-memory related operations, that is, interprocessor communications, SA Bus 206 and SD Bus 206 operate differently in certain respects from that described above when an interprocessor operation is to be performed. As described above interprocessor operations are treated as a default from memory related operations. That is, a single MC Bus 202 code indicates the entire class of non-memory type operations. As also described above, upon the appearance of the interprocessor communication code on MC
8us 202 the elements connected to System Bus 104 must refer to information presented on SA Bus 204 and SD Bus 206 by the requesting element to determine the type of interprocessor operation to be executed.

Referring to Fig. 2A, therein is represented the information which may be presented upon SA Bus 204 and SD Bus 206 in an interprocessor operation. As shown therein, the information appearing on SA Bus 204 ~2~7~

includes a 4 bit Target Address (TA) Field 214 identifying the target, or intended recipient of the message, a 4 blt Message Type ~MT) Field 216 identifying the type of message to be sen~ to the target, and a 16 bit Message (ME) Field 218 which may contain a message. In certain interprocessor communication operations, wherein data is to be transmitted from one element to another, SD Bus 206 may contain a data field of up to 8 bytes.

D.2.b.1 AT Field 214 Codes ~!
Considering now the various interprocessor communication fields ,r, appearing on SA Bus 204, the TA Field 214 may, for example, contain the following target identification codes:
CODE TARGET IDENTIFIED
O Support Control Unit (E.g., LSC 112 or RSC 114);
1 Broadcast to all CPUs 110;
2 CPUl 110;
3 CPU2 110;
4 CPU3 110;
CPU4 110;
6 CPU5 110;
7 CPU6 110;
8 CPU7 110;
9 CPU8 110;
A Reserved for future use;
B SBIl 116;
C SBI2 116;

i2~7Z4~

D SBI3 116;
E SBI4 116; and, F Broadcast to all SBIs 116.
It should be noted that the above codes are presented in hexidecimal format.

It is apparent from the above code formats that the exemplary system envisioned in the above code assignments includes a single Support Control Unit 112 or 114, up to 8 CPUs 110 and up to 4 SBIs 116. The assignment of target codes may be altered at will, depending upon the envisioned configuration of the particular System 102.

It should be noted that SPUs 118 are targeted and messages transmitted thereto through the SPU 118's associated SBIs 116. It should also be noted that the interprocessor communications allow the simultaneous broadcast of messages to all elements of a given type, for exa~ple, to all CPUs 110 or to all SBIs 116.

There are no target identification codes for memory elements, that is, for MEMs 108, provided in the exemplary TA Field 214 codes. As described previously, all memory related operations are ~nitiated at the MC Bus 202 code level and the target MEMs 108 identified by addresses concurrently appearing on SA Bus 204.

D.2.b.2 MT Field 216 Codes The contents of the MT Fields 216 depend upon the particular type of recipient identified in the associated TA Field 214, that is, in the 7~

present example, whether the targeted recipient is an SBI 116, that is, an SPU 118 connected from an SBI 116, a CPU 110 or a Support Control Unit 112 or 114. As will appear in the exemplary MT Field 216 codes presented below, an MT Field 216 code may identify a message as being the transfer of a message, the transfer of data, or a command for an operation or change of operating state on the part of the recipient element.

Considering first examples of the types of MT codes which may be transmitted to an SBI 116 type of element:
CODE MESSAGE TYPE
O Message transfer to target SPU 118;
Data transfer to target SPU 118;
8 Reset target SBI 116;
9 Reset target SPU 118;
A Turn InputlOutput I/O)) protection of~;
B Turn I/O protection on;
C Enable I/O access to speciFied memory page; and, D Disable I/O access to specified memory page.
Again, the MT Field 216 codes above are presented in hexidecimal h format and codes 2, 3, 4, 5, 6, 7, E and F are reserved for future use.

Considering now examples of the MT Field 216 codes which may be used when the targeted recipient is a CPU 110:

Z~9 CODE MESSAGE TYPE
O Class 1 ~lO Interrupt;
1 Class 2 IIO Interrupt;
8 Interprocessor communication; and, 9 Synchronize clock.
Again, the codes are presented in hexidecimal format and codes 2 to 7 and A to F have been reserved for future use.

It should be noted that the above CPU 110 message types provide for two classes of I/O interrupt, Class 1 for when no error has appeared in the I/O operation and Class 2 for when an error has occurred in the I/O operation, for example, in the data. The two classes are provided because of the different handling of these events by the targeted CPU 110.

In the Interprocessor communication type of message, MT Code 8, only SA Bus 204 is used for the communication and the recipient CPU 110 refers to the accompanying ME Field 218 appearing on SA Bus 204 to determine the message, examples of which are presented below. The occurrence of such a communication causes an interrupt pending flag to be set in an interprocessor communication register internal to the recipient CPU 110, described below. The recipient CPU 110 micromachine will then read the interrupt pending flag, execute a macrointerrupt at the next opportunity, and execute a routine to appropriately handle the received message. If the interprocessor communication interrupt flag in the recipient CPU 110 is already active~ the recipient CPU 110 will send to the transmitting CPU 110 acknowledgement and busy signals, as described below. It should be ~ 7 Z ~3 noted that in communica-tions from SBIs 116 to CPU's 110, the SBI 116 receiving a busy response will handle the busy response in a manner appropriate to the SBI 116's bus protocol, that is, its protocol d with respect to its SPUs 118.

Again, the particular message type codes appearing in MT Field 216 and their meanings may be determined at will, depending upon the configuration and function of the System 102 envisioned and the elements appearing therein.

D . 2 . b.3 ME Field 218 Messages The 2 byte interprocessor communication messages appearing ln ME
Field 218 are, again, dependent upon the function and configuration of System 102 and the elements comprising System 102. Examples of such messages include, in the present implementation, communications between the system elements and the System 102's operating system being executed in the CPUs 110. Such operating system communications may include communications between an IlO device, that is, an SBI
116 or SPU 118, and the operating system and communications between the operating system and a Support Control Unit 112 or 114. For example, the operating system may send a message to a Support Control Unit to read or update an error file or requesting the Support Control Unit to examine some aspect of System 102's operation and report the findings of the examination to the operating system, for example, in diagnostic operations.

Other examples include communications between the CPUs 110 of a multiple-processor conriguration of System 102. For example, a particular CPU 110 may wish exclusive access to a given page of memory and may inquire of the other CPUs 110 whether they are using that memory page. The requesting CPU 110 may then, if that page is not being used by another CPU 110, inform all CPUs 110, by a broadcast communication, that it is claiming exclusive access to that memory page. In a further example, a CPU 110 may wish to open and modiFy a particular file and will inform all other CPUs 110 that it is obtaining exclusive access to that file for that purpose. Yet other examples are messages coordinating the activities of the CPUs 110; for example, a first CPU 110 may assign a task to a second CPU
110 by an interprocessor message and the second CPU 110 may send a communication informing the first CPU 110 when the task is completed.

Still other examples of interprocessor messages occupying the ME
Field 218 are I/O messages, essentially commands from the CPUs 110 to the SBIs 116 or SPUs 118 to initiate or control the operations of these elements.

Finally, and referring again to Fig. 2, as described above with reference to memory related operations certain single line sub-busses of System Bus 104 are associated with interprocessor communication operations. Among these are Acknowledge (ACK) 220 and Target Busy (TB) 222. ACK 220 is asserted by tne target element of an interprocessor communication when that target exists and acknowledges that the sending element is attempting to send an interprocessor communication to that target element. The sending ~LZ~7~9 element monitors ACK 220 to determine whether the attempt to send an interprocessor communication was successful.
.
TB 222 is asserted by the target element of an interprocessor communication to indicate that the target element is busy and cannot accept the interprocessor communication. The sendlng element monitors TB 222 and, if the TB 222 is asserted by the target element, will handle the condition depending upon the nature and function of the sending element.

Also associated with both interprocessor communications and memory related operations is LOCK 224. LOCK 224 may be asserted by the initiator of a memory related operation or interprocessor communication to lock out all other users of System Bus 104. LOCK
224 may be asserted, for example, when an element wishes to communicate a series of interprocessor communications or a series of memory operat~ons. LOCK 224 is monitored by all elements connected to System Bus 104 and no user will attempt to obtain access to System 8us 104 while another element is asserting LOCK 224.
I

Finally, as indicated in Fig. 2, System Bus 104 may include a System Clock (SYSCLK) 226, which is provided to all users of System Bus 104, thereby achieving common timing for all such elements.

Having described the operation of the bus structure of System 102, the bus interface logic residing in each of the elements connected to System Bus 104, and their operation, will be described next below.

~'7Z~

E. System Bus Interfaces (Fi~s. 3A and 3B) Referring to Figs. 3A and 3B, therein are presented diagrammic i~
representations of the interface circuitry provided in each element connected to System Bus 104 to interface these elements to System Bus 104. It should be noted that the logic and circuitry presented herein are illustrative and representative only and may be replaced by any logic or circuitry performing equivalent functions. In addition, where the design and operation of the logic and circuitry presented herein will be well understood by one of ordinary skill in the art, that logic or circuitry will not be described in detail.
The operation of the element interfaces will, however, be described in detail where relevant to the present invention.

E.l SBP Bus 106 Interface (Fig. 3A~

Referring first to Fig. 3A, therein is presented the interface circuitry primarily concerned with requesting and gaining access to System Bus 104. As previously described, each element connected to System Bus 104 is connected in series along the SBP Bus 106 loop.
Other sub-busses of System Bus 104 concerned in obtaining access to System Bus 104 are, as indicated in Fig. 3A, Busy 210 and LOCK 224 and, in certain cases, LPL 124.

As previously described, when an element currently having access to System Bus 104 releases System Bus 104, the opportunity to gain next access to passed through S8P Bus 106 to the next element along SBP
Bus 106. That next element may take access to System Bus 104 or, if ~7Z9c~

it does not do so, passes the opportunity for access to its next element along SBP Bus 106, and so on untll the element originally having access is reached again or some elemen-t along SBP Bus 106 P
takes access to System Bus 104.

As shown in F7g. 3A, the input of SBP Bus 106 from the previous element along the SBP Bus 106 loop is designated as SBP Input (SBPI~
and is connected to a first input of Request Gate (RG) 302. It should be noted that SBPI is shown as an active low signal, as are all other overlined signals. The output of RG 302 is connected to the continuation of the SBP Bus 106 loop to the next element along the SBP Bus 106 loop and is designated as SBP Output (SBP0).

A second input of RG 302 is connected from the Request (REQ) output of Request Enable Gate (REG) 304. REQ will be generatedt as described below, when the associated element wishes to request access to System Bus 104 and other conditions, determined by the various inputs to REG 304, also described below, do not prevent the element from requesting access. REQ operates as an enabling signal with respect to RG 302 and SBPI, that is, SBPI will be passed through RG 320 as SBP0 and thus the SBPI of the next element along the SBP Bus 106 loop if REQ is not asserted. If, of course, REQ is assertedt SBPI is prevented from passing through RG 302 to become SBP0 and the next element along the SBP BUs 106 loop will not receive an SBPIt the present element having claimed access to System Bus 104. RG 302 thereby operates as a gate to prevent the propagation of the opportunity to obtain access to System Bus 104, that is, the element may "capture" the opportunityt and as a signal '7~4g level restorer for the SBPI/SBPO signal propagated along the SBP Bus 106 loop.
~i It should be noted that due to the structure of the SBP Bus 106 interface logic, the SBP Bus 106 access signal will propagate around the SBP Bus 106 loop at a speed determined by the propagation time around the physical loop and the delays through the RGs 302 of the elements connected therefrom. As such, delays in gaining System Bus 104 access due to the operation of SBP Bus 106 and the element interfaces thereto are minimized and the overall speed of operation of System 102's bus structure enhanced.
.

Referring now to REG 304, as shown in Fig. 3A REG 304 receives a Bus Request (BREQ) signal from the element's internal control circuitry, described further below, when the element ~ishes access to System Bus 104. BREQ is gated, in REG 304 by inputs BUSY and LOCK, previously described, from respectively BUSY 210 and LOCK 224. In certain cases, also previously described, REG 304 is provided with a Local Priority signal from a previous element through an LPL 124.
The enabling of RE~, and thus the capture of SBPI/SBPO thereby requires that BUSY, LOCK and a Local Priority signal7 if any, not be asserted.

E.2 System Bus 104 Interface (Fig. 3B) Referring to Fig. 3B, therein is presented a diagrammic representation of the general interface to System Bus 104 of an element connected to System Bus 104. As shown therein, the interface ., ~ 2 ~'7 ~ ~ 9 -30- ..

logic includes an Interprocessor Communication Control ~IPCC) 306 having inputs connected from the sub-busses of System Bus 104 as indicated. Included among these inputs are the 4 bi.ts of MC 8us 202, ii WAIT, BUSY, VMA, ACK AND TB, the functions of which have been described above. Also included in IPCC 306's inputs are the TA Field 214 from SA Bus 204.

These inputs essentially define the occurrence, type and state of execution of interprocessor communications and IPCC 306 is essentially comprised of decoding logic for decoding these inputs and providing corresponding outputs to the internal control circuitry of the element, for example, microcode control circuitry.
The design of such internal control circuitry and of such decoding logic as IPCC 306 is well understood by those of ordinary skill in the art, especially after the previous and following descriptions of the operation of System 102's bus structure.

Among the outputs provided by IPCC 306 are an Interprocessor Interrupt (IPCI) to the elements interrupt handler, as previously described, and ACK and TB signals to ACK Bus 220 and TB Bus 222.

IPCC 306 also provides an output indicating the occurrence of an interprocessor communication (IPC) to Interprocessor Communication Register (IPCR) 308 which, as shown in Fig. 3B, is a register connected from the MT Field 216 and ME Field 218 portions of SA Bus 204. IPCR 308 thereby captures and stores the MT and ME fields of an interprocessor communication appearing on SA Bus 204, and provides , Z~3 -31- ~

these fields as inputs to the elements Internal Control Logic (Internal Control) 310.

As is well known in the art, Internal Control 3iO may be comprised, for example, of dedicated control logic or a microcode programmed microprocessor controller. Internal Control 310 operates in a first respect to control the operation of the element in response to a received interprocessor communication and, in a second respect> to initiate and control interprocessor communications from the present element.

In this second respect, Internal Controller 310 may generate interprocessor communication control signals to Interprocessor Communication Output Controller (IPC0) 312. IPC0 312 may in turn be comprised of registers and decoding logic to generate MT and ME P
fields to the SA Bus 204 ~Jhen the element is initiating an interprocessor communication, and may generate the BREQ signal, described above, when the element wishes to request access to System Bus 104 for an interprocessor communication.
p Finally, the interface circuitry may include Data Registers and Drivers ~DR) 314 connected with SD Bus 206 to communicate data between the element and SD BUs 206. Such data communication may occur, as previously described, in a memory related operation or when data is transferred in an interprocessor communication.

The interface between an element and System Bus 104 may differ from that described above when that element is a memory element such as ~7~

an MEM 108. In such a case, the circuitry lllustrated in Fig. 3A may be modified accordingly, that is, may contain only those functions necessary for memory related operations.

For example, an MEM 108 IPCC 306 may be provided only with inputs from MC Bus 202, which are sufficient and complete to define all memory related operations. The MEM 108's IPCC 306 may, accordingly, provide outputs ~AIT, BUSY and VMA to the appropriate System Bus 104 sub-busses, rather than those shown in Fig. 3A. Similarly, the IPCR
308 of an MEM 108 will be comprised of an address input register connected from SA Bus 204 for receiving memory read and write addresses, and the MM 108 will include a bidirectional data connection to SD Bus 206.
,, Having described the bus and interface structures of System 102's bus structure, and the functions and operation of the signals involved therein, the operation of System 102's bus structure will be described further below with reference to flow charts illustrating these operations.

F. Flow Chart Illustration of Operation (Figs. 4A and 4B) Referring to Figs. 4A and 4B, therein are presented flow chart illustrations of the operations of System 102's bus structure. Fig.
4A is an illustration of a system bus operation from the viewpoint of the bus requestor, while Fig. 4B is an illustration from the viewpoint of the target element.

~'7Z~9 F.l Bus Requestor Operat~on (F~g. 4A) Referring to Fig. 4A, an interprocessor bus operation, either a memory operation or an interprocessor communication, is begun with the requesting element generating, through its internal control logic, an Access Request. In the first step, the requestor determines whether System Bus 104 is available; that is, and as previously described, the requestor determines whether BUSY, LOCK
and LPL 124, if any, are asserted and whether an SBPI is available.

In the second step, and if the conditions for bus availability are met, the requestor seizes System Bus 104 by capturing the SBP Bus 106 signal and places an appropriate code on MC Bus 202 to indicate the type of operation to be performed. If System Bus 104 is not available, the requestor takes what actlon is necessary to wait until the bus is available and retries access.

If access to System Bus 104 was gained, and the operation is a memory related operation, the requestor places the memory operation address on SA Bus 204 and the information to be written, if any, on SD Bus 206 and monitors ~AIT Bus 208.

If the responding memory element is busy, the memory will assert WAIT on ~AIT Bus 208 and the requesting element must enter a wait mode until the responding memory element is available.

If the responding memory element is available, and the memory request is valid, the memory element will assert VMA on VMA Bus 212 and the operation will be completed. If the memory request was not ~2~-~7Z~'~

-3~ .

valid, VMA will not be asserted and the requestor must refer to its error handler facility to resolve the problem.
, If the requested operation was an interprocessor communication, the requesting element will place the appropriate flelds, as previously described, on SA Bus 204 and SD Bus 206 and will monitor ACK Bus 220 and TB Kus 222.

If the target element asserts ACK and does not assert TB, the operation will be completed.

If the target element asserts ACK and TB, the requestor must wait and retry the communication as the target element i5 busy.

If the target element does not assert and does assert TB then the target element is again identified as busy and the requestor must retry the communication. If the target element does not assert ACK
or TB, the the requestor must refer to an error handler to resolve the problem.

F.2 Tarqet Element Operation (Fig. 4B) Referring now to Fig. 4B, therein is presented a general flow chart of the operation of a target element for both memory related operations and interprocessor communication operations.

At the first step, the target element monitors SBP Bus 106 and identifies the initiation of a System Bus 104 operation by the state .

~L~ ~'7 Z ~9 of SBP Bus 106. At this time, the element reads the code on MC Bus 202 to determine the type of System Bus 104 operation to be executed.

Considering first a memory operation, and assuming that the target element is a memory element, the element then reads SA Bus 204 to determine the target address in System 102's address space and SD
Bus 206 to capture the data to be written if the operation is a memory write.

If the target element is busy, it will respond by asserting BUSY on BUSY Bus 210. If the target element is not busy and the memory request is valid, the target element will assert VMA on VMA Bus 212 and will perform the requested operation.

Considering finally an interprocessor communication, the target element determines, by reading the MC Bus 202 code, that an 1-interprocessor communication is to be performed and then reads the r TA Field 214 from SA Bus 204 to determine whether that element is the target element.

If that element is the target element, it asserts ACK and, if not busy, does not assert BUSY.

Assuming that the element is not busy and is the target element, the element reads the MT Field 216 from SA Bus 204 to determine the type of communication and the ME Field 218 from SA Bus 204 to capture the message. The element then responds to these fields by executing what . .

7,~Z~g operatlons are required by the message, including reading data from the SD Bus 206 if the communication requires the transfer of data.

The invention described above may be embodied in yet other specific forms without departing from the spirit or essential characteristics thereof. Thus, the present embodiments are to be considered in all respects as illustrat~ve and not restrictive, the scope of the invention being indicated by the appended claims rather than by the y foregoing descriptions, and all changes which come within the meaning and range of equivalency of the claims are therefore intended to be embraced therein.

Claims

THE EMBODIMENTS OF THE INVENTION IN WHICH AN EXCLUSIVE
PROPERTY OR PRIVILEGE IS CLAIMED ARE DEFINED AS FOLLOWS:

1) A system bus for coupling components of a data processing system which includes a memory component and other components and carrying a communication between at least two of the components, the system bus comprising:
a first plurality of lines for carrying first codes including a plurality of memory operation codes specifying a plurality of types of memory communications having an other component as a source and the memory component as the recipient and a single inter-processor communication code specifying a non-memory communication having one of the other components as a source and another thereof as a recipient;
a second plurality of lines which, when the first plurality of lines specifies one of the memory communication types, carries an address in the memory component and, when the first plurality of lines specifies a non-memory communication, carry one of a plurality of type codes specifying a plurality of types of non-memory communications and one of a plurality of recipient address codes specifying an other component as recipient;
a third plurality of lines for carrying signals specifying the status of a current memory communication; and a fourth plurality of lines for carrying signals specifying the status of current non-memory communication.

2) A system bus as set forth in claim 1 wherein:
the system bus includes a fifth plurality of lines for carrying data;
the plurality of types of memory communications includes write communications wherein the source component provides data on the fifth plurality of lines to the memory component to be written at the address specified by the second plurality of lines and read communications wherein the source component receives data stored at the address specified by the second plurality of lines from the memory component on the fifth plurality of lines; and the plurality of types of non-memory communications includes types of communications wherein the source component provides data to the recipient component on the fifth plurality of lines, but does not include communications wherein the recipient component provides data to the source component.

3) A system bus as set forth in claim 1 and wherein:
the third plurality of lines includes a wait line for carrying a wait signal indicating that the source component is to wait for completion of the memory operation specified in the communication;
a busy line for carrying a busy signal indicating that the memory component is busy; and a valid memory access line for carrying a valid memory access signal indicating that the memory operation resulted in a valid memory access.

4) A system bus as set forth in claim 1 and wherein:
the fourth plurality of lines includes an acknowledgment line for carrying an acknowledgment signal indicating that the recipient component received the communication and a target busy line for carrying a target busy signal indicating that the recipient component is busy.

5) A system bus as set forth in claim 1 wherein:
the codes specifying the non-memory communication further include a fifth code specifying a message in certain of the communications.

6) A system bus as set forth in claim 1 wherein:
the system bus includes a fifth plurality of lines for carrying data: and certain of the non-memory communications include data transferred from the source to the recipient on the fifth plurality of lines.

7) A system bus as set forth in claim 1 and further including:
a system bus priority line for carrying an access signal indicating which component of those which presently can be sources of communications may have access to the system bus; and a lock line for carrying a lock signal indicating the one source component is retaining access to the system bus.

8) System bus interface apparatus for coupling a component other than a memory component of a data processing system which includes both memory an other components to a system bus which has a first plurality of lines for carrying first codes including a plurality of memory operation codes specifying a plurality of types of memory communications having an other component as a source and the memory component as the recipient and an inter-processor communication code specifying a non-memory communication having one of the other components as a source and another thereof as a recipient, a second plurality of lines which, when the first plurality of lines specifies a memory communication, carry an address in the memory component and, when the first plurality of lines specifies a non-memory communication, carry one of a plurality of type codes specifying a plurality of types of non-memory communications and one of a plurality of recipient address codes specifying an other component as recipient, and a third plurality of lines for carrying signals specifying the status of any non-memory communication, the system bus interface apparatus comprising:
control means coupled to the first plurality of lines, the second plurality of lines, and the third plurality of lines for responding to the interprocessor communication code on the first plurality of lines and to a recipient component address on the second plurality of lines specifying the component containing the system bus interface by producing a first signal indicating the receipt of the communication and second signals on the third lines specifying the status of the communication;

means coupled to the second plurality of lines and responsive to the first signal for retaining the type code in response to the first signal; and processing means coupled to the type code retention means for receiving and interpreting the retained type code.

9) The system bus interface apparatus as set forth in claim 8 wherein:
the second plurality of lines further carries a message code specifying a message in certain of the non-memory communications;
the type code retaining means further retains the message code:
and the type code interpreting means interprets the message code as well as the type code in the certain non-memory communications.

10) The system bus interface apparatus as set forth in claim 8 wherein:
the third lines include an acknowledgment line for carrying an acknowledgment signal indicating that the component to which the bus interface apparatus belongs has received the communication ad a recipient bus line for carrying a recipient busy signal indicating that the component is presently busy: and the control means further responds to the interprocessor communication code on the first plurality of lines and to a recipient component address on the second plurality of lines specifying the component to which the system bus interface apparatus belongs by producing the busy signal on the busy line when the component to which the system bus interface apparatus belongs is busy.