CA2257511A1 - Digital data processing methods and apparatus for fault isolation - Google Patents

Digital data processing methods and apparatus for fault isolation Download PDF

Info

Publication number
CA2257511A1
CA2257511A1 CA002257511A CA2257511A CA2257511A1 CA 2257511 A1 CA2257511 A1 CA 2257511A1 CA 002257511 A CA002257511 A CA 002257511A CA 2257511 A CA2257511 A CA 2257511A CA 2257511 A1 CA2257511 A1 CA 2257511A1
Authority
CA
Canada
Prior art keywords
bus
error
pci
board
address
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Abandoned
Application number
CA002257511A
Other languages
French (fr)
Inventor
Conrad R.. Clemson
William I. Leavitt
Jeffrey S. Somers
John M. Chaves
David R. Barbera
Shawn A. Clayton
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Ascend Communications Inc
Original Assignee
Individual
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Individual filed Critical Individual
Publication of CA2257511A1 publication Critical patent/CA2257511A1/en
Abandoned legal-status Critical Current

Links

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00Error detection; Error correction; Monitoring
    • G06F11/07Responding to the occurrence of a fault, e.g. fault tolerance
    • G06F11/16Error detection or correction of the data by redundancy in hardware
    • G06F11/1608Error detection by comparing the output signals of redundant hardware
    • G06F11/1625Error detection by comparing the output signals of redundant hardware in communications, e.g. transmission, interfaces
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00Error detection; Error correction; Monitoring
    • G06F11/07Responding to the occurrence of a fault, e.g. fault tolerance
    • G06F11/16Error detection or correction of the data by redundancy in hardware
    • G06F11/1629Error detection by comparing the output of redundant processing systems
    • G06F11/1633Error detection by comparing the output of redundant processing systems using mutual exchange of the output between the redundant processing components

Abstract

A fault-isolating digital data processing apparatus includes plural functional units (12-18) that are interconnected for point-to-point communications by a plurality of buses (20a-20d). The functional units (12-18) monitor the buses (20a-20d) to which they are attached and signal the other units in the event there are bus communication errors. The functional units (12-18) can simultaneously enter into an error isolation phase, e.g., in response to a bus error signalled by one of the units. In addition to signalling bus errors, the functional units can signal unit-level (or "board") faults when they detect fault in their own operation. Each unit (12 or 14) includes error isolation functionality that signals a fault based on (i) whether that unit (12 or 14) signalled a loopback error with respect to its own operation; (ii) whether that unit or another unit signalled a bus error during the error isolation phase; and/or (iii) whether any other functional unit signalled that it was faulty during the error isolation phase.

Description

DIGITAL ll~A P~ G MEl~ElO~S AND ;~PPAR~l'US EOR F~ULT ISOIAlICN
Reference to Related Applications ~s application is a continuation in p~t of United States Patent Application Serial No.
081309,210, filed ~b ~.. 20, 1994, the teachings of which are i~o.yuldted herein by l~r ,~.

Reser~atioD of Co~

The disclosure of this patent docwnent contains mat~ial which is subject to co~light protec~on. The owner ~ereof has no vb;:~i~ to facsimile l~.~du~,lion by anyone of the patent document or of the patent disclosure, as it appears in the United States Patent and T~ .L (~ficepatent file or records, but U~ 'CS all rights under co~ h law.

Background of the Invention The ...~ r10~ and fl~ -Pd herein relates to digital data pl~cf~ g and, moreparticuhrly, to . ~e~h~l~ and a~ t~-c for fault ~PtP~inn and isolation.

A sho.lc~....;.~ of many co~.e~ 1 digital data ~lOC~ is their inability to detect and isolate a range of data h~f~ and op -~ l faults. Personal computers and wo,l- ~
for example, are typically ~--r~ .A to detect only a single class of data tIansfer faults, such as, panty e~rors. When llet~ ~h~, such a fault can cause the computer or workstation to halt , or "crash. " Larger computer systems often m~.dte er~r~l.~li.lg codes to detect and cor~ect single bit errors. These systems can be c ~ y~d to c~ ; ---e c~ation even in the event of fault.

Computer systems maTloeted and d~ - ibed in prior patents to the ~ gnr~ hereof are capable of both ~ hP;flL, correc~ng, and continuing opelation after faults. U.S. PaIent No.
4,750,177, for example, ~ s a fault-tolerant digital data ~ or having a first r,~ 1 unit, such as a cent~al ~ocessor, with duplicate P1VC~ L ~;O ~ coupled to a co.. o-~ system bus for identically ~l~f s a~ signals ~ ed from other ~u~e~iolal units, CA 022575ll l998-l2-03 WO 97/46941 rCT/US97/09781 such as the memory or y~ ,dl device controller. In addition to chP~l~ing the received data, the first f ,~ ;o~ unit CO~Ilpal~S the output g~nel~t~d by the sections while one of them -- the so-called adrive" section -- trqncmitc J,.ocesse~ data to the bus. When the section outputs do not agree, the funr-tionql unit drives an error signal onto the bus and takes itself off-line. Accol.ling to the patent, a filn~tionql unit such as a central ~ cessor can have a l~ nl partner that is consLlu~l~d and u~ lf S j(1pnti~qlly to the o~iginql In such a CO~ u.n';~ , if one of the pa~ is taken off-line due to error, processing co~ uf~s with the partner.

According to U.S. Patent No. 4,931,922, also accignfcl to the qCcignpe hereof, re~
pe- ;l)kf -, l control units, each with ~up1ir~~ pJuCe,,;~i~g 5e~ s, control the l~tnhing of data and cign~ling of errors vis-à-vis data ll~,sr.,l~ with ~ttP~h~l pf-;l.h- ."1 devices. For this u~ûse, data signals applied to the pelipL~al device bus by either the control units or h~lal devices are captured and Cu ~IlJalCd by ~,l.,cessih~g secti~ nc within each control unit. The results of those co...~ -- ;cons are shared between the controllers~ If the col.l~alisons inr1i-~tf- that the data captured by both control units agrees, then control units gene.a~e a Ustrobe" signal that causes the data to be l~t.~hfd If the units do not agree after succçccive retries, tbe control units withhold icsl~n~e of the "strobe" signal and enter an error-h~n~ling state for det~.. il.g the source of the error~

Co-pendi~g, commonly ~cci~l U~S. Patent Applir~tinn 07/926,857 ~liccloses, in one aspect, a digital data processing system having dual, redlln-l~nt processor el~nnP.ntc, each inr~ ing dual, l~.nrl-nl procçscing sections that ûperate in lock-step ~yllcllluni Failure det~,cti-)n stages in each section ~ccPmblP signals gen~ ed by that section into first/second and third/fourth groups"G~;lively~ Normally, signals in the first group match those of the third group, while the signals in the second group match those of the fourth group. The groups are routed ~l~n the sectionC along conductor sets Sf~ e from the system bus.
To detect fault, each stage co~ s a ,.,i,~,ecli~re one of the groups with its coullle~ . If either stage detects a ...i~ h, it signals an error to its l~,~ecti~e processing section, which can take the colle~onding processor cl~",~ off-line.

While the above-mentioned patents and patent appli~ ~ionc describe systems with high degrees of fault de~rc;~ n and fault tolerance, such capabilities are not required for all digital data ~ systems. While fault tolerance can be sacrificed in more mode~.ly priced systems, fault d~tecti-n cannot be. Indeed, in systems that lacking fault tol~n~ it is decin~bl~ not only to detect fault, but to isolate its source.

An object of this invention, the"~ , is to provide digital data proces~inp a~pal~lus and m~th~s with improved fault--le~e~ c~p llilities A related object is to provide such ~ 5 and m~th~l~ with fault i~ inn r~ilitiPs, i.e., t_at of ~ ; .g r~ units that are the sources of faults and ta~ng those units off-line.

Further objects of the invention are to provide such a~aldus and methods that can be readily implP.m~nted without excessive cost.

A rehted object is to provide such ~ s and m~thori~ as can be imrlPm~nt~1 in high-speed hald~ cle.../~ , such as ASIC's.

CA 022 7F. l l 1998 - 12 - 03 Summary of the lnvention The afor~mentinn~ objects are among those met by the invention, which provides in one aspect fault-isc-l ~ing digital data pluc~s~ g apparatus inr1-ltling plural functional units that are .hlt~ ~t~d for point-to-point co~ ;nnc by a plurality of buses. A system with two cent~al p.oces~ g units (CPU's) and two input/output (I/O) intcrf~~e units, for eY~Am can employ siY b~ 1 buses to couple each unit to the others. The r ..,nl ;. ,ul units monitor the buses to which they are ~tt~~hed and signal the other units in the event there are bus co...~u~ic~ti~ n errors. Thus, for ey~mrle, if one of the f ~nction tl units detects a parity or error cu~ g code (ECC) error during the l~ ---is.:on of data, that unit can signal a bus error the other units.

The functional units can ~im~ Aneuu~ly enter into an errûr isolation phase, e.g., in ~o~se to a bus error ~ign~ A by one of the units. During this phase, each unit tr~ncmit~ test data (e.g., pre~lrn --.;.tfA ~ Ile~ of O's and l's) onto at least one of its ~tt~h~f1 buses. Though more than one unit can 1~ test data at a time, only one unit can do so on a given bus.
For ~Y~mp1~., in a system with two CPU's and two I/O interface units, both CPU's can s;mlllt~n~o{)usly drive test data over the buses tbat couple them to the int~rf~ce units. Later in the phase, both VO units can drive test yall~lllS to the CPU's back over those same buses.

The fimctil~n~l units continue to monitor the buses and to signal bus errors wnile the test data is being t.~ ed Each unit does this not only when another is ~ ...;II;ng test data on a common, point-to-point bus, but also when the unit itself is driving such test data. The latter affords each unit the OyyOllu~uly to ~lru~ a loopback co...p~ o~ to match the data t'nat it int~n-led to drive onto a bus with that actually "received~ from the bus through ..-o~ ,. ;..g.
For ~-A...~lF, in a system as de~cri'necl above, each CPU mnnitorS its buses for error whi}e the I/O units are driving test data and, in ~lrlition, while the CPU itself is dIiving such data.
Moreover, each CPU COIllpal~ s the test data that it int~n~ied to drive on the bus with that which it receives from the bus.

In ~ litinn to ci~q11ing bus errors, the fi)n~tionq1 units can signal unit-level (or Uboard") faults when they detect fault in their own u~Gl~lioll. To this end, each unit inr1.-~1es error isolation funf~tionq1ity that signals a fault based on (i) whether that unit .ciEnqll~d a loopback error with respect to its own operation; (ii) wh~ that unit or another unit cignq11~ a bus error during the error i.col~qti~nn phase; and/or (iii) whcll~er any other fi~nctirmql unit cign that it was faulty duIing the error ico1 ~inn phase. By way of example, in a system as ~es(~ above, a CPU can signal the others that it is faulty if it detects a loopback c~ .;.con mismatch while driving test data onto the buses. By way of further example, that CPU can signal the others that it is faulty if it and the other f~mrti~nq1 units detect a bus error during a phase when that CPU drives test data, but not when the I/O units drive test data onto that bus. A unit that d~ s itself fault can, in addition to cignq11in~ the other units, take itself off-line.

Acco~l;ng to further aspects of the invention, any of the rclcgoulg functional units can include a ~ces~ g section that g. ~ f,S data for co~ r-q~tiQn to another unit and that inr1udec dual int~ s for applying that data to the qc.cociqtrd buses. The interfq.~.es, l~,fc.,Gd to as the Udrive side" and Ucheck side" interfqces, apply to the buses comp1iment~q-ry portions of each datum ge~ ed by the processing section. In the ~going eYqmrl~, a CPU can use dual intPrfP-~e secti~ nc to drive one half of each datum g~ led by its processing section onto the buses.

The drive side and check side int~.Tf~~~5 also receive data driven to the buses, whether by that r!~--cl;-)n~1 unit or another. With respect to data driven by the r~-~.cl;nn~1 unit, data received by drive and check sides can be mqtrh~i against the data intrnf~ed to be driven by the unit for lJul~o~es of the loopback co~ ;.con. Thus, for example, portions of a word received from the bus by the check side intP~ce can be colul alGd with the po~ ns which had been app3ied by that inte~fare as part of a Uloopback drive check." likewise, the other portions of a word recc;ved from the bus by the check side intrrf~e can be cou-~ul against those which had been applied by the other inte~ce (i.e., the drive side intr~e) as part of a Uloopback cc "~y~ue check. "

wo 97/46941 PCT/USg7/09781 Still further aspects of the invention provide mPthr.-~ for fault-i~o~ digital data processor operation pq~l1P,lin~ the yl~cesses d~Psrnhe-~ above.

These and other aspects of the invention are evident in the dlawin~,s and in the des~;liy~ion that follows.

. .

Brief Description of the Drawings A more ~.I.~lete un~l~t~n~ling of the invention may be ~tt~in~d by l~,fe.~nce to the L~Y~ings, in which:

Figure 1 depicts a digital data l~luces.~ g system e~ )yh:~g the il.~e.l~ion;

Figure 2 depicts the data buses of an "Xbus" accol.ling to the ,.~ lion;

Figure 3 depicts the control buses of an Xbus;

Figure 4 depicts a bit ~.. he.;i~E~ scheme for an Xbus;

Figures S and 6 ill--~tP simple word, read and line write tr~n~lrtiQn~ on an Xbus;

Figure 7 depicts the phases of an ~ lion on an Xbus;

Figure 8 depicts the effect of bus BUSYs on the basic bus cycle of an Xbus;

Figure 9 depicts the effect of bus errors on an Xbus;

Figure 10 depicts the basic state rn~-~hinP and state l-..n~:l;on~ for a bus handler used in c~nnrr,~ n with an Xbus;

Figure 11 depicts the inl~ l~;o...~f~ n between the Xbus and two functional units, or boards, connP~l~d Ih~,l~..ilh;

Figure 12 depicts self-c~ parity logic used in cf~nnP~ with an Xbus;

Figure 13 depicts loopbac~ connp~lh~ity used in conn~P~tinn with pr~cti.~e of the invention;

Figures 14 and 15 depict the timing and 1~ ing q~ociq~ with the breaking of boards in connr~tion with practice of the invention;

Figure 16 depicts the timing of board breaking in ~ ~nse to loopback errors acccldil~g to a p~a.çtire of the invention;

Figure 17 depicts timing of board breaking in r~sl~on~ to l-..~ l;r. or arbitrary breaking in accord with a pr~~tire of the invention;

Figure 18 depicts p~~ g of the Xbus in accord with a p~actice of the invention;

Figure 19 depicts CL ~;uil,~ for routing board_not broken in c~n~F~ n with practice of the invention;

Figure 20 depicts the routing of i-lrO..-. ~ion lines in accord with practice of the invention;

Figure 21 depicts routing of 3-way voted lines in co..n~l;on with prq,ctire of the invention;
and Figure 22 depicts error Çl~n(L;i~g logic in accord with p..~ e of the invention.

~ , . ~ ~

Detailed Description of the Ill~t~ d E mbodiment Figure 1 depicts a digital data ylvces~ine system 10 accor~ing to one pr~q~ctire of the ion. The system 10 in~ des mllltirle filnrtionql units, namely, central processing unit (CPU) 12, CPU 14, input/output (I/O) unit 16 and I/O unit 18. Each CPU 12, 14 is coupled to each I/O unit 16, 18 via buses 20A - 20D, as ill--~tr.qted These buses can include data lines, as well as control lines. CPU 12 is also cuu~led to CPU 14 via bus 20E, while ItO
unit 16 is coupled to I/O unit 18 via bus 20F. These buses include control lines, although they can also include data lines. Each r. - l ;0l~l unit inrl ~(ltes a procP-~cing section, 12A, 14A, 16A, 18A, ~ ly, and tvo intprf~qce s~PctionC 12B, 12C, 14B, 14C, 16B, 16C, 18B, 18C, l~s~;lively, as shown in the ~llawih~. In a plef~ d embo~1im~nt each r..n. ~ l unit is co~ &~l on a printed circuit board that couples to a bus bac~plqne in the system cabinet ~not shown) in the co-l~e-~l;nn~l manner.

The processing sc~ c 12A, 14A car~ out convçntionql ~nctionc qccoriqtP,d with central plu~s~ing units, e.g., instruction ~Yecuti~n To this end, the pl.,ces~mg sectinn.c 12A, 14A
can include co~,~f ~linn~l microprocessors or other central execution units (not shown), as well as other conventinnql el-~-.t..~l~ (e.g., on-board l,.ellluly) typically employed for those r... ,;. ~C. Interface s~ n~ 12B, 12C (con.l;l.,l;.~g the drive side and check side ;."~lr;.r~s for unit 12) Lla~f~ i~fo~ ;nn belw~~ q-cso~ ,ucec~ g section 12A and buses 20A, 20B, 20E which couple CPU 12 with I/O units 16, 18 and with CPU 14. ~kewise, int~rf. ce sf~1;f~..c 14B, 14C (c~ ti~ g the drive side and check side intPrfaces for unit 14) L,~srel inrC~ n ~t~vf~ n ~Jl~;Ssing section 14A and buses 20C, 20D, 20E which couple CPU 14 to those same I/O units 16, 18, and to CPU 12.

I/O units 16, 18 serve as bridges to ~ ;~h~ device buses 21A - 21D and, in turn, to ~,e-;~ ~,.1 devices 22A - 22D, as j~ $l.;.ted In the ill.J31. ~~ecl embo-1imP.nt pe- ;lJ~tf -,.l device buses 21A - 21D opeIate in accord with the il~dusL~y s~dar~ PCI Bus ~p~i~ tif~n ...;pkf .,.l devices 22A - 22P are also PCI co.-.p~ c. I/O units 16, 18 can include ce3~;~ secti~-m~ 16A, 18A, l~ecli~ely, for carrying out ~~lmini~t~tive tasks, if any, required by the I/O units in their roles as PCI brid~es. Tnt~rfnce section~ 16B, 16C

CA 022',7',11 1998-12-03 W O97/46941 PCT~US97/09781 (c.,~ ul;.-~ the drive and check sides of unit 16) and, 18B, 18C (co~ liu~ the drive and check sides of unit 18) provided in each of the I/O units 16, 18, respectively, tlansrer i"ro----~linn bel~n buses 20A - 20D and buses 21A - 21D. This inr~ ps convertingr~llcd illrol",alion to and from the PCI protocol used by buses 21A - 21D. ~lthollgh each pair of ;.,l~.r. ~e sections 16B, 16C and 18B, 18C intPrf ~e with a l~ ecti~,~e pair of internal buses 20A, 20D and 20B, 20C, respectively, the sections 16B, 16C, 18B, 18C
interface ~e~lir~rl PCI buses 21A, 21B, 21C, 21D, l~ ~e~ /ely, as illll,ctr~t~A

Though the illustrated I/O units 16, 18 serve as PCI bridges, those slcilled in the art will d~pl~ciale that the invention has application to I/O units capable of ;-~1~.. r.--~;n~ and/or controlling pe~il)h~als that co.. ~ e via any protocol. More gPnPr-dl1y, it will be appreciated that, although the illl-~ct~tP~ functional units are central processing units and I/O
units, the invention can be applied to any class of f~mrtion~l units, e.g., central proceccing units, memory units, I/O units, co.. ~ ir~ti-ns units, etc.

In ope.dion, the CPU's 12, 14 drive i.,rol"lalion to I/O units 16, 18 over their res~e~;live buses 28A - 20D, and vice versa. Each of the units 12 - 18 also monitor the buses to which they are ~tts-~hed and signal the other units in the event that there are bus co-.. -.~-ir~tion errors. Thus, for example, if CPU 12 detects a parity error or an error cull~;Li~g code (BCC) error during a tr~n~mi~sil~n of data to I/O units 16, 18 over data lines in buses 20A, 20B, respectively, CPU 12 can signal that bus error to the I/O units over control lines in buses 20A, 20B. CPU 12 can also signal a bus error to CPU 14 over bus 20E. The other fu~clional units 14 - 18 can likewise signal bus errors in the event they detect tllo,leous t~dn~mi~ic n~ over their l~ ;live buses.

In l~,i~nse to a bus error signal by any of the functional units 12 - 18, each fUnrti~m~l unit enters into an error isolation phase. During this phase, each of the units 12 - 18 lld~
p~ttP.rnc of test data onto its l~ ~eeli~/e buses 20A - 20F while, at the same time, moni~ g those buses for error. For example, in ~ )ol se to a bus error ~ l~ by I/O unit 18, CPU 12 can drive test p~ ..s onto buses 20A, 20B, while CPU 14 .~imnlt~nPously l~d~
test p ~ on buses 20C, 20D. During these t~n~ on~, each of the functional units , CA 022575ll l998-l2-03 12 - 18 mnnitor the buses 20A - 20D for parity and ECC errors, while CPU's 12, 14 u~ loopback ~ ;n~ on their ~ e buses 20A, 20B and 20C, 20D, respectively.
Once the CPU's 12, 14 have comrlet~P,d their test cycles, I/O units 16, 18 take their turns at driving test data onto the buses.

The fi~nrti~nq1 units co.~ ---e to monih~r their ~ ;ti~e buses 20A - 20D and to signal bus errors while the test data is being ~ ~. Each unit 12 - 18 does this not only when another unit is l-, A~ data on a c~ bus, but also when the unit itself is driving such data. This permits the units to ~lru~ loopback eu~p~;conc to match the data that it inl~.n~P~ to lrive onto a bus with that actually ~ ed from the bus ILluugh .~n ;~o.;~g.
For example, while CPU 12 is driving test data onto buses 20A, 20B, it ~imllltqnf~oysly colllp~es data values received from those buses to ensure that they are iflentirql with the driven data. As during normal u~ ;n~, the r,--- I;on~l units 12 - 18 signal bus errors to one another in the event that they detect cv--~ nC faults during the error i~olqtinn phase.

In ~ itinn to ~i~qline bus errors, each of the funrti-nql bus errors 12 - 18 can gem..~r. a default (or "brokenn) if it detects that its own ol~f..,.l;nn iS faulty. Thus, when~ ~er a fu-lrtinnql unit 12 - 18 detects a loopback error, it will signal the others that it is broken.
L~ ore, if any of the units 12 - 18 detect a bus error when it drives test data onto the bus, but not when another unit drives such data, the unit can signal the others that it is broken -- so long as none of the other units has, first, si~rqlPcl broken, e.g., as a result of its own l~opb~ error. By way of r~ le, CPU 12 will signal the other r....~l;n~.~l units 14 -18 that it is broken if CPU 12 detects a bus error during a cycle when it is driving test data onto buses 20A, 20B, but not when I/O units 16 - 18 are driving test data onto those buses.
On d~ ~;n~e that it is faulty, any of the Ç,....~;nA~1 units 12 - 18 can take itself of-line.

A more cn~ e Of tne construction and op~,"-l;on of the ilhl~tr.q-t~de~llbodi~ t may be A11; ;A~A by lef~ ce to the section that follows, in which digital data p,u~ssing system 10 is l-f~lcd to as "Polo," buses 20A - 20F are referred to as the "Xbus," buses 21A - 21D are referred to as the "Ibuses," CPU 12 is refe~ed to as "CPUl,"

CPU 14 is l~fe,l~d to as "CPUO," I10 unit 16 is referred to as "PCI Bridge 1" or "PClB 1,"
I/O unit 18 is referred to as "PCI Bridge O" or "PCIB 0."

Bus Naming Convention The Xbus actually consisls of 4 data buses and 12 control buses, which are named as desçrihe~ below.

Data Bus Nam ng Convention Figure 2 shows a block dia~.~... of the 4 data buses in the Xbus system. The number for the bus is taken from the CPU slot number shown in the illll~tr?tion; th~rc,l~ data buses co~-.P~Ied to CPU O end in O and data buses connf~t~ to CPU 1 end in 1. The letter for the bus is llrle~ r~ by whether or not the bus is a Cli~S~luss bus (i.e. connects an even slot number to an odd slot number) or a straight bus (i.e. com-ect~ an even to an even or an odd to an odd slot). Based on this convention, CPU O has colm~ -n~ to data bus AO and BO.
CPU 1 has COI~f.~ionC to data bus Bl and Al. PCI Bridge O has connPcti(n~ to data bus AO
and Bl. PCI Bridge 1 has con~ ons to BO and Al. In the ill~ d embo~limPnt~ the PCI bridge cards do not run in lock-step.

Control Bus Naming Convention Figure 3 shows a block ~liq~m of the control buses in the Xbus system. The control naming con~ ~n for the b ~~l~nP signals uses the signal name followed by the source of the signal and then the ~les~; ~A~;~ n of the signal. The naming convention for the associated ASIC pins uses the signal name, the direction (in or out), and the c~ n (n for neighbor, o for opposite, and p for peer). ~;Y ~ s of this naming convention are shown in Figure 3.
Table 1 lists the names of all control buses in the Xbus.

Table 1. Control Bus Names Control BusControl Bus Control Bus ASIC Driving ASIC Receiving Source Desl; ~ Name Pin Name Pin Name CPU O CPU 1 control_O_lcontrol out pcontrol in p CPU O PClB Ocontrol 0 2control out_ncontrol in_n CPU O PCIB 1cont~l_O 3control_out_o control_in_o CPU 1 CPU Ocontrol 1 0control_out pcontrol_in p CPU 1 PCIB Ocontrol 1 2control_out_ocontrol_in o CPU 1 PCIB 1control 1_3control out_ncontrol in_n PCIB O CPU O cont~ol 2 0control out ncontrol_in_n PCIB O CPU 1 control 2_1control out ocontrol in o PC113 0 PCIB 1control_2 3control out~ control in p PCIB 1 CPU O control 3 0control_out ocontrol_in_o PCIB 1 CPU 1control 3 1control_out ncontrol_in_n PCIB 1 PCIB Ocontrol_3_2control out pcontrol_in p Bit Numbering Figure 4 shows a des~ of the bit ~ bG~ scheme used on the Xbus.

Terminology bus cycl~-the 24MHz (--41.67 ns) building block from which all Xbus ope~ti~nC are built. A
bus cycle is the time which a valid logic level driven by one board on the backplane is seen by all other boards. Two bus cycles compose a bus phase, 4 bus phases colllpose a bus oy~ l ~;r)l~, and one or more bus op~ innC cG~.Ipose a bus tran~actiQn.

. .

bus phas~-the 12MHz (--83.3ns, 2 bus cycle~ building block from which all bus operations are constructed. There are logically 11 types of bus phases on the Xbus; "Arb", "Info", "Postl", and "Post2" are the phases that occur during normal operation. When an error is detected, the special phases "Errorl", "CPU test", "CPU Post", "IO Test", "IO Postl", "IO
Post2", and "Error2" are inserted in the protocol for fault isolation. tThe error phases are somP~tim~s collectively ,.,fe,l~d to as "Post3"). During each bus phase it is possible to two sets of i~u. .~ ;on on a physical set of barl~rl~n~ lines (i.e. "double ~u-llyi~") though this is not done for all bus phases and/or signals.

bus operation--A bus upe~aLiOn iS the basic unit of address and data tr~n~nlic.cir)n and çhrrL ;..~ on the Xbus. It is g~nPr~lly compose~i of at least 4 phases: an Arb phase followed by Info, Postl and Post2 phases. Bus errors cause the ih~SGllioll of the error phases after Post2 and i~e ~ the number of phases required to complete a bus operation. Bus ope~ti~ nC may consist of m-~ltirl~. info t-~ ol-c in the case of a block transfer. A bus operation can be thought of as a full one way l~ Çer on the Xbus.

bus sub-operation--A bus sub-ope~tiQn is an operation initiated by a bus master during subsequent phases of a block transfer. Sub-ope~tion~ always carry data and BUSYs are ignored. It is ~ene~lly co~osGd of 4 phases: an Arb phase during which grant inhibit is used, followed by Info, Postl and Post2 phases. Bus errors may incl~ase the number of phases required to complete a bus sub-ope~tion. A sub-operation is dilr~,,~..li~led from an oy~ tion in that a bus opeption for a block ~ sr~,l consists of the first l1~CS~ plus a number of sub-ope-i~t;on~ concicting of multiple data Ll~r~

bus transaction--a complete high level çY~h~r~ge of iufo.~ ti-)n on the Xbus. Examples include reads and writes. A read l~,~n~ .clion l~lween CPU and PCIB is co~yosed of a of two bus Op~ ; one operation provides the address and function code, and one or more Oye~aliolls provide the return data. A write t.~ns~~tinn to a PCIB is composed of a ,-.h.;.-.~ of two uy~ l;on~; one operation provides address and Çunt,lio~ code, and one or more ~(iitinn~l oper~tionc provide the data.
.

.....

Figures S and 6 illll~trAte the t~rminnlogy ~ unding simple word read and line write t~n~inn~.

Bus Master--A board that has won &~ ;on This board drives the info lines in the info phase of the bus ope~ A bus master can be a ~ ..c ~ m master or a l,, n,s~ inn shve.

Bus Slav~-A board that has clel~ ...;nf~l the info lines carry i~o....-lion that it must receive.
A bus slave can be a t~mC~r,1ion master or a tn~n~ ti~m slave.

CD dift'erent read--A read in which the C and D side ASICs each provide haLf the data e.g.
when reading error status regi~tfrs Loopback cher~ of the bytes driven by the other side lS Su~)pl~ Sed.

Cydops--The Xbus to Ibus intP.rface ASIC on the CPU board in the Polo system.

echo transaction--The second half of a peer-to-peer bus trAn~tion between CPUs. Send and echo trAn~rtions are not split; grant inhibit is used to ensure that no other tri~n~action~
occur bel~ the send and echo. This is to prevent re-ordering.

EFQ--Eviction-Flush Queue. This queue exists only on CPU boards. l~efer to the Cyclops (Bus T.~ e) Spe~ifi~ti~n for details.

EFQ-Freeze State--Set via bit 19 of the Bus TntPrf~re State ~.gi~t~r. This mode only exists on CPU boards. When in this mode, a board will busy all ~ ces directed to its EPQ
except those from its partner unless it is a data return of any size.

Fast Duplexing--Fast duplexing is a form of duplexing in which no memory is up~
This is done when both boards are coming up for the first time as c~osed to up~ting a new board from a lu~ll,ing OS.

... .

CA 022575ll l998-l2-03 Fr~ Stat~-Set via bit 14 of the Bus Tnt~ce State Register. This mode only exists on boards that can be duplexed. When in this mode, a board will busy all accessçs directed to its RWQ except those from its partner.

Gambit--The Xbus to PCI ASIC on the PCI Bridge card. It jrlterf~ ; to the Xbus and the PCI bus.

I/O virtual address (lOVA)--An IOVA is an address ~en~ t-~ by a PCI card. This address is ~ n~ r~l across the Xbus to the Cyclops ASIC. Inside the cyclops ASIC, the address is -c.1~t~d into a valid system address. The IOVA is used to provide fault tol~r~n~e. It es that a PCI card will e- n~"~P a correct address range.

loopback ch~l~i~g This is when an ASIC checks that the value it sees on a pin is equal to the value it thinks should be on the pin during nonnal op~or~tif~n loopcheck ~ dtion--This is when the bus ASICs drive test palL~llls of 55/AAs as part of the error protocol in order to cle1r....it~ the site of a fault.

peer to peer transaction--A two part l.,.,-c~ ;nn between two CPUs. The Xbus does not have fuUy il~te~ nn~lP~ data buses, and ll~lSrrl~ bet~ the two CPU boards must occur in two steps: first a send bel~lt the CPU and the PCIBs), then an echo from the PCIB(s) to all of the CPU(s). The requesting CPU drives a cc,~ )lete tr~n~,tinn on its A and B
buses. The PCIBs look at the address, and det~ ;nfs whether the tr~nS~ti~n is directed to the CPUs. If it is, they buffer the t~nS~ n in order to repeat the tr~nC~tti~n on their A &
B buses once Post2 of the hst info phase has passed with no bus errors. In this way, both CPUs see the tT~nS"r'til~n at the same time. Peer to peer t~n~'~ti~nS between CPU boards require a .. ;n.. --~, of four oper~tinnc.

~WQ--Read/Write Queue. This queue exists only on CPU boards. Refer to the Cyclops (Bus Tnttorf:~ ~e) Sperifir~tion for details.

WO 97t46941 PCT/US97/09781 Reg..~ ted Info--A ~ ;urgi~d info is a Cougar rConrad, what is Cougar?] gener~t~d cycle used during the update process. It is gen~led by the update-on-line board and n.~ to the update off-line board. It is unique bP~--se an update off-line board accepts info cycles even if the base address does not match the base address of the board.

send transaction--The first half of a peer-to-peer bus tr~n~a~tion b~weel~ CPUs.
single side operation--An ~pP~ion with data supplied entirely by either the C or D ASIC;
e.g. a read from a PU card. Loopback çhPrL ;.~ is ~.~ul~ncd only by the side supplying the data.

TRIl)--T~n~~tion i~ ;r~r. This is a unique binary number used to tie together two bus o~ nc during a split tr~nc~rtir~n and to identify the source for write l..~n.~r.litm~ (TRIDs on write t~n~rtinn~ are strictly for debug). A TRID is unique only while a giventr~nc~.tir,n iS still ou~ n~ and will be re-used for later transactions by a tr~nc~ctir~n-master. Note that trid bits 02-00 are used for the slot number of the tr~n~ctiQn-master and t~d bits 06-04 are g~ aled by an on-bûard master - thus allowing a board to have 8 unique masters with I~"QC~'1;On~ uul!~ti.nf~ g. Trid bit 03 is a new bit for Polo that intli~tr.,5 the format of the address; a zero int~ir~tPs a Jetta style system address is being ~a~ ~ and a one inrlir~trs an IOVA (I/O Virtual Address) format is on the backplane. The IOVA address needs to be t~n~l~t~l into a system address via the map RAM.

Transaction Master--The specific l~,suu~;e on a board that ~f ne . t~l a tran~ fir,n. The tr~n~ctir)n master is l~ onsibl- for e~"~r~ E the TRID of the l""~c~cl;~ n Transaction Slav~-The board on which a t~n~rtir)~ was directed to~ d~, i.e. a board in which the function code and address of a bus operation has ~ecode~ to.

Xbus Sig~lal Description The following section ~les. ~il~s in detail the various signals that CC....I-.;cf, the Xbus. A
filnt'.tit~n~ f.S~ d;~ iS in-~1u~erl for each type of signal, though not for every individual signal (for e ~ l,1f there is a TRID field for all 4 buses, however there is only one de~cli~)tion that covers both). The follo~ring rules were used in C~ g the signal names:

~ all lower-case çh~"~
~ the "_" is a ~ ,it,l when it is not at the end of a signal name ~ the "_" at the end of the signal name in~ tes that the signal is low-tIue ~ a indicates A_bus signals ~ _b ;.~ s B bus signals ~ _x_, ~_, and _z_ are used for the three low-true signals on a Ll;~.licated net~ [n:m~ is used to de~ ~ il e a multi-bit field Signal Description The bused signals are imple-n-~nt~ as four point-to-point bidirection~l signals. The ability to drive these signals is controlled lL~ugh an ~I.;I-,-~ n n_~wolh. These signals are pl~ll~t~d by a single parity bit that aCcon~ s the bus. The buses are ih~ ~ so that each CPU iS CO - ~f~'ted to both PCI bridge boards and each PCI bridge board is con~ ;led to both CPUs. Enor recovery is accomplich~d through the XBus error protocol.

The control signals are cl~ PA in a set of poiDt to point u~ ;o~l buses. Each ofthese buses is ECC p~l~t~d. These buses carry control signals which are not govemed through ~l,;t~"~;o-~ Unlike the bused signals, there is a control bus in each di~
between every board. This is ~-P~ / in order to ensure the single bus view of the system.
For example, if one PCI bridge card sees a bus error, that i~o....~1;ol~ must be t~,.l.c...il~rA to all three other boar~s in order for the boards to all pulrO~ the error se~lu~ ~ce.

The bused signals and control signals are double l~u~l~ped at 24MHz each cycle. That is, they carry Lrr~,-t data during the first and second 24Mhz bus cycles that make up a single phase. All buses and control signals are active high on the b~~~pl~ne.

A small ~ of reset and broken related control signals are buffered by the 26SlO
~.,.n~ rer and rerlir~t~l for th~ree way voting. These signals are active low on t_e b;l~1rrlqnf-~

The Info Bus The follov.~ing signals are collectively referred to as the info bus. ~lthough there are actually 4 sets of these signals (aO,al,bO,bl), for simplicity's sake only the aO version is listed. For ~lc, when the TRID field is ~es~-~ihe~, it should be understood that there are actually 4 TRID buses: trid aO, trid_bO, trid al, and trid_bl.

Table 2. Xbus Bidi,~lional Buses signal width des~ ! inn info aO[31:0] 128 Xbus info bus - Info is driven during the info phase (32x4) of a bus OpC~ by the current bus master. This field may contain either an address (phy~ical address or virhlal index with rwl~lion code) or data, ~.n~ling on the fund_op control line.
trid aO[6:0] 28 Xbus transaction id (TR~)) - The trid lines carry the (7X4) ~D (t~n~f~.tinn ID) during the first cycle of a phase. During the second cycle, it carries the number of phases lG~I.si~.;..g, the first op bit, and cache cohe~n.;y bits.

WO 97/46941 PCTtUS97109781 signal width rl~.s. . ;~.l ;nn func op aO 1 Xbus func op - This line carries the func op signal which in~iir~t~~ that the current ih~ n on the info bus c~ c a function code that should be decode~d This signal is low t~ue so that an idle bus will indicate that a function ne~ds to be decod~ and thus a no-op r-...~;on This bit is valid during an info phase and is ~ ~;~d by paIity along with the lower half of the info field. During the second cycle, it is unused (driven to logic O on the backp}ane).
paIity aO 1 Xbus parib - This parity signal covers all of the bid r~Lional signals on a bus; info, trid, func op .

The Control Bus This section ~es~ ~~ the control signals. There are actually 12 control buses, but again only one is de~..il~l here. The names of the twelve contro} buses are listed in Table l, above. For si~ )li. ily of doc~lmP.~ ;ol-, the control bus is id~ntifiPd by bit numbering, similar to the trid field. However, since the mP~ning of the control bus bits is very ~;C"ir.~ each one is de~ e~l in detail here.

The control buses are ~l~L~led by a single bit correction, double bit detP~tion ECC code.

Table 3. Control Bus Signals signal width dc.,clil)tion control[O] 12 bus req and ack - During the first half of the phase, this bit(12xl~ is used for bus_req. During the second half of the phase, this bit is used for ack.

During the first half of the phase, this bit is driven by a board when it is ~ "es1 ;ng the bus. As df.c.~ .d in later c~l;nf~, the bus uses a .l;.cl~ib~t~~ ;.oll model loosely based on the Golfbus. Each board in the system drives the bus req and tests all of other boards bus req signals to .l..t~ .. who will drive the info signals during the next phase.

During the second cycle of the phase, this bit is used to ..ch~u~.ledge a bus tr~n~~.ti~ n. Ack is asserted in Post2 by the target board of the 1.,,~ - This signal is the result of an address decode, so it is only valid in Post2 of an ~ n that is Llar~S~ ing an address. Ack provides an intlir~tion of whether or not a tr~nc~~~tit)n is plU~ ,s;~ g.
This is relevant in a Polo system, since a PCI card may go away ~,.clllting in a no ack for a ping operation.

Acks in Polo system also let a PCI Bridge know that a CPU's map RAM has ..1~l.~1 a PCI initiated access to a valid CPU address. If a PCD3's read or write is not Aed, the PCI slot that ill;l;~led the access may or may not be set off-line dr.~)e~ -g on bits in the Gambit's configuration register.

Writes fmm the CPU to the PCIB are acked to facilit-~~
debug, but are otherwise unused. A peer to peer CPU write is not ACKed by the PCIBs, but the echoed operation is CA 02257~11 1998-12-03 control[1] 12 grant inh and main_int - During the first half of the phase, (12xl) this bit is used for grant inh, during the second half of the phase, this bit is used for main_int.

During the first cycle, the grant inhibit control bit is driven by the current bus master to extend the info cycles when the bus master is moving a block of data. The ~l,:~alion logic will not issue a grant to any other board when a board is driving the signal. T~is ensures that the current bus master will retain o~. ne~Li~ of the bus for a.~olhcr cycle.

During the second half of the cycle, this bit is used to signal a ~ t~ re in~ ,e Il~t~ )l in~ t~s that some board in the system is re~esting attention from the software ~ ;n~ nce process. Any board in the system may drive this signal during the second half of a bus phase regardless of bus ~..~tr~ ). All boards in the system will sample ~~ tun~ hlt~"lu~ and use it to reset their albiLlaLion priority.

controlt2] 12 bus err a and busy - Assertion of this signal during the (12x1) first half of Post2 signals that a bus error was dettrtçd on the info bus ~C~i~t~d with this particular control bus (n,o,p). Any ~1~ in Arb, Info, or Postl will be aborted. Ope~tion~ in Post2 are s~nd~A while the error protocol runs, and then will return to the Info phase.

I~e CPU ~i~i~tor of a peer-to-peer operation must track the entire operation to see whether the cycle is errored in either the send or echo portion of the 1~ m Assertion of this signal during the second half of Post2 indicates to the bus master that the operation should be aborted and re-tried at a later time.

Busy is ignored for the send portion of a peer-to-peer op~ n The CPU in;l;~ r of a peer-to-peer operation must track the entire ope~ti~ n to see wh~ lh~ the cycle is busied.

control[3] 12 bus_err_b and funny state - Assertion of this signal during (12xl) the first half of Post2 signals that a bus error was dt;lP~trd on the B bus conllecled to this board. Any operation in Arb, Info, or Postl will be aborted. Opr~ in Post2 are su~pended while the error protocol runs, and then will retum to the Info phase.

The CPU initiator of a peer-to-peer operation must track the entire Opf..~ ~;on to see wL~Iel the cycle is errored in either the send or echo portion of the tr~n~a~tion During the second half of the cycle, this bit is used to signal that a board has ~ust gone unbroken. The board will contim~e to assert this signal until it has seen eight phases without a bus error occurring. At that point the board will stop asserting this signal. Then all other boards will treat this board as an active, l~ondillg board. This pl~velll~ a board that is going ll~n from n ~on~ g to bus errors in the middle of an error se~uenr~ that is already w~d~l~ay.

control[7:41 48 checkbits - The top 4 bits of the control bus are clleckl ;
(12x4) g~ 1~ from the lower 4 bits of control signals.

The cll~il ~1~. ilh-.. is:

control[41 = control[O]~controltl]Acontrol[2];

controlt5] = control[O]~control[l]~controlt3];

control[6] = control[O]~control[21~contlol[3];

control[71 = control[l]~control[2]~control[3];

The Voted Signals 'rhe voted signals are the only signals through 26S10 ll~nscGi~ers. These signals are point to point and t~ d at each end, so that insertion of an u~lpu~ d board does not disturb the th...;..~ion (and timing) of a net in use. Only the x versions are listed; there are y and _z signals making up the triplet.

Table 4. 3-Way Voted Si~nals signal total desc~ ion reset_O_l x_, 18 reset - There are separate 3-way voted tripletsreset 0 2 x, (2x3x3) from each CPU to the other three boards in the reset 0_3 x_, system. When a RECC needs to reset the reset_l_O_x_, system, all lines go active. When a CPU wants reset_l_2_x_, to reset another board, only the lines going toreset 1_3_x, that board are active.
board not broken O 1 x 36 broken status - This three way voted signal is _ board not broken_O_2 x(4x3x3) driven from each board to each other board. It board_not broken_O 3 x is driven when a board is alive and not broken in board not broken 1 0 xthe system and is used to dPle.. ~ine which buses _ board_not_broken 1_2_x are active. The C-side ASICs drive the signals board not_broken 1_3_x and the D-side ASICs drive the output enables board not_broken 2_0_x for the 26SlOs. This o~ ;t)n ~ ~s board not broken 2 1 x that board not broken is deasserted whenever _ board not_broken_2_3 x either side of the board thinks that the board is board not_broken_3_0_x broken. The lGceiving board votes the x, y, andboard not broken 3 1 x z signals. CPUO in slotO drives board not_broken_3_2_x board not broken[O], etc.
sync_x 3 sync status - These signals are on the CPU only(lx3) and are used when sync~u~ing a pair of CPUs to enter the d~lplPYe l state.

even online_x_, 6 on-linc status - These signals are on the CPU
odd_online x (2x3) only and are used by the CPU boards to Co"~ ..te to each other which CPU board(s) are in the on-line state. Even_online_ is asserted when the CPU in slot O is on-line; odd_online_ is asserted when CPU in slot 1 is on-line.

CA 022575ll l998-l2-03 signal total des~-- .pl ;ol-Other Control Signals Table 5. Miscellaneous Control ~nql~
lot ida 2 slot id - The slot ID signals are hard wired on the slot_idb b~ p1qne for each slot. There is one duplicated slot id. Since the slots are d~Ai--qtP~ in Polo, it is only n~ to det~rmine if a board is in an even or an odd slot. These two bits will be l~is~ ,d and ~h.or~ by each ASIC at reset and will not be sampled again. If an enor is d~ ;t~ at reset, the board will bre. k and hence will never be capable of being brought on-line.

xb clk8 1 system clock - This is the system clock received by the Sentry clock chip and used to genc alG the board clocks. It is ~e~ cl by the bac~l~n~o. clock osci11-q~Jor. This clock runs at 8MHz, so every board in the system will be in sync with each other and there will be no need for ~ itinnql ~y~c~ inl. clocks to be passed along the bar~ n~ The clock is pulse width mod--lq,tçd so that 4Mhz can be ~;e.~. "~t~

slotO ta d, 4 ta signals - These "ta" signals are only present on slotO ta c , CPU boards and are sent bG~l,. e~n a dl~rl~Y~d board slotl ta d, pair for early d~tecti~)n of the boards going out of slotl ta c lockstep.

Xbus Protoco Overview The Xbus is a point-to-point ~ynt Ll~ us, pipelined, mllltirleY~d address/data, error clele~ bus with split-~ ;on c~r~iliti~s I~u~clioll-code, address and data are parity and loop-back ~ . Cont~ol lines are ECC pi~,t~ted and loop-back chP~l~ Three-wayvoting is ;~ ted on t_e reset, clock, and broken inrlir~tor lines.

The bus ~u~ word ~cces~s and has a block tlansrcr capability for support of cache line acce~ses. The Xbus has a logical 32-bit wide data/address bus.

Bus Operation The basic CC~o~C-lt of all Xbus ~ C~ ion.~ is the opc~alion. An operation is composed of four phases as ilhls~t~l in Figure 7: arb, info, postl, and post2. Two inÇolll,alion transfers can occur on the bus during each phase; this is referred to as "double ~u~ g". The double pump rl~quen;y is a~)pl. ~;...~tely 24~Iz. The figure ill-lst~tP~s the logical activity on the bus. All il~..~ is directly l~ lc~l in the ASICs without any eYt.o.rn~l buffers.
The phases are used as follows:

A~rb phase: Boards drive their ~I,ill~lion request lines during the first half (cycle) of the all,iL-~tion phase. During the second half they (~rl~ ...;nr, whe~ler they won ~biLl~lion and ~.l~alG for the info phase.

Info phase: For non-IOVA address llansr~l~, boards drive the virlual index, function code, remote/coh~llt bits, and byte enables during the first half of the info phase and the physical address during the second half. For IOVA address ll~sr~ OVA
bit in the trid is true), boards drive the IOVA during the first half of the info phase and ~lrl~ tic data with good parity during the second half; the physical address is . .

gotten from the I/O address map RAM look-up. For data transfers, data is driven during both the first and second halves of the cycle. Note that non-cache con~if t~nt address 1., n~r~ need not supply a virlual index through the driven i~...--~';o~ must be ~let~ ;r and parity will be cc...-~ ~l across it.

Postl phase: During this phase, boards are d~ ;..g whether any error con~.1ition~
- existed with the info phase and whetll~ r there is need to BUSY the u~, ' ;~ n CPU
boards map the device index portion of the IOVA to obtain the full physical address and virtual index of an I/O board's ~"~sr~,l for IOVA address transfers.

Post~ phase: Any board ~le~ an error with the info phase drives the error lines during the first half. If a board does get errored, it next goes to the error se~uPnre phases to d~ the source of the error. Any board ~etPcti~ the need to BUSY
an address/ru ~ code driven during the info phase drives BUSY during the second cycle of this phase. It is also during this phase that ~ec~es are acl~owledged.

Bus Busies Figure 8 ill~ ,s the effect of bus BUSYs on the basic bus cycle. As shown in the figure, BUSY has no effect on a bus ope~i()n during any phase except for post2; a BUSY during post2 will cancel the bus opP~tion Note that busys for m-lltipl~P cycle bus o~eri ~iOn~, such as block II~Ç~,~, have the special rules. Should a cycle be both BUSYed and ERRORed, the ~RROR takes prec~Pnre Bus E~ors Figure 9 shows the effect of bus errors on the basic bus cycle. As shown in that figure, the board that was ~ during the error ;---lol..~ lly gets the first available info cycle following eYp~utirm of the error protocol. The ~b;l.i.l;r,n iS ignored in the previous cycle.

, . .

CA 02257~11 1998-12-03 Since the Xbus has no L,~s~;~ers, the lou~ fl ~ phase of the Golfbus error protocol (Post4) has been m~ifi~d to allow each board an op~llunily to verify its t.~ ...;l and receive capabilities. This has resulted in new states being added to the bus error olx~
These states are de~ç.il.e~ below:

Errl: The Errl state is entered on the cycle after a bus error is detecte~l This state is used to allow for time to tum off the info bus before the lou~bacL checks are pc.rul,l,ed. A board that is in its info phase during Errl will disable its output enables half way ll-luugh the phase.

CPUTest: The CPUTest state is used to test the CPU's ability to drive p ~I....c on the Xbus. On the first cycle of the phase the CPU will drive 55 on the info bus, 55 on the trid bus, 1 on the parity line and 0 on the func op line. On the second cycle of the phase the CPU will drive AA on the info bus, 2A on the trid bus, 0 on the parity line and 1 on the func op line.

CPUPost: The CPUPost state is used to turn the bus around bel~ the CPU's loupba~check and the I/O boards loopback check. This phase is also used as a Postl cycle for the CPU's loopback pattern.

IOTest: The IOTest state is used to test the I/O board's ability to drive ~ on the Xbus. On the first cycle of the phase the I/O board will drive 55 on the info bus, 55 on the trid bus, 1 one the parity line and 0 on the func_op line. On the second cycle of the phase the I/O board will drive AA on the info bus, 2A on the trid bus, 0 on the parity line and 1 on the func op line. Bus errors from the CPUTest phase are ,~olled during this phase.
This illro... ~ n is used to evaluate the bus, CPU, and VO board at the end of the error se~ue.n-~e. The last VO ASIC to drive the data bus drives the bus during the IOTest phase.

IOPost: The IOPostl state is used to evaluate the IOTest data.

_ _ . . . . . ..

W O97/46941 PCTrUS97/09781 IOPost: The IOPost2 state is used to ~ n;l any bus errors from the IOPostl state. This il Çol,l.alion will be used to make an intel1igtent dec;s;on about how to deal with the error.

Err2: The Err2 state is used to evaluate the i~ ;on from the loopback checks. Bus errors from CPUPost and IOPost2 as well as n~...\~inn shared ~l..~n the C and D sides of each board are used to d~,t~ what course of action to take. This set of actions will be desr.;1led later in this section.

Figure l0 shows the basic state ~qrhinP and state tr~nsiti- n~ for the bus error handler. The key çhq11P~e for the bus error algol;lh~l, on the Xbus is to ~ii~nose errors so that system opçrati-)n can c~ e. Unlike previous systems that use duplicated buses to allow all fim~tinnq1 units a ~ .. t~,p~J path for co.. ~ rqti~mc~ when the Xbus removes a bus from sen~ice, it must also remove one of the two units ~ rl~rA to that bus. In some cases, the right thing to do is obvious. In other cases, it is not. The following sections analyze various faults, how they are h~n~l1Pd and how they l..al~ir~ sl themselves.

Bus Error Broken Conditions At this point it would be helpful to classify the dirr~lcl~t types of con~litinn~ that cause a board to go broken when a bus is ~1etect~.d bad.

~ loopback on control - C and D ASICs must always agree on what to drive on the control lines, in~h1Aing whether or not to assert bus error. If one ASIC asserts bus error and the other side does not, the board breaks.
~ Ioopback on data - C and D ASICs must always agree on what to drive on the duplicated info lines. When driving CD same data, ASICs co,l,p~e the data they drive with the data they receive. An ASIC asserts bus error on parity errors when receiving data, and parity errors and loopback errors when driving data. Loopb. ~~
Pr~ g is ~ qhle~1 when a board drives "CD dirr. n ~It" data, such as the co-~e~ of error l~pO~ lg l~iStl l~ or data f~m PCI cards. The board breaks if the two sides disagree on which bytes have or do not have errors.

.

~ arbitrary - Break the ~~ n~t~ board in Err2 when there are bus errors ~i~nqlffl during CPUtest and IOtest and no board has broken by the end of Iopost2. This iscalled an ~l,hl~ shoot because the fault is most likely on the backplane, so it is ~ as to which of the two bo. rds col-ne~;led to the faulty bus to break.
Typically, the CPU is set broken, so that the system can c~ ,e with all of its ItO
available, but if bit 21 of the Bus ~nt~rfqr~ State register is set then the PCIB board will be the ~s;gr~ l board.
~ heurific - a board breaks itself during Err2 if there is a bus error when it drives, but no bus e~Tor when the other board drives, and the other board did not break by the end of IOPOST2.

Xbus Fault Analysis In order to u~ various faults and what they can mean, it is h,.~l~l to present a ~let~ d block ~ &~m of the Xbus int..lconnc~;l. Figure 11 shows the ih~Gl~;onnect for a typical Xbus line.

The black dots in figure 11 lc~l~S~ the conn~< rS to the backplane. Por fault tolerance and fault isold~ion reasons, it is hlll)oll~l that the boards should be routed so that the etch between the D-side and the C-side runs lh~u~h the co~e~or co~ )n. This limits the amount of etch on each board that cannot be ico1~ted to a ..-i";.~ ... On the CPU board, one ASIC both drives and recehes a given net while the other ASIC only receives tbat net. On the VO board, each ASIC can potr~ lly drive every net. The CPU ASICs are always in ~ and lh~lefon~ each ASIC is capable of sharing the data out load. However on the VO board, each ASIC co-~ to a dilf~ l PCI bus so a signal ASIC may need to drivethe entire Xbus. There are cases in normal opeldlioll when only one CPU ASIC will drive the entire bus.

.

Fault Conditions The following se~!;n.~ identify all known fault conditions and descrihe their hqn-lling. Refer to Figure 11 to dtte~ r- the loc~ n of the fault site i~ vqt~ That figure depicts two r~ l units (or board), e.g., units 12 and 16, as well as t_eir ~ e drive and check sides 12B ,12C, 16B, 16C.

CPU Board Faulb Input Circuit - CPU Driving ~ fault site 31 ~ b~ak via loo~ae~ on control fault This fault deals with a fault in the input section of one of the CPU ASICs. In t_is case, the fault oc~;ull.,d during or just before a cycle in w_ich the CPU drove the info bus. The error is ~el~ when the CPU drives the bus. The ASIC with the faulty circuit will signal a bus error during the Post2 p_ase of the cycle and the other side ASIC will not. The board will go broken and drive bus error during Errl. The error sequP-n~e will be r~P~;uled, and the o~.. ~;nn will be retied by the partner CPU with no error.

CPU Board Faulty Input Circuit - I/O Board Driving ~ fault site 31 ~ brealc via loo~ac}~ on control fault This fault deals with a fault in the input section of one of the CPU ASICs. In this case the fault occurred during or just before a cycle in which the VO board drove the info bus. If the e~or is a multi-bit error that evades the parity logic, the error will be caught internql to the CPU board and the CPU board will go broken. If the error is a single bit error the faulty ASIC will detect a bus error during the Postl phase of the ~ r~. The ASIC will drive bus error during Post2 of the l~ s~e~ and the other side of the board will not. The board will CA 022.,7., l l 1998 - 12 - 03 break with a loopback on control failure in the next phase. After the error se~ , the operation will be retried by the partner CPU with no error.

CPU Board Dill~ t Data C-Side and ~Side ~ fault site 32 ~ break via loopback on data fault This fault deals with an internal CPU fault that results in dill~ data being driven out of each ASIC. The error is dete~te(l when the CPU drives the bus. The C and D ASICs will trade error status and disagree on where the error is during Postl; both C and D-sides will see an error on bytes the other side drives but no error on the bytes it drives. The board will go broken and drive bus error during Post2. The error se~quence will be t~x~uted and the operati~m will be retried by the partner CPU with no error.

CPU Board Faulty Output Circuit - Buffer to Pad Fault ~ fault site 33 ~ break via hen~i~tic broken This is a fault in the output section of the CPU ASIC reS~lting from an output driver circuit fault that blows in a n~ ~r that causes an ir~te~nql open b~ n the output driver and the ASIC pad, while not dis~ g the filnt~ti-~nqlity of the input receiver. All ASICs on the bus will detect a bus error during the Postl phase of the transfer. The ASICs will drive bus error during Post2 of the ll~s[~l~ All ASICs will detect a bus error during the CPUPost phase.
No bus errors are d~ ~ during the Iopostl phase. The CPU board will go broken during Err2 based on the fact that it (let~t~d a bus error when it drove the bus, no bus error when the I/O board drove the bus and the I/O board did not go broken after its er~or ~ ue ~ce.

. . .

CPU Board Open - CPU Board Driving ~ fault site 34 ~ break vh loopback on data fault This fault results from an apen due to either a broken etch or a lifted pin on he CPU board.
The routing is very important for this class of fault. The etch 1~1..~l, the C-side and the D-side should be routed IhlY.u~;h the cQ~ D~ pin. This limits the pos~i) ilhy that an open on the CPU board is miQ~lrPn to be an open on he l ~ ne. In this case the fault occurred during or just before a cycle in which the CPU drove the info bus. The error is d~ te~l when the CPU drives the bus. During the Postl phase, the driving ASIC will not see an error but the ~ hD L ;-~g ASIC will signal a co~p~ error. During Post2 the board will go broken and drive bus error. The Opf~ will be retried by the partner CPU witb no error a~ter the error s~ .nr~

CPU Board Open - I/O Board Driving ~ fault site 34 ~ break via loo~ on control This fault results from an open due to either a broken etch or a lifted pin on the CPU board.
The routing is very i~ l~l for this class of fault. The etch b~ . ~n the C-side and the D-side should be routed l~uugh the c4~ Qr pin. This limits the posc;~ hy that an open on the CPU board is mi~ n to be an open on the b~ In this case the fault oc-,ull~,d during or just before a cycle in which the I/O board drove the info bus. The ASIC with the open bcl~ it and the c~ 10r will detect a bus error during Postl. During Post2 one ASIC will assert bus error and the other will not, causing the board to break. The Opf~Ation will be retried by the partner CPU without any error after the error se~"~"~r~, .

CPU Board Short ~ fault site 34 ~ break via a~ broken This fault deals with a short on the CPU board. When the fault occurred and who was driving the info bus during the fau1t are not relevant to this class of fault. During Postl of the ~ r~, all ASICs on the bus will detect a bus error. The ASICs will drive bus error during Post2 of the transfer. Both ASICs on the CPU will detect a loopb~ error during the CPUPost phase and the ASICs on the I/O board will signal a bus error during IOTest. The ASICs will detect a bus error during the Iopostl phase and signal a bus error during IOPost2. During Err2 the ~es;gn ~r,cl board will go broken based on the fact that it has d~t~ d bus errors during the error s~uen~ and no other boards went broken after the error seq~l~n~'e Backplane Open Etch ~ fault site 35 ~ break via all,itl~al~ broken When the fault occulled and who was driving the info bus during the fault are not l~lc~ant issues for this class of fault. During Postl of the ~ r~, some ASICs on the bus will detect a bus error and drive bus error during Post2. Both ASICs on the I/O board will detect a bus error during CPUPost. The CPU ASICs will detect a bus error during IOpostl. During E~r2, the designated board will go broken based on the fact that it has d~ d bus errors during the error s~~ and no other board went broken during that error s~ue l~-e.
Backplane Short ~ fault site 35 ~ break via ~l,;LI~ broken W O97/46941 PCTrUS97/09781 When the fault occurred and who was driving the info bus during the fault are not relevant issues for this class of fault. All ASICs on the bus will detect a bus error during Postl and drive bus error during Post2. All ASICs on the bus will detect a bus error during CPUPost and IOPostl. During Err2, the ~e~;gn~l~ board will go broken based on the fact that it has dete,ct~ bus errors dunng the error se~u~nre and no other board went broken during the error s~en~.

I/O Board Faulb Input Circuit ~ fault site 36 ~ break via loopback on control This fault deals with a fault in the input section of one of the I/O Board ASICs. For this particular fault, it is irrelevant who was driving the b ~kpl?ne when the fault was detected The faulty ASIC will detect a bus error during Postl. During Post2 of the transfer, the faulty ASIC will drive bus error and the other ASIC will not, causing the board to go broken. If it was a CPU initiqte~ re~quest the opelation will be retried by the CPU with no error after the error se~uence. If it was a request inhiqt~l by the I/O board, then the request will be dropped.

ItO Board Output Circuit Fault - Buffer to Pad ~ fâult site 37 ~ break via htouri.~tic. broken This is a fault in the output section of the I/O board ASIC. This class of fault results from an output driver circuit fault that blows in a manner that causes an internal open between the output driver and the ASIC pad, while not disrupting the filn~ionqlity of the input receiver.
All ASICs on the bus will detect a bus error during Postl of the transfer and drive bus error during Post2. No error is ~etected during the CPUPost phase. The I/O ASICs vill detect a bus error during IOpostl. During Err2 the I/O board will go broken based on the fact that it has detected a bus ermr when it drives the bus, no bus error when the CPU board drove the bus and the CPU board did not go broken after its error se~ue~e.

VO Singl~side Access - ASIC Parib Gen. Fault ~ fault site 38 ~ break via internal parity g, nt-,lt~l broken A fault in the parity ~n~ Lion logic during single side q,~ cçc~es (data driven entirely by either the C or D ASIC) could cause the system to bus error forever. The othP.r~i~le ASIC
doesn't know the data, and has no way of cl'f~~ e the parity. For this reason, the info bus parity gel~ .,O~ n iS duplicated and selÇcl.Pr~ e inside of the Gambit ASICs.

I/O Single-side Access - ASIC PCI Data Path Fault ~ fault site 38 ~ break each PCI slot due to ~hPrL~.J.., and may RAM errors This is a fault within the ASIC's PCI data path. This hanl~ is not rul.ni,.g in locl~t~p with the other side of the board and I~ erol~; is not self-cl-~r~ . EventuaUy bad addresses produced by the PCIB wiU cause map RAM errors and/or data clle~ errors in the CPU
ASIC. The CPU ASIC noACKs q-rcesses that cause map RAM errors, and the PCIB setsoff-line the PCI slot that o.;~ ed the noACKed cycle based on an option bit in the Co~fig~ n l~ in page zero of SAM col~aLil)le I/O space. EventuaUy, . U PCI slotshqn-llP~ by the detective ASIC wiU be set broke. In the mpqntir~e co,.ul~d I/O data is dete~tçd by che~ ms and hqn-ll~ by _igher level ~n,locols.

I/O Non-singl~side Access, Different C-D Data ~ fault site 38 ~ b~ak via loopback on dat., , ., This fault deals with an int~rnql I/O board fault that results in dilr~ l data being driven out of each Gambit ASIC during a regular (not single-side) read return. Each ASIC will disagl~,c with the data driven by the other side and the board will break with a loopback on data error. The error se~ will be ~Y~ ted and the operation will be retried by the CPU and get noACKed.

I/O Board Open ~ fault site 39 ~ break via loopback on control This fault deals with an open, either from a broken etch or a lifted pin on the I/O board.
The routing is very u~o~ l for this class of fault. The etch between the C-side and the D-side should be routed Il~ u~ll the conl-P~Ior pin. This limits the possibility that an open on the I/O board is mi~tqken to be an open on the b-q~ qn~. The driving ASIC sees no errors and the outside ASIC asserts bus error; the board breaks the following phase with a loopback on control fault. After the error se~luence the oper~~ion will be retried with no error and get noACKed.

I/O Board Short ~ fault site 39 ~ breakviaa,l,:h~u~ broken This fault deals with a short on the I/O board. When the fault occurred and who was driving the info bus during the fault are not relevant issues for this class of fault. During Postl of the transfer all ASICs on the bus will detect a bus error and then drive bus error during Post2. All ASICs on the bus signal a bus error during CPUpost and IOpostl. During Err2, the designated board will go broken based on the fact that it has detected bus errors during the error S~l~ nce and the other board did not go broken after its error se~.,c~

Transient Fault ~ fault site ~ywh~,lc ~ action depenAing on fault site and timing This fault deals with a ~ fault. A l- ~ n~ fault is defined as a fault that is detect~d but cannot be l~,~uduced by the error se~ . If the fault is on the driving board and h is caught by loopback check, that board will go broken. If this is not the case then no board will break until Err2, then the ~1P~ t~3 board wil~ go broken.

Slow Driver Fault ~ fault site 34, 39 ~ possibly break the wrong board with a loopback on control fault When an ASIC with a ,.,~ginally slow driver drives the bus, four ASICs clock in data from the net while the net is ~ n~ g, Due to dirr.".,.lces in speed between ASICs from dirr~
lots, its pcss ' ~ that some of the ASICs will detect a bus error, and some won't, resulting in a loopback on control fault. This may result in one or both of the boards on the bus going broken.

Board Not Broken Generation The board_not_broken_out signal is ge l~ .t~~:d by all boards. If a board is going to break the normal operation is for the board to assert bus error. All boards will enter the error seq~lense and the board that was going to break will de-assert board_not_broken out during the error se~ e. It is possil~lc that a board could break without asserting bus error. This could only happen if there is a problem with asse.~ g bus error on the d-side gate array (for inst~nce the clock line going to the flip-flop that p~.luces the control signal (bus error) opens).

Arbitrary Breaking There are a series for cases ~ ~ before that error logic cannot .1~.. ;.. f where the fault is. For these cases the goal is to not use the bus that the two boards are co~ led to.
The default is to break the CPU board (~1Pcign~ board). This makes sense because there is a partner lUlL~h~g in lock-step so no CQ~ P~ ity is lost. It is poscih1~ that the CPU board will break but the fault really lies with the PCIB. The only way this wiU pose a problem is if the failure in the PCIB effects the other bus that is still c~nP~ed and is being us d.
Normally it is the CPU board that is the designated board.

Xbus Fault Tolerance The goal for fault tolP~nr~ on the Xbus is for no single point of failure to cause a system crash or allow a 1~ lion to comrl~~e without ~ Q~ g the correct data. A single point of failure could be any co..~ f.-~ any signal, or any pin on a colll~ne~l.

The Xbus fault tolerance scheme ~sumP~ that one of the cc,l-lpon, lls CQ~ d to the bus is always ~ .c;~.le for hard bus errors. To this end, the design has been ~ .pl;r.f d aTound the ~ mpti-~n that when an error occurs, an orL..(~ g board, or Field R~lnrP-IhlP Unit (FRU), will be removed from service. This extends to include all buses that the FRU is connP~ct~ Thus when an Xbus fault occurs the faulted bus service is removed taking at least one FRU with it.

Both sides of the CPU board drive signals to the bus and both sides of the board ~elro"., various checks to ensure that the board is functioning normally. This is illPnti~l to the Golfbus methoclQlogy. The PC}B board behaves dirr~ lly; both sides of the board drive and check the bus when driving data from Xbus related IO legist.,l~, and when driving data orig;-.-l;-.g on a simplPY~A PCI card, only one ASIC drives and checks the entire width of the bus.

Info Bus ~.~ ~ti~n The info bus is ~,ot~ted by parity and loopback rllf~ g. However there is no A to B bus cross chr~ by anyone on the bus other than the bus master. When a PC IB is bus master only one of the two buses ~ h~ to a given CPU module will be active because the PC~s do not duplex and th~ ., the bus from the PCIB that did not win ~billalion must be idle.
Likewise the PCIBs cannot know if two du~ ed CPU modules are driving the bus or a single simplexed CPU has o..-~el~hip (two s;~-pl; ~ff~ CPUs could have started ~billalion at the same instant). Thus receivers on the Xbus will only listen to the bus that is driven by the module that won ~I,;~

-CA 022575ll l998-l2-03 Parity Parity is g~n~-~t~A and !~ across each of the physical buses. The 32 bit bus, trid field, and funcop bits are all covered by a single parity bit. The yu~osc of the parity check is to protect against etches that open bcL~ the tr.q-n~ and receher. Boards in the system check the !.,..~ paIity th'at they receive with parity that they g~..f.iur thP,m~,lveS.

A bus error is ~i~qlP~ on any parity error. The bus err signals are also effectPA by other error chfrL ;-~e that is ~es~ d later in this do~ 1 In an effort to sihl~ylily these de.,~ ion~ the reader may assume that the actual error signals are a logical OR'ing of the various outputs from di~ nl chfr~; ~e blocks.

Both the C and the D-sides of the boards will p~ru~ the check and con~a,c results individually and decide on any action that must be taken. In the event that the two sides of a board do not agree upon the statute of a bus cycle, the board will break in the following cycle in one of two ways.

~ The D-side of the board ~ e~i~d an error and the C-side did not. In this case the D-side will drive a bus error and the C-side will disagree with what the D-side has done and break the board. Here the trqnC ~tilm is errored and repeq~, but the broken board should no longer be ch~L ;~g and the tr.qn~ ction will complete.
~ The C-side of the board d~tc~1~d an error and the D-side did not. Here the C-side will once again break the board, but this time b~.se h e~ to see an error on the bus and didn't.

The trqn~~fi~n will co~ l le normqlly and the bad board will be removed from service.

CPU boards co--~ e parity in both C and D ASICs and break if a defect arises in one side's parity E; ~e~ "~. Most of the transfers from a PCIB board, however, ol;~ind~ from a PCI
bus con~ ed to a single Gambit ASIC. When the Gambit drives this single sided data, it CA 022~7~11 1998-l2-03 wo 97/46941 PCT/USg7/09781 drives all the bits on the Xbus, and an error in the simplexed panty E,. ~ "Qr would hang the system. Ful~unalely, there is no need to dUF~ A the entire Gambit ASIC, only the parity ~"...-.,~;nn section.

R~Çe.ling to Figure 12, a fault at site 41 causes e~luilcous data to be ~ nc -.;~ with correct parity. This will be det~te l by higher level c~ L~-~-.-c. A fault at site 42 will cause the board to break with a data loopback fault. A fault at site 43 will cause the board to break with a parity ~ n~ l fault. A fault at site 44 or 45 will break the board with either a data lO.,~back fault or a parity ~en~lor fault.

Loopback C~- -ki.l~

The loopback CGll~~ ity is shown in Pigure 13. On normal acces~es (non-CD dilre.enl, non-single sided), each side of the board drives half of each field to the bus and receives all of both busses. Thus there are two checks that each ASIC can do on the data ~.,lu...;..g from the bus.

1: Does the ~ g data match the data that was driven from this ASIC. This will be known as the lnopb~ r drive check.

2: Does the l~l~JI~ g data match the data that was driven by the other ASIC.
This will be known as the loo~back CGlul)~; check.

Two error signals are generated inside of each ASIC for each physical bus that it is driving.
In order to know the tme state of the board each side of the board must know the results of the C~...p3.;SOnC on the other side as well as the cr....p~ Qnc it has done. Thus two signals are passed in each direction, as shown in figure 13, so that the each side may coll~ ly ~,te-...;-.e the state of the board.

.

CA 022575ll l998-l2-03 Each ASIC will d~ wl.~lh~ the data on the bus is faulted or not, as well as if this board is broken or not. Table 4 below shows what co.~ ;( n~ of the loopback signals for each bus inrlir~tr that the board is broken.

Table 4. Board Broken Matrix for a Sin~le Bus drive err from comp_err from my drive err my comp_err the other side the other side brd broken of the board of the board O O O O O

O O

CA 022F,7., 1 1 1998 - 12 - 03 A double bus error will be as~ t d and the not_broken signal de-asb~.led if a condition exists that should break the board. In addition a bus error will be asserted for any condition where a co~ ale error of any type has occurred and the board has not broken. The bus error will only be on the bus while the error is dete~tP~I

Loopback on CD Different ~('CPReÇe When an ASIC drives data from a CD ~rr~ register on the bus, it pe~ s loopback ~hPrlring only on the trid, func_op_, and parity lines. It asserts bus error if it sees an error, and will go broken according to table 4.

Loopback on Single-side ~(~rP~ee.e When a Gambit ASIC drives single-side data from the PCI it p~rOlllls loopback c-h~r~ing on all of the bits; it does this only for the ~ul~ose of asserting bus error, not to see if it should go broken. It will not break according to table 4, because the cmp_err and drive_err signals from the ~lhf~ e are invalid. However, if the driving ASIC sees a bus error, and the otherside ASIC does not (parity received good), the board will break due to a loopback on control fault.

Loop_ck_ops Loop_ck ops are used during the error sP~uPnre to make intPlligPnt choices about the state of various boards and the buses that they are ~~tt~l-hed to. A loop check op is a cycle in which a board drives an ~11P-.~ pattem of AA's and 55's on the b~pl~ne. The board evaluates the pattem and similar pattems driven by boards on tne other end of the bus. Based on this ih~"..-~ion, bus state is altered.

... .

Control Bus ~ote~ion The Xbus collects control signals into a series of point-to-point unidirecti(mq1 signals plo~ ed by an ECC code. Each board drives a S~ ualt; control bus to each of the other boards.

There are two completely indepe~d. -~1 types of checks pe.~-n~cd:

1: Every ASIC col~ for any single point failures (shorts, opens, dead drivers or .~;~e.~) with a single error coll~ling, double error delec~ g ECC code.
Single bit failures cause a I~U;~tr~ t~llu~)l and are logged, but otherwise have no effect.

2: If a driving board detects a double bit error on the oul~oing control signals it will attempt to assert bus error and then it will break in the error se~rnre (itwill break regardless of its ability to drive bus error). If a receiving board detects a double bit error on any i~ in~ control signals it will attempt to assert bus error then it will break in the error s~lu.-~.r~. If the error was actually on one board and only one board saw the error then the correct board will break. If the error was such that both boards saw it then both boards will break. This is to limit the pos~ihi1ity that data gets cc.ll~ted.

Three-way Voted Signal Protection Some broken and reset related control signals on the Xbus are tr-r1ir~t~1 and voted. The reset and board not broken_out signals are burr~Gd through lla"sceivers because they must be valid during power up or power down events. The CPU signal sync out is buffered through t.a"sce.~ers to reduce ASIC pin count. The CPU signal my online_out is sourced directly by the ASIC as a cost reA~.ction.

The D-side ASIC drives signals to the ~ nc~:~ers and to the C-side ASIC. The C-side of the board checks what was driven by the D-side with what it thinks should have been driven.
If the C-side disagrees with the D-side the board breaks.

Each ASIC that receives a three-way voted signal will pe~ru~ a ma~ority voting al~oliL~Lu, on the three signals to d~ .;. r the value. In normal operation all three lines will be the same. If there is a fault on one of the three lines (either on the ~ nF or on the board), two of the three lines still will agree.

When a voting error occurs, the device de~ g the fault should send a ...~;.,t~ n.~
int~,.lu~ and log which ll~s~;~,er has the non--,n~ ous vote. The logged inro....~ n is stored in the Vote Enor Transceiver Status l~,gi~lt;l(s) and will not be over-written until tne status l~;sh~. is ~ lici1ly r~ ~ ~b1-~ by s-rl-.~e.

Error ~'~r ~

A ~lu1llber of errors can be ~PtPrt~d by a board's bus intP,rf~ce logic:

l: A loopback error that breaks the board. This ~erifir~11y refers to loopback errors on the info, parity, TR~, and func_op signals which are seen by one side of the board and not the other during bus cycles in which the board drove the bus.

2: A disagl~.lle~ the two-sides of a board on wh~lL~.r a l~ .dy voted signal should be driven.
3: A parit,y error on the bus, either on a cycle where the local board was driving the bus or another board was driving the bus.
4: A loopback error on the bus.
5: A three way voter error in which not all three signals are in ag 6: Any bus error ci~lP~i by any board in the system.
7: A thermal fault (~ugh t - ~ etect~P~d on the board).

Some of the above errors may break a board (e.g. ,Y31 and 32), some may cause a bus error to be generated (e.g. .~Y33,34 and 36), and some have no effect on normal m~hinP. ope~3tisn (e.g. #37). The Xbus intPrf: ee co.~ two classes of status l~i;~t~.;j to record i lru~ l;on sullu ~ ~ the above errors: Broken Status and ~Tor Status. Broken Status l~ iSt~ are only activated when a board goes broken, and are deci~Pd to p~hll the logic that set broken. Error Status l~ist~ are for l~ liUg non-fatal errors such as faults on the All of the Error Status legiste.~ are ~I'd-l~d cimlllt~neoncly on any bus error ci~n~lPd by the system, or a voter or thermal error on the board. Specifically, when all error ~ lg is enabled, any of the above error cause:

1: A l~ t~ n~'~ illt~lU~ to be g~ t~,~
2: The type of error condition to be stored.

In the above case, sLbs~u~nt error state ~co~ g and the resulting ll~ lrl~nt'~e hllt;llU
are disabled until eYrliritly re-enabled.

It is not always optimal that all the above errors trigger the error l~;iSt"~ and ~e-~,-,.~P. a "~ n~nne i~ ul)l. The l~ol~in~, of new errors can be effectively ~lorl~P~ by an excessive number of reports on a known error. Hence, it is posc;bl~ to turn off error latching due to dirf~ nl suu,ces:

1: Pe~rul u error l-t- h;ng and m~;..t~ n~n~ e hlt~,.lupls on all err~rs.

2: Inhibit error latching and ~ ;nlf ~nce i~ u~ls of incoming three-way voter errors.

3: Inhibit error l~ ;.,g and ~ t ~~n~,~e int~lu~ls on thermal failure related errors.

The error l~rl.;..~ &ble bits inhibit those faults from ~ ~ ;gge- ;,.g the error registers.
However, if a ~ error is present and non-disabled error occurs, both errors will be eco..led in the error l~gi~t~

Producing Errors The Cyclops and Gambit gate arrays have specific registers which allow software to produce various errors on the Xbus.

The basic logic to p-u~ce an error consists of XOR gates on all the info, trid, func op, parity, control lines going out and all the same lines coming in. One input on the XOR is the data line the other input is the error line and the output is the res~lt~nt data. If the error line is high the data coming out will be in error. If the error line is low then the dah coming out will be in the for n of a loopback error. If the data is errored on the input and the output then the error will be in the form of a bus error since it will not be a loopback error.

Table S shows all the error cases and the nP~$~ settings from a h~Jw~c point of view to produce the error. 'Receive error' means the XOR gate on the receive side is inverting.
'Transmit error' means the XOR gate on the l~ .c..~;l side is inverting. The cat~oly descrihes if the error occurs only on a particular pattern match or if the error occurs just after the register is written. For many cases it would be possible to produce an error with CA 022575ll l998-l2-03 dirf.,~ cases than those llesc~ ;he~ below. The goal is to only plodu2e the error in one manner.

Table 5. Error .Settin~
CPU Board ~or Type of ETror Category D-Side C-Side CPU Board Faulty Input Circuit - CPUTmm~liqte receive error no error Driving CPU Board Faulty Input Circuit - I/OData Match receive error no error Board Driving CPU Board Dilr~.lt Data C-Side and D-Immediate fr.q-ncmit error no error Side /receive error CPU Board Faulty Output Circuit - Buffer TmmPAi~~ trqncmit error no error to Pad Fault CPU Board Open - CPU Board Driving I~ ,eliate no error receive error CPU Board Open - I/O Board Driving Data Match receive error no error CPU Board Short Immediate 1~ error no error Backplane Error Type of E~r C~t~,oly D-Side C-Side Rp,rlplqnP Open ~tch Data Match CPU and IO CPU and IO
board board t~,..-~...il error receive error /receive error R~q~rl~lqn~ Short Immediate CPU and IO CPU and IO
board board I-,..~.~...il error no error CA 022S7Sll 1998-12-03 WO 97/46941 PCTtUS97/09781 IO Board Error Type of E~ror Cat,~ y D-SideC-Side I/O Board Faulty Input Circuit T.. ~ P receive error no error I/O Board Output Circuit Fault - Buffer to Immediate !~,-n~-il error no error Pad I/O Single-side Access - ASIC Parity T.. ~ , parity gen. no error Gen. Fault fault VO Single-side Access - ASIC PCI Data Tmm~ P bad address no error Path Fault VO Non-single-side Access, Dilr~nl C- T.. F,l;~t~ ~.,.n~.-.il error no error D Data Ireceive error I/O Board Open T~ .Pr1;~.~ no eIror receive error I/O Board Short T.--n.~ ,te ~-, n ~.il error no error Tlansienl Error T.,.n?;~ Fault This causes a bus error, but when the error se~ is ~ u~l there will be no error.
This fault wil} not l)luduce a maint. int. from this board, but all the other boards will assert maint. int.
Bus Busy Bus Busy This causes a bus busy.

Board Breaking Timing Note in the figures Figures 14 - 17, Info_O's shows the time the info bus is driven to zero lming that the bus is not already triCtj~ed from this side, xcever disable shows the time the drivers from the broken board are disabled from the bus. Bus error is also shown when valid on the Xbus.

Board Breaking and Information Latching There are several status bits in the Broken Status register on each board to ~3el~ --n~ the reason a board went broken. These bits are frozen after a broken COndiLiOIl occurs and will remain frozen until after a cold reset. All bits in the common leglst.,l are straight ~~
with the eYc~ption of board speci~lc set broken. Board specific logic may have several reasons to assert this signal and may also have a similar register for the reasons it has asserted this signal. If board_s~ r~_broken is set in the common register, the board specific broken register will contain the real reason the board asserted the board spel if i- set broken signal. If a board breaks for a reason other than board specific broken, the board ~l,e- :rc_broken bit will not be set in the broken status register, However, board spe~ if ic set_broken may have been asserted after the original broken condition. This would have caused the board specific broken status register to latch and hold a reason it had set broken. It is ~ )oll~l to understand that becal.se the board sp~ifi- broken bit is not set in the cornmon status register, this was not the reason the board went broken. The reason the board went broken is logged in the broken status register and the board specific status register may be ignored.

In the timing (lia~m of Figures 14 and 15, someone_set broken is the logical OR of the following signals:

cold reset, wann reset, slot parity_error_set_broken, control or_3way vote sig error, asic specific set_b-ol-~n, ecc_dbl_error_in. In addition Gambit contains the following signals: break pcib, xb parity_gen_set_blo'-~n. The board will break if any broken signal is asserted on any phase (i.e. any 48mhz edge). my_set broken_reg is the equivalent of otherside set broken_reg on the other ASIC. my_set_broken reg is ORed with otherside set_broken_reg to create set_broken_sync.

The commnn broken status ,~ister is frozen when latched_broken_status is true. Note, All status signals for the I/O register have the same timing as broken.

Note the bus error driven on the bus at the time the board went broken. This timing is Ol ~1~.

Board Breaking Timing on lnfo Loopback Error As shown in Figure 16, if a loopback error occurs that will break the board, bus_error will be aas~ d in POST2 and the board will break in ERRl. Both ASICs will know of the error be~ e of there are four lines for each info bus that co~ r~te loopback status between the ASICs (cmp err_in, cmp_err_out, drive_err_in, drive err_out.

Board Breaking Timing on ~Ieuristic or Arbitrary bn ~ n-As shown in Figure 17, boards that break due to a ~e~ri~i~ or an Arbitrary broken will participate in the entire error seJq~en~e and then deassert board_not_broken in ERR2.

. .

CA 022S7Sll 1998-12-03 Xbus Physician Partitioning This section gives a high level overview of the physical partitionine for the int~ ~e ASICs to the Xbus. It d~scrihes the co ..po.~ ~e5C-~y to in~rf~ce to the Xbus as well as a r-...~ n~l des~ n of how the in~ ~e is ;..t,..~ to operate. The fault tolerant ~l~at~g~
is also desc.il~d Info Bus Partitioning As shown in figure 18, the C and D ASICs on the CPU board each drive half of the bits on the info bus when they are bus master. The C and D ASICs on the PCD~ drive either half or all of the bits, depf~ g on the type of ~a. sr~r. During normal ;Icc~es, they drive half the bits as shown; during single-side ~r~s, they drive all of the bits, in~ ding the parity bit.

Control Signal Partitioning The D-side ASICs drive the control buses, and both C and D-side ASICs check.

Xbus Voted Signal Partitioning The 3-way voted signals in a Polo system have varying topologies ~le3~en-iing on funt~ti-m In genP~al, each ASIC on a board drives half of the 3-way voted signals o.;g;~ l;..g on a board, and the other side ASIC sits at the end of the net and checks what is driven.

Figure 19 shows the 3-way voted signals for board not_broken routing. Each hansoei~
section (shown as a NAND gate or i~ ) of a 3 way voted triplet is in a .~,~ ..lr.
p~ ~e, to prevent a single failure from di~lulb~g more than one bit of the triplet.

5s .

CA 022575ll l998-l2-03 W 097/46941 PCT~US97/09781 Signal Routing The bidi,~lion~1 info buses are routed point to point b~L~n boards. On each board, 5~-"~l~ traces run from the con.~;Lor to each ASIC; this allows the error protocol to del~ f if a fault is in the b~ n~ or in the board. Series l~,sislul~ are used to control signal quality, nd CMOS levels re used for inc.eas~d noise margin and reduced ~usceptibi1ity to cross-talk. Pigure 20 shows the routing of info lines.

As shown in Figure 21, the three way voted signals are routed as se~ point to point conn~r,ti~nc The reset_x,~_,z_ signals are driven by the CPU boards, wire-ORed on the backplane, and ~ ;ved by all boards. The board _not_broken_ signal is driven by a single board and received by all boards. These signals are drive by norrnal 4mA drivers of the ASIC, and l,urr~l~d by 26S10 I-~lc~e;-~ers. The voted signals are t~-.ni~ ~1 with a pull-up and i~~ n diode on both the receiver and driver side. This means that it is possible for there to be one or two pull ups on the line. However, whenever the line is being used for activity ~ignq11ing, there are exactly two If ~ , one at either end of the line.

Fuoctional Description Figure 22 is a block rliq~m of the error check logic used by the ASIC's. The Error ~hP~ki~ and P~girt~ring logic p~r ~llUs two basic fUnctions It controls most functions or error ch~- ~ ;ng and it ~ ;n~4~ E and oul~oing Xbus signals. The error chro~ ;ng ru~lC~O~ Çululs the fo~owing tasks:

~ ECC g~ ,d1;o~ and ch~rking/co,l~ting ~ Parity generation and l.hf~l~;ng ~ Loopba~ ~hfr~ g ~ Thl~-~.ay voting ~ Controls error IJlOlocol (inr~ es det-ici~-nc on breaking board) -~ Provides logic for error insertion ~ Records error illrO...-~I;o.~

The ~ u-;~ logic p~o~ s the following t sks:
~ ,~ist..~ g bus signals uulgolng bus signals ~ comhin~s (logical 'or') n~igl-bor, ol)posi~, peer versions of l~ gi~ ;d ECC coln~t~d control signals coming into the ASIC with the delayed oul~g cc ~ ondi~g sign. l to produce a Cf'~ f~l control signal for int~.rrql ASIC module use The Xbus can be broken into four classes of signals when ~ clls~ing error cl-P~ g:

Xbus Signql~: these signals are ~l~oteeled by one parity bit, this detects an odd number of errors. During normal ~rC~cces the D-side gate array drives applok;...--rly half of the signals and the C-side drives the other half. Please refer to Section 11.1 on page 96 for more detail on the signals that are driven by D-side and C-side. The D-side checks the data it has driven (loopba~ drive check) and the D-side checks the data that the C-side had driven with the D-side would have driven (loopback con~p~ check). During single sided ~~cess~.s (PC~3 only) the driving ASIC drives all of the bus signals. The driving ASIC can do a loopback drive check but the other ASIC cannot do a loopback co",l,a,e check because it has no knowledge of the exact data being driven.

Control Bus ~ these signals are pl~t~led by four bits of ECC, this detects a double bit error and can correct a single bit error. The D-side drives the signals and the C-side does the loù~a~1~ coll,p~e check. The D-side does the loopl)d~ drive check.

Voted signals: these signals are triplir~ted and protected by three-way voting, tbis allow for one out of the three lines to be in error. The D-side drives the signals and the C-side does .

WO97/46941 PCTrUS97/09781 the loopback cc,l~ , check. The D-side does the loopback drive check. The one çYcepti~n to D-side driving all signals is the board not broken_signal. The C-side will drive the board not_broken signa~ and the D-side will drive the enable for the board not_broken signal.

Mi~cell~nPous signals: The slot id signals are duplicated and they are in disag,cen,~ the board will break.

Figure 10 shows the Bus Error flow chart,, ~ g the fol~oing. A further un~ I;n~ of the invention and of the constluction and operation of the illllstr~tçd embodiment may be ~tt~in~l by l~Çe~_nces to the Appen~lires filed h.,.~iwlh.

Summary Descril)ed above is a digital data proces~ing system m~ting the aforementiQn~d goals.
Those skilled in the art will a~l~cide that the illll~t~trd enlho~limrnt is but one example of the invention and that other embo-l;--.P~ incollJol~ling mo~lifir~tions thereto fall within the scope of the invention, of which we chim:

W O97/46941 PCTrUS97/09781 This document consains confidenbal and p,u,~liet~,y infonnation of Stratus Computer, Inc., and any reproduction, rlicclo~Jre, or use in whole or in part is e)~,esslyprohibited, except as may be specifically authorized by prior written ay-~:e",~nl or pc---,;~ .;on of S~atus.

Polo Programming Guide "A SAM is a logical cu.,ce, t"

Jeffrey Somers Gregory Green Michael I lo.-,Lerg Conrad Clemson Will Leavitt Joe Lamb 26 December 1995 Revision: 3.2 Notice:
Released Version (c) Stratus Computer, Inc. 1995 Appendix I ~ ~

.

Polo So1tware Progran ning Guide Stratu~ ompany Confiden~ial 1. Revisi~n History .................................... 7 1.1 Open Isc~s - 7 1.1.1 Polo UPS communicaffons . - - 7 2. Intror~ucti~n......... .- ..... ~............. .................. 8 2.1 Applicable ~)ocuménts . - 8 2.1.1 Stratus Applicable Dowme~tc 8 2.1.2 Other Applicable Documents - 8 2.2 Te minology - .- 8 3. Functional Overview... ~ ...... ........ ~... .... ............. ~.- - .. ~ 10 3.1 Polo ~unctional Overv~ew 12 3.1.1 Logical Summary t2 3.1.2 Physical Summary . 12 3.2 Je~alFTIO Functional Overview 15 3.2.1 Logical Summary . 15 3.2.2 Physical Summary 15 3.3 Jetta/HSC2 Functional Overview ......... 16 3.3.1 Logical Summary. 16 3.3.2 Physical Summary ~ 16 4. Xbus/Gol~bus to PCI Address Mapping ................ 17 4.1 Add~ssi. ,9 Ovs~view ~ . . 17 4.2 Address Re-mapping . ..............18 4.3 Per-Space Ordering of l/O Arr~esses. 18 4.4 Address Decode 18 4.4.1 Per-Slot l/O Space (Local, Paired, Non-Paired) ~n 4.4.2 Global G~o~
4.4.3 IllegaUAbnormal l/O Access.~3s. , . ~1 4.5 PCIB MlO/lOBus Compatible System Address Map 4.5.1 lOBus or Xbus to PCI Local Bus Requests . ~ - -- -24 4.5.2 MlO Compatible Register Space .. - ~ ~5 4.5.3 lOBus Compaffble l/O Space Msp . - ?5 4.5.3.1 SAM l/O Address Space - ~6 4.5.4 lOBus Compabble Memory Space Map . -. --~
5. Xbus/Golfbus PCI to XbuslGolfbus Address Mappin~ .. .. ............ 295.1 Software View of Re-mapping 79 5.2 PoloRe-mappng , , 79 5.3 PCI Memory and l/O Address Mapping 31 5.3.1 PCI to Xbus/Golfbus l/O Reques~ ~. 31 5.3.2 PC1 to Xbus Memory Requests 32 5.4 AddressRe-mapping 33 5.4.1 System Address Generation~ . - 33 5.4.1.1 Polo System Address Generation .... 34 5.4.12 HSC2 System Address Gene,_ Gn 35 5.4.2 lO Address Table Entry Invalidate . 35 5.4.3 Address Error Checking. ....... 35 5.4.3.1 lO Device Address Checksum ~ - 36 5.4.3.2 Outof Bound AccessCher,~ .~ 36 5.4.4 Control ~its.. -- . - - - . ~ - 375.4.4.1 Valid Entry bit .. -- ~ ~ - 37 5.4.4.2 DataPre-read...... - - ~ 37 29 Dece,.,~r 1995 2 ~'o .
_ .

CA 022~7~11 1998-12-03 Polo Software P ,gramlmng Guide S .dtUS Company Confidenbal 5.4.42.1 Cohere:h. y Resl.i~.~ons . 37 5.4.4.3 Lock Cycle. 38 5.4.4.4 Swap 32 bit Endian - 38 5.4.4.5 Incohe~enl Memory Access 38 5.5 Software Map Manag~",enl 39 5.6 ByteAlignment 39 6. Interrupts ......................................... 42 6.1 PCIB/HSC2 Interrurtc . 42 6.1.1 Introducbon - 42 6.1.2 Implementation 90AIC 42 6.1.3 l la,~~~e Implementation ............ 42 6.1.4 Sug~c~ Software Imple."e"t~tion 44 7. Boot................................. .. . . ...... ... 45 7.1 Boot Altematives 45 7.1.1 Baby BIO Boot .. 45 7.1.2 Bio Firmware on Raid Controller Boot 45 7.1.3 CPU Prom r.~ : ~nl PCI Card Boot. . 45 - 7.1.4 E,.~ns;on ROM Resident PCI Card Boot. . 46 7.1.5 x86 Emulator PCI Card Boot. . .46 7.1.6 PCMCIA Flash PROM Based Boot . - .. ..... 46 7.1.7 Conclusions - 46 7.1.8 Open Roo~ 47 7.2 PCI BootProcess. 47 7.2.1 Find and Configure the Boot SAM 47 7.2.2 ConfigurePCMClA Bridge Chip 48 72.3 Configure PCMCIA Bus 49 7.2.4 Configure PCMCIA FlashCarl . . 50 8. PCI Configuration ...... . . ... ......................................... 51 8.1 PCI Configuration Register Sp~ce 51 8.1.1 Gambit PCI Configu~ a~i~ n S~r~ . 51 8.1.2 PCI Adapter Configuration Sp~ce 53 8.1.3 PCI Configuration space header descripbon 54 82 P,~posed configuration sequence. 56 8.3 OS Based full system Configuration 56 8.4 PCI devire detection 56 8.5 PCI Address forrnation 5 8.5.1 Configurabon cycle generation 57 8.5.2 Special cycle gt:"e,~,lion - - 57 8.5.3 PolotHSC2 Configuration Address Generation. 58 8.6 Multi functionAdaptors 59 8.7 SAM and PCI Adaptor Configuration 59 8.7.1 Base Address lle!J;;,Ia,~................ 59 8.8 PCI Address Space - . 60 8.9 PCI Bus Target and Master At)orts .60 8.9.1 PCI Bus Fault Dete~ tion and Tolerance 61 8.10 On Line Adds 61 8.11 On Line Failures. . . 61 8.12 Board removal 61 8.13 Special Requirements 61 29 De~.,l~e~ 1995 3 6'1 , . .

CA 022~7~11 1998-12-03 Polo ~ P~ugldl""lingGuide Stratus~ )mpanyConfidenbal 8.13.1 Lock requirements ..... ............ ....... 62 9. PCIB Maintenance & Diagnostics ... 63 9.1 Overview ~ . . 63 92 ID Proms - , . 63 9.2.1 Fault Logging . , 63 9.2.1.1 PCI~ . . ......... 63 9.2.1 ~ HSC2..... . 64 9.2.2 ItO Power Supply Faults . . 65 9.3 Inse,~on ~ Removals... - 65 9.3.1 PCIB devices - ~.. 65 9.3.2 Polo PCI devices 66 9.4 Reset Overview . 66 9.4.t Polo Rese~ - - 66 9.4.1.1 CPU Peset - . 66 9.4.1 2 PCIB . . . 67 9.4.1.3 PCI cards.~ . . 69 9.4.2 1~SC2 Resets 70 9.4.2.1 HSC2. ~-- - --70 9.4.22 PCI cards. . 71 9.5 Detemlining ~he source of a Maintenance Interru~ -~ -- 72 9.6 Faults&Errors . . . 72 9.6.1 PCI card faults . 72 9.6.2 PCI busfaults , - - -- 73 9.62.1 PERR#- 73 9.622 SERR# - - 73 9.62.3 Polo(Gambit)/HSC2(Mirage) Error logic.. - 73 9.6.3 Disk bus faults - - - 76 9.6.3.1 LED Initialization .. - ~ .. 76 9.6.32 DiskLED Control . 76 9.6.3.3 Disk ll,se.lion and Removal - . 76 9.6.4 SCSI bus faults ~ - - - 77 9.6.5 PCle/HSC2 faults . - .......................... 77 9.6.6 Xbus/Golfbus f~ . 77 9.7 Diag,ID~ L s .................................. 77 9.7.1 PCI Card . 77 9.7.2 PCIB . 77 10. ReCC ~unctionality on Poio ...... ........................................ 78 10.1 Remote Power control . 78 102 RS-485 bus 78 10.3 Powerfail . ., . .. 78 10.4 Cl~clc~rd, bacl~al-el power supply failures -78 10.5 I/0 power supply failures 78 10.6 NVRAM , 78 11 . Board States.......................................... 79 11.1 PCIB Board ~t;~ 79 112 HSC2 BoardStates.............................. ~ - 79 11.3 SAM Board States .. . ....................... ..................... 80 12. Register Definitions ..................................... 82 12.1 XBusJGolfbus Bus Interface Registers .. . ............ 82 29 Dece"~ 1995 .
~2 -. ~ . . ...

CA 022~7~11 1998-12-03 Polo Soflware P, ~rammmg Guide S .dtUS Company Confidential 122 Reading Registers with Different CID Inforrnabon (Polo Only) 82 12.3 Common Xbus/Golfbus Register Definitions (Polo Only) 83 12.3.1 ID Prom Instr. . . 85 12.3.2 ID PROM Address Dat~ .. 86 12.3.3 Board Reset - .. 88 12.3.4 Bus Inter~aoe State go 12.3.5 Board Sync Register= 93 12.3.6 LED Control gs 12.3.7 Slot ID 96 12.3.8 Read Ping Interval 96 12.3.9 General Purpose Communicabons ~7:0] 97 12.3.10 Memory Size/Location 97 12.3.11 Test Control 97 12.3.12 Bus Intsrfaoe Fault Reporting 98 12.3.13 Common Broken St~' ~s 100 12.3.14 ASIC Specfflc Broken Status - 101 12.3.15 Bus Info Error Status . . 102 12.3.16 Misc. Error Status . ... 103 12.3.17 Control Bus Error Status 104 12.3.18 Bus Error Byte Status . . 105 12.3.19 Voter ErrorT,ansceiwr Status . 106 12.3.20 Bus ASIC Chip ne. ~ ~n . 107 12.3.21 Pe,f~.. ,lance Counter . 107 12.3.22 Pe,~u,,,,dnoe Counter Trig[1:0] .. 108 12.3.23 Pe,fo.. "ance Counter Mask[1:0~ . . 108 12.3.24 Pe,f~ ,ance Counter Controi . 109 12.3.25 Fault Bit[1 :0] . 109 12.326 Data Match 110 12.3.27 Error Control 110 12.4 CPU Specific Xbus ne~;ste~ Desc.i~lions . 112 12.4.1 ASIC Specific Configuration ..... 112 12.4.2 I/O Address Map Error 1. 112 12.4.3 I/O Address Map Error 0 . 113 12.4.4 Time Of Day. . 113 12.4.5 Quicktime . . 114 12.4.6 Master Jiffy Counter 114 12.4.7 Jiffy Control 114 12.5 PCIB/HSC2 Spscific Xbus Register Descriptions . 115 12.5.1 Gambit Maintenance Attention Request [31:0] 115 12.5.2 lOBus S~ s 116 12.6 SAM Interface ne!J;stor Descriptions - 117 12.6.1 SAM Interface Registers - Page 3 119 12.6.1.1 Disk LED Control 119 12.6.12 Disk Status 119 12.6.1.3 Power Supp~y LED 121 12.6.1.4 Power Supply Status(Vanguard powe supply) 121 12.6.1.5 Fsn Speed Control 121 12.6.2 SAM Il lt~- idce Registers Page 2 122 12.6.2.1 Address Table Co""nand and IOVA Register. -122 12.6.22 Address Table Data 2 Register 122 12.6.2.3 Address Table Data 1 Register . . .123 12.6.2.4 Address Table Data 0 Register .123 12.6.3 SAM l"t~.idce Rey;ste,~- Page 1 . .124 12.6.3.1 PCI config_addr 124 29 Dece,-lber 1995 5 CA 022~7~11 1998-12-03 WO 97t46941 Polo So1tware Plug~ ing Guide Stratus _Jmpany Confidential 12.6.32 PCI config_data ... 125 12.6.3.3 Test Control - . .12~
12.6.3.4 PCI Error -- . . 126 12.6.3.5 PCI IOVA Error 127 12.6.3.6 SAM S~t~. - . - 127 12.6.3.7 Arbitration Freeze Count Max. \/alue 128 12.6.3.8 Host Request FIFO Timeout Value Register .. 128 12.6.4 SAM l,lt~,lace Registers- Page 0 129 12.6.4.1 Board Reset 129 12.6.4.2 LED Control .130 12.6.4.3 SAM Host Intsrrupt Bit 130 ~2.6.4.4 SAM Interrupt Mask- - 131 12.6.4.5 SAM Interrupt Souroe 131 12.6.4.6 SAM Host Intenupt Address Pointer 131 12.6.4.7 SAMHostlntenuptTablePointer#2 .~ 131 12.6.4.8 SAMHostlnternptTablePointer#1 131 12.6.4.9 SAM Configurabon 132 12.6.4.10 PCI IO Space Offset 132 12.6.4.11 PCI Memory Offset . 132 12.6.4.12 Bus l,~te.ldc~ State - 132 29 De~,-l~er 1996 6 ~1 Polo Software ~. Jrammlng Guideatus Company Confidenffal 1. Revision History Rev 0.0: April 26, 1994 Created.
Rev 1.0: Aug 8,1994 ~ Many updates for s ~r_.e Rev 1.1: Aug 10,1994 ~ Updates from intemal review for secbons up to interrupts.
Rev 2.0:
~ Release Version Re\l 2~: March 21, 1 9g5 ~ FTIO updates Rev 3.0: July 19, 1995 ~ llo,l~oIcd ohs~,' te FTIO ill.~ Il~Lon Rev 3.1: September 14 1g95 ~ HSC2 Updates Rev 32: SDecelll43r 26, 1995 ~ Intemal Review HSC2 1.1 Open Issues Open Polo related issues are also tracked in problem under pci_c o, ~I~ llon. This list reflects open issues that will not be closed for the current release of the specification along with a jusbfication for the issues to rernain open.
Open HSC2 related issues are tracked in prob~em under hsc2_issues. This list tracks Ul ll~sol\rod issues during the devebp,l~ellt cycle of the HSC2 project.
1.1.1 Poio UPS con"-,~nicdtions Polo will specify, support and qualify an extemal UPS solubon. The actual vendor selecbon for the UPS has not occurred yet. Only vendors with an RS-232 intelc,onnecl are being cons;clell:d. When the vendor has been selected, the secbon on UPS support will be updated to provide the communications ~pec ;~ic~bons.

26 Decelllbe( 1995 6 ~ 7 Polo SoftNare r~o~ ing Guide Stratus ~mpany Confidenffal 2. Introduction Thls specifi~tion is the Polo i-'luyld~ lling Guide tor the Polo PCIB based system. It provides both the functional model for the hardware and the soflware detailed design s~sc fiL . 'ic ns. It is intended to be the primary working model forthe plo~la"ll,ling guide, and it specifies the sofh~are visible hardware implementation.
2.1 Applicable Documents 2.1.1 S~ratus A pplicable D oc~ ts Cougar Specific~bon, JED-0005 Xbus Functional Sp~,ilication, JED-0155 Cyclops ASIC Functional Spe~fi~,ation, JED-0159 ~ Garnbit ASIC ~unctional Specification, JED-0160 ~ Polo Product SpeC;f;C;~Ii')ll, JED-0151 ~ Polo D.~ ne Specification, JED-0158 ~ Xbus Functional Specif~tion, JED-0155 ~ Mirage ASIC Functional SFe.,i~ ,n, J~D~1??
2.1.2 Other t~ppl~c~ble D oc~ ts ~ PCI Local Bus Specifi~tion F~ev-2.0, April 30, 1993 PCI Special Interest Group ~ PCI System Design Guide Rev-1.0, September 8, 1993 PCI Special Interest Group PCMCIA 2.1/JEIDA 4.1 Specif~ ;on 2.2 Te~inology Cyclops - The Polo Ibus to Xbus ASIC. This ASIC contains many of the features of the Gofer ASIC.
~ Gambit- The PCIB based PCI/Xbus inl~.ldce ASIC.
~ Mirage - The HSC2 based PCllGbus interfaoe ASIC.
G81- Gotfbus(Xbus) Interface ASIC. This is the Gofer in Jetta systerns and the CycJops in Polo systems.
~lash ram - Flash RAM in this spe~,ifi~tion refers to a PCMCIA flash ram card. The flash ram card is used for boot functionality.
~ FTIO - FTIO is the name for the overall l/O subsystem that encG,-,uasses the MIO board, the fault tolerant l/O bus, and the SAM and PCI cards.
~ Goter- The .letta based GBI ASIC.
~ HSC2 - The Jetta Golfbus/PCI interface board ~ IOVA - IIO virtual address.
~ IOBus - The taul~-tolerant bus ~tv._en the MIO board and the SAM modules.
~ MIO - The ~letta GolfbusllOBus i"le, lace board.
~ O~,Qn0Ool - This is a l~roposed slan~lar.~ i"~t:,lace for booting any OS trom a PCI card. It is independe"~ of the ope~ alin9 system, the system hardware and the ~" u~u,ucessor. This s~nda-cl is still under rego~;3tion.

26 Dece"lber 1995 8 ~;~

. . .

CA 022575ll l998-l2-03 Polo Software P,~ramming Guide ~ ..atus Company Confidential ~ Polo - The new low end system. The Polo system is fully desc,il ed in the Polo Product Specifi~tion.
~ PCI - P~ .ipl,e,al C~"-~,one~lt Interlace; PC Bus under development by Intel and others as a ~n-hld for PCs. The expectation is that p~liplleldl device controller chip sets will be available which can reside directly on the PCI bus ~ PCIB - The Polo equivalent of a SAM module.
~ PCMCIA - PC Memory Card Int~:r"dtional As~ ~ ~ ~ 'ic n. PCMCIA is an industry :~ndal~l bus which is used for boot functionality on Polo and FTIO.
~ PMI - P,ùcessu,/Memory Interface ASIC. This is the Cougar in Jetta and Polo systems ~ SAM - Stratus Adapter Module - A Jetta-based board which provides the fault-tolerant multi-drop interface to the lOBus. SAM is a generic term for lOBus adapter (not just PCI board adapters). Thls specification deals only with the SAM/PCI version of the SAM adapter.
~ TRID - Transaction ID - This binary number provides an identifier for Golfbus and Xbus operations.

26 Dec~"lber 1995 9 ~'~

Polo Software ~uy.dn-~, ,g Guide Stratus ~ompany Confidenbal ~. Functional Overview Figure 1 below shows a block diagram of a one SAM bucket system that both the FTIO, HSC2, and Polo systems present to sofl --are. This diagram is meant to show only the p,og,a"-,uing model and not the actual imple",e,l~t,on.
For a Polo system, soflware should view the system as a 4 slot Golfbus based system. There are Mercury CPUs in slot 0 and 1 and MlO lO boards in slots 2 and 3. The MlOs in a Polo system will never be in a duplexed state.The MlO can connect to one PCI bucket with up to 16 SAMs in it.
For an FTIO system, there can be CPUs in slots 0-3. MlOs can reside in slots 2-11. The system can support multipe MlOs. Each MlO can support 1 to 64 SAM cards residing in 1 to 4 SAM
buckets.
For an Jetta based HSC2 system. there can be CPUs in slots 0-3 and MlOs can reside in slots 4-11. The MlOs in a Jetta system will never be in a duplexed state, The MlO can connect to 1 PCI
bucket with up to 4 SAMs in it.
Throughout the docurnent, many of these basic blocks are referred to in both Polo and FTIO
descriptions. Although mentioned in the terminology section, several of these co""~onents are e~belllely illl~llnllt in u"d6,sla,-ding of how both systems work and present a co",pat;l,le view to software. For this reason, these blocks are des~ il,ed below. In some cases, these units are real hardware, in others these blocks are pure dbsh~ L~ ~s"eplesent~ti;c of ~Gotential hal~.~lè, or merely first i"s(antiations of potenbal hal~ha~.

29 De~..,ber 1995 10 ~'8 .

Polo SoftHare ~ .,..".ng Guide ~ atus Company Confidenbal Figure 1. Logical High Level Diay~d"~ of the Polo and FTIO System Mercury CPU Board - t301~b Xbus lOBus MlO lO Board ¦ SAMO H SAM8 ¦ SAM 1 H SAM9 ¦ SAM2 H SAM 10 ¦ SAM5 H SAM13 ¦ SAM6 H SAM 14 ¦ SAM7 H SAM15 Mer~ury CPU board: the mercury CPU board is the first CPU board in the Jetta product family.
The Mercury CPU board is e~ d to be the tirst CPU board in the FTIO product. Whiie a modificabon of this CPU is used in Polo, it is identical from a software ~.;.~cb~c to the Mercury CPU. When the term CPU or Mercury CPU is used in this document it refers to either the Polo or FTIO CPU and any follow on CPU product.
FLASH Card: PCMCIA card containing Flash (non-volatile) memory. Inforrnabon can be o,yar,i~ed as an ATA disk or as mernory.
FTIO: Fault-Tolerant lO subsystem con I ~9 of an MlO pair, lOBus, Repeater pair, SAM and SAM bac~nel. The subsystem offers fault-tolerant conne.,b~ y to an open bus :,~ndald, namely PCI.
~:olfbuc~Xbus/System Bus: FTIO and Polo use different system buses. Golfbus is the Jetta system bus. Xbus is the Polo system bus. Except for M&D these buses are idenbcal from a software perspective. The M~D differenoes are derived form the different app,oach the buses must take to bus errors. Golfbus, Xbus, and system bus are used inte,~;l,angeably in this document to refer to the system bus inte~onneuL
lOBus: lO Bus is a Fault Tolerant lO bus unique to the FTIO im~e--,en~lion of the system. This bus is tlans~,a-e"I to software except for M&D handling of error condibons.

29 Dece-ll~( 1995 11 ~ .

WO 97146g41 Polo Software F,og,d.-...~ing Guide Stratu_ _.ompany Confidenffal MIO: MIO is the FTIO inl~, lace board which connecl~. the system bus to the lOBus. In the FTIO
subsystem this is a single board duplexable unit. In the Polo system, the MIO is an at~.~d..~o".
Although not imple.,.e.lted as a unique board, its software visible aspects are implemented in dfflerent co",~one(,t~. of the system.
PCI Bus: the PCI bus is an Intel des,y.,ed pe,i~l,erdl bus which is used as the industry s~nda-d IO bus for both FTIO and Polo.
PCI Card: A PCI card is a 3rd party card which interfaces to the PCI bus and an ItO specific interface. A PCI card is the l/O adaptor for both systems.
PCIB: A PCIB (PCI Bridge) is unique to Polo. The PCIB pe,f~-",s the funcffons of the FTIO MIO, lOBus, and SAM. It is invisible to software, exoept from an M&D standpoint.
PCMCIA: Pt:,2.onal Computer Memory Card Interface Associaffon. Interface Standard used typically for lap tops and notebook pcs for memory and i/o devices. The cards are 54mm wide by 85.6 mm long (credit card size). The cards are available in 3 heights, 3.3mm, 5.0mm and 10.5mm.
SAM: the SAM is the interface and control unit for the PCI cards. In the FTIO system, the SAM is a hot pluggable card which pelfo~ isolabon and fault tolerance for the PCI cards. In the Polo system, the SAM is an a~2.0 a-,~on tor the logic and software visible aspects of PCI card control. In this document, the terrn SAM applies to the sofl~/are visible features asboc~ ?d with control and fault tolerance with respect to the PCI bus.
SAM Repeater. The SAM le~ is a unique SAM which i,lt~"aces to a PCMCIA bus. In the FTIO this card also pelhlllll~t some low level ~le~,bi~al and M~D functions.
3.1 Polo Functional Overview The Polo Product is a Jetta Mercury and FTIO only based system. From a high levet, some of the key features of the CPU board are:
~ PA7100 Based Mh opr~oessol, 72 or 96 Mhz, 256KB or 1 M data and instruction cache.
~ 1 or 2 miclo~,,ucessul;. per board ~ 128MB, 256MB, 512M~ or 1GB of DRAM memory This implell,~"_L~n of the MIO subsystem has the f~"c/:\~,g key features:
~ Standard UO Sus support ~PCI) ~ 1 to 16 PCI cards supported.
3.1.1 ! ogica~ Summary Figure 1 above is a logical diagram of the overall system. As pointed out, many of the specific features, SAM, lOBus, and MIO, are a~.l, dcled in the Polo system.
3.1.2 Physlcal Summary While the logical view of the-system is t~tat of a s~n.~d~ Golfbus machine, t~te physicsl view is very d:fferent. figure 2 below illustrates an overview of the Polo system.
There are two major logical cG-,,uoner,~. that will be designed by Strstus in this figure. The first major COIll~ onel ll is the mother board consi2.~ ~g of the Mercury core ~snoop subsystem, cougar, 1 29 Dece"-be~ 1995 12 ~V

Polo Soflware . .ogramming Guide .,tratus Company Confidential or 2 Lisbon memory cards, and on or 2 logical PA modules) and ReCC. The new AStC on the e,L~.d, the Cyclo;os ASIC, contains all sofi.~a.e visible aspects of the Gofer and MlO
Golfbus ASlCs. From a software p~lap~b~c, the GoHbus is inside Cyclops. As explicitly defined in later sections. all intemal registers are either implsmented or will return intelligent ~a~onses to software pe ~ ons.
The second major cbl,l~nel~t is the PCI interface board. This board contains two i"a~n~:s of the CPU-PCI interface ASlCs, the Gambit ASIC. Each Gambit acts as the ir,t~ ce h5: _en the CPU
and one PCI bus. Each Gambit emulates the functionality ot 4 SAM cards (one for each PCI slot).
The inte,~nne~l t: _~n the CPU boards and the PCI interface boards is a set of cross-couple buses. These buses are fauit detecbng, but not correcbng. The buses loolc like a hybrid Golfbus and are named ths Xbus. A complete descripbon of the bus i,)~.~dce can be found in the Xbus Funcbonal Specificabon. Memory updates, Regurgitated infos, and other CPU to CPU traffic is run across the Xbus.
In Polo, the MlOs will always appear to be simplexed. All Prcesses can be pelf~ d to either slot 2 or slot 3 address spaoe, but must be pt, f~" " ,ed as non-paired ~rce~ses. Paired ac:cesses will t~
respor,ded to as TBD. Just don't do it...

29 Deoe"liJer 1995 13 .

W O 97/46941 PCTrUS97/09781 Polo SothNare r?~ug~a~ ing Guide S~atus u~ompany Confidenbal Figure 2. Polo System Oveiview Bo~rd O Board 1 Power Supply Power Supply Non-i T pow r tor board ONon~ T power tor board 1 Coolln~ Unlt Coolln~ Unlt Non-FT coolln~ tor board O & PCINon~FT coolin~ tor board 1 ~ PCI
....... ... ..... .....

, ........ ,~, .. Y .. ~ -- .. i I-Bus P~U8 Cy¢lops Cyolops Amy Array CPU (MAP) - CPU (MAP) Xbu~ .
-PCI ~lot 3 -- , _Pcl Slot 3 PCI slOt 2 PCI slOt 2 PCI Slot 1 -- PCI Slot 1 .

¦ PCI-PCMCIA ¦-- --¦ PCi-PCMClA ¦
¦G~mblt A~lC ¦ D-side D-side l 6amblt ASlc ¦
PCMCIA ~mblt ASlC ¦ C-side C-side j GambU ASIC ~ PCMCIA
tla~h ram tlash ram PCI Slot 4 _ PCI 81ot 4 PCI Siot 5 _ _ PCI Slot 5 PCI 81ot 6 _ . PCI Slot 6 PCI S~ot 7 _ iCI Slot 7 Power Supply Powor Supply Non-FT power tor Non- T power tor PCI slots and dlsk PCI 810t~; and dl~k 29 Dect"-~Lar 1995 14 .

Polo Software Ptogramming Guide ~.~ atus Company Co,lfi~er, 3.2 ~lef~ta/FrlO Functional Overview The FTIO project was can~ mi~project. Although it is no longer a planned project there was a bght co""~atibility goal ~ ~' ecn polo and FTIO. The r~""~a"h of this remain in the design. For this reason the tJ~ Wjn9 sub-sections are included in the ~pecifir~tion to provide bachy,ùund to the reader.
This FTIOIMIOISAM i", 'e "~"ldtion has the l ~ 9 key features:
~ Standard l/O bus support (PCI and PCMCIA~.
~ Up to 64 SAMs (Total of Repeal~. ~ plus SAMs) 3.2.1 I C'~iC?~ Summary The logical view of the FTIO sub-system is consiste, t with Figure 1.
3.2.2 Physical Summary FTIO is a sldrlJdlJ Jetta Golfbus interface Su~system. There are four major logical co",poner,t~
that will be des;y"ed by Stratus in this figure. The first major co",~ùne"t is the MlO primarily consi ,Li"g of the IF ASIC. The IF ASIC provides a bridging function bct~ een the Golfbus and the lOBus. There is no plocessor or memory resident on MlO.
The second major co""~onenl is the nepeater. The Repeater boards receive the point-to-point dfflerential ECL lOBuses and converts the levels to BTL for use in the multi-drop SAM bachplane.
The Repeat~. . provide power for the SAM backplane termination and also re-broadcast the differential ECL lOBuses to allow daisy-chaining up to 4 SAM chassis. (Each SAM chassis contains 2 Rep~at~, ~ and up to 14 SAMs) Each Repeater consists of ECL and BTL bus transceivers and an IAM ASIC. The IAM ASIC provides the host-bridging function from lOBus to PCI Local bus. The IAM also provides Repeater specific support when used in the Repeater mode. The Repeater uses a ~way voted clock gene,dtion circuit. The Repeaters also support a type ll or lll PCMCIA i"l~,ldce.
The third major CG"",onel~l is the SAM. Each SAM consists of BTL bus l,ansce;~r;, and an IAM
ASIC. The IAM ASIC provides the host-bridging function from lOBus to PCI Local bus. Each SAM
supports a single 32-bit PCI slot. The IAM provides system memory p(.:c tion for bus ",a~le,ing PCI adapters and also SAM specific support when used in the SAM mode (as opposed to repeater mode). The SAM uses a ~way voted clock generation circuit.
The fourth co""~onent is the SAM ~ach~lane/SAM clock board combination. The SAM bach~Jlane is a 16 slot back~Jdnel which supports 2 Repeater and 14 SAMs. The bach~,anel will conduct the bulk power to each SAM and Repeater eliminating the need for bus bars. The bach~.anel will also provide the cable attaches for the loaus cables bulk power connec~ions CDC co~me~ 'hns and a JTAG port for scanning each slot individually. The backplane will also have conne. tu, s for mounting the SAM clock board. The SAM clock board will provide point-to-point triplicated clocks to each SAM and Repeater board. The first iteration of the clock board will be a high-reliability model. A follow-on project can replace the highreliability clock with a fault-tolerant clock. The high-reliability clock cannot be ,e~.Jdced on-line.

29 Dece"l~r 1995 15 ~3 PCT/U~97/09781 Polo Software ~oy-d",.. l ,9 Guide Stratus ~;ompany Confidential 3.3 Jet~ll :''C~ Functional Overview The HSC2 project staned during first release of Polo hardware. One of the goals of ItSC2 is software compatiblity with Polo for use in Jetta-12 and Jetta-6 configurations. Sections of this document are being updated to highlight diff~ r~n~es and give the reader insight The JettalHSC2 imple."e"~on has the f~'h .~;ng key features:
~ Standard l/O bus support ~PCI and PCMCIA).
~ 1 to 4 PCI cards supported per Jetta slot.
3.3.1 Logical S~,.n.~a"~
Figure ~. is a logical diagram of the overall system. As pointed out many of the specific features;
SAM lOBus, and MlO are aba~a~lad in the Jetta/HSC2 system.
3.3.2 Fh~sical Summary The logical and physical view of the system is that of a :,lan~. d Golfbus machine. Figure 3 below illustrates an overview of the Jetta/HSC2 system.
~igure 3. HSC2 System Overview Mercury CPU Board GoUbus HSC2 DCI 81ot 3 ~cl S~o~ 2 Pcl slOt 1 Mlrage ASIC
--¦ PCI-PCMCIA ¦
I

~Mlroge ASIC
PCMCIA
nash ram The mapr co.--ponent that will be des-y"ed by Stratus is the HSC2 Jetta l/O board. The HSC2 is a simplexed board so all nr~s~es must be pe, ~u""ed as non-paired a~ sses The new ASIC on the HSC2 board the Mirage ASIC contains all the software ~lisible aspects of the MlO Golfbus ASIC and also emulates the functionality of 4 SAM cards (one for each PCI slot).

29 Dece"lLer 1995 ~6 ~(1 .. . .

Polo Software ~.ugramming Guide _.ratus Company Confidential 4. XbuslGolfbus to PCI Address Mapping This section outlines many aspects of addles~ g on the Polo PCIB and ~letta HSC2 systems. The overall high level address map is presented and PCIB/HSC2 specific aspects are outlined. The address translation map is presented and its outline and management are discusse-l 4.1 Ad~r~ssing Overview The Polo system address map is de~igned to preserve maximum compatibility with the Golfbus addre~ map. As such, it preserves the supportfor 15 logical slots, etc. even though Polo is a fixed configuration of two CPU boards and two IIO boards.
The PA7100's address map is broken down into two seg",t:nts, main memory and l/O space. All of main memory is cacheable and all of IIO space is non-cacheable. There are no limitations as to what virtual address is mapped into what physical l/O spaoe.
Rgure 4. PA7100 Fl.,~ ' Address Map OxOOOOOOOO
OxF0000000 Main Memory ,~
Address ,' Spaoe "' I ~ ~ s OXI-I-tt~ttt ' ' - ~ .. .
... .
OXtttl-l-ttl-By mapping IIO space into the virtual address space and using the same page translation ",echd";s~" as that employed for memory, the PA7100 architecture allows the fbxibility for non-privileged users to be given access to some regions of l/O space without co,,,p,u,,-;s;ng system security. All duplexed boards' I/O register definitions can be ar~ssed through local, paired, non-paired, and global space and all simplexed boards' I/O register definitions can be A~ce~sed through local, non-paired, and global space. MIO mirrors it~s Golfbus ,~.s~.;, so that they can be Acc~ssed in ~freeze space" to allow a CPU controlling the duplexing process to access MlC)'s registers while MIO is in freeze state.
Local Space - Resource is local (same-board) to the requester and lI,e.ufure does not require an XbuslGoffbus cycle. Registers such as CPU interval Rmers, Interrupt 8, Mask Registers are good examples of frequently Accessed registers that are u~tilllally handled locally. When a board is off-line it can only access and test l/O on-board through local space.

29 Dece"ll~er 199~ 17 '~ ~

.

Polo Software ~oy~d~m;ny Guide Stratus ~;ompany Co"fi~le,ltial ~ Non-Paired Spsce - I/O space is Accessed explicitly by slot number. Note that boards that are duplexed are required to go through the motions on non-paired l/O but only the addl ~ssed slot will perforrn the write for write operations or drive data onto the )~bus~ lflous for read ~ope. t;~ns.
~ Paired Space - I/O space is accessed explir;itly by slot number; however the least significal,l bit which diffe~e~ ales the ever~odd slot of a duplexed pair is ignored. There are registers in IIO space that are not allowed to be read through paired space such as a per-board broken status register.
~ Global ~ d cas,t Space - Global l/O space is defined as writ~only space ~s ~,art: needs to enforce this). It's primary purpose is to provide a means of bl'l)Af~c~F~;n9 sy"~;l"onous c~",ll,an~ to multiple boards in the bach~la,~e simultaneously.
~ Free2e Space (Paired and Unpaired) - This UMIO only~ space resides at offset 7FExxxx and allows a CPU that is handling the duplexing of the MlOs to access the Golfbus IIO registers while MIO is in freeze state. Normal paired and unpaired Arr~sses (offset 7FFxxxx) get busied.
4.2 Address P~mapping Polo/HSC2 use address re-mapping to protect the system from errors caused by simplexed l/O
devices. Note that the l/O Virtual Address or IOVA fommat used by Polo & HSC2 systems differs from the IOVA format used on BIO systems.
In HSC2 systems the IOVA is translated into a system address inside the Mirage ASIC on the HSC2 board. The Map RAM is also intemal to the Mirage ASIC which is self-cl-e~ ing.
In Polo all system Ar.cesses from the l/O ASlCs are pel ~OI l "ed using the IOVA (IO virtual address); these are rnapped to the final Xbus style physicaUvirtual index ad~l~sses (CPU
a.ld~e~ses) by a map RAM in the Cyclops ASIC. Both IOVA and CPU ad~l~sses are passed on the bachpldne with a bit in the trid field distinguishing b~ een them.
4.3 Per-Space Ordering of 1/0 Accesses 10 Accesses requiring the Xbus/Golfbus (~Accssses to paired non-paired or global space~ are ordered relabve to each other for a given plocessor. I/O A~cesses to local space are only ordered relative to other local l/O ~esses trom the same ~)f~ c~ssor. In particular if a write to a register in global space is 1~ 'lo~ d by a read of that register in local space it is possible tor the read to occur before the write takes pace. Ve,i~iLation of global writes should be pe,~orl"ed by non-paired reads.
There is no ordering guaranleed bel~ on I/O space accesses and memory space ~cessPs A
an;~.l, for making l/O ~cce~ses sequenffal to memory a~esses is board specific and desc,il,ed in the relevant board s~.ecifi.,ations.
4.4 Address Decode The Golfbus UO address map used by Polo and HSC2 systems is suL~ ad into 16 seyn~t:'ns of 16Mbytes each. The first 15 seylllents corl~soond to Slots 0-14 in the bacl~JIane slot 15 is reserved to map local and global l/O Ad~t~sses. A Polo system looks to the p,oyl~lll,ner like a four slot Golfbus bscl~plane. Note that the slot ide" ~ ~r field of the l/O address is inverted to allow local address space to begin at ~hsolut.t address 0xF0000000 which c~"~sp~ ri~ls to the boot address issued by the PA7100 upon being " leased from reset.

29 Dece,-~ber 1995 ,~ f 18 C~

Polo Sof~Nare P,_,Jramming Guide ~, atus Company Confidenbal Figure 5. Physical l/O Address Decode.

1111 ssss Offset Where: ~ is the slot # jIIJ
1111 -> Slot #O
0000 -> Slot #1 5 Each slot with the e~r~pt;orl of slot 15 is suWiJ:ded into paired and non-paired space which effecbvely halves their lo~ical address space from 1 6Mbytes to 8Mbytes. All registers on duplexable boards are addl~ssable through either paired or non-paired space. ~ccssses to paired space on a non-d~r' ~ le board are ignored.

29 Dect~ er 1995 ~ 19 ~' Polo Sot~Nare Pluglall"";ng Guide Stratus .,ompany C~ f ,~

Figure 6. Physical l/O Slot Decode _ , OxFOOOOOOO

0o'xFF~~~o~~~~0 PER;SLOT ~ OI~FODOOOOO

0xF2000000 Slot 13 Ox~ur~ t OxF3000000 Slot 1 2 OAr~ OOOSlot 11 ~ p~j~ OAF1000~00 ~ Space OxF4FFFFFF

OxFBOOOOOO
Slot 4 .
OxFCOOOOOO _ ~ OxFDOOOOOO

O~FDOOOOOO ~ 2 ~ Sp~e O ~u OxFFOOOOOO
Slot O

4.4.1 Per-Slot llO Space (Local, Paired, Non-Paired) In general the view ot Xhus~.sl~bus l/O space is identical whether it is add~t:ssed through local non-paired or paired space. All board l/O, ~y.;,l~l a will reside in the last 16 pages ot l/O space and each board in the Polo/HSC2 a,.;t,it~cl,Jre will have a CG~ llOn set of registers that reside on the last physical page ot the per-slot l/O seg..,e, ll to simplity the M&D (Maintenance & D;ay..~,alic) interface. Addfflonally the last 1 Mbyte sey~ "l ot a board s address space can be touched via global b,oad~al write operat;ons which are des~ ed in greater detail in the next section.

29 Dece",ber 1995 ~ 20 Polo Software ~ .uyr~"""i"9 Guide ,uatus Company Confi~en~ia Fi~ure 7.110 Space Map RESERVED
uArûSFOûûû - Global Broadcast Reoen~ed\ ûxF?7FOûûû - ~ r ~ space p~r-board\OxF?F~Oûûû - Paired Spaoe 8MB _ I/o r~gister ~ Defini~on 64KB IIO Registers _ St~ndard uxFû~FFûOO - Global2roadcast Gol~bus UO \ OxF?7FFûOO - ~: r ~ Spsoe Regis~ers \Ox~?FFFOûO - Pair~d Spaoe ? = inverted board ~lot #

4.4.2 Global Br~ c-sls Global blu~ are write-only and are visible to all boards through slot 15 as illustrated in figure 6. Global Inoad~ can be issued to all boards simultaneously in the backplane or to classes of boards such as all CPU/MEM slots. Global bfoad~l ope. ~s must be architected in such a manner that all boards that receive them perform the opelcllioll sy"chlunously relative to one another. Figure 8 below indicates the decoding for global brsa~r~sk Figure 8. Xbus/t:olfbus Global IIO decode 31 28 27 24 2 22 2û 1 9 0 1111 0000 1 ccc Offset CCC = CG~ and Type 000 = B,oadca~l to all Boards 001 D~oadcasl to CPU/Memory Boards 010-111 = ReseNed for Other Board C!~ s 4.4.3 llle9allAl~aGr~al UO /\Cc~cses Table 1 summarizes the behavior of various t\tpes of illegal/~b"~""al ItO a~;cesses Note that as desc,iLed in section 12.1 on page 82 though all cG"""on registers are 32 bits they are spaced at 64 bit (8 byte) inteNals to aid future board designs which may make use of 64-bit internal buses.

29 Dec~"~ber 1995 21 Polo Software rluyld~ ing Guide Stratu~ ,ompany Confi~J~n~al Also note that the cu,.llnon logic plo~/;des an indication to board specific logic that a read access has tirned out/no-ACKed, or that a write access has failed (i.e. a simpex source broke after su~ssfully driving info but before driving data). Different boards handle these condfflons in dilferent ways. Mercury boards for example:
~ retum O's and signal LPMC's (HP-PA low priority machine checks) for timed-ouVNACKed reads ~ do nothing for failed writes other than to write O's or nothing (per the table below).
Table 1. Illegal/Abnormal l/O Regisle,~

Bus ASIC Registers Non-Bus ASIC
32-bit read on an odd 32-bit boundary retum O's if within a unsupported page that has other ra~;at~,s, else NACKed sub 32-bit read 32-bits returned 32-bits returned. Byte enabled bytes guar-anteed to be valid, all byes guaranteed to be deterministic.
32-bit write on an odd 32-bit boundary ignored unsupported sub 32-bit write ignored ignored for ASIC l/O, t e- f ~ ed as speci-fled by byte enables for PCI 1/0.
failed write (i.e. a wnte trom a simplex board ignored Ignored which breaks after suc~s'ully driving address but before driving data~
write to non-existent register ignored ignored for ASIC l/O, PCI will Master Abort read of a non-existent register return O's if with~n a Can return O's if the page that has other address de~des to a registers, else page that has others NACKed registers, e~se, NACKed, PCI will Master Abort 4.B PCIB MlO/lOBus Compatible System Address Map All of the MlO ~,-,palible address space can be add~essed direc~y through the XbuslGolfbus l/O
space. Each l/O boards' address space can be a~ssecl directly via set size ~ n~ s, where the base address of the window can be ployr,l"",led individually on each board.
The Polo version of the MlO compaffble map is a subset of the map presented in the t~ wlng sections. As previously stated the t'olo appears to s.~ as a system with simplexed MlOs in slots 2 and 3.

29 Dece"~ 1995 22 .
. . .

CA 022575ll l998-l2-03 Polo Software Pr~,.Jramming Guide ~uatus Company Confiden~al The Polo PCIB and Jena HSC2 will only operste in simplexed mode. All lOBuslPCI slots are cJec~ded according to the address map defined bellow.
Figure 9 shows the MlO/lOBus co",jua~L)le address map in context of the Xbus address space.
Figure 9. M10/lOBus Compatible Address Map in Xbus/Golfbus Space 16MB of Slot-? I/O Space (fiyure 6) oxF~ "~- r~ or Palr dAddr. MdOr/eOssBuMsacpo8mpaBible Paired Space OxF ~ or 0x~ rF~ I ~ MIO Registers t64 K~
Oxl~800000 r ~ " ruuù0 or OxF'~FI-~OOO
Ox~
~ Reserved Non-paired " ~
Space O'x~.600000 or 0xF?E00000 ox~oo0000 ~ ~ lOBus -IO Space 2 MB
~~. ~~~~Oo~ COoooo ? . inverted Xbus Slot # ~ " lOBus -Slot-O, OxF j~ Memory Space 4 MB
Ox~ /uOOOOO or OxF?80~ uB~
The breakdown of the Xbus/Golfbus address bits is illustrated in table 2.
Table 2. Xbus/Golfbus address decoding XbuslGolfbus Address bits Address Space 31:28 27:~ 23 22:20 19:16 4'hF !slotP/NP 3~h7 4hF MIO P~egisters 4'hF tslotP/NP 3'h7 4'hE MIO Registers (treeze space) - Mirrored area that allows access while MIO is in freeze state 4'hF !slotP/NP 3~h7 4~hO-D n~scrv~d 4'hF IslotP/NP 3'h6 xxxx nc erved 4'hF !slotP/NP 3'h~4 xxxx Access to SAM IO Spaoe (32K direct - 1 6K IAM IO/
16K PCI IO) (Address bits 31:15 are re~apped before furU er usage.) 4'hF !slotP/NP 3'h3-O xxxx Access to SAM Memory Space (64K direct PCI
Memory) (Address bits 31:16 are rc l"~ped betore further usage.) MIO Compaffble lle3 t -- Space:
~ The register space includes all r~g;ster~ ot the l/O board, including the slanclarcl Xbus/Golfbus 1/0 registers, the ASIC registers, and the ID PROM space. Many register localions wlll be c~"""on)y defined for all l/O boards. See section 4.5.2 for details.

29 Dec~ , 1995 23 Polo Software P~ al~ ,ng Guide Stratu~ vompany Confidential lOBus Compatible UO Space:
~ This space includes rt:giste~ and memory on the PCIBIHSC2; a portion of this address space will be tumed into llO cycles on the PCI bus. Only a 32 KB window of the PCI13IHSC2 llO
space can be ~ssed from the XbuslGolfbus at a bme. The base offset of the window can be ~l ,anged on the Mirage or Gambit. See section 4.6.3.
IOBus Compaffble Memwy Space:
~ Acce~ to the memory space portion of the lOBus compatible address space will be tumed into memory cycles on the PCI bus. Only a 64 KB window of the PCIB/HSC2 memory space can be n~ss~d from the XbuslGolfbus at a time. The base offset of the window can be cl~an~ed on the Mirage or Gambit. See section 4.5.4.
4.~.1 lOBus or Xbus to PCI Local Bus Requests Base address registers have to be set up in either the Mirage or ~ambit ASIC and in the PCI
configuration space of the PCI adapter card in order to allow l/O or memory (non-configuration) a~sses to the PCI adapter across the PCI local bus from the Goflbus/Xbus. One set of registers (and hence one llO and memory windowt exists for every PCI slot. Figure 10 illustrates a high-level view of this mapping proce~.
Figure 10. SAM address window scheme Go fBuslXbus address s~ce SAM c ~ PCI
per-slot lO space 18MB) address spaoe awress spaoe _ .
, ---- - - ~ MlOa ~ loBus ~~~ PCI

lOBus smp8mocry - - ~
Ory ,~ windrw ~I spaoeb . "
rlld, ~ME~ n n~air~d '4MB whdr~w PCI Memory oflset register ~J

Direct memory ac~ces to the PCI adapter must be made through the 64KB SAM memory spaoe window which can be shifted through the entire 32-bit adapter memory address space. I/O
~~~sses to a PCI adapter likewise must be done through the 32 KB SAM l/O space window. The adapter card has to be ul u~d~ led to respond to a specfflc memory or l/O area where it will be add~3ed by the system. Each adapter type may ha~e different memory or l/O range requirernents. These requirements can be determined from me base address registers in the adapter's PCI configuration space. The PCI Memory Offset register of the Mirage or Garnbit ASIC
has to be set to point to the bottom of the selected memory range of the PCI adapter. The memory can start anywhere in the 4GB addre~ range on a 64KB boundary. Also the PCI lO Space Offset register of the Mirage or Gambit ASIC has to be set to point to the bottom of a 32KB window within the l/O range of the PCI adapter. See the specific PCI adapter card~s :,pec;~ ;cn and section 4.5 PCIB MlOllOBus Compabble System Address Map for more on this. Thus the procedure for au;essing either memory or l/O space on an adapter card is;

29 Dece"~l~er 1995 24 Polo Software ~loy,~"",ling Guide ~itratus Company Confidential ~ initialize PCI Mernory or ItO Space Offset register in the bridge (Mirage or Gambit) to the bottom of a 64KB (memory) or 32KB (I/O) window.
~ initialize PCI Mernory or l/O Space Offset register in the PCI adapter.
~ access memory or l/O window through Golfbus/Xbus l/O space (secbon 4.5). Any access to the PCI address windows is illt~ tt:d as legitimate access using whatever address happens to be in the Offset Registers.
At a minimum the software should guarantee that the PCI slot memory space ~; :. Ich.ls and the host bridge memory space do not overlap tor any group of four PCI slots aaaigned to a single PCIB
card. Figure 11 shows an example for a group of four PCI adapters. All four PCI adapte. a reside on the same PCI bus so care has to be taken in laying out the address windows.
Figure 11. PCI Bus Memory Space Map 32-blt PCI
System Memory Address Msp SAM Slot~n Memory Spsce ~\\\\\\~
Host Brid e Configura~le Memory Space PCl slot3 ~ PCI Memory offset 166hFKFBFF~I~t~ , P.~ e~ S '~
16'hOOOO~I- r ~ .... ?:.. r ¦i :::::~:::: x PCI Memory offset register ~ Memory Space ~ ... ~
166hFKFBFF~ rh~ PCl slot 1 --¦ PCI Memory off~et ¦
1 6'hOOOO ~ ~ ~\\\\\\~
PCI Memory offset register ~~ ~\\\\\\\\\~

~1 = NOT MAPPED
4.5.2 MlO C~ pdtil,le r~ist~r Space 0x~%6F0000 - OxFYJ6FFrFr; Xbus/Golfbus co"-",on and MlO specific compatible registers including access to the ID PROM. For a detailed register address map and descripbon see secbon 12.
The ~/O is equal to lhe inverted Xbus/Golfbus slot number. The ad~\~ sses listed are for non-paired space for paired space addlasses change bit 23 from a zero to a one.
4.5.3 lOBus Co~ ,d~ble 1/0 Space Map The f 'o~ ;ng lOBus Compabble ItO Device address map allows for 64 add~~ssable l/O slots per lOBus. The address space is divided to l/O Space and Memory Space.
The Polo system only supports slots 0 to 15. The other 48 slots behave as un-popu~ 1 slots in a 29 Dece"ll er 1995 25 ~3 Poto Software P~uy,a,,,-,,;ng Guide Stratus ,mpany Con~ al Polo system.
The HSC2 system only supports slots 0 to 3. The other 60 slots behave as un-pop~ ed slots in a HSC2 system.
Figure 12. IOBus Compatible l/O Space ~onR~s address lOBus Address Space 0xFYo5FFFFF 32 KB lOBus Slot 63 0xFYo5F8000 - IO Space 0xF~o5F7FFF ~ 32 KB lOBus Slot 62 oxFFY~o5E0~$~$ 10 Space O~t~ .G .~000 0xF~~41FFFF ~ ~2KB lOBusSlot3 0xFYo418000 ~ IO Space 0xF%417FFF ~ 32 KB lOBus Slot 2 ~ IO Space % . inverted Golfbus/Xbus Slot #
0xF9~410000 Slot-O e 0xF
0xPYc40FFFF ~ 32 KB lOBus Slot 1 s~ot-3. oxc 0xFYo408000 ~ IO Space The ad~ ses listed are for 0xF%407FFF _ lOBus Slot 0 non-paired spaoe, for paired 32 KB space ad~ change bit 23 0xF%400000 IO Space from a zero to a one.

Table 3 illustrates the Xbus/Golfbus address lo" "dLon for the lOBus ~"-~,dti~le l/O space shown in the figure above. The slot field in the address is the inverted Xbus slot number.
Note that SAI~IUPCI slots 16 to 63 do not exist and the MIO ~",patil~le space exists in Xbus slots 2 and3.Fu,UI~,,,,c,dthePoloPClBdoesnotsupportduplexingsoitwillnotrespondtopaired space ?~cessss Note that SAM~PCI slots 4 to 63 do not exist and the MIO c~"".dt;l)le spaoe exists in ~ISC2 Fl"U,~""o,e ~e ~et~a HSC2 does not support duplexing so it will not respond to paired space A~sses Table 3. IOBus compatible llO space slot and offset bit positions 1 0 9 8 7 6 5 4 3 2 1 0 9 8 7 6 5 4 3 2 1 o 9 8 7 6 5 4 3 2 1 O
lalo~ ~ 1 0 6 blt alot SAM ott~et Into 32K IO Space (Xbu~) number a. pai~ed/ non ~ed bjt 4.5.3.1 SAM llO A~ ess Space SAM l/O space is divided up into two spaces (see figure 13).

29 Dece,nbeY 1995 26 Polo Sot~Nare Prugramming Guide S"atus Company Conr~den -PCI UO Space: The PCI Local bus can support a full 4GB of l/O space but the hislu"cal maximum is 64KB due to x86 lilll ti~ ns. Access to the 64KB PCI l/O Space is made through a 16 KB
window defined as SAM PCI/IO Space. Ac~esses to the SAM PCI/IO Space are passed along to the PCI bus with the l/O Read or l/O Write function codes. The position of this 16 KB window can be changed on the16 KB boundary via the PCI l/O Space Offset Register. The base l/O address of the PCI adapter must align with the PCI l/O Space Offset Register of the Mirage or Gambit ASIC.
A read that maps to the PCI l/O space of the SAM and ge"e,ales a PCI cycle ~at does not map to the l/O space of an adapter will cause a master abort and return zeros. The PCI bus read could also map to the l/O space of the next adapter if the l/O spaces of the adapters are contiguous.
When a sub word access is made to PCI lO space the byte enables are passed through from the read. Since the address from the CPU will have 2'bO0 in the address bits [1:0~ these two address bits need to be recons~ cted from the byte enables as shown in table 4.
Table 4. Lower a~dl ess bit construction AD1 AD0 C/BE3# C/BE2# C/BE1 # C/BE0#
O O X X X O

SAM compatible r~ e- Space: Ope,alions to this 16 KB spare provide access to SAMcompatible ASIC intemal registers. Reads to ~;sla,~ that are not imple",e"lt:d in this space retums zeros.
Figure 13. SAM l/O Space Map PCI llO Address Map SAM and PCI Slot-n l/O Space ~, 16'hFFFF

16 KB SAM ~ ASIC ~";~
1 6'h4000 ~ Reglster S~ .. ~ ~ KB
16KB PCI/IO ~ ~ _ _~

¦ PCl llO Space Offset ~ 1 6'hOOOO

The i. ~ ~i ,g table illustrates the fo",~tion of system bus ad~n:~s~s within the PCIB/SAM l/O
address space.
Table 5. SAM Compati~,le Register System Address Formation System a 1 0 SAMIPCI Slot b Ot~set bus slot number Inverted 29 Dec~,ul~l 1995 27 ~5 W O 97t46941 PCT~US97/09781 Polo Software Fl.)g~ "l ~9 Guide Stratus ~,ompany Confiden ~' a.p~s~L~->-l;unpuxds~e->_0 b.l-SAM- , '' ASIC~gs~rs~e:0-Pa ~Os~e 4.5.4 lOBus Compatible Memory Space Map Once again the Polo system only supports 16 slots and the other 48 spaces behave as un-p~ru'~tsd slots in a Polo system.
Figure 14. IOBus Compatible Memory Space Xbus address lOBus Address Space OxF~o3FFFFF ~ 64 KB lOBus Slot 63 OxF%3FOOOO ~ Memory Space OxF%3EFFFt ~ 64 KB lOBus Slot 62 OxF%3EOOOO Memory Space OxFYo03FFFF 64 KB lOBus Slot3 Memory Space OxF%030000 % . inverted Golfbus/Xbus Slot #
OxF%02FFFF 64 KB Memory Space Slot-3 OxC
oxnLQ~nooo OxFYoO1FFFF ' 64KB lOBusSlot1 Memory Space The ~ 31isted are for OxFYoO10000 non-paired space, for paired OxF%OOFFFF ~ 64KB lOBusSlotO spaceR~3cschangebit23 oxrl~oooooQ ~ Memory Space from a zero to a one.

Table 6 illustrates the XbuslGollbus address formation for the lOBus CCil I Ir ~ ' ' ' ~ memory space shown in ~e ~ure above. The slot field in the address is the inverted Xbus slot number.
Note that in Polo systems, SAMIPCI slots 16 to 63 do not exist and ~e MlO compabble space exists in Xbus slots 2 and 3. Ful U ,~, l nore the Polo PCIB does not support duplexing so it will not respond to paired space ~nre~ces either.
Note that in HSC2 systems, SA~NPCI slots 4 to 63 do not exisL Ful U,e. ..,o,~ the Jetta HSC2 does not support duplexing so it will not respond to paired space ?.cc~sses either.
Table 6. IOBus compatible memory space slot and offset bit positions lalot ~ 0 6 blt slot SAM otfset Into 64K mQmory sp~ce (Xbus~ number a.~nonpux~t 29 Dece"lber 1995 28 Polo Software P~og(~.""ling Guide Stratus Company Confidential 5. Xbus/Golfbus PCI to Xbus/Golfbus A~ s Mapping The ha(~a~e takes a PCI gene~led address which we refer to as an IOVA and maps it to a system address.
5.1 Software Vlew of Re~mapping Figure 15 shows the sofhvare view of re-mapping.
Figure 15. IOVA to System Address Map~ g 31 28 27 20 19 16 15 1413 12 11 o Chunk¦ l~pper device ¦ Check ¦2ibOo¦ Page ¦ Line [11:0] ¦ IOVA
Cllecks ~ e8~1on 5.32 ~Itsi2720]
map ram entry 8/
map Ra-n index (1024 entries) startaddr ~ ~ range enaaaar ~ ,~ checking J

optlon bftl~ error Vlrtu~l Index _ ~
refe- to ¦ Block\~ rKb~ 5o~ 11:5] ¦ ~3 S5.e4.4~n 31 + 12 11 + Xbus PhysicalAddress[31:12] 1 I Physical ~ tc ~ ten~b coh~rant j~e~ Goi~s Functbtlal '',~ ~ ' ' for det~ib) 5.2 Polo Re~mapping Polo suppoits both an addlessing scheme compatible with Jetta based processc ,;" 32 address mode and an add~:ss",g scheme co",~ible with Runway based p~ocesso,~ 40 bit rr~de. The sequence starts with a PCI Address or IOVA. Gambit takes the IOVA and sends it across the Xbus. Cyclops re-maps the address to a system address and sends it to the CPU via the IBUS.
Figure 16 illustrates the address re-m 1,~ lg for the 32 bit address mode used by Jetta systems and currently used in Polo systems. The line offset shown in the system virtual index of figure 16 is the line offset taken from the Xbus physical address driven during the Xbus IOVA cycle.
HSC2 supports only the 32 address mode co" ,~alible with Jetta based pfocesso, ~. The sequence 29 Dece"ll~l 1995 29 8~

Polo Soflware B,oylalllllling Guide Stratus ~,ompany Co"~identi31 starts with a PCI Address or lOvA. Mirage takes ~e IOVA and r~maps the address to a system address and sends it to the CPU via the Golfbus. Figure 16 illustrates the address re-mapping for the 32 bit address mode used by Jetta systems and curren~y used in Polo systems.
Figure 16. Polo System Address generation (32 bit address mode) L Address [31:0] ¦ IOVA

~ [31 :0]

~ addr~s C~lclopslMllage g ~Flgu~e 15) 1'~0 ~bus/Gbus sddress[31:0]¦ ¦

~1 30 ~ 19 18 ~ 1211 6 5 ~ 3 0 ¦ ¦ Virtuallndex[11:0] ¦ 1ineoffset[1~ func ¦ rc byteen ¦ Vl~ual 31 ~1, 0 Xbus/Gbus Physicsl Ad~ress[31:01 ¦ Sys1ern Addreas Figure 17 illustrates the address re-r"z ~ : ,9 used when 40 bit address mode is sel~ct~d (Polo only). This is the addle~",g mode that will be used by Runway systems and is included for upward cu,,,patibility purposes. Bit 2 of the ASIC specific configurabon register wiU~in the Cyclops selects b ~ en 32 bit and 40 bit address mode. The mode bit is driven on bit 31 of the virtual address to indicate which scheme is being used.

29 Dece"~ 1995 30 ~'~

, WO g7/46941 PCT/US97/09781 Polo Software P.uy-d,---,ling Guide ~..atus Company Confidential Figure 17. Polo System Address generation (40 bit address mode) Address [31:0] ¦ IOVA (from PCIB) xeUS into ~ [31:0]

f addr~ ~ Cyclops r-,. ,, (Fl~ure 15) 1'-1 ~ J
Xbus aW~ 1:0~ ¦
~'-0 :1 30 28 27 1211 6 5 3 0 8ydem func byte en Vlnual Virtual Index [15:0] ffrom Xbus) rc tfrom XbuS) Index O Sy~tem Xbus Physical Addressl31:0] ¦ Ad~r ss 5.3 PCI Memory and llO A~lr~,ss Mapping The PCIB/HSC2 has no local memory spaoe. All requests ~u,) the Xbus/Golfbus and the PCI
adapter are passed through the GambiVMirage ASIC. The address mapping between the two buses can be best des-,-ibed according to the direction of the request.
5.3.1 PCI to XbuslGolfbus llO neq~ests l/O Devices have the same view of the whole 4 GB Address Space as the boards on the Xbusl Golfbus do. Cunently the need for a PCI adapter to generate PCI IIO spaoe a~ sses is not a~arenL The GambiVMirage will issue a maintonance d~t~ , n if a PCI card does an IIO space ar,cess. This will happen through the Master Abort ~-~I,ar;_.-,. The GambiVMirage will also capture the address on the PCI bus which caused the master abort.

29 De~l-lber 1995 31 8~

Polo Software P"~yla"",ling Guide Stratus ~,ompany Confidenbal Figure 18. 32~it PCIB/HSC2 Memory Map System Memory Space System r1~ ~.o~ Address Map 32 I ,FrrF.FFFF . \~ ~ 32 h~0Dn ODOO
~ - . 32'h1 FFF.FFFF
256MB Memor~ AdY~re5s Address Main Memory 256M~
~ 32'h1 000.0000 \~,~\~, 32'hOFFF.FFFF
32'hOOOO.~OOO ~
PCI 5101 n PCI slot 3 PCI slot 2 PCI slot 1 ~ 32'1~nO8n nooo 32'hO07F.FFFF
~ 32'hOOOO OOO~O

5.3.2 PCI to Xbus Memory Re4uests The Gambit or Mirage ASIC only accepts 32-bit PCI memory a~ sses in a 256 MB window (32'hXOOO OOOO-XFFF.FFFF). The 256 MB block is movable on 256 MB boundaries as defined by the base address ley;st~la in the PCI configuration space. The PCI adapter a~ zs~es this 256 MB window with the ap~-op(;at~ Chunk Address which is the high 4 bits of the IOVA. Memory accssses created by the adapters that are not within t'nis 256 MB block are terminatsd through the Master Abort ~"ecl ,dn;.,",. There does exist the possibility of accessi"g a peers memory range on Polo.
The base address registers for GambiVMirage and the PCI adapters must not create a conflict in PCI space. All devices and this 256MB block must not overlap in PCI address spaoe. The hardware cannot detect overlaps so the base registers must be set up correcUy. The resulting behavior will be unpredictable. The driver must also set up the address translaUon map using the address table registers des..,il,ed in section 12.6.2 bsfore any access from the PCI adapter can take place.
In a Polo system the l/O vir~al address (IOVA) is placed directly on the Xbus. Responding agents on the Xbus are told that the address on the Xbus is an IOVA by the requester's having set a bit in the Xbus TRID field. The responding agents then put the IOVA through their local address t~ahslalion ",echan;sn, to check and generate a standard Xbus address.
In a Jet~HSC2 system. the IOVA to system address l,a"sh~ion is done entirely with the Mirage ASIC. The location of the address lldil ~. ~n whether in the Mirage as in HSC2 or in the responding agent as in Polo -is completely l,ar,~,~arent from a soft Nare pel;,~ tiJo.

Figure 18 shows an example of a PCI memory map with 4 adapters mapped. The GambiVMirage PCI base address register is set to 32 h1000.0000 and the limit is set to 32 h1 FFF.FFFF. This is the bridges 256 MB window to host memory. There are 1 K 16 byte entnes in the t~ ~.n '~ - n ta~e 29 Dece"ller 1995 32 Polo Software F ,ogramming Guide ~,.ratus Company Confidential which are indexed by the llO Virtual Address gene,dt~d by the PCI adapter.
5.4 AJ~ gs Re-mapping Polo and HSC2 use address re-mapping to protect the system from enors caused by simplexed l/
O devices. Note that the l/O Virtual Address or IOVA format used by Polo ~ HSC2 systems differs from the IOVA format used in BIO and PKIO systems.
In Jetta systems the lO~tA is ban~lalt:d into a system address inside the Mirage ASIC on the I ISC2 board. The map RAM is also located inside the Mirage ASIC and the Mirage ASIC is self-cl,e~ hing.
In Polo all system ~C~5S95 from the l/O ASlCs are ~, fu- ",ed using the lO~tA (IO virtual address); these are mapped to the final Xbus style physicalhirtual index ad-J~esses (CPU
a.ld~e~es) by a map RAM in the Cyclops ASIC. Both IOVA and CPU ad~t;sses are passed on the Xbus with a bit in the trid field disbnguishing between them.
The PCI adapters access main memory via the XbustGolfbus for co""-,and data and status t~al l~fel s. The address gene-dted by a PCI adapter originates from a non-self-cl)ecl~i"g source. It has to be checked to protect self-cl ,eching (safe) " ,e" ,ories.

5.4.1 5y~t~.,. AW~..S Ge~crdtion There is an ItO Address Tat)le a ~cated in intemal memory. This memory resides within the Cyclops ASIC for Polo systems and on the Mirage ASIC for HSC2 systems. In the table there is an entry for each block ~ 1c ~ ~ted for l/O in system memory. An entry does not assume any ad~JIe~s;l ,g method beyond the basic 16KB block size. It contains the starting virtual index the starting physical address, and the 12 bits of the ending physical address of the block. The next field is the PCI slot number. This field is used to verify that only the e-l,b.t~d slot can access a particular IOVA entry. Figure 19 illustrates the encoding of this field. it is checked against the encoded slot ill~ l"alion in the TRID generdted by the Gambit.
Figure 19. PCI Slot [3:0] IOVA Map Field Definition - Polo only 3 2 ~ O
Xbus dot Sld~ PCl Slot ~ . 'h. T.. ~ n Trld[6:5]
~slot 2 = 2, 810t 3 =1 D slde = 0 8 ot ~qual to Xbus TRID O] C ~Ide _ 1 ot 1~O
PCI Bus number5 ~~ ' 1 Note that the HSC2 system only uses the PCI Slot 11:0] bits. PCI Slot 13 2] bits are ignored and are not saved in the Map RAM located inside the Mirage ASIC.
There are another four bits allocated for board specific opbons and a final bit to indicate validity of the entry. Figure 20 shows an entry example using the Xbus/Golfbus address naming conventions.
Each imple",en~on will drop unused bits for this table in ha~ a.t: to reduce the RAM required.

29 Dece"lLer 1995 3 ~3 ~

Polo Software rloylallllning Guide Stratus ~;ompany Con~id6l, -Figure 20. I/O Address Table Entry/ Block 31 o Block Starting Physical Address e Physical Cadle Tag (31:12), Line Offset (11:2) Block Virtual Index (15:0) ¦ Reserved ¦ Block Ending Address (11:2) Reserved (2 bits) ¦ PCI Slot[3:0] ¦ n ~ d ¦ IC ¦ SW ¦ LK ¦ PR ¦ VA
VA = \~alld Entry OPTION BITS - see ~ection 5A.4 PR = Dsta Pre-recd LK = Lock Cycle SW = Swap 16 or 32 bit Endbn IC = Ir.~ol.er~ Memory Acceo~

This table will be indeterminate at power on. Software should walk through the table and invalidate each entry. S ~.v~ ar~ then must write a valid entry for each device index before any of its les~"tiJc lOVAs can be used. A mail~t~nance alt~lltion will be gene,aI~d if an entry is used without the valid bit set.
This table is d~l~alll ~lly managed by the l/O driver on a per CGIllll ,and basis. The driver has to set up the l/O address table entry for each block before an l/O co" " . ,and that requires memory access can be inibated. The data block can be any size up to 16 KB. Upon co"""and completion the l/O
address table entry has to be invalidated by the driver.
There is a map table entry for each 4K page (there can be up to 1024 re-mappable 4K pages). The page bits of the IOVA bits 12 and 13 are used to address the map RAM. If the plugla,-""er wants to allocate a page in PCI memory space larger than 4K with one device index there must be a map entry for each 4K page. This scheme allows up to 4 contiguous 4K pages in PCI memory space without forcing them to be contiguous in PA memory space.
The dnver software gives the l/O Virtual Address (IOVA) to a PCI adapter with every command.
The IOVA is bhe combinabon of the device index to the l/O address table entry the page within the device index and the starting Page Line Offsets[11 :0] into the block. Figure 21 shows the IOVA for a 32-bit PCI adapter card. It allows for a maximum of 1024 Entries in the l/O Address Table. The next two secbons describe the step-by-step process used to create the system address.
Figure 21. 32-bit PCI adapter l/O Virtual Address Reserved ¦ Devicelndex ¦ Checksum ¦2bOO¦ Page ¦ LineOffsetlll:O
max. 256 Indexes 4 pages 4KB blocks 5.4.1.1 Polo System Address Generation The Gambit will use the iOVA as is in an Xbus request cycle. The Cyclops in tum will take this .l"at,on and pass it to the address ",~p, lg logic where it applies the ap~,fo~,fiaSe al~ori.."" to perform the checking and address mapping.
The l/O device driver goes through the following steps to set up ad~esses for an l/O blOCk transfer:
~ Finds an available slot in the l/O address table.

29 Dece",l~er 19g5 34 W O 97/46941 PCTrUS97/09781 Polo Software i-.~,an,l",.,g Guide _. atus Company Confidenbal ~ Sets up the entry for the block in the l/O address table.
~ Sets up the llO Vir~ual Addre~ (IOVA) (including the generabon of the checksum).
When a PCI adapter inibates a transfer, the Polo system goes through the i.," ~v :ng steps of addre~ c1,6~,hi"g and re-mapping:
~ Gambit ASIC gets the UO Virh~al Address.
~ Gambit checks that the access is to the Xbus memory range. If there is no match the access is halted and the error state is yene,al~d. The Gambit does a master abort and causes a main~enanoe attenffon pm the Xbus.
~ Gambit checks the checksum within the IOVA and will cause a Target Abort if it is not OK. This will also cause a maintenance attenbon.
~ Gambit issues an Xbus cycle using the IOVA.
~ Cyclops checks the IOVA against the l/O address table entry for the device index and page. If the valid bit is set snd the lOVA's page and line offset checks out against the entry~s start and stop add~ ses, the combination of the entry and the lOVA's Line Offset~ 0] create the system address. The Cyclops then looks at the system address to deterrnine if the cycle was directed to its memory spaoe and behaves accordingly.
5.4.1.2 HSC2 System Address Generation The Mirage will use the IOVA to perform the address map look-up. The address ge"~,all:d from the look-up will in tum be used to gene-d~e the Golfbus address.
The l/O device driver goes Shrough the following steps to set up a.Jul~~sses tor an l/O block transfer:
~ Finds an available slot in the llO address table.
~ Sets up the entry for the block in the llO address Sable.
~ Sets up She l/O VirSual Address (IOVA) (including the generabon of the checksum).
When a ~CI adapter inibates a transfer, She HSC2 system goes through the IG"D~ing steps of address cl,e~,ki,lg and re-mapping:
~ Mirage ASIC gets She UO VirSual Address.
~ Mirage checks the checksum within She IOVA and will cause a Target Abort if iS is not OK. This will also cause a maintenance interrupt.
~ Mirage checks the IOVA against the l/O address table entry for the device index and page. If the valid bit is set and the lOVA's page and line offset checks out against She entry's starting and ending add~esses, She combination of the entry and the lOVA~s page and line offset[11:0]
create the system address. Othenu; ~e the access is halted and She error state is gene.ab3d.
5.4.2 lO A~d~.,ss Table Entry Invalidate Upon completion of the l/O, the driver s,. ~a~ has to invalidate the co-.esponding l/O address table entry in the table. The table~s entry is invalidated by simply clearing the valid bit. The entry cache's valid bit can be cleared by using the address table le4;~t~,.a des.i-iLed in section 12.62. If this index ~.~t~,hes the entry cache's index, the valid bit is cleared.
5.4.3 Address Error Checking Three other address ~;h6CI~il ~3 features are employed in addibon to the index~entry validity check 29 Dece--~ 1995 '~3 W O97/46941 PCTrUS97/09781 Polo Software Plugla~ g GuideStratus ~,ompany Confidential as described in the next two secbons.
5.4.3.1 lO DeYice Address Checksum Even though the IOVA is re-mapped it is still used to index into the l/O address table. This leaves a small possibility of an erroneously gene-aled IOVA indexing into a valid but in~"~;~ l table entry.
To reduce this probability there are four bits of checksum allocated in the IOVA to cover the index.
This number is gel~e,al~d by the driver so~ when seffing up the l/O Vinual Adaress. It is checked by the Cyclops when the l/O deYice ~,~sent~ the IOVA for data access.
SU~PGI lillg the 16K page size in PCI memory space impacts the fault tolerance. In order to keep the 4-4K pages that make up the 1 6K page contiguous in PCI memory space the 2 pages bits of the IOVA bits ~13:12] are not checked under the checksum. In the case of an enor this could allow a co"t~.l'e to write to the wrong 4K page within the 16K page. Ityou want cov~.a~e on these hNo IOVA bits then only make one 4K page valid in the 16K page. Also note that this ~~d ~ c e s the number of usable ta~le entries to 256.
The algo,iU"" to generate checksum (C) on the index (I) is the f~ ;ng.
C 0l = 1[0 ~ 1~4] ~IOVAl14~
C1]=1[1 ~15]~IOVA~15]
C2]=1~2 ~16]
C 3] = 1l3] ~ 1 71 This code will detect all single bit errors and all adjacent double triple and quad bit errors. Over all the code detects 94% of all possible errors in the 8 bit index (table 7). Note: that the logic exclusive ORs IOVA bits 14 and 15 into the check bits since they should be zero this does not change the outcome but makes insures that the addl~sses don't inc.~",e,lt out of the 16K page.
Table 7. 4 Bit Parity On 8 Bits F~ult Detection Summary # of faults# of possible faultfaults % of unde~ected faults 1 bit 8 o o%
2 bit 28 4 14%
3bit 56 o o%
4 bit 70 6 9%
5 bit 56 o o%
6bit 28 4 14%
7 bit 8 o o%
8 bit 1 1 1 oo%
total: 255 16 6%

5.4.3.2 Out of Bound Access Check The ASIC performing the address r~",apping checks to make sure that the physical address l1 t :2] of the l/O virtual address is larger or equal to the starting and smaller or equal to the ending physical address 111:2] of the table entry on both read and write Arcesses Thus checking is done on 32 bit boundaries. This check is automatic. If you don't want checki"g set the starting and 29 December 1995 36 .. ..

CA 022~7~11 1998-12-03 Polo Software . Jy,ai""""g Guide _Lratus Company Conti~ential ending add~sses at their limits.
5.4.4 Control Bits There are five control bits in the address table entry as illustrated in figure 20 and desc-iL,ed in the f.'Ic.~i ,g secbons.
5.4.4.1 Valid Entry bit The valid entry bit in the table entry indicates whether the entry is valid or not. The table should be fully invalidated by inibalization software at power on. This involves resetting the valid bit for each of the 1024 entries in the tabie. When an IOVA is mapped sof~ware must write a ~,-esponding entry into the IO map table with the proper system addl~:.ses and set the valid bit. If a device uses an IOVA with an invalid entry the Xbus request will be no-ACKed resulting in the Gambit performing a target abort and generating a maintenance a~ ti~n.
5.4.4.2 Data Pre read This bit is ignored on writes.
In a HSC2 system the Mirage ASIC will perform a data pre-read from the host system if this bit is set and the PCI adapter is performing a read. The Mirage will begin by reading a cache line for the current request and then continue to read succescive cache lines until the pre-read is completed.
The data pre-read will continue until the pre-read limit in the Mirage is full the block ending address is reached, or a cycle is issued from the PCI adapter that is not to the block being pr~
read. In the event of a non sequential address the pre-read data will be flushed. If this bit is not set then the Mirage will only read the 32 bit memory word reques~e~ by the PCI adapter and nothing else until an additional request is made. We may make use of the PCI ~""-,an~b read and read-multiple.
In Polo. all memory accesses through the PCIB initiated by a PCI adapter are issued as 32 bit reads on the Xbus. A full cache line is returned if pre-read is on lock is ofl the starting address is on a cache line boundary and the ending address of the cache line is less than the block ending address (if the starting address of the access is out of bounds then the access is illegal and not fesponded to~. If all of these con.li~ons are not met on a legal access then the Cyclops will only retum 32 bits of data. The Polo system will only use this bit as a q~ ~a~ific~tion for retuming either 32 bits or a cache line on a read. The Gambit ASIC will accept either a word or a full cache line back in .t:sponse to a cache line read request.
5.4.4.2.1 Coherency ne~.lriclions sof~ware note:
I,..~urDper use of the pre-read option can result in i"consi~tenl data in the system. It is Su~ 'b l~:aponsibility to guarantee data coherercy by insuring that the PCI adaptor never receives stale data from a pre-read buffer.
On an HSC2 system pre-read data sits in the Mirage until one of the F~ J::ng conditions are met:
-~ The data has been co"~ ~1et~ 'y read by the PCI adaptor ~ The host updates the Map RAM and the map table entry matches the Map cache entry beingused by the Pre-read state machine.

29 Dece,.lber 1995 ~ 37 , WO g7/46941 Polo Software rtoyfa~ ll;ng Guide Stratus ~ompany Confidential ~ The host explicitly flushes out the pre-read buffer by writing to flush pre-read data bit in the Test Control Register.
Note that PCI adaptor wntes while pre read op~ dtiO.~S are taking place do not cause dsta to be flushed trom the Mi~age even if ff~e write ~ 3~ tion is to the same page as the data p.~ ..3d 5.4.4.3 LockCycle This bit is ignored on writes.
The data ,ore-read bit is ignored and should be set to zero if the lock cycie bit is set to one.
In a HSC2 system any read access to the block will go out onto the Golfbus with a lock cycle function code (a load/clear operation) if this bit is set. The data pre-read bit is ignored and should be set to zero if the lock cycle bit is set to one.
In a Polo system the l/O address map RAM resides on the other side of the Xbus so the cycle will not have a lock function code when it is on the Xbus. I lo~.e~Er if the l/O address map look-up reveals that the lock cycle option bit is set then the Cyclops will turn the cycle from a cache line read into a load/~lear op-erabon ~s~"~Jhu,~: lock). The Gambit ASIC will accept either a full cache line or a 32 bit word back on a read access so the Cyclops will only send back the 32 bit word pfo~:Jed by the load/clear. The data pre-read bit is ignored and should be set to zero if the lock cycle bit is set to one.
Note: The swap bit must be tumed on when using the lock bit to avoid the possibility of ' ~conr st~rt ;~ , 1 .9 L~h ~en two boards in a duplexed pair.
5.4.4.4 Swap 32 bit Endlan When this bit is set to a one the data asso. ;~d with this IOVA address will be big/iittle endian byte :-appe~ If this bit is set to a zero then there is no byte swapping is p~,fi,,.ned. Refer to section 5.6 for a detailed desc, i~ion of byte swapping and when this bit should be set.
5.4.4.5 IncoherentMemory~ gss This bit allows software the option of generabng incohél e"l (don't snoop don't set remote tag) as oprosed to r~l)ere"l (snoop but don't set remote tag) Xbus/Golfbus memory read and write transacffons. Refer to the secbon titled Remote/Coherent bits in the Xbus/Golfbus s~cifi~ ol~ for more details.
~ O = cohe-e,ll ~ 1 = in~;ol-~ nl In a HSC2 system if this bit is set to a one then a read or write Opel ~lion will go out on the bus with the remotcl~hefe, It bits set to indicate an ,ncohe, e~ 1l operation. If this option bit is set to zero then the lelllut~ ohef~nl bits indicate a cohe,ent cycle.
In a Polo system the snoop results will be ignored if this bit is set to a one. All reads and writes will be issued from the PCIB Wittl the remote/cohe,e,~l bits indicating a cohe,tz"l cycle. The remotel coh~ "l bits sent across the Ibus will reflect the state of this bit.
29 Dece"lber 1995 9 ~, 38 PoloSoftware~ alll~ lgGuide _ atusCompanyConlider,tial 5.5 Software Map Manas~ t The four addrsss table ,tsgisl,:,~ that control reading and writing of the l/O address map RAM (see secbon 12.6.2) will require the use of locks to guarantee exclusive access. There is only one set of these leg;3t~ ccessible from 16 different add~e~ses so a single lock should be used to control them.
5.6 Byte Alignment The GambiVMirage is the interface from the big-endian byte order Xbus/Golfbus to the little-endian byte order PCI Bus. Signiflcant endian issues exist within the PCI Bus as data structures (including SCRlPTs code) are required to be in little-endian format while disk and tape data are required to be in big-endian format (Stratus currently writes to disks ~K1 05/K12 1~ and tapes (K116) in big-endian forrnat).
Many data structures (including 82596 SCB's) intermix 32-bit entibes with 1 ~bit and ~bit entities which causes further byte ordering conversion issues.
The PC18JHSC2 system does not attempt to resolve these endian issues, rather it supports a method of moving data into and out of the PCI adapter with the byte swapping under software control. This is configurable on a page by page basis through the option bits in the address map (refer to secbon 5.4.4.4). It is the r~3p~nsll~ility of solt . 1,~ to resolve any remaining endian issues.
The GambittMirage will always perform a byte swap on data going from the lOBus/Xbus to the PCI
bus. When going from the PCI adapter to the Xbus the byte swapping is software programmable via an option bit in the system IO address table re-" a ~ ,9 entry. Figure 22 fe~ s~no, the byte s::al~ ~9 pe,lu,,,,ed for a little/big endian byte swap. A byte entering a specific byte lane (i.e. byte 0; big-endian) on one side (Xbus or PCI bus) will be sent back out on the same byte lane tbyte 0;
little-endian) on the other side. Thus when a 32 bit word is byte s::al~ped its forrnat changes from big to lit~e endian or vice-versa so that the data is identical from a slJIh,.~ ~ pe.xpe~e when viewed from either the little endian (PCI) or big endian (Xbus) side of the system. This is the default configuration and the one which should be used when moving data fromtto disk and tape devices. Pl.g~a~ a~ byte w aF.)iLIg is not supported for requests coming in from the Xbus/
Golfbus to the PCI bus - data is always byte s~.~pped so that it appears the same in either format to software.

29 Dece")l~( 1995 3 Polo Software F,og,d""";ng Guide Stratus _ompany Co"fiJel, ?

Figure 22. Little/Big Endian B~ne Swap incrrasing order Goltbu~ dats bi~_ 31 2 23 16 15 7 ~t ~
Big-Endian byteO byte 1 . byte2 byte 3 (Golfbus) ~ J
. .J
_-- ~ ~
r.- , ~ ~
PCI bus data b~ ~1 ~ 24 23 ~ ~ 1615 ~ ' 7 ~ ~ ~
Littlc [n~lian b~ne3 byte2 byte 1 byteO
(PCI bus) onfCrea~int9 )order Figure 23 ,t:~,,esen~ the byte swapping pe-f ,""ed when the option bit in the address fbld is configured for no endian swapping. A byte on bits 7:0 traveling from the PCI Bus to the Xbusl Gotfbus will remain on bits 7:0. In this mode an integer will hold the same value whether it is in PCI
memory or in system memory.
Figure 23. No Little/Big Endian Byte Swap increa~ing order Go~Dus/Xbusdabbib~ 31 2423 1615 87 ~0 ~i~Endian ¦ byteO byte 1 . byte2 byte3 (GolfbuslXbus) PCI bu~ dats bits ~ 3~ 24 23 1615 ~ 8 7 ~ O
I i~? [nJian ¦ byte3 byte2 byte 1 by~eO
(PCI bus) increasing order ot _ . ,c~c The fc awi.1g table illustrates the byte written or read based on the system bus byte enables. Note that the big endian point of view is the standard system bus point of view and the one that should be used when ~- f~ renc;ng the byte enat 3s 29 Dece"ll,er 1995 9 40 Polo Software ~.. gramming Guide _.,atus Company Confidential Table B. Big Endian to Little Endian Byte Swap Xbus/ big endian data (Xbus/Golfbus) little endian data (PCI bus) Golfbus byte bits bits bits bits bits bits bits bits enables 31.. 24 23.. 16 15.. 8 7... 0 31.. 24 23.. 16 15.. 8 7.. 0 8 blt ~F ~
1000 0 1 2 3 NCa NC NC 0 ~100 0 1 2 3 NC NC 1 NC

16blta, ~ 6 24 blt cp r~ ~r 32 blt ~ ~ tl~ r a. NC - no chaDge. w~ites - this byte holds the plevia~s value; o~ leads this data is ~ot vali~
Table 9 illustrates how various HSC21PCIB data cydes are affected by byte s Ja~,)i. I9.
Table 9. BiglLittle Endian Byte Swapping Based on Access Type access type swapping ~I h~ d MIOISAM ~,-" atible Xbus/ no swapping, registers accessible from Golfbus/Xbus only and Golfbus registers stored in bi~endian format PCI adapter and bridge always s:/apped when read/written, only accessible from configuration space registers Golfbus/Xbus.
Host access of PCI adapter always s~:.Ap~,ed when read/written PCI Access of Host Memory Swapped (svf; ~a,!b COI~t~ d currently on a page basis, may change on a con: ~"e basis) 29 Dect"n~e~ 1995 c~c~ 41 .. ..

Polo Software rn~y,a,("~ling Guide Stratus ~,ompany Confidenbal 6. Interrupts Interrupts ge,le~dt~d by cougar ReCC and Sable remain ~nchânged and are well specified in the app,u~,,iate specs. This secffon defines the interrupt structures ~soc;a~d with the PCIB and HSC
subsystem.
6.1 PCIB/HSC2 Interrupts 6.1.1 Introduction The PCI bus sl~-;'ie~ the .,,eul,dn;~.-l through which PCI devices can interruptthe host. The PCI
interrupt n~e~ l-an;~,.,l defines 4 level sensitive interrupts (IMA INTB INTC and INTD). Three of these interrupts (INTB INl C and INT~)) can only be used by mulU-tuncbonal devices. Non-multifunctional devices must use IMA. The interrupt output structures are defined to be opendrain signals with no resl,i~tions on grouping lugJ~; le~ or interrupts. The implelllel l ~aLon only supports the one INTA# interrupt (actually a combination of all four lines tied tog~U,e, in hal.lw...t:).
This saves 3 pins and only marginally co,ll icates sof,~ .e for mulbfunctional device drivers.
The HSC2/Polo interrupt scheme is based around the method that HP uses to implement intenupts on the PA-RISC chip. The HP scheme is a llan~a~;tion based intenupt ar~;h;~ct-lre.
Interrupts are signalled by sefflng a bU in the EIR (External Interrupt Register) inside the PA chip.
In HSC2tPolo intenupting dsvices may select to interrupt a specific p,ucessor or all ~ cessGI ~ by writing to either one or all EIR leg.st~

6.1.2 Impl~ o~tiongoals The spedf~ goals of the PCIB/HSC2 interrupt scheme are:
~ C .~cl~d InterruptCapsbilities It should be possible to select the target host on a per-PCI adapter basis. This allows tor s~lection ot either a particular CPU or all CPUs on an individual SAM basis.
~ Minimal host o~_.hq~i The host should not have to poll SAM modules in order to determine the source of the interrupt.
~ simultaneous Interrupt su~pG~l Since multiple SAMs may share the same EIR bit any implementation must be able to support simultaneous interrupts. Schemes that depend on only one interrupt being presented until that interrupt is sel~;~d are not accep~hle.
~ Simple Complexity in effler i.a..lwa,e or software is ul~desi~dble.

6.1.3 HardwarelmplQ.~lentati~n In Polo/HSC2 there are some severe system limi~lions on how interrupts are managed within the EIR register of the PA chips. Each PA can have 32 interrupts but of these only a small num~er can be ? " ~ ~ ?ted for a given slot. Without some kind of ~ ~ lic ~al hardware support it would be necessary tor the interrupt handler to poll all PCI adapters on the interrupt. This would not make for very optimal pe"ur,llance.

26 Dec~",ber 1995 42 ~ ~c~
.
.. ...

Polo Software ~ ~gramming Guide _.ratus Company Confidential For that reason the ha..lwi ~ must implement more than a write to an EIR register to perform an interrupt. In order to prevent polling of all thé PCI adapters each SAM writes an interrupt into the EIR register and to an OS specified byte (the Host interrupt table) in cacheable memory. The OS
can specify any location in memory for this byte. This allows the OS a great deal of flexibility. The OS can then scan the table of interrupts at memory speeds to detemmine which PCI adapter interrupted it. In the suggo~ ~ ~d software implementation some possible architectures for this table are discu~se~l In order to implement this scheme the hardware makes use of a series of registers operations and memory tables. The software team should evaluate this p,oposal to ensure that it is both necessa,y and sufficient for their needs. Six registers have been implemented on the SAM for hardware support of interrupts. See the cc--~;.ponding register descripffons in section 12.6.4 on page 129 for a bit-by-bit description.
SAM Int~ ,p~ Source Register~0~
This read only register reflects the state of the interrupt pins on the PCI bus. The signal is high tnue and used to trigger interrupts. This bits reflects the sy"ch,oni~ed value of the intemupt signals on the PCI bus.
SAM Interrupt Mask Registe-10~
This register is used to enable and disable individual interrupts in the interrup~ source register.
Asserbng a bit in this register enables intenupts in the interrupt source register to cause an intenupt event.
SAM Host Intermpt Table Pointer[47:0]
This 48 bit address points to a locabon in rnemory that will be used as a table entry to identify the interrupt source.
SAM Host Interrupt Address Pointer[31:2~
This 32 bit l/O address idenbfies an EIR register address. The interrupt bit register value will be written into this register.
SAM Host Intemupt Bit Registerl5:0]
Bits [4:0l Of this register point to the bit to be set in the host intenupt register EIR and bits 5:0 are written to the interrupt table pointer address. Bit 5 is always a one and functions as a valid bit for the table entry.
Intenupts are gene,aled by either of two conditions:
Cor.~i~on 1: If there is a zero to one transition on the interrupt bU and the intenupt mask bit is set to one an intenupt action will be l,igge,~d.
Cor.dition 2: If there is a zero to one b~n s ~ian on the intenupt mask bit and the intenupt bit is set an intenupt action will be lliggel~d.
The hardware pel h ll IlS an interrupt ope,aliol1 in the following manner:
Step 1: A byte write is pe, f~" ",ed to the address indicated by the SAM Host Interrupt Table Pointer. This write is pe, f " ",ed as a cacheable memory write so the software must use a valid memory address for this location not an l/O address. The data pattem for the write is taken from the least sig"ili~anl byte of the SAM Host intemupt bit register. No validity checks are pe, f-." "ed on the write.

26 Dece"l~r 1995 4 ~ o ~

PCTrUS97/09781 Polo Software Flog,~.,.,.,il,g Guide Stratus ~ .npany Confidential Step 2: A 32 bit l/O write is perlo"~ed to the address indicated by the SAM host interrupt register pointer. The data pattern written is all zeros except for the bit indicated by the SAM host interrupt bit register. It is intended that the software set the address of this write to point to either an individuat Cougar EIR register or to all CPUs' Cougar ~IR register.
No further interrupts will be transmitted to the host by this SAM until one of the 2 conditions listed above is meL
These two writes are not atomic. Sofhvare running on the PA can make no assumptions conceming timing of these two writes. Fu. ll ,e- " ,~fe it is possible for two SAMs to perforrn these op6r~ 'ic ns at the same time. When this occurs any mixing of write operations is possible between the SAMs. If there is a c~-",-lon interrupt bit shared by the two SAMs s~A: - are must ensure that it correctly deco~es all of the interrupts. I lo.~J_~er the writes are strong ordered on a per SAM basis that is the two writes from a particular SAM are guaranteed to be seen in the correct order.
For these reasons, it is sugge~ed that software follow the algorithm below for decoding and handling internupts.

6.1.4 Suggested Software Imple...~ alion As part of the PCI interrupt handling l"echan.sm the OS must manage the host interrupt table which consists of a one byte add~essable piece of in ,-.~ ~n per PCI adapter that is used by the hardware. Since this table is in cacheable memory software must keep the virtual index up to date.
This is p- obably best accb~ ed through static ~ 1oc~tion of the table. Since the SAM register that contains the pointer to this byte is justified down to the byte the sofivrare has a great deal of flexibility in how they orga~iLe the host interrupt table. While there are a number of possible l,r~ani~,lions the su~gested imple~ ticn is to use one host interrupt table per system. This table should be organized as 64 contiguous bytes of PCI interrupt information. This ofgan:za~ ~n allows for the fastest possible scan of the interrupt table. Optimally all interrupts should be directed to one of the CPUs local to the host interrupt table. That CPU can per~orm the steps below and then d~le~ the servicing of the interrupt request to the applupr~ CPU.
The order in which various ope,~ons asso~ ed with the interrupt handling are pe-~u-,,,ed is also illlpOI L~nt. Deviations from the sugges~~~ i"~pleu-eR~ t ~ n may result in either eAl,~neous null interrupts or lost interrupts. The basic steps are outlined below:
Step 1: internupt is received Step 2: intemal interrupts are masked.
Step 3: the ~p,ulu,ùridl~ bit in the EIR register is cleared.
Step 4: The host interrupt table is harvested for any pending interrupts.
Step 5: At this point or later in the sequence internal interrupts can be LIUI I .asked.
Step 6: Gambit or Mirage interrupt mask register is set to O ~d;sabling Gambit or Mirage interrupts~.
Step 7: PCI card specific actions are pe,~o""ed to clear the interrupt.
Step 8: Gambit or Mirage interrupt mask register is set to 1 (enabling Bambit or Mirage interrupts) Step 9: The interrupt is serviced.

26 December 1995 4 , Polo Sof~ware ~ .Jgramming Guide _,ratus Company Confid~" ~' 7. Boot This section of the spec outlines the boot process for booting an OS on either a Polo or HSC2 machine. The boot process consists of the ope,a~~n of loading an OS image into main memory and beginning normal OS eY~cuffon. Before describing the boot, lleth~ aY in detail, the boot altematives that have been cons;de,ed will be desc-ibed.
7.1 Boot All~. "alilles PCI boot is a challenging opportunity for both systems. Since FTIO will be offered in an MIO only configuration (i.e. no BIO) and Polo's only confburation is a PCI only configurabon. A ~ lelh~ d~ 'c ~y for booting the OS in a PCI only configuration must be supported. A number of possible approaches for accomplishing this task are ouUined below.
7.1.1 Baby BlO Boot The Baby Bio boot altemative assumes that Stratus commits to designing a BIO .~pla~,e,.,enl with a PCI i,lte-bce. If Stratus follows that app-ua~ , then the Baby BIO would be the only bool~b'e PCI device and all boot functions would be pe.lu...,ed by the Baby BIO.
The major trade-off in conside- i"g boot altemabves is the balance of hal ~ ~. e effort and software effort. A baby BIO approach to boot le,ulesenl~ a trade-off that minimizes software effort at the expense of hardware. In this approach, a PCI card would be dasi,~"ed that mimics that BIO
functionality but has a PCI i"t~,.hce instead of a Golfbus inte,lace. Much of the BEtll and BIO
software could be recycled, and changes to the boot prom would be minimized. The baby BIO
would respond to a boot culll-lland in the same fashion that it does today A fullda",ental problem with this approach is that it probably could not be ready in bme for inibal power on.
7.1.2 Bio Firmware on Raid Co.l~lol;~r Boot This is a variation on the Baby BIO boot. In this altemative, Stratus would buy an i960 based raid co,)t,."er with 720 SCSI scripts processor;. and ,oort the Bio firmware to it. In this approach, mosV
all of the firmware would need to be rc w. itlen, but the CPU Prom and the BEHI could be maintained.
This app, ua~;l, has the advantage of minimizing the hardware and boot prom effort. I lo.~ ~vcr, it is not clear if this card exists. Also, the firrnware effort would be very large.
7.1.3 CPU Prom Resident PCI Card Boot In this boot all~.llati~c, the CPU PROM would contain all i"'~,r~" ~ie a and algo"~l""~ necessary to perform a boot of either OS off of a given PCI card. Given that PCI cards are not very intelligent, this may be a suLal~lltial amount of code. The advantage of this approach is that the system would be able to boot off of any PCI device. There are several disadvantages to this ap~.,uacl-.
Stratus would have to write PA7100 specific code for each boot device in the system. Every ffme a new bootable device is released, a new boot loader would have to be written and qualified. Also, every time that a new card is ,.lea~d a new boot prom would have to be ~lea3e~ as well.

26 Dece"lbdr 1995 45 Polo Software Plog.dn"" ~9 Guide Stratus ~,ompany Confide"bal 7.1.4 Expansion ROM Resident PCI Card Boot A variation to the CPU Prom resident boot is to use a PCI card resident boot. This method makes use of the eA~,ansion ROM on each PCI card. PCI cards have an expansion ROM that can be used to store self-test boot and other code. Stratus could write PA7100 boot ~t~ rping code for each PCI card that is supported that contail ,s the required boot code. The CPU prom would only need to know how to per~orm a generic ROM copy and then execute that code. This ap,u,ua~;l, requires a small ",odi~,cation to the CPU boot PROM but it would require every new bouldb:E PCI
card to have a special boot driver written for it. This work is e-f~d to be sul~r,~ial.
7.1.5 x86 Emul~tor PCI Card Boot.
A third variaffon on the boot from PCI card theme is to make use of not only the e,.pansion ROM
on the PCI card but the vendor provided boot code as well. This could be done by s~pporL. ,9 an x86 emulator in the CPU PROM. This emulator would then run the PCI BIOS code directly. The advantage to this ~ 'c ~y would be that the installed base of PCI card code is primarily x86 code and this would allow us to make the maximum use of that code. I IOJ~ /er it would take conside, ~ work to support the emulator. Also ha, ~v:_. e changes must be made to support such features as the DOS ~-"t~atibility hole. Finally the address mapping scheme may have to be a~ ed to allow for some address li,.,i~t;ons in the PCI code.
7.1.6 PCMCIA Flash PROM Based Boot A totally different variation on boot is to choose to never boot off of a PCI card. In order to accomplish this boot method a tb-~ic~tPd Flash PROM card would be used instead. This functionality would be dupicated and located on a PCMCIA bus on each ,e~uea~r in MIO and the 2 PCI8 cards on Polo. The CPU Prom would configure the PCMCIA bus and then initiate the copy of the boot-loader or OS into main memory. This ap~,,uacl, has the advantage that the number of boot devices that need to be supported is minimized. The boot process is kept simple. It has the disadvantage that it requires some specialized hardware. Also some new utilities will have to be written to manage the OS images and the tlash prom.

7.1.7 Conclusions Table 10. summary o~ pros & cons Bio CPU Expansio x86 Flash Baby Bio Firmware PROM n ROM
Boot RAID Resident Resident Emulator Prom Boot PCI Boot PCI 'doot PCI Boot Boot I lafL'wa~e strong strong strong strong weak weak effort nega~ ~c positive positive positive negabve negabve inibal Prom weak weak strong weak strong weak effort positive positive negabve negabve negative negabve sul~se~uent strong weak strong strong strong strong prom effort positive positive negabve positive positive positive 26 Dec~"lber 1995 46 ~~ 0(~

.

W097/46941 PCT~US97/09781 Polo Software Prugrsmrning Guide ~uatus Company Confidential Table 10. summary of pros & cons Bio CPU ~ ,ansio Fl Baby Bio Firmware PROM n ROM EmXU8l6ator praOsrn Boot PCI 800t PCI Boot PCI Boot aOot iniffal driver/ weak strong strong strong strong weak os effort positive neg~o negaffve ne~atiJc negaffve negative s~hsen~s~lt strong weak strong strong weak strong driver/OS posiffve positive nes~ negaffve positive posiffve effort cost strong strong strong strong strong weak neg..~ negaffve positive positive positive negaffve The table above lists the potenffal pros and cons of each solution. The standing ~roposal is to use the flash prom boot. This decision is based on the belief that the 2 dosest altematives to the flash prom boot are the baby bio and the e,.~,ansion prom boot. The baby bio minimizes the sofh~lare effort at the extreme expense of the ha,du .,, e efforL While the e,~ansion prom boot minimizes the hardware effort at ffhe extreme expense of the sof~ware effort. The flash prom altemative seems like a good ~
7.1.8 Open Boot A note on Open Boot...
Open boot is a p.~,~osed sLandd,d to allow for a proces~oi and OtS indepan~en~ boot process. It is currenffy undear what direction this process is taking. We have currently chosen not to count on open baot being availab~e in the inibal PolotHSC2 time-frame. I lo.ve.~er, should this change, it may be adva,ltager,us to .~co,.sid~r some of our current directions.
7.2 PCI Boot Process 7.2.1 Find and Configure the Boot SAM
Onoe the PROM has accomplished its normal initialization and is ready to boot ffhe system, it checks ffhe boot list in the ReCC NVRAM. There should be two SAMs idenbfled in ffhs boot list as primary and seodhcl~ly bootable devices. The SAMs and PCI buses for the devices need to be configured before they are used. The h'h . ing steps are followed to do this:
1) Set the HSC2/PCIB on-line by writing to the HSC2tPClB bus interface state register.
2) Set the SAM on-line by wribng to the SAM bus interface state register.
U~lth both the PCIB and SAM on-line, PCI configuration cycles r;an be run to inltialize the PCI bus.
The PCI configuration cydes are accomplished by ~irst writing to the PCI Configuration Address register and then wriffnglreading tolfrom the PCI Configuration Data register. For Configuration cycles to the bridge, the enable bit (bit 31 ) should be enabled, the bus number (bits 23:16) should be 0, the device number (bits 15:11) should be 0, the function number (bits 10:8) should be 0, the register number should be set to the register number for the desired configuraffon register, bits 1 26 Dece"~l,ar 1995 47 ~ ~S

PCT~US97/09781 Polo Software P,ug,an-l-ling Guide Stratus ~,~Jmpany Confidenbal and 0 should be 0.
WARNING: Confburation arJdtess",g is little endian. See the endian des~ io,) for a ~fi~cu~sion of how endian works. The summary is that byte opelalions work with no intervention, 32 bit operations must be byte :~\r-_ppe~l!
1) Per~orm a 32 bit read from configuration address 0. Verify the manufacturer ID, 16 bit word at location 00, is 1107h. Verify the Device ID,16 bit word at locaticn 02, is 0700 for IAM or 0600 for Gambit.
2) Perform a 16 bit write to location 04 of 16'h127. This write disables fast back-to-back enable, enables SERR, disables ECR0, disable address s~p~ , enables parity enor ,~:spol1ses, disablss VGA palette snooping, disables memory write and invalidate cycles, enables special cycles, enables bus " -asl~ring, enables memory space and enables l/O space.
3) Pertorrn a 16 bit write of location 06, the status register, verify that it is 0.
4) Set the memory base address register to 0 by performing a 32 bit write of Oh to address 1Oh.
5) Set the ~is~,onn~ timer to 0 by perforrning a byte write ot 0 to 40h.
6) setthe retry abort timer to 8 by performing a byte write of 8 to 41h.
7) set the trdy timer 10 20h by performing a byte write of 20h to 42h.
The bridge is now configured.
7.2.2 Conffgure PCMCIA Bridge Chip Once the PCI bridge is configured it is necessaly to configure the PCMCIA bridge chip by performing a series of configuration reads and writes to the PCIMCI bridge. For these writes, the enable bit ~bR 31) should be e n~ d, the bus number ~bits 23:16) should be 0, the device number (bits 15:11) should be 1, the funcffon number (bits 10:8) should be 0, the register number should be set to the register number for the desired configuration register, bits 1 and 0 should be 0. The PCMCIA bridge chip ~PPEC~ is a mulfffunctional device l lowe~,rer Polo only uses the first function.
1 ~ Perform a 16 bit read of register 00h. Verify that the Vendor ID is 8086h, intel.
2) Perforrn a 16 bit read of register 02h. Verify that the device ID is 1221 h, ppec chip.
3) Perform a 16 bit write of 14bh to location U4h.111is write disables fast back-to-back enable, enables SERR, disables ECR0, disable address stepping, enables parity error ~es~ unses, disables ~tGA palette snooping, disables memory write and invalidate cycles, disables special cycles, disables bus ~"a:.tt";"g, enables rnemory space and enables l/O space.
4) Perform a 16 bit write of c800h to location 06h.
~) Initialize the PCI-PCMCIA bridge base address register. This register deterrnines ~e starting address of the PCMCIA inde~ ata socket configuration f~:yi5t~ in the PCI I/O space. It is n:~""-~nded that 0 is used is for the base address. This is accomplished by wriUng a 1 to location 1 Oh. After boot this base address can be deconfigured or moved out of the way for norrnal c,c . ,~
6) set the PCI Configuraffon control register. Perforrn a byte write of 39h to locabon 40h. This 26 Dece-nber 1995 48 ~ 0.~
-, .... _. ~, CA 022575ll l998-l2-03 Polo Software t~ nllng Guide _ .atus Company Confidential enables enhanced PCMCIA timing mode enables PCMCIA rea~~ f._tc~, )g buffers enables write posting butfers, set the PCI clock speed to 25Mhz.
7) set the PCI interrupt routing register. Perform a byte of 3h to location 50h. This routes both of the PCMCIA interrupts to the interrupt line and disables interrupts from all other (unused PCMCIA) sockets.
The PCI side of the PPEC chip should now be configured.
7.2.3 Configure PCMC~A Bus The PPEC chip is run in modeO. This oper~;onal mode plulide~ two fully buffered PCMCIA
sockets. Polo and HSC2 only popu'~~~s socket A so all ac~3~ses should be pe,lor",ed to socket A
register and ~ d~v ~ Like the PCI side of the PPEC, the PCMCIA side re~uires some iniffal set up. The set up is accomplished through performing l/O writes to the PCI bus. The PCMCIA chip uses an index port and a data port to access its registers. The index port is a byte located at the base address loaded into the base address register in step 5 above. The data port is a byte located at base address ~ 1.1f the ~'9~ i;h", was foilowed as outlined in step 5 then the base address is at O in PCI I/O space. If the algorithm was not followed then you must do your own nslalidn. Writes can be pe,lo""ed to the PPEC chips by performing a 16 bit write with bits 7:0 cu~ lespûhding to the desired register and bits 15:8 co"e~ponding to the data. Reads may be pelh---led by performing a byte write to the index port and a byte read to the data port.
WAP~NING: this register and all de~riptions are little endian. Byte swapping must be pe,lu-,,,ed if anything other than byte ope,~l;uns are pe-fo....ed.
1) Configure the PCMCIA power. Perform a 16 bit write of 1000h to PCI I/O space Oh. This will disable the output enable for the PCMCIA bus tum off auto-po.. ~ring tum the socket on to 5v and turn VPP on to 5v.
2) Enable the PCMCIA bus. Perform a 16bit write of 9000h. This write enables the PCMCIA output enables for actual writes to the PCMCIA bus.
3) Set up the l/O window data side. The PCMCIA card has an 8 bit l/O bit data path. This is configured by perforrning a 16 bit write of 1007h.
4) Setup l/O windowO. The PPEC chip contains two l/O space ~.;nd~ . . The l/O space is passed directly through the PPEC chip. l/O ~r~.esses are used to setup and control co...~Jon~.n ...anage",~"l reg;~.te.a starting at address 4000h. We will setup 4000h through 41ffh as the llO
space for the PCMCIA bus. These may be decont~ured after boot. Set the low byte start address for l/O window O by perforrning a 16 bit write of 4008h. Set the high byte start address for l/O
window O by performing a 16 bit write of OOO9h. Set the low byte stop address by performing a 16 bit write of ffOah. Set the high byte stop address by perforrning a 16 bit write of 41 Obh.
~) Setup Memory Address M~rr ~9 Window 0. The PPEC contains 6 memory v~ind~ ~ The memory v.;..doJ~ are used for accessi,.g the co"",-and register and the flash array itself. ~or simplicity's sake we will map this address space starbng at O and running through the fu1120MB
range. After boot this address can be deconfi~.lred. The window allows for a 16Mbyte region of PCI address spaoe to be mapped at one bme and for the full 64Mbyte region of PCMCIA space to be mapped. the mapping granularity is a 4K page. We will setup a window of 16Mbytes of PCI
space and 16Mbytes of PCMCIA space and then reset the v:;..do.:s when we roll over 16Mbytes.
Set the system memory address mapping window O start low byte (bits 19:12) by performing a 16 bit write of 0010h. Set ~e system memory address "-ap~ ~9 window O start high byte (bits 23:20) by performing a 16 bit write of 8011 h. Bit 7 of this write sets the data path to 16 bits and bits 3:0 26 December 1g95 49 PCTrUS97/09781 Polo Software Fluglall"" ,9 Guide Stratus ~,-)mpany Confidenbal set address bits 23:20 to 0. Set the system memory address mapping window O stop low byte (bits 19:12) by perforrning a 16 bitwrite of ffl2h. Setthe system memory address n ~p,: Ig window O
stop high byte (bits 23 20) by performing a 16 bit write of 6f1 3h. Bits 7:5 set the timing of the card to 1 50ns. Bits 3:0 set the upper address to f. Setup the PCMCIA card offset address. This will initially be 0. We will bump this up when we roll over the 1 6Mbyte region. Set the card memory offset address O low byte register by perfomming a 16 bit write of 001 4h. Set the card memory offset address O high byte register by performing a 16 bit write 001 5h.
6) Enable the address Ill-.r~ 9 ~;.ldo- ;.. Perfomm a 16 bit write of 4106h. This enables l/O
window O and memory address window 0.
With these sefflngs:
1/0 ad.l~ses O and 1 map to the PPEC chip.
1/0 add~t:sses 4000h - 41ffh map to PCMCIA llO space Memory ad~b~sses 000000001~ - OOfflfffh map to PCMCIA space.
7.2.4 Configure PCMCIA Flash Card.
The PCMCIA tlash card is ~ces~;' le as a slave on the PCMCIA bus. The card is o,yan,~ed into an attribute space a~essed through l/O space and a memory space arcessed through memory space. The first step is to configure the l/O space to the proper configuration.
1 ) Configure the voltage register by performing a byte write of 01 h to 4000h. This sets VCC to 5v and VPP to undriven.
2) turn on VPP generdtion by pe~lu"~ ~y a byte write of 01h to register 410ch. This tums on the intemal VPP geuerdtur in the PCMCIA chip.
The card is now ready to be read for boot. This ope,dt;on is started by performing a co"""and write. The co"""al)d write sets the mode of the flash array. Perform a 16 bit write of hex fflf to memory space Oh.
The card is now in read mode and memory reads wiîl be p,ocessed directly to the array. The array may be read in 32 bit chunks.
Notes:
This description did not describe setting up the Gambit/Mirage offset window registers to the PCI
bus. These must be setup.
This description did not explicffly explain the ~ep~ugldlllllling of the card offset register when the prom rolls over the 16Mbyte PCI page.
Writing the PCOM is not deswil~d and will be covered in a later revision of the spec.

26 Dece"lber 199~ 50 .

WO 9714Cg41 PCT/US97/09781 Polo Software P~ramming Guide ~,.,atus Company Confidential 8. PCI Configuration The Polo and HSC2 PCI bridge and adapters have to be configured. This wiil need to be done at boot bme. There are also configuration issues that need to be addl~sses when new devioes are plugged in, pulled out or when they fail.
PCI is defined to allow the PC to do an auto configuration. The PC BIOS ~;S~J~ which devices are on lhe PCI bus and configures the b~idges and a~pt~ There is some user intervention required in some cases whsre it cannot be completely automatic - see PCI ECR Status document dated 5111194.) When a configuration is complete the inforrnation is stored in non-volaUle memory and is used in s~hseql~ent boots as long as the same devices are present We need to support a configuration similar to the auto configuration done in PCs. Cards may have s~v.n~sd since the last booL We also need to support hot-plug re~,lfiyuration due to board pseudo-hot plugs in the Polo and HSC2 systems.
8.1 PCI Configuration Register Space Each PCI agent, including the host bridge within the GambiVMirage ASIC, has a 25~byte configuration space. The frst 64 bytes of this confburation space ~"""ises the required PCI
confiyuration registers. These are a predefined header containing ID, status, ~"""and and other information. The optional configurabon space from 040h-FFh is available for use by bridges.
GambiVMirage will not use this area. All PCI adapte,~/dger,~ must respond to the entire 256 byte configuration space. All writes to undefined areas must have no effect and all reads to undefined areas must retum zeros.
The Gambi~Mirage ASIC implements Accesses to the configuration space using a method similar to the PC/AT. ~.r~Qssss to the 32 bit IIO registers, config_addr and the confb_data, defined in the GambiVMirage l/O register space are used to generd~e PCI cycles with the Configuration Read or Write funcbon code. First an address is set up by wribng to config_addr then a read or write is done to config_data. See configuration section 8.5, PCI Address forrnabon, for spsr,i~il,s on how the config_addr decodes to the individual PCI devices on the PCI bus.
8.1.1 Gambit PCI Configuration Space The PCI configuration space of GambiVMirage is not available to other agents on the PCI bus, but only to Xbus generated re-lu~ A~sses to these reyist~.;. are made through the 'config_addr' and 'config_data' registers in Gambit IO Register Space. The Gambit PCI Configurabon Space is available as 'Device Number 0' for type '00' ACc9sse\s~
Tabie 11. GambiVMirage PCI Configuration Space Header Offset Register Name RNV RvaelSueet Des~ tion 00h \fendor ID R 1107h Unique manufacturer ID - Stratus PCI ID
02h Device ID R 0600h Identifies the device as per vendor 29 Dece"lber 1995 51 ~R oo~
, . .. .

Polo Sottware ~,~,g,d.,.nling Guide Stratus ~-,,npany CorfiJe" -' Table 11. GambiUMirage PCI Conffguration Space Header Oltset Register Name R/W RvaelSueet Des~"i,~ion 04h Co~ ldnd 15:10] R OOh lle~e.~ed 9 R O Fast back-to-back Enable 8 RNV O Global SERR# Enable. Discrete enabJes in ECRO.
7 R O Wait cyde control - Address/Data Stepping 6 R/W O Parity Error Res~ se Enable R O VGA Palette Snoop - Not used.
4 R O Memory Write and Invalidate - Not used.
3 R/W O Special Cycles- Special Cycles.
2 P. 1 Bus Master Enable '1 R~W O Memory Space Enable o R O l/O Space Enable 06h Status 15~ RNYl O Detected Parity Enor.
14 Rl\N1 0 Signaled System Error.
13 R/W1 0 necei~J0d Master-Abort.
'12 R/W1 0 Receive~ Target-Abott.
11 RIW1 0 Signaled Target-Abort.
10-9] R 01h DEVSEL#hmins: OO=tast,01=medium,10~slow.
y RfW1 0 Data Parity Error Reported.
71 R O Fast Back-to-BackCapable.
6:0] R O Reserved 08h ncvi~'~n ID R OOOOh IAM Rev number.
O9h ClassCode R 06h Bridge Device OOOOh Host Bridge (GambiVMirage) OCh Cache Line R OOh Gambit/Miragenotsl"~po,i ~9 cache pr. ~,a' Size ODh Latency Tlmer R OOh No Latency Timer.
OEh Header Type R OOh Header layout as per PCI spec.
OFh BIST R/W Built-ln Self-Test 10h- Base Address RrW Refer to secbon 62.5 PCI Specification Rev 2.0 27h 28h- Reserved 2Fh 30h E~pansion R/W Refer to secbon 6.2.5 PCI Specification Rev 2.0 ROM Base Address ~4h- Reserved 3Bh 3Ch InterruptLine R OOh Noti",,1~ ~.e"t~d 29 Dece"lL)er 1995 52 Polo Software ~ ._,aramming Guide _..atus Company Confidential Table 11. GambiVMirage PCI Configuration Space Header Offset Register Name RNV RvaelSueet Description 3Dh Interrupt Pin R 4h Not implemented 3Eh MIN_GNT R 0 Not implemented 3Fh MAX_LAT R 0 Not implemented 8.1.2 PCI Adapter Configuration Space The PCI conf~urabon space of the mated PCI adapter is available to the GambiVMirage ASIC
across the PCI bus. Ar~sses to these rey;~tt5ls are made through the 'config_addr' and 'config_dataH~y;~te;~;~ in GambiVMirage IO Register Space. The PCI Adapter Conflguration Space is available as 'Device Number 1 ' for type '00' ac~sses. For adapter specfflc details on the Configurabon Space Header, refer to the PCI Specification Chapter 6 and the app,op,iale Adapter Spscih~tion.
Table 12. PCI Adapter Configuration Space Header Oflset Register Name De~ripbon 00h Vendor ID Unique manufacturer ID
02h Device ID Identifies the device as per vendor 04h Co,-""and 06h Status 08h Revision ID Rev number.
09h Class Code Class of devioe OAhOBh Sub-ClasslProg l.F. Sub Class / ~oyldlllllling Interface OCh Cache Line Size Not supporting cache protocol.
ODh Latency Tlmer Refer to secbon 62.4 PCI Specificabon Rev 2.0 1 Eh Header Type Header layout as per PCI spec.
OFh BIST
1 Oh-27h Base Address Memory and IO Offset Add~sses as per PCI
spec.
28h-2Fh Reserved 30h EA~,anaion ROM Base Refer to secbon 62.5 PCI S,r s~ 'iuabon Rev 2.0 Address 34h~Bh Reserved 3Ch Interrupt Line Identifies interrupt co"~-.11er input. Refer section 6.2.4 of the PCI Specificabon 29 De~"~er 1995 _ 53 Polo Software Pluy.c""".;ng Guide Stratus ~,ompany Confidential Table 12. PCI Adapter Configuration Space Header Offset Register Name Description 3Dh Interrupt Pin Ider,tlfies Which INT# signal the PCI device asserts 3Eh MIN_GNT Suyg~ l- d Latency timer value 3Fh MAX_LAT Sug~esl~d value for tuning the PCI bus.

8.1.3 PCI Configuration space header d~-~cri~ n These registers are defined in the configuration space for system buslPCI host bridge and for each PCI adapter in the system.
The Vendor ID field is read in each slot to check for card e,.iat~nce. It is a 2 byte field. The PCI
spec. states that if the read returns 16 hFFEF this is an indication that the device does not exist.
On Polo/HSC2 the XbustGolfbus l/O read will be aborted and cleared out of the PING table in Gambit'Mirage. This will result in a PING timeout which will then cause a data return of zeroes with an LPMC to be sent to the Cougar. The OS should search for available cards and not the driver code. Otl,er~ e the vendor ID should be checked for a valid Stratus supported device.
The next feld read should be the Device ID. Again it should be checked against known devices for the vendon The Revisfon ID may have meaning in some cases. We need to see what specific adapte.a do here and what aiyl ,ifi~ance the number holds.
The Header Type idel~ti~ies the layout of bytes 1 Oh through 3~h and bit 7 indicates if the device is a multi-function device. Type 00 is a norrnal card and type 01 is a PCI-PCI bridge header fommat.
The other encoci;n~s are reseNed. If the device iâ multi-function the all of the function numbers have to be polled for e~islence. If it is a PCI-PCI bridge there are 2~ Qaal type 01 configuration cycles required to configure the lower level PCI buses The Class Code field is a 6 byte field, the first two bytes indicate the base class (VGA, Mass Storage, Network, Bridge...) The next two bytes are the Sub-class code (SCSI, IDE, Floppy...~.
The next 2 bytes are the ~loy,c",-,uing Int~ ce which indicates a register level p(oyldlllllling interface for device inclependent solh~_rt:.
The Con"~and register control the major functions of the PCI card. Writing all zeros to this register logically di~.conne-,~ the card from the PCI bus.
Bit 9 - Fast Back-to-Back Ensble - Reset since GambitlMirage will not support it.
Bit 8 - SERR# enable - This bit will be turned on.
BR 7 - Wait cycle control - Reset to zero if writable.
Bit 6 - Parity Error Response - Set to enable parity cheching Bit 5 - VGA Palette snoop - Not supported Bit 4- Memory Write and Invalidate - Not supported Bit ~ Special Cycles - Reset to zero to disable. I la, Jua~ is capable if Software wants to tum this on.
Bit 2- Bus Master - Set to 1 Bit 1- Memory Space - Set to 1 Bit 0- lO Space - Set to 1 on ada~ , Set to 0 on the bridge.

29 Dec~"lber 1995 54 ( ~l q, Polo Soflware P~g-a,~ ing Guide~ratus Company Contidential The Status register Bit 15- Detected Parity Error This bit indicates that a devioe ~J~ 'd a parity error. A Maintenance Internupt is gene,dt~d from GambiVMirage. An adapter is .t:sponsible to report a parity error to the system most likely via an interrupt to its driver.
- Bit 14 - Signaled System Error This bit gets set whenever a device asserts SERR# which is a global error signaldirected to the Ope, ati"g System. SERR# indicates an address parity error data parity errors on a special cycles or other non parity errors. SERR# will cause aMaintenance Interrupt to be gene,at~ from GambiVMirage.
Bit 13 - Received Master Abort GambitlMirage causes a Maintenanoe Interrupt.
Adapter - TBD
Bit 12 - Received Target Abort GambiUMirage causes a Maintenance Interrupt.
Adapter - TBD
Bit 11 - Signal Target Abort GambiUMirage causes a Maintenance Interrupt.
Adapter - TBD
Bit l10:9] - DEVSEL timing 00 - fast 01 - medium 1 0 - slow These bits are read only and indicate the speed at which a device will decode es to its address spaces. It can be used by the subtractive decode agent to speed up se'e t; ~ ~. GambiVMirage does not do subttactive derode so the Master will have to use the Master Abort Mechanism.
B-lt 8 - Data Parity Detected This bit is only set by a bus master when: 1. When ~e master asserts PERR# (on aread l,a,~baction) or when a master samples PERR# (on a write l,ansa~ion) and 2.The Parity ertom~:aponse bit is enabled.
Bit 7- Fast Back-to-Back Capable The Cache Line Size register - This register only needs to ~e implemented on devioes that are partidpating in the caching protocol which we do not suppott.
The Latenc~ rmer- register must be writable on all master devioes that can butst more than two data phases. It may be tead only on devices that butst two or fewer data phases but must contain the number 16 or less.
The Built-in Self Test (BISJJ register Bit 7 - BIST Capable Bit 6 - Start BIST
Bits ~5:4] - Reserved - Bits l3:0] - Completion code: O indicates test passed; non zero device specffic failure codes.
The Intenupt Line register - Use of this register is TBD. In a PC it co"espondb to the IRQ level of - the 8259.
The Interrupt Pin register - This register indicates which interrupt line the devioe will use. It will contain O if no interrupts are used. Polo will tie all intetrupts to INTA#.

29 Deoe.. ,~- 1995 55 ~ 3 CA 022575ll l998-l2-03 PCTrUS97/09781 Polo Software F~,yf~l-"lling Guide Stratus ~.o-l"~an,~ Confidential The MIN_GNT register- This register s~cifies how long a burst period the device needs assurning a 33 Mtlz clock. This value should be the fecGIl,,,,en-Jed Latency rlmer value or 0 if there are no spedal requirements.
The MAX_LAT register - This register contains the bme this device can wait for a grant. This number is used to set the latency ffmers of other devices. There must be some algorithm/
equation but we don't know it.
8.2 r-up~d conffguration sequence 1. Boot system - See Section 7.
2. Start OS based config-for each PCI slot and sub-function a. Device d~ Configure host bridge if it exists b. Match device against requirements - then config the device i. Run the post code on the devices r~ansiùn ROM
ii. or Run the cards initialization code stored in the OS files system iii. or Restore the previous configuration (unlikely to be choice. diccll~sed later) c. Shut down any u,,h,ù.~,,/unconfigurable devices d. Update host bridge configuration with new address infonnation.
- e. Run any didyl IOaliCS on device i. ~Apdnsion ROM based diagnG:.tics ii. or OS based diagnostics f. Handle special configuration i.e. dual initiated scsi host ids etc.
~q. Set up device drivers 8.3 OS Based full 5y~ Configuration The PCI config_addr and config_data r~g;~t~, s are in Gambit/Mirage and there is one register that ,onds to the address cG"t:sponding to 4 SAMS. They map as ~ollows:
MlO Polo HSC2 SAM0 PCIB Gambit~side HSC2 Mirage SAM 1 PCIB GambitC-side HSC2 Mirage SAM2 PCIB Gambit~side HSC2 Mira~e SAM3 PCIB GambitC-side HSC2, Mirage SAM 4 PCIB Gambit D-side does not exist SAM 5 PCIB Gambit D-side does not exist SAM 6 PCIB Gambit D-side does not exist SAM 7 PCIB Gambit D-side does not exist NOTE: To control access to these multi-,esponse register the OS needs to lock the group of 4 SAM reg;~u ~ since they are physically one register.
8.4 PCI device detection All PCI devices a~Jlo, ~ and bridges must implement the first 4 fields in the header Vendor ID, Device ID, C~"",~nd and Status. The additional fields are optional but if used must be at the apprup(idle memory address.
In a PC all possible PCI slots must be polled to determine the configuration. The polling starts at device 0. If a multi function device is detected the all other functions in the slot must be polled. Our system will avoid polling since each SAM will supply a " ,ai. ,tenance interrupt when it powers on or 29 Dece"lLer 1995 56 . .

Polo Sof~ware P~g~a~ uing Guide ~atus Company CGnf,~ential after a reset. When the OS is ready to configure the system it will reset all of the SAM slots. Each exisUng SAM will supply a maintenance interrupt.
Polo and HSC2 have 4 PCI slots per PCI bus. There will be one ",aintenal1ce interrupt per PCIB/
HSC2. Software must poll each PCI~/HSC2 slot for ~he E~islence of a PCI adapter. The PCI
adapters are a~ed as though they are pseudo SAMs (see Section 8.5). Polo needs to check each slot for rrndff-function adaptors see section 8.5.1.
8.5 PCI AWress forrnation GambiVMirage will format configuration ad.h~sses from config_addr as explained in the PCI rev 2.0 spec.
8.5.1 Configurationcyclegeneration The ha.~.e will co-"pare the ~Bus Number'l trcm the address to the bridges secor,~,y bus number which in our system is 0. It will gene,~t~ a type O configuration cycle when the ~Bus Numbe~' is zero ~ e~ 3e it will generate type 1 configuration cycle. When a type 1 configuration cycle is created the lower 2 address bits ADl1:0] need to be .;I,anged from 2'bO0 to 2'bO1.
For ele~G i~al reasons we require that the address for a configurabon cycle be Pre-driven. Since the address lines will be r~sis~ l~1y coupled to the IDSEL lines going to the adapter the IDSEL
lines will have a low slew rate. The address must be stable TBD clocks before FRAME#.
8.5.2 Special cycle generation When the confi~ addr register is loaded with the Device and Function numbers of all ones and the Register number is zero the bridge is ~ - i" led for a special cycle. When our config_data register gets written we genet~te a special cyde if the bus number equals zero. Other~- ;..e we will send it on as a config_write. Lower level PCI-PCI bridges that match the Bus number will then create special cycles on their secondary buses.

29 Decelllbet 1995 ~ ~ 5 .. . ..

WO g7/46941 PCT/US97/09781 Polo Software r~oy~a"l.. ;ng Guide Stratus ~,ompany Confidenbal ~igure 24. GambiVMira~e Bridge Translation con~ addr type 00 -~ PCI AD
config_addr 31 30 24 23 1615 1110 8 7 2 ~ o Bus Device Function Register 0 Reserved Number Number Number Number ~

~ Enable bit 10 ~o- ~lS:16] PCI Addr ~s ~31~
etAD11 (Gambithostbfidg~) \
00 ~et AD 12 01 ~t AD 13 10 ~etAD 14 11 ~tAD 15 ~

Device Number-OnlyOne 1 Number Number ~ ~
PCIA~ 1110 87 2 1 o Figure 25. Bridge Translation config_addr type 01 -> PCI AD
conflg_addr Bus Devioe Funcbon Register 0 noser~0d Number Number Number Number ~

Bus Device Function Register 8 hOD Number Number Number Number 31 24 23 15 1110 3 7 2 1 o PCI AD
8.~.3 Polo/HSC2 Conffguration A~h~ss Generation Polo/HSC2 have 4 PCI buses with 4 PCI slots each. The devices will be numbered 1-4 on each PCI bus. For type 00 configuration cycles the Gambit will form PCI address bits l31:11 ] from the devioe number in the PCI config_addr register along with bits l16:15] of the XbuslGoHbus IO
Space ad~ress from the cycle that ar~essçs config_data to mimic which of the 4 pseudo SAMs is being a.hll~ssed. If the Device Number equals zero the bridge is aoeessed if it is non zero devioe 1 is A- oessed Therefore device 1 should always written to confb_addr to access an adapter. See example below. The IDSEL[3:0] will be tied to AD~15:12] ~spe~.~, ly again see section 3.6.4 of the PCI spec for ele~ al details. Polo/HSC2 also needs to check each slot for multi-funcffon adaptors see Section 8.1.2.1.

29 De~"~ber 1995 58 ~ , . .

Polo Sof~ware rlu~ ing Guide Slratus Company Confidential PCI Device Field lO Space 116:15] PCI Address [31:11]
O XX setAD 11 (GambiVMiragehostbridge) !0 00 setAD 12 01 setAD13 setAD 14 ~O 11 setAD 15 8.6 Multi-function Adaptors A PCI adapter can have mulUple funcbons on one card. For example SCSI and enet or multiple SCSI ports. These sub functions are add~essed through the function number field in the confiQaddr register. All cards must use function number 0. If they have mufflple functions then bit 7 of the Header Type configuration register indicates whether a device is a mult~function device.
If a multi-tunction device is d~ ed all possible function numbers must be polled for existence.
The PCI spec. allows them to occupy any function number after function 0. Each function has its own configuration header and thel~ol~ has to be configured into PCI address space as an individual device.
8.7 SAM and PCI Adaptor C~ ration Once the bridge device is detected the bridge can be used for PCI configuration cycles to detect resident PCI devices without any bridge configuration. Sof vqre then reads the header infomnation for PCI device 1. The PCI device needs to be mapped into PCI address space. First ~he sofl.._.t, has to determine how much memory and lO address space the device needs. This is done by writing all ones to the base address registers and reading them back. The device will retum O's in all don't care bit positions effecUvely sper~fy;ng the address space required.
Note: The typical device will have two base address ,~g,ste,s in the header i.e. requiring one memory range and one lO range. The low order bits distinguish between the memory and lO. See PCI local bus spec section 6.2.5.1.
8.7.1 Base A~ ss Regis~ers The number of high order bit that an adapter impements detemnines how big its memory space is.
For example Gambit/Mirage ~e:.~,or,d~ to 256 MB so it implements the 4 high address bits. If software did not know how big the range was it could write all ones to this register and it would read back fOOO_OOOOh indicating the size and the fact that it is a memory space base register. See figure 26 and figure 27 for the formats of base registers.

29 Dec~"lber 1995 59 ., PCTrUS97/09781 Polo Software ~,vg/a,-""ing Guide Stratus Company Confidential Figure 26. Base Address Register for Memory r, v~ l ,able t Type Memory space indicator Figure 27. Base Address Register for 1/0 Reserved t IO space Indicator 8.8 PCI Address Space XbuslGolfbus addl~s.,-g of PCI address space is well defined. Inside each SAMs address space there is a 64KB memory window and a 32 KB IO window from the XbuslGolfbus. The PCI card's memory may be much larger than these ~;n~ s and the PCI card needs to mapped into a complete range of PCI address space. Fl.,U,e,."o~é for Polo we need for these ranges to be non-overlapping from SAM to SAM (,ell,e"ltJerthattherê are 4 pseudo SAMs per PCI bus in Polo).
This non-overlapping address space needs to be tracked by the configuration sof~ware. There are a number of issues around this PCI address space related to board inserlior~e".o~al. When a new board is added to the system its memory and IO ranges have to be mapped so that ~t does not overlap with any previously mapped devices. When a board is pulled it will leave holes in this PCI
address space. This brings up the need to d~ haylllent the PCI address range if new devices cannot be mapped into the existing ranges.
(There are other attemative here like a fixed per slot allocation which may not work due to large possible memory and IO ranges. The contiguous range has to work per PCI bus. Four separate ranges can be tracked one for each Polo PCI bus or one per bucket if that is easier.) 8.9 PCI Bus Target and Master Aborts The bus master detects a master abort when no target r~:,pon~ to its request. The master sets bit 13 in the its PCI configuration status register. The Bridge (GambiVMirage) also watches the PCI
bus for master aborts and sets a bit in its PCI Error register. This condition will cause a mainlenance interrupt if it is not masked off in the SAM IO space Configuration register. If the 29Dece"~ber1995 ~l~ 60 CA 022575ll l998-l2-03 W O 97/46941 PCTrUS97/09781 Polo Software ~.~,gramming Guide _..atus Company Confidential bridge acting as a master detects a master abort it sets bit 13 in the its PCI configuraffon status register and may cause a maintenance interrupt if it is not masked off in the SAM lO space Configuration register.
A target signals a target abort when it gets an enor on a ~cnsa Lon. The target sets bit 11 in its PCI Configurabon status register. The master sets bit 12 in its PCI Configuraffon status register. In Polo and HSC2 the bridge is either the master or target in every transaction. The bridge is aware ot the abort and sends a maintenance interrupt again based on the mask bits in the SAM lO space Configuration register.
8.9.1 PCI Bus Fault Det~~ti~.- and Tolerance See M I D section 9.
8.10 On Line Adds The PCI configuration will have to be updated when a JettatHSC2 or a Polo/PCI card is added to the system. This, ne~l ,an,~., l is different tor HSC2 and Polo.
Although the HSC2 board itself can be inserted, PCI cards lhelllselv0s cannot as they are ~"ne~lèd on the HSC2 board via PMC (PCI Mezzanine Card) Adaptors.
Polo requires the PCI bay to be po. erèd down for a board insertion. When a bay comes back on line after a power down the OS should go through a full configuration sequence.
8.11 On Line Failur~s An on-line failure is a condition that breaks the SAM. These include SAM errors and PCI errors.
See the error leg;slel;~ in section 12.6.
1. Reset the failed device.
2. Check the failed device.
3. Configure it back into the same place it used to be.
8.12 Board n,.-,oval Polo requires the PCI bay to be ~.. ~red down for a board removal.
Although the HSC2 board itseU can be removed, PCI cards Ulelll~,GlJes cannot as they are cunne.,tod on the HSC2 board via PMC (PCI Mezzanine Card) Adaptors 8.13 Special Requirements On reads from bit encodëd fields with reserved bits software may not rely on the value in these fblds. Software must use the a~ pro~liate masks. On writes software must read the register and merge in the reserved bits to the new data to be written then write back the merged word.
We need a ~"~.,I,anis.,- for ~,uy,~l"",ling and storing config information for PCI cards with l~ser sele~ e config. opbons. For example we need to change the SCSI host adapter id number for the second host on a dual initiated bus. There is additional information on this topic in the PCI ECR
Status document dated 5/11t94.

29 Dec~"ll~r 1995 61 ~19 CA 022575ll l998-l2-03 PCT~US97/09781 Polo Sofbvare r~oyra.,.,ning Guide Stratu~ ~,ompany Cor,fide~ial When an e.~nsiun prom is decode enabled the s.lf; a, e must not access the device through any other base address It:yiste,~. This is bec~se PCI adapt~,rs may save gates and share the address ~le-~ for the e~.ansion ROM with another base register.
See the byte order dica~ss~on in section 5.6.
8.13.1 Lock requi,~
The page 2 SAM It,y;Sld,S (section 12.6.2) need to be locked due to Polo multi registers. See section 5.5 for more information on locking these registers.

29 De~,.k,e- 1995 62 .
. . .

Polo Soflware R~ "~ ng Guide S~ratus Company Cor,fiJe"tial 9. PCIB M&i--t~-,ance & Diagnostics 9.1 OYenriew The Maintenance and Dia~nG~li~ (M&D) subsystem is a major part of the value-added in a Stratus system. It is ,~sponsilJle tor managing the configuraffon of the machine as parts fail and are replaced, and as the machine is upy-c.ded on-line. It handles the ~ ~fti. lg and logging of error informaUon, but does not attempt to diag,.ose the root cause of a system problem by interpreting the error status.
Polo's PCI sul~/it~", consists of 1 to 14 PCI cards a"an9ed in two 7 slot groups. Each group has 8 controller card (the PCIB) bridging betw3c.1 the Xbus and two multi-slot PCI buses, and a PCMCIA flash card on the PCIB. The HSC2 su~r~tt"" consists of at least one ~simp~exed) HSC2 board with 1 to 4 PCI cards. On Polo and HSC2, the "SAM" is just an a~t,_ - ~ n in the address space; 8 ~'SAMs" are implemented on a single PCIB card, 4 "SAMs" are implemented on a single HSC2 board, and all power up and down simultaneously. The only time new SAMs (PCI cards) appear is when a PCIB/HSC2 is pu.~ered up.
9.2 ID Proms There is a Jetta-style ID prom on the PCIB and HSC2 for tracking board revisions and for error logging. The ID proms on the PCIB and HSC2 should not be p,uy,a,,,,,,ed with ~"'~ " at~n conceming the PCI card in the SAM's PCI slot.
There are no Stratus ID PROMs on the PCI cards ll,e",~,elves; these boards are identified by reading their PCI-defined configurabon space. All PCI cards provide a Vendor ID (ass;gned by the PCI Special Interest Group to assure uniqueness), Device ID, and Revision ID. The PCI cards will not contain individual serial numbers in an on-line readable fommat. Most PCI cards have bar-code serializabon only.
The Polo ba-,h~lane ID Prom is ~ssed through the ReCC, identically to Jetta. Additionally, the Polo l/O power supply has 8 bits of ID information in the register space of the gambit ASIC.
9.2.1 Fault Logging An i",~,~nt function of the ID proms is the logging of fault informabon to aid in ,Jiagnosis when a defective board is retumed from the field. Each ID prom records faults in a 512 byte parbbon. The partition initially starts out empty, and it is filled in a round-robin fashion as faults occur.
Note that PCI cards do not support memory dumping. Any infomlabon from a stuck PCI card must be copied prior to resetting the card. Virtually all PCI cards zero memory on reset.
The ~ ..;.,g ~eg;~ are logged for each board type. Note that different boards record different ,ey;ste,;,.
9.2.1.1 PCIB
A single PCIB is a bridge to 8 PCI slots. Accordingly, the relevant per-slot SAM registers appear 8 times in the fault log. In the f~"c .. ;, ,9 table, 'xxx~ takes on the hex values 404, 40C, 414, 41 C, 424, 42C,434, 43C. The four config space registers are the first 4 DWORDs of the PCI config space in the PCI card in the co"es~,onding SAM; the M~D perforrning the logging must access them using 29 Dece,-lber 1995 63 ,,~\

PCT~US97/09781 Polo Software P~ugrd~ ing Guide Stratus u;ompany Confidential the PCI configuration space n,eul,anis",~ None of the per-bus registers are judged useful for diagnosing tailures and they are not included.
Table 13. PCIB registers logged Offset Register Narne size t8 Board ~eset 8 7FFFE0 Buslnterface State 32 7FFFC8 Slot 1[) 8 7FFF78: Gen. Purpose Comm 13:û] 32 7FFF38 Bus Interface Fault Repor~ng 16 7FFF30 Common ~roken Status 16 7FFF28 ASIC Specific Broken Status 16 7FFF20 Bus Info Error Status 16 7FFF18 Misc Error Status 16 7FFF10 Control Bus Error Status 16 7FFF08 Bus Error Byte Status 16 7~FF00 Voter Error Tnanscei-Jer Status 16 7~F008 PCIB Status 32 7FF000 PCIB Configuration 32 xxxFV8 SAM l~CI trror Register 32 xxxFB8 SAM Status 32 xxxFB0 SAM Configuraffon 32 config space 00 Device IDNendor ID 32 config space 04 Status/Co"""and 32 config space 08 Class Code/nevis ~n ID 32 config space 0C BlST/header/latency ~il"~,/cd(;he 32 line size g.2.1.2 HSC2 A single HSC2 is a bridge to 4 PCI slots. Accordingly the relevant per-slot SAM registers appear 4 times in the fault log. In the following table 'xxx takes on the hex values 404 40C 414 and 41 C.
The four config space r~y;~te. :. are the first 4 DWORDs of the PCI config space in the PCI card in the co" esponding SAM; the M&D performing the logging must access them using the PCI
configuration space ",ecl,ar,;_.-,. None of ~e per-bus ~eg;~.t~..s are judged useful for diayllosil~g failures and ~ey are not included.

29 De~--lLer 1995 64 ~, .. . . ..

Polo Software Pl~ramming Guide Suatus Company Confidential Table 14. HSC2 r~g;sle,~ logged Offset Register Narnesize 7~t8 Bosrd Reset B
7FFFEO Bus Interface State 32 7FFFC8 Slot ID 8 7FFF78: Gen. Purpose Comm l3:0l 32 38 Bus Interfar,e Fault Reporting 16 7FF~30 Common Broken Status16 7FFF28 ASIC Specific Broken Status 16 7FFF20 Bus Info Error Status 16 7FFF18 Misc Error Status 16 7FFF10 Control Bus ErrorStatus 16 7FFF08 Bus Error Byte Status 16 7FFFOO Voter Enor T.ansce.ver Status 16 7~008 PCI~ Status 32 7FFOOO PCIB Configuration 32 xxxFV8 SAM ~CI tnor Register 32 xxxFB8 SAM Status 32 xxxFBO SAM Configuration 32 confi s~ace 00 Device ID/\/endor ID32 confi s~ace 04 Status/Co--",~nd 32 confi s~ace 08 Class Code/nevi~i?rl ID 32 config space OC BlST/headerllatency ti."el1cscl,e 32 Iine size 9.2.2 llO Power Supply Faults There are 2 status bits that identify that an l/O Power supply in a Polo system has failed. This inforrnation will k:ll~ti~_ly be stored in ReCC status registers. Spe~ "y, I/O power supply fault status will be stored in the ReCC bus and Local fault status register (address OxFOOOOB). Bit 2 will be used for l/O power supply 1 fault, and bit 1 will be used for l/O power supply O fault.
9.3 Insertion & Removals 9.3.1 PCIB devices After insertion, power-up and release of reset, a PCIB issues a maintenance interrupt to the host CPU. The host then goes through the process of setting the board on-line:
~ read the id prom and verify revision co,,-~uatiLlility with ~e rest of the system ~ set the yellow led on ~ TBD
When the lid to one of the Polo PCI card bays is opened, the power to the PCIB and 8 ac,cot;~l~d SAM slots (7 physical PCI slots and the flash ROM) is switched off. To sof~ware, this is identical to 29 Dece:"lber 1995 65 ~ ~ 3 CA 022~7~11 1998-12-03 Polo Sottware ~lug,d""ning Guide Stratus vompany Confidential a simplexed MlO being pulled from a system. In theory if all disks are duplicated on different controllers and if comm is duplicated in both PCI bays as needed opening a PCI bay lid on a running system should cause no l nùble",s. I loJ~evcr it may be advdr~eous to include an M&D
funcbon that 'ql'iL S S e s all operation in a PCI bay in preparation for po. ~,i"g it down. From a signalling sldndpo:nL the yellow L~D will be illuminated on the remaining power supply.
9.3.2 Polo PCI devices In the Polo and HSC2 systems there are no SAM w~d~,ped around the PCI cards to provide hot plugging. As a result new PCI cards appear in the system only when a PCIB or HSC2 powers on and the code to add a PCI card is simply run atter the code to add a PCIB/HSC2. In Polo there is a single MAP ram in the CPU board; in Jetta there are sep~(ate MAP rams in each HSC2.
9.4 ResetOver~iew All Xbus/Gbus boards support at least two levels of reset a Uwarm'' reset and a ~cold" reset.
Cold reset resets as much state as possible on a board. It is always applied to a board by hardware upon first powering it up so that the C/D sides are sutficiently similar to permit operdlion of the board without b,~aki"g.
Warm reset resets as little state as possible. It is intended to unwedge a broken board with destroying as little state as possible so as to permitthe diay,losis of fault coh.lilions.
As part ot the reset process hardware first puts a board into the broken state and then about 1 " ,: ;, osecond later unb~al-~ the board. This has two consequences:
There is an app,u~i,,,alely 1 mi~usecond period during reset that a board will not respond to bus activity.
A board gene,ales a ",ai"~nance interrupt when it breaks or unbreaks. This means that a healthy u~ rùken board will generate two mainlenance interrupts as part of reset process and a broken board l" ,bfoken by reset will generale a single maintenance interrupt.
This next section will attempt to Sl""" lal i~e the effect of reset on the major c~l"pone, lls of the subsystems. For detailed bit-by-bit desul r ~ 1S please refer to Register Dsfinitions on page 82 of this docurnent.

9.4.1 POlO n~
9.4.1.1 CPU Reset A Polo cPu can be warm or cold reset through two l"echa~,is" ,s. First an Xbus reset can be issued to a Polo CPU through an l/O write across the Xbus to the CPU's reset register. This will cause a reset only to a non-broken CPU. Since broken boards do not respond to Xbus aocesses a reset to a broken board will not reset it. Polo i"~ 1er"er,~ a second reset function de~.g"ed to reset a board regardless of its state. This is the suggest~ method for ,es~ti"g boards in a Polo system although the Golfbus method is supported for colll~ ibility. This reset is acc~",. " hed by perforrning a local l/O write to the ap~,,op,idle reset bits in the local Board Reset register. Setting the appro~,idl~ bit in this register causes a reset to be pe,loR"ed using a special set ot ~edicated three way voted lines on the Xbus. For more details on the protocol of this reset line please refer to the Xbus Functional Specifi~lion. This reset does allow for cold and warm resets to be b,uad~sl to any board in a Polo system including CPUs.

29 Decen,ber ~995 ~ 66 , .

Polo SomHare P .uy.a~ uing Guide ;~ ratus Company Confidenbal The effect of a reset on a CPU is idenbcal to a reset on a Golfbus system and is documented in the Golfbus Functional Spesific~tion.
9.4.1.2 PCIB
In the PCIB subsystem there is a 2 level h e ~-~;hy of resets. The two levels are the one MlO
cu,.,~tible Xbus Board Register, and the eight page O SAM compatible Reset registers. Figure 28, below shows a block diagram ol the Resets relative to Polû.

29 Decenlber 1995 67 Polo Software F~uy(alll"ling Guide Stratus ~,ompany Confidential Figure 28. Polo PCIB reset Hierarchy ¦ MlO Compatible Reset Register ~ warm reset O
SAM O PageO reset register _cold reset O
PCI slot O reset t warm reset 1 SAM 1 PageO reset register _cold reset 1 PCl slot 1 reset warm reset 2 SAM 2 PageO reset register _cold reset 2 PCI slot 2 reset warm reset 3 SAM 3 PageO reset register _cold reset 3 PCI slot 3 reset warm reset 4 SAM 4 PageO reset register _cold reset 4 PCI slot 4 reset warm reset 5 SAM 5 PageO reset register _cold reset 5 PCI slot 5 reset warm reset 6 SAM 6 PageO reset register _cold reset 6 PCI slot 6 reset cold r~set warm reset 7 SAM 7PageO reset register _cold reset 7 PCI slot 7 reset warrn reset .

The MlO COlllr ''~ '~ Xbus/Golfbus Board Register documented in the Golfbus Functional SpecifioaliQn allows for either a warrn or cold reset to be pe, ~.r,.,ed. The effect of this reset is to reset the entire PCIB subsys~em. From an FTIO standpoint this reset is the equivalent of resetting 29 Dece"lbe~ 1995 68 Polo Software ~ rd~ 9 Guide .atus Company Confidential an FTIO and every SAM conne~,~d to it. Warm and Cold resets leave the PCIB unbroken, off-line, and with only the red LED on. Warm reset leaves most register state (pa- b~ 1y error register state, PCI configuration space and the PCI boards II,~".~hres.) intact; Cold reset clears nearly everything. This reset will reset the MlO co,l.p~lil)le Xbus/Golfbus fey;~t~,.s, all 8 sets of page 0 per slot registers, both per bus registers, the two PCI buses connec~.ecl to the PClBs (cold reset only), all PCI devices conne~bd to the PCIB (cold reset only), both sets of PCI configurabon space inforrnation (cold reset only), and any associated state with the PCIB. This reset is intended to allow for resetting of a broken PCIB. As with the CPU resets, this can be accomplished either across the X8us or from a local CPU write. As with the CPU, the local write is the p.e~e--ed method.
The page 0 SAM co.., ~ ~- Board Reset Register is supported to 811OW for resefflng an individual SAM. A reset to this register leaves the PCI slot in the Off~ine not ready state.This reset resets the pagO SAM state for that slot, the PCI board in that slot (r old reset only), and any state associated with that slot. This reset is i, It~nded to allow software to re~/iYe a dead PCI card, recover from a single PCI card hang, and provide any co,--pa~;bilities required between an FTIO system and a Polo system with regard to reset functionality.
9.4.1.3 PClcards PCI cards are provided by third party vendors, and do not support the warm and cold fbYors of reset. All PCI cards implement a specified reset behavior for their PCI illte"ace, the effects of reset on the remainder of the card is carr~spec~fic.
Typically, resetting a PCI card dears all usefubegis~ and zeros out on-board memory. Please refer to the manufacturers sp~;~ic-~l;on for each PCI card.
This reset is accomplished by writing a PCI resetto the approplidle page 0 SAM compatible board reset register. It has the effect of only resetLng the PCI card and not effecting an SAM coll)~,dtible or PCIB board state.

29 Decei-lL~r 1995 6 .

Polo Software R~ug,c""-""9 Guide Stratu ~,ompany Confidential 9.4.2 HSC2 n~-~e~

9.4.2.1 HSC2 In the HSC2 suL~.,te", there is a 2 level hi~dlul,y o~ resets. The two levels are the one MlO
compabble Xbus Board Register, and the four page O SAM cu-,,~tiWe Reset registers. Figure 28, below shows a block diagram of the Resets relaUve to HSC2.
Figure 29. HSC2 PCIB reset Hierarchy MlO CompaUble Reset Register wamm reset O
cold reset O
SAM O PageO reset register PCI slot O reset ~ warm reset 1 SAM 1 PageO reset register _cold reset 1 PCI slot 1 reset warrn reset 2 cold reset 2 SAM 2 PageO reset register PCI slot 2 reset cold r~set ~ wamm reset 3 SAM 3 PageO reset register _cold reset 3 PCI slot 3 reset warrn reset The MlO compaUble Golfbus Board Register, documented in the Golfbus Functional ~Specifir~tion.
allows for either a waml or cold reset to be pe,lu,.--ed. The effect of this reset is to reset the enbre HSC2 subsystem. From an FTIO ~la" 9~ ,~, this r~set is the equivalent of resetting an FTIO and every SAM conneu~d to it. Warm and Cold resets leave the HSC2 ul~ , off-line, and with only the red LED on. Warm reset Ieaves most register state (parUcularly error register state, PCI
configurabon space and the PCI boards lh~ cl~Es.) intact; Cold reset clears nearly everything.
This reset will reset the MlO cc ,~ dtible Golfbus registers, all 4 sets of page O per slot registers, both per bus ~y;St~S, the PCI bus connected to the HSC2 (cold reset only), all PCI devices connected to the HSC2 (cold reset only), the PCI configuration space inforrnation (cold reset only), and any ~-soc; ~tC'd state with the HSC2. This reset is intended to allow for resetting of a broken HSC2.
The page O SAM co""~lible Board Reset Register is supported to allow for resetting an individual SAM. A reset to this register leaves the PCI slot in the Off-line not ready state.This reset resets the pago SAM state tor that slot, the PCI board in that slot (cold reset only), and any state associated with that slot. This reset is intended to allow software to revive a dead PCI card, recover from a 29 Dece"lL~r 1995 70 Polo Soflware P~oy,a-,.."ing Guide ~tratus Company Confidenffal single PCI card hang, and provide any con.~3tibilibes required t - \ sen an FTIO system and a Polo system with regard to reset functionality.

9.4.2.2 PCI cards PCI cards are provided by third party vendors, and do not support the warm and cold flavors ot reset. All PCI cards implement a specified reset behavior for their PCI interface; the effecls of reset on the remainder of the card is card-specific.
Typically, resstbng a PCI card clears all useful registers and zeros out on-board memory. Please rsfer to the manufacturers specificabon for each PCI card.
This reset is accomplished by wribng a PCI reset to the ap~..vp,iate page O SAM compatible board reset register. It has the effect of only resefflng the PCI card and not effecting an SAM co" "~ble or HSC2 board state.

29 De~,-ll,er 1995 71 ~ ~9 CA 022~7~11 1998-12-03 Polo Software Pl~yldllllll;ng Guide Stratus ~;ompany Co"fide~ltial 9.5 D~t~.,.,..,ing the source of a MaTI,t~.,ance Interrupt Main~enance interrupts provide notice that there one or more co,'~ ~ions requiring action from the M&D process. In general, it is impossil~le for software to distinguish between a single maintenance interrupt, and multiple ",air~t~nance interrupts in quick sl~ccess;on. Upon seeing a main~nance interrupt in a system, the l- llDu~ing process can be used to detemmine the source(s) of the intenupt.
If an Golfbus/Xbus board that was in the system is no longer visible to software then that board gene,at~d a ~ai~)tendnce interrupt. (The reco.."..ended way of determining if an Xbus board is Lubf~hcn is by reading the Bus Interface Status Register; it is gua-a,.~ed to return non-zero on a u,l~.l.en board). To get more detail on what l a~.,~ned to the board, it r;an be reset and its Common Broken Status and ASIC Specific Broken Status registers examined. If multiple reset attempts have no effect, the board was either removed or is too broken to be read. The OS should then use the board pieaence ill'c Illaliùn from the RECC to detemmine if a board has been removed.
Golfbus/Xbus boards that have not broken or been removed from the system will have either the "Bus l,.l~.làce Maintenance Interrupr' bit orthe ~'Board Logic Mdintenance Interrupr bits set in the their Bus Inle"ace Reporting register if they activated mainl~:nance interrupt.
If the "Bus Interface Mai..l~:nance Interrupr bit is set, then Board Reset register may be read to see if a maintenance interrupt was due to a board reset. Similarly, the Bus Info Error Status and Misc. Error status rey;~le.~ may be read to det~-",: ~e if the maintenance internupt was generated by any non-fatal errors (the board would be broken u:hen~ise). If no bits are set in either of these two f~g;~l~r!i, the mair,t~,nance interrupt was software gener~ted.
If a UBoard Logic Maintenance Interrupt" is set on a PCIB or HSC2, then the interrupt originated from SAM slots a~soc :a~d with this board, Ot from a disk being inserted or removed from the Stu-ag.,~:Orks disk shelf. The next step is to read the SAM Mainlenance Attention Request register. If any of bits ~0 in SAM Maintenance Attention Request register is set, all four PCI Bus 1"le"ace State register for SAM3 - SAM0 should be read. If any of bits 11- 8 in SAM Mainle,lance Attention Request register is set, all four PCI Bus ll l~el lace State register for SAM7 - SAM4 should be read tPolo only).
In Polo, if a UBoard Logic Maintenance InterruprUs set on a Polo CPU, then the interrupt originated from an IOVA map error and the IOVA Map Error 1 and 2 ,egisle,~ should be examined ~section 12.4.3 and section 12.4.2). The TRID can be used to determine the PCI
slot that sourced the Xbus IOVA operation. The IOVA map error main~ndnce interrupt can also be masked from the IOVA Map Error 1 register.
The next step is to examine the SAM Status and Configuration reg;~le,~ for all PCI slots that issued maintenance interrupts. Bits 7:21 specify various sources of maintenance interrupts;
only some bits are i",; '- llellled in Polo & HSC2 systems.
On Polo systems, the final place to check is the Polo Disk Status register. This indicates whether a maintenance interrupt was issued due to a disk swap event. Unfortunately, it does not specify whether a disk was added or removed; only that there was a change of state.

9.6 Faults & Errors 9.6.1 PCI card faults Typically, there will be no Stratus style fault ~l~te~: ~ comparison logic on 3rd party PCI cards.
Some boards may compute check-sums or utilize parity on memory arrays; support for this level of fault ~pGI ~ng iS thC responsibility of the card driver. i lowevcr, there should be an interface for drivers to communlcate a text string of fault i"lu,l"dlion to the M&D, so it can be inco",o,d~ed into 29 Decem~er 1995 72 ~3~a _ . . .
..

WO 97/46941 PCTtUS97/097B1 Polo Software Fluylallllning Guide Stratus Company Confidenbal the SYSERR log and be available to the CAC.
The p,bsenoe or absence of a PCI card is indicated in the PCI config space Vendor ID register.
The PCI bridge will retum a value of all 1 s (an invalid vendor ID) on read ac~sses to configuration space registers of non eAis~nt devices.
9.6.2 PCI bus faults 9.6.2.1 PERR#
The PCI bus is protected by a single Even parity bit over 36 bits of inforrnation (32 addressldata and 4 CtBE). The parity is driven with the same timing as the addresstdata, but delayed by one clock. All agents on the PCI bus must yenel dte parity.
Data parity errors are indicated using the PERR# signal. Only the device whidh daims the cyde may drive the PERR# signal on the PCI bus. The GambiUMirage ASIC will continuously monitor PCI b ansa tions and the state of the PERR# signal, regardless of the selected device. Any PCI
parity errors observed during non-Special Cycles will result in a maintenanoe internJpt action back to the host CPU.
PERR# is a non-recoverable condition and causes a state reduction in the Gambit/Mirage ASIC
from on-line to off-line, not ready. This rer~u ~n in state will cause a maintenance interrupt. It is up to s~ft~ e to test and bring the card back into service if it can.

9.6.2.2 SERR#
System errors which could be cd~as~opllic are indicated on the SERR# signal. Any PCI agent which detects a non-recoverable condibon can assert SERR#. The GambitlMirage ASIC monitors the state of the SERR# signal and will issue a maintenanoe interrupt up to the host ,c, ocessor for any ass~"ti~n of SERR#. The state of the SAM will then be reduced to Off-line, not ready. The Gambit/Mirage ASIC does not generate target-abort for this case.
A PCI adaptor non-recoverable condition includes:
1. Address parity enors on all cycles.
2. Data parity errors on Special Cycles.
3. A ca~ u~ adapter fault.
SERR# can be disabled via config space on all PCI devioes (including the Gambit asics). All PCI
devices power-up with SERR# disabled; it is up to M&D software to enable SERR# assertion.
9.6.2.3 Polo(Gambit)lHSC2(Mirage)Errorloglc Pob(Gambit) detects several enor conditions on PCI bus and takes different actions depending upon error kind. Actions taken for different kind of enor is given below.
1. Target Abort If the bridge is current bus master, bring the SAM who drove the devsel line to OFFLINE-NOT-READY state. If this is a read access by host, retum failed op.
If the SAM is current bus master, bring the cunent bus master to OFFLINE-NOT-READY

29 Dece..~ 1995 73 ~ ~\

Polo Soft Nare P~oyrd~ ng Guide Stratus ~,~,mpany Confidential state.
Set the value in PCI Enor register and PCI IOVA Enor register.
2. Master Abort If the bridge is current bus master, and this is a read access by host, retum failed op.
If the SAM is current bus master, bring the current bus master to OF~LINE-NOT-READY
state.
Set the value in PCI Error register and PCI IOVA Error register.
3. Parity Enor If the bridge is cunent bus master, bring the SAM who drove the devsel line to OFPLINE-NOT-READY state.
If the SAM is wrrent bus master, bring the current bus master to OFFLINE-NOT-READY
state.
Set the value in PCI Error register and PCI IOVA Error register.
If the bridge is bus master and is reading trom the SAM, failed op retumed to the host.
If the bridge is bus master and is writing to SAM, it is up to the SAM to take ap,u~op~idte actions.
If the SAM is cunent bus master and is writing to the host, the transactions is dropped. No tldi~sd~bon happens on X bus.
If the SAM is current bus master and is reading from the host, it is up to the SAM take ap~,u~id~e actions.
4. System Error If the dont_break_on_serr bit is not set in test control register, bring all SAMs to OFFLINE-NOT_READY state. If the dont_break_on_serr bit set in test control register, nothing happens.
On power on, dont_break_on_serr bit is cleared.
Set the value in PCI Error register and PCI IOVA Error register.
If the bridge is bus master and is reading from SAM, failed op retumed to the host.
tf the bridge is bus master and is writing to SAM, it is up to the SAM to take ap~u~upliate actions.
If the SAM is current bus master and is writing to the host, the bdn a~ ns is dropped. No transacbon hap~ns on X bus.
If SAM is current bus master and reading from the host memory, the ~ar~s~ ~ n is dropped.
No transaction happer,s on X bus. The SAM receives target abort.
Note: If the host read of PCI bus gets SERR and retry, but do not get SERR for retried transaction, data is returned to the host. I lo/.~.Jcr PCI Error and PCI IOVA Error register is 5. nm~ Out Error (TRDY bmer enor) If th ~ bridge is current bus master, assert PCI reset to the SAM who drove the devsel line. If t~tis is a read access by hos~, retum failed op.
\~t/ the value in PCI Error register and PCI IOVA Error register.

29 Dec~"~l et 1995 74 ~ ,~, ~., _ .

CA 022575ll l998-l2-03 Polo Software Plvglalllllling Guide :jlratus Company Conlide"- -6. Drive Check Enor Break PCIBISAM board.
Set the value in PCI Error register and PCI IOVA Error register.
( ~etry Count Error Bring the- SAM who drove the devsel line t~lNE-NOT-READY~. If this is a read \ ~access by host, retum failed op.
Set the value in PCI Error register and PCI IOVA Error register.
r~ 5Q~)y T~n~o~ ~~,~. G~,-8.~Di~ ~unt Error ~l~ue PCI reset to the SAM who drove the devsel line. If this is a read access by host retum ailed op.
Set the value in PCI Error register and PCI IOVA Error register.
9. Peer to Peer Error Bring the current bus master to OFFLI NE-NOT-READY state.
Set the value in PCI Error register and PCI IOVA Error register.
10. Protocol Error If il ,e bridye is current bus master, issue PCI reset to the SAM who drove the devsel line. If this is a read access by host, retum failed op.
If the SAM is current bus master, issue PCI reset to the SAM.
Set the value in PCI Error register and PCI IOVA Error register.
Note: If a PCIE~ is issued a warm reset while PCI trattic is in ~ruy~C:ss~ a ~.ulocol error may be d~tçct~d by Gambit error logic.
In certain cases ot Frut.,tol error, write buffer data in the Gambit ASIC goes on X bus even if there is 8 p. ~tr~ol error. ( e.g. A pci adapter does a write to gambit, but does not assert irdy_ and deasserts trame_ causing a ~.o1~ ool error) .

11. Host Request FIFO Tlme-out Error This error is detected when Host Request FIFO (inside Gambit ASIC) requesffng to access PCI bus fails to advance by the number of clock ticks s~e,,;fied in Host Request PIFO Tlme-out Value Register. In this case PCIB broken so that CPU does not hang waibng for data to return from PCIB.
Break PCIB/SAM board.
Set the value in PCI Error register and PCI IOVA Error register.
Note: This error l-al,p_ns in some rare catastrophic cofi-l;tion(e.g. frame_ signal stuck low on PCI bus). When ~is error condltlon happens, the PCI IOVA Error neS~;ster and PCI Error Register may or may contain the info about the tra. ~ ~ I o n that caused this error, since it Is h~.p o sr ;hle to figure out which l. dnsa~.tion caused this error.
The PCI enor logic in Gambit implements PCI IOVA register and PCI Error register for diaynosing errors. The PCI Error Register is read only by host. This register has arm/rearm wntrol. This register could be a""ed~ a""ed by host. When PCI Error Register is armed the PCI IOVA

29 Dece,-,ler 1995 75 --~3~

Polo Software ~,uyl."""ling Guide Stratus~,ompany Confidential register contains the PCI bus address that caused error. When an error occurs, the PCI IOVA
register and PCI Error registers are frozen so that it holds the address that caused error.
Subse~ ent errors does not affect the register. It needs to be ,ea",-ed before it could store address for next error. For desc, il,tion of PCI IOVA register and PCI Error register, refer to section 1 3.5.5.
9.6.3 Disk bus faults The DEC Storageworks disk solution provides the necessaly features for Stratus disk M8D. In pa,i ~u'~, St~,dga-.orks accomplishes live i"se,liQn, inse,lion/removal d~ b~ 1, and fault LEDs.
In order to perform these tasks DEC has i" ,ple" ,ented two bssic features. The first one deals with live inse,L~n. Live inse,b~n is ac~"l~lished through some current limihng and power sequencing.
It seems to work. The laner two features are accomplished through DEC's fault bus. The fault bus consists of a swap interrupt and an LED bus. Both of these features are i,. l 1~ llellled on Polo.
There are a series of oper~tions which should be pe,lo""ed by the M&D p,ocesses ass~cict~d with the various M&D features of the disk subsystem. These are ~lisrJI~sed in the sections below.
9.6.3.1 LED Initi~ tion LED initializabon should be pe~r~""ed at power up time. The LED initiali~lion roubne provides a simple way to tum off all of the LEDs. At power up time, the LED state is non-deterrninisffc, and must be set by setting/resening each LED individually, or as a group by using the broadcast bit.
The broad cast is pe,lurl,,ed by setting (turn on LEDs) or clearing (to tum off LEDs) bit 4 and setting bit 3 (the b-oad~sl bit) in Disk LED Control register. This register is des~iLed in section 12.6.1.1 of this document. In a broadcast write to the disk LEDs, the SCSI device target ID (bits 2:0) are not meaningful. For exa"-~'E, to set all ot the LEDs in a SCSI bus, a 1 8x would be written to the register.
g.6.3.2 Disk LED Control Individual LED control is p~,fL""ed on an as needed basis by the s~'~,a~e. Software has control over one of the LEDs, the fault indicator, on the disk. In order to address a pa, lic l~r slot, bit 3 (the b,oadcasl bit) must be clear, bits 2:0 should be set to the SCSI device target ID, and bit 4 should be set (to turn the LEDs on) or cleared (to turn the LEDs off) in the Disk LED Control register. This register is d~sc,ibed in section 12.6.1.1 of this document. The SCSI devioe target ID is the same as the SCSI devices normal SCSI device number. For ex~",, ~e, to set the led for SCSI device 2, a 012x would be written to the Register. to clear the LED for the sam devioe, a 002x would be wrinen to the register.
9.6.3.3 Disk Insertion and Removal Finally, disk insertion or removal is d~t~ d through the swap has occuned bits. For the ~side shelf, this is on bits 1 and 17 and for the D-side it is on bits 9 and 25 of the Disk Status register. If the c~,-t sponding intenupt disable bit is cleared, then a maintenance interrupt will be generdted due to this event. The internupt is cleared by clearing the applu~(id~ bit in the register, bit 1 for G
side interrupts, and bit 9 for D-side intenupts. The swap interrupt reflects only that a change has occurred in the state of the disk shelf, the M&D so~ must perforrn a SCSI poll to d~le~ e the nature of the change.

29 Dece"l~er 1995 76 .

CA 02257~ll l998-l2-03 W O 97/46941 PCTfUS97/09781 Polo Soth~are P~uy.a~ ing Guide Stratus Company Confidential 9.6.4 SCSI bus faults Faults on the SCSI bus are handled in a manner specific to the PCI card.
9.6.5 PCIB/HSC2 faults 9.6.6 Xbus/rclP~!~sfaults Xbus faults .t~pG-~d by the PClBtHSC2 are handled like those trom any other Xbus/Golfbus board. Please reter to the XbuslGolfbus Functional Specification for more intormation.
9.7 Diagnostics 9.7.1 PCI Card The level of .liayno sL ~ support tor the PCI cards will be consistent with the PCI Third-Party Adapter Requirements Speri*r~tion. Most PCI cards will be very highly integrated (co"si~ting of 1 or a few chips), and all are anticipated to be very reliable. In any case, they are non selh;l,e~ing ~."ponen~, and the system is protected from i--~.,ecl operation by higher level ..,e~.l-an;s"- ,.
address re-mapping, check-sums, and time-outs.
The system must be able to isolate faults down to a field l~ e~l le unit. If it is not possible to isolate taults to this level via the error, epo. Iing re~;sta~, specific fault isolation diay. ,os~ics will be needed.
9.7.2 PCIB
The PCIB/HSC2 board has no local pfocessor and an exbt5--.ely low chip count. It will be tested and diag--osed exclusively by scan.

29Dece--lber1995 ,--Polo Software P,og,d"",ling Guide Stratus ~,ompany Confiden~ial 10. ReCC Functionality on Polo This section de:,.;l ibes key changes envisioned for the ReCC finnware on Polo. On Polo it is a goal to isolate the OS from many changes in the M~D subsystem ~at are particular to Polo.

10.1 ne.,.ote Power control One feature on Jetta that is not supported on Polo is that ability to cycle power from the ReCC.
The Polo machine does not implement house keeping power or the power supply control necess~y to perform this function. Therefore it is not supported on Polo. ReCC firrnware should p.obably stub out this co" " "and set. It has been 5~ ge~ ~e~ that the ability to power down the machine from the ReCC be supported. This feature would require spinning the else gate anay on the ReCC and therefor will not be supported.

10.2 RS~85 bus Polo does not implement the CDC. In the initial release of Polo there is no 485 bus devices. The 485 is in the system for future e~par,sion but with no devices in the 485 system it will not respond to any polls. Therelo,~ the ReCC lillll:.. e should return intelligent r~sponse to M&D requests for 485 i"lo""ation.
10.3 Powerfail Polo does not support p~ ail. Polo s solution to AC failures is accomplished through an interconne"l to an extemal UPS. The i,lt~,~""ection to the UPS will be documented in a later release of this specifieation. It is e~ ed to be RS-232 and it will be conne- t~d to the secondary console when it is used.
10.4 Clocl~c~r~ ckr~nel power supply failures The clock card and backpanel power supplies are not implemented in Polo therefore there will be no signalling of these faults.
10.5 IIO power supply failures The 1/0 power supply fault signals may be rooted in place of the backpanel power supply signals.
If these lines are activated it signals that the 1/0 power supply has determined that a fault exists in the 110 power supply.

1 0.6 NVRAM
It would be p,e~e,~b ~ to replace the full funcbonal EEPROM with flash prom. This may effect the writing -1g~riL""forthe PROM.

29 Dece"lber 1995 78 ~3k, W O 97146941 PCT~US97/09781 POtO SL~ O P,~.~...",ling Guide Suatus Company Confidenbal 11. BoardStates This section d~sc,;Les the board states for the PCIB and SAM modules in the Polo system and the HSC2 for the Jetta system.
11.1 PCIB Board States The Polo PCIB module would gain little by s~ ,ing duplexed operation for the f ~ ;..g ~~asol~s.
1. All paired SAM space ar~sses whether to memory or l/O must be converted in hardware to non-paired. This is be~ce the Polo PCIB does not have the beneft of going through an MIO
before drNing the Xbus so the two PCIB boards cannot know the same information.
2. Many of the registers in Gambit could only be 0~ssed through paired spaoe due their definitions already.
The task of implementing a duplexable PCIB in hardware would add work with little or no gain so the PCIB can only be run in simplexed mode.
11.2 HSC2 Board States The HSC2 is similar to the BIO in that it does not support duplexed cp ~ ~ ~n. The HSC2 is similar to the Polo PCIB in that it shares the SAM board states model detailed In SAM Board States on ~ page 80.

26 Decel,lber 1995 _- 79 Polo Software Ploy,a"""ing GuideStratus Company Confidenffal tl.3 SAM Board States Figure 30. SAM-PCI Board State Transition Insert ) Warm Cold ~ ~ Reset J
Reset J

~/no er~r ~tate f Off-line ~ Off-line ~
Ready J r~arm_~n_rogs~cl~arorror6)~ Not Ready J

~rbiter ~nabbd ~ ~ ~ r~cl er~or albiter disabled\ ~"
\~ ONLINE )/

Board States:
OFF-LINE - Not Ready Board capable of rece.l;.,g and responding to XBus/Golfbus requests but is incapa-ble of pe(lur",;ng PCI adaptor initiated reql~es~ There is non-zero error state in the SAM.
This state indicates the occurrence of a PCI Bus related hardware fault which was d~ t~ d by the cl ~eclu, ~9 Iogic. This state cannot be entered by a s~fl~, a. e set off-line cu,,,..~nd as there must be non-zero error state in the SAM.
OFF-LINE - Ready Board capable of receiving and responding to XBus~Golfbus requests but is incapa-ble of performing PCI adaptor initiated reques~. No error conditions are present.
This state is entered automaffcally after a co~d or warm reset (provided there are no errors de~ t~d by the SAM) or when the error state of the chip is cleared via a Re-arm Error Co,---,.and. Also so~ ar~ can enter this state by either a set off-line com-mand or by a clear on-line co"-"-and.
ONLINE
Board capable of receiving and responding to XBus/Golfbus requests. Board capable of performing XBus/Golfbus initiated PCI requests. PCI adapter enabled to post re-quests.
26 Deo6",ber 1995 g 80 Polo Software ~ gramming Guide ~tratus Company Confidential Software can enter this state with a set on-line oo..u,,and to the Bus Interface Sate Register provided there is a non-zero error state.
Figure 31. SAM board states.

u~
u~
Z s O ~ a ,,~ m, ~ S ~ U~
m ~

~n o ~ c:
J~ Q ~ ~
ZJ ~ 6 ~ ~r ~ J ~ o O o 1 1 1 0 o 1 1 1 1 0 KEY:
~ 1 -- State bit true.
~ O -- State bit false.
~ X -- State bit true or false.

26 Dec~"ll~( 1995 81 ~3~

Polo Software Fluglall~ ing Guide Stratl,_ ,ompany Confidential 12. Reyi~t~. Definitions The f~ l~..i lg section outlines the ItO space rt;g;;,t~.a in the Polo system.

12.t XBus/Gotfbus Bus l~n~ ce R~ist~.~
All IO registers in the Xbus/Golfbus interface are designed to be a~ sed as 32-bit l~g;ste,a so as to be ~a~ffl~le by all devices on the bus. Writes to any srnal~er data size Xbus/Golfbus registers are ignored and reads retum 32-bits even if a smaller size datum was a~ed Though all registers are 32 bits, they are spaced at 64 bit (8 byte) intervals to aid future board designs which may make use of 64 bit intemal busses.
Writing any non-existent IO register has no effect fi.e. unused add~esses do not wrap around and map to used advl~Yvssea). Reading a non-existent IO register either retums O s or causes a NACK, depending on the partiwlar register. The read never has any side effects.
There are two types of IO registers supported in the Xbus interface in additional to interrupt rey:~.te.a intended to mimic the HP-PA interrupt sv 1 ) read/writs registers These registers are readable/writable 32 bit registers for which no bit encoding ocrurs, i.e.
assuming ha~dwa,-v events have not altered any state, what is written to bit 0 is read from bit 0, what is written to bit 1 is read from bit 1, etc. For upward cv~ ~tivility, unused bits should always be written with O s, though 1 s have no effect. Reads of unused bits always return O s.
2) r~a~/~ ,tclear registers Read/aYvVelaar ~e~a;shra are used when individual bits need to be atomirally updated. Reads retum 32 bits; writes specify one of the 32 bits to be ei~er set or deared. Specifically, writes are pe,fo""ed by selevlill9 one of the 32 read bits bit to be atomically set or cleared with the five least si~llifi~an~ bits (4:0) pointing to the bit and using the 8th least signilic~nl bit (7) to set (=1) or clear (=0) the s ~ ec~qd bit. For example, to set bit 2 ot the read register, 00000082x is wrilten to the register. To dear bit 2 of the read register, OOOOOO~Y is written.
Unused bits always retum O s and for upward cvn,~libility unused bits shouid always be written with O s, although writing 1 s has no effect.
3) .-vadU~.laar registe~
fiead/.lea~ registers are used when individual bits need to be atomically ll~t~d Reads retum 32-bits; writes specify one of the 32 bits to be cleared. Specffically, wfites are pe,frv,,,,ed by selecting one of the 32 read bits bit to be atomically cleared wi~ the f~e least siyl,if~ t bits (4:0) pointing to the bit and using the 8th least significant bit (7) clear (=0) the selected bit.
For exam~e, to clear bit 2 of the read register, 00000007Y is wri~ten. A~L.Il~ to set the register will have no effect.
Unused bits always retum O s and for upward r~".~dtibility unused bits should always be written with O s, although writing 1 s has no effect.
12.2 Reading Registers with Dlll_r~"~ CID Inh,."dt~cn (Po~o Only) Note: This secffon does not apply to the HSC2 as there is no board p,o~sor to read the error re~;~.t~., in local space.
There are certain C and D r~g;st~, ~ that can have different values ~for example the error ~~g;it~, :,).
In a duplexed pair of boards the register values be~een board pair may also vary. There must be a way of read ng these f~g;stt:l ~ via either the local or a remote ~, uCe5~01.

29 Dece,nl,er ~ 995 82 Polo Software P.uy-a,-~ ing Guide ~tratus Company Confidenffal These registers should not be read in local space be~ se this could bring duplexed boards and/
or the C&D board halves out ot lockstep when the register values are different.
The access to these registers is similar to unpaired reads in that part of the data cl,e~,kil,g has to be ignored. During an unpaired read the board not providing the data has to ignore the ch6cl~i.)g but go through the mobons to remain in loc~ep In this case the conoept is taken a lit~e further.
Each board half (except the board half actually being read) goes through the motions but ignores the error ~I,ecl~ing on the info; however the trid, func_op, and parity are still checked.
Since some of the bits are driven from the C-side and some are driven from the D-side and the data from the two sides can dffler, there is a complication in generaffng good parity. The solubon to this problem is to place the diflering error bits in the regions of the info bus that are driven by the ASIC responding to the read (see figure 7). For example ~side error bits have to be in the region 7:0 or 23:16, and the D-side from t5:8 and 31:24. To get good parity each side drives the same data on each of the bytes that the ASIC drives and puts zeros into the bytes that the ASIC does not drive. This will result in an even number of ones in the parity chain.
These registers must be readable when the board is off-line which is not an issue bec~use the board r~spond~ to unpaired reads when it is off-line. There is a problem reading these ~t,g.st~,~ if the board is broken. When the board is broken, it cannot be read or written by s.~fl.~ . To access these .~g.ste.;. on a broken board there must be a warm reset which brings the board to the off-line state.
Figure 32. Bus Driver Definition 6 0 0 31 24 2316 1~ 8 7 0 0 I [1 ~ ~ I ~
trid fuoc op InfoBus Parity Bits are sssigned starting from the least ~ bit [~ Driven by the D~side ~SIC ~¦ Dri-en by the Cdde ASIC
during nonnal acces#s, ~nd during normal a~ esses, and by the dri-ing ASIC during by the dri-ing ASIC during -' g~ -' ~ ' acc~sJes. ! - llca~ses ~3 Parity is dri ~n by the D~sde on thc CPU
lt is dri-en by the D-side of the PCm during normal aa ~ses, and by the dri~ SIC dunng Si~gle Side aoc~

12.3 Common Xbus/Golfbus Register Definitions (Polo Only) Note: HSC2 supports the set of c~"""~n Golfbus ~oglsters. F~efer to the Go~fbus Speb;f~ on for details c onceming the co"u"-~n Golfbus r7e~..,t~i~ for the HSC2 board.
The Xbus supports a set of Co,-,---ol) registers that are present on all Xbus boards. These are ~--Gdeled directly after the Golfbus interface r~g;s~c.~, and preserve ~-"paliL~ility to the bit level whenever possible. ~ lo.~ cr, due to the different physical partitioning of the Polo system, there are necessa- ily some changes. To ease the task of porting sofhNare to the Polo system, we define four levels of compaffbility for each register:

~ exact: this register preserves exact read,'w~ile functionality with the Golfbus defill: ~n, and may be proy,~-",.,ed in exactly the same way. Examples include the ID PROM Address Data 29 De~--~ e( 1995 83 , _ .

Polo Software l'~ g,~l""~ing Guide Stratu~ Company Confidenbal register.
~ usage change: this register is bit for bit idenbcal to the ~, ~e~ponding Golfbus register but the usage in practice may be different. For example: the pe,lo-nlance monitor registers can be p~og,d"""ed to monitor the function codes on the bacl~Jlane; however the Xbus is 32 bits and does not use the Golfbus 64 bit ba. h~lane function codes.
~ su~set/eul~er-set: the Xbus version ot this register is a subset or super-set ot the Golfbus register. It may be p~uy~d~ ed in the same way but bit values that make sense in Golfbus systems rnay not be supported in Xbus systems or visa-versa. It may also haYe additional bits that do not exist in a Golfbus board. An example is the Bus 11 ~te- l~ce State register: the Xbus does not have the equivalent of the bus obey, so sefflng the "obey_a" bit has no effect.
~ incompaffble: this register has new meanings and bit definitions in an Xbus system.
The Xbus/Golfbus c~""~ ility column in table 15 indicates the level of compatibility with the co,,~apunding Golfbus register. Descripffons of the registers follow the table Table 1~. I/O Space Map.
Offset Register Name Type C/D different compatibility 7~ 8 ID PROM Instr ~ead/Write no exact 7FFFF0 ID PROM Address Data Rea~.ite no exact 7FFFE8 Board Reset Read/SeV no superset Clear 7FFFE0 Buslnterface State Read~/SeV no subsetlsuperset Clear 7FFFD8 Board Sync Register Read Only non- exact deterministic;
data MUST
be discarded 7FFFD0 LED Control Read~ite no exact 7FFFC8 Slot ID Read Only no exact 7FfFC0 Read Ping Interval Rear~Write no exsct 7FFFB8 Set Intem"~/lnt~"upt Status I teadNU\ite no not impl.
7FFFB0 Clear Interrupt Write only no 7FFFA8 Set Interrupt Mask/lnterrupt n~ad~'~ite no not impl.
Mask 7FFFA0 Clear Intermpt Mask Write only no not impl.
7FFF98: Gen Purpose Comm [7:0] ReadJWrite no exact 7FFF~8 Memory SizelLocation ReadJ\Nrite no im~emented tor S/W c~ dt~l~ility, but ignored by HIW
7FFF48 Test Control Read/SeU CD dfflerent subset Clear if difference bit is set 7FFF40 ParityTest nead/W~ite no notimplemented 7FFF38 Bus Inte~tace Fault Reporting Read/SeV yes i,,~-,-ualible Clear 7FFF30 Common Broken Status Read Only yes in~."~a~ble 7FFF28 ASIC Spedfic Broken Status Read Only yes inco,--l i~'o 7FFF20 Bus Info Error Status Read Only yes subset 29 Dece"l~e~1995 84 CA 022575ll l998-l2-03 Polo Software P~yl~"",l;.lg Guide ~tratus Company Confidenbal Offset Register Name Type C/D different compabbility 7~18 Misc Error Status Read Only yes subset 7FFF10 ControlBusErrorStatus ReadOnly yes newregister 7FFF08 Bus Error Byte Status Read Only yes usage change 7FFF00 Voter Error T,ansceiver Read Only yes incompabble Status 7FFEF8 Bus ASIC Chip Revision Read Only no new values 7FFEF0 Perf_Counter Read/Write no exact 7FFEE8: Perf_Counter_Trig~1:0] Read/Write no exact 7FFED8: Perf_Counter_Maskl1:0] neaWili~e no exact 7fFEC8 Perf_Counter_Control Read/SeV no exact Clear 7FFEC0: Fault Bit[1:0] ReadfWrite no new register 7~FEB0 Data Match Read/Write no new register 7F~EA8 Error Control ReadlSet no new register General Waming: Many of these l~:y;~t~l~ have bits which can be toggled during self-test and left in varying legibrnate states. Prior to or as part of duplexing, all bits must be ensured consisle"~.
Second General Waming: Several of the error reporbng ley;st~"~ are UCD differenr' and can legibmately contain data that is different between C and D sides of the board. They must only be read via unpaired space, or they could break the board or hang the system.
The fomnat of each register d~sc,i,~ion is as follows. The pa(aylapll header for each desc,i~ion provides the register name. The bold faced text i"""e~li..lJly f " ~ .~;. ,9 contains the register type, address(es), reset states, and a lisbng of boards types it is present on.

12.3.1 ID Prom Instr Type: ReadlWrite Offset~22:0~ B
. ~.,senl on all Xbus boards Warrn and Cold have no effect on this register C~ ility: exact Wribng to this register scans an instrucbon into the extemal .1TAG ID Prom logic. Reading this register retums the status of the scan. It is only used for initializing the ID Prom logic after power-up or reset.
31:8 Reserved 7:0 JTAG instrucbon/status for the extemal ABT18245 scan register.
Refer to the next section for usage. Valid instructions are SAMPLE_PRELOAD = 92x EXTEST = 00x Valid retum status O~_STATUS = 91x 29 Dece-,~ 1995 85 ~ 3 WO97146941 PCT~US97109781 Polo Software ~lu9la-"",;.,9 Guide Stratus Company Confidenbal 12.3.2 ID PROM A~lr~s Data Type: ne~ e Oftset [22:0l~ 0 P~ent on all Xbus boards Warm and Cold have no effect on this register ~:orlr~ihil~ty: exact 31:28 ID prom opcode. This controls the WEJ CE_, and OE_ lines of the ID Prom id_READY = 6x id_PRESENT_DATA = Ex id_WRITE = Ax id_OFF ~ 7x id_READ = 4x ~,7:20 ID Prom Data. On writes, this is the data to be written into the id prom. On reads, it is the data retumed 19:11 Reserved ~0:0 ID Prom Address.
Every Xbus board has a 2Kx8 writable ID PROM. The PROM is intended to store various board status informabon such as serial number, eco rev level, PCB rev level, etc.
The actual PROM used is the X281 6C 2Kx8 EEPROM from Xicor. This part is ~Ar~ssed though a JTAG compaffble serial scan interfaoe to save pins on the bus ASIC and to allow the part to be Ac~sc;ble via the board's scan test logic. A Texas Instruments ABT18245 scannable buffer is used to ban51d~ from the scan intbllace to the ID Proms b,oatl~ide pins. More detail on the use of this part is available in section 11.3.1 on page 98 (ID PROM Partitioning) in the Xbus Functional Spec~ tion.
Logically, the ID PROM appears as two virtual 32-bit registers, an instruction register for sending initialization c~ "----andb to the ABT18245 scan chip, and an address_data register for reading and writing address/data pairs to the id prom itself. These two virtual registers are actually imptemented as a single 44 bit scan register in the ASIC. Writing to the instruction register initiates an 8 bit instruction scan to the ABT18245, and writing to the address_data register initiates a 44 bit data scan which drives and sa---, '~s the pins of the id prom. Reading the address_data register retums the bits of the scan register co,-t:spo--ding to the addreb-s, data, and opcode fields.
Reading the instruction register retums an 8 bit value co--~3~.ponding to the results of the instruction scan. Note that there is only one scan register; a read of id prom instruction reg should only follow a write to the id prom instruction, and a read of the id prom address data reg should only follow a write to that address. Ol~elwise, detemministic gibberish will be retumed. Once initialized, reads of the instruction and address_data registers will always retum non-zero data; ff a zero is ever retumed, it is due to the target board going broken or a simplexed fault in the ID prom hardware.
The ID Prom hardware needs to be initialized after power-up or a wamm or cold reset. Once it has been initialized, the prom can be read or written. The ID prom is a fully random access device, with no sectors or erase modes. When a single byte is written to the X281 6C, the device begins an intemal ~cnoy-~..---ning cycle that takes -5mS. While the pl~ldllll uing cycle is in ~,.og-~ss, bitpl of the byte written to the part will appear inverted when read; software needs to poll the device to deterrnine when the program cycle is finished. The ID prom has an endurance of 10,000 writes.
To initialize the ID Prom hardware:

ID_prom_instr = SAMPLE_PRELOAD; //(0000_0082x) ID_prom_instr = EXTEST; //(0000_0000x) 29 Dece,-lber 1995 86 '~'1(1 Polo Sottware P~us~ uin9 Guide Stratus Company Confidenbal if(ID_prom_instr != OK_STATUS); //(0000_OOBlx) ~hardware failure in ID prom"
ID_prom_address_data = {id_OFF,28'hO000000 ?
To read the ID prom add~esses start_addr to end_addr ID_prom_address_data ~ {id_READ,8'hO0,start_addr}
for tl = start_addr; i<= end_addr; i++) ID_prom_address_data = {id_READ,8'hO0,i+l}
read_data[i] ~ ID_prom_address_data[27:20]
ID_prom_address_data = [id_OPF,28'h0000000 To write mybyte to the ID prom address start_addr ID_prom_address_data = {id_PRESENT_DATA,mybyte,start_addr~
ID_prom_address_data = {id_WRITE,my~yte,start_addr~
ID_prom_address_data = {id_PRESENT_DATA,mybyte,start_addr}
~ /~ poll to wait for proyLr lng cycle to complete ~/
ID_prom_address_data = {id_READ,~'hO0,start_addr} /* once */
ID_prom_address_data = ~id_READ,8'hO0,start_addr} /* again ~/
isbyte~ ID_prom_address_data[27:20]
while ~isbyte[7] != mybyte[7]) {

wait l mS
ID_prom_address_data = {id_READ,8'h00,start_addr]
i6bytec ID_prom_address_data[27:20~
ID_prom_address_data = {id_OFF,28'hO000000}
Note that two surr~ssive writes of the id_READ opr,ode are required betore read data can be examined. This is because the boundary scan ring in the ABT18245 acquires the data on its pins before applying the new data. The first write ar,quires old data, then applies the id_READ opcode and address to the ID Prom. The second write acquires the results of the read opcode and flrst address, then applies the id_READ opcode (and opbonally a new address).
Note that ID PROMs dffler in contents trom board to board. Boards o~.~,ti.,g in lock step should always read and write them in an unpaired fashion. An unpaired read to an ID PROM belonging to one of a dupex board pair causes both board's state machines to perfomm ths actual read, though only one board actually drives the bus with the read data. An unpaired write causes both board's state machines to go through the motions, though TCK is inhibited on ths board not being written.
This JTAG scan occurs at 12 MHz, hence each write of these registers takes about 5 usec. The bus ASIC circuitry autornabcally delays a sllt~e~ ent access if an earlier inibated write is still in p-ug,ess. When the ID Prom is ~r~ssed via global, paired or non-paired space, other lO
2"~a55es can take place while an ID prom access is in ~nu9feS5. When the ID prom is Arce~sed via Local space, all other lO space access to Bus Interface reg;~,t~";. are held off unUI the access completes.

: =

29 Dec~,ul,er t~5 _ ~ 87 CA 022~7~11 1998-12-03 Polo So~tware ~luglall",ling Guide Stratus ~ompany Confidential Figure 33. Internal Implementation of ASIC scan register ctl data Oo reserved address tdi t~
A2 o A1 B2 B1 ~ ~ ~ , 31 Z (~
reserved op/status Waming: Re~ se multiple p,ocessG,~/p.o~esses simultaneously acces~i"g the ID PROM may intertere with each other, 5vf vare must manage access to the ID PROM to ensure only a single ,~-u~sso,tpn,cess is accessing it at a Ume.
Waming: The ID PROM is a simplexed ~non error p.uleclad) cc.-.~n~nL A checksum or other similar protection scheme should be utilized by sofhvare to ensure the integrity of the data read from the ID PROM. Also, bec~se ID PROMs on pa~ d boards will have different contents, ID
PROM Data registers should always be read in an non-paired fashion.

12.~.3 Board Reset Type: n~QVClear Offset 122:0~ t8 Present on all Xbus boar~s Cold an~ wann reset anect this ,egist~r as ~oc~ ,nted below CG~ a~l~llitY: super set Polo boards have two resets, a cold reset applied to the board at power up (or via software or ReCC) and a warm reset applied via sof~Nare or the ReCC onoe the rnachine is running. Cold reset brings all registers to a known deterrninistic state, warm reset generally resets state machines and cancels ope,dbons in p,uyless but leaves as much of the register state untouched as possible to aid in debug.
The acth/abon of any of the bits in this register (excluding test cases) mean that a reset pulse was earlier driven on the board, not that a reset line is currently asse,led. It should be noted that as part of the normal reset process, the board goes broken for the duration ot the reset pulse, and then u.,b,o ~en when the reset de-asserts.

29 December 1995 88 .. . . ....... ..

CA 022~7~11 1998-12-03 Polo Software ~uy~a,""ling Guide ~tratus Company Confidenbal Cold resets clear all bits in this register except for the bit idelltAy;ng the source of the cold reset warm resets leave bits in their current state (except for activating the bit that idel ,L ~~s the source of the warm reset).
This is a Read/SeVClear register, co"""an~ are pe~,---ed by writing to this register with data bits [4:0l pointing to a bit to set or clear and bit 7 determining whether the bit is set (=1 ) or cleared (=0).
Reading this register retums 32-bits. Setting effler~Software initiated Cold Reser or~Software initiated Warm Reser actually causes a reset even if the bit was already set. Setting (to test the register read path) the other bits cause the bits to go to 1 but no reset takes place.
Polo is dfflerent from Golfbus systems in that writing the Roard Reset register of a broken board across the bus has no effecL For this reason Polo has 4 additional reset bits that when activated send a reset pulse to the co,.esponding siot. This reset is interpreted as a cold reset if it is three 4 Mhz periods long and as a warrn reset if the pulse is two 4Mhz pulses long. Software Initiated Cold Reset and Software Initiated Warm Reset still work as before when pe,~o""ed locally and work across the bus when the target board is unbroken.
31:13 These read bits/write encodings are reserved for additional board specific resets.
12 Cold Reset Slot 3 Writing to set one of these bits sends a cold reset pulse to the board in the co"esponding slot. These bits retum 0 when read. These bits are only implemented in the CPU.
11 Cold Reset Slot 2 10 Cold Reset Slot 1 9 Cold Reset Slot 0 8 Warm Reset Slot 3 Writing to set one of these bits sends a warrn reset pulse to the board in the ~,-esponding slot. These bits retum 0 when read. These bits are only implemented in the CPU.
7 Warm Reset Slot 2 6 Warm Reset Slot 1 5 Warm Reset Slot 0 4 Software initiated Warrn Reset When reading this register this bit z1 indicates that the board has ,eceived a software generd~d warm reset since this bit was last cleared. Writing a 00000084x to this reg-ister yene,dlds a warm reset pulse on the board and sets this bit. Wribng 00000004x to this register clears this bit. The activation of either of the cold reset bits also clears this bit.
3 Reset line initiated Warm Reset When reading this register, bit 3 =1 indicates that the board has received a warrn reset from the reset line since this bit was last cleared. This bit is cleared by writing 00000003~t to this register. The activation of either of the cold reset bits also clears this bit. This bit can also be set by writing 00000003x to this register.

2 Reset line inibated Cold Reset When reading this register bit 2 =1 indicates that the board has received a reset line initi-ated reset when the board was broken and lheleforè treated it as a cold reset. Writing 00000002x to this register clears this bit, as does a power up initiated or software ini-tiated cold reset. When this bit is set by a reset line reset all other bits in this register are cleared. This bit can also be set by writing 00000002x to this register.
29 December 1995 89 CA 02257~11 1998-12-03 Polo Software P~oyfa"",ling Guide Stratus L~ompany Confidential Soflware initiated Cold Reset When reading this re~ister, bit 1 c1 indicates that the board has received a software gen-erated cold reset since this bit was last cleared. Wribng a 00000081 x to this register generates a cold reset pulse on the board, sets this bit, and clears all other bits in this register. Writing 00000001x to this register clears this bit, as does a power up initiat-ed cold reset. This bit is NOT cleared by any warm reset.
0 Cold Reset due to Power up When reading t'nis register, bit 0 =1 indicates that the board has received a power-up gen-erated cold reset since this bit was last cleared. This bit is cleared by writing 0000000QY to this register or by a software initiated cold reset. When this bit is set by a power up reset all other bits in this register are cleared. This bit is NOT cleared by any warm reset.

12.3.4 Bus Int . I~ce State Type: Re~llSeVClear Offset [22:~3: 7FFFE0 P.~s~nt on all Xbus boards Cold and wsrm reset sftect this register as documented below - Com~ . sul,~Us.Jper-set This is a ReadtSetlClear register; co,-~l,land~, are pelfu~ ed by wribng to this register with data bits [4:0l pointing to a bit to set or clear and bit 7 deterrnining whether the bit is set ~=1 ) or cleared (z0). Reading this register returns 32-bits.
Note that since this register always retums 1 s in bit ~ ic ns [3: 1] , reading this register will always produoe a non-zero result assuming the board is not broken. Stratabus boards used the ID PROM
for this purpose. It is I t Cul l ll l l~ncled that this register be used tor that purpose on Xbus systems due to the large latency ot accessing the IDPROM,.
31:23 These read b;t~w~it~ encodings are reserved for additional board specific purposes.
These bit positions always retum 0's when read unless assigned a board specific purpose.
22 Force Peer-t~Peer Mode - When this bit is set the operaffons in the second column of ta-ble 10, Peer to Peer and Non Peer-to-Peer Operabons, will be tumed into peer-to-peer cycle. Refer to secbon 6.8 for details. Warrn or cold reset clears this bit.
21 Break PCIB board on backplane failure When a failure on the backplane brings down a bus, it is nece~saly to break one of the two boards conne~;~d to it. The default is to break the CPU. When this bit is set, the PCI8 board is the one desiyllal~d for breaking. This Polo feature is descnbed in full detail in section 6.4, Bus Errors, on page 44. This bit is cleared by cold reset.
20 Enllal)ced EFO/RWQ Nesting Disable - this mode is not implemented in Polo (Polo doesn't support enhanced nesffng) - writes are ignored, reads retum zeros.
19 EFQ Freeze State This bit only exists:on CPU boards. When set, the board busies any Xbus t.~nsa L'~ de-coded ~r the Cyclops E~Q (Eviction Flush Queue) EXCEPT ~If it is a data retum ofany size. When read, this bit will not appear set until the Cyclops's EFQ is empty or is currently being emptied onto the Ibus. It is set in the i4'1c .:;.,g cases:
Set when an Xbus or local write of 00000093x is perforrned AND the board's Freeze State bit is NOT set.

29 Dece"~ 1995 ( L~ g 90 .. ~ _ . . . .

CA 022~7~ll l998-l2-03 Polo Software ~ r~y~ ning Guide ~tratus Company Confidenbal Set when an Xbus or local write of 00000093x is pe.h"",ed AND the board's FreezeState bit is set AND all Cyclops and Cougar incoming R/W pipes are empty.
This bit is cleared by writing 0000001 3x or by warm or cold reset. It is not cleared as a re-sult of the board going broken.
18 Break Even Board on TA Failure This bit only exists on duplexable boards. Setting this bit causes the even board to break when a TA failure is detected; uUIe~i5e the odd board breaks. Software should set this bit so that the most recently added board of a pair is the one that breaks during a TA failure. The bit is set by wriffng 00000092X and cleared by wriffng 00000012X to this register. Wamm reset has no effect on this bit, cold reset clears it. This bit always retums 0 on those boards which do not implement iL
~7 Simplexed mode - this mode is not implemented in Polo - writes are ignored, reads return zeros.
~6 Nested OCU Op Disable - this mode is not implemented in Polo (Polo doesn't support nested OCU ope-ations) - writes are ignored reads return zeros.
~5 On~ine For Dumping This bit only exists on some boards which have system memory. Setting this bit permits the memory from an otherwise off-line board to be read. The bit is set by wribng000000~x and cleared by wriffng 0000000Fx to this register. Cold and Warm reset clear this biL This bit always retums 0 on those boards which do not i.. r1e "e"t it.
14 Freeze State This bit only exists on some boards which can be duplexed. On CPU boards it is used in con~unction with the EFO Freeze Mode biL As outlined in detail in table 15 on page 72 in the Xbus Functional Specif~Rhon wriffng to set this bit (0000008Fx~ causes a board to busy all a~esses directed to it except those from its partner unhl the board passes through the sync point. Wriffng 0000000Ex Warm and cold reset cold reset clear this bit. This bit always retums 0 on those boards which do not implement it.
13 Update Mode This bit only exists on boards which have system memory and lock cycles. As outlined in detail in table 15 on page 72 in the Xbus Funchonal SpecificAhon~ when set this bit in-dicates that the board is the "update" mode, a mode used during the board synchro-nization procedure to update memory on a new board. When in this mode the on-line board sends its oU,e,~;se local writes to the Xbus and the new partner board (in the off~ine/update mode) updates its memory from these writes. Wriffng to set this bit ~1000008Dx) puts a board in update mode. Writing 0000000Dx Wamm and cold reset cold reset clear this bit. This bit always retums 0 on those boards which do not imple-ment it.
12 Any CPU On~ine This bit indicates that some CPU board in the system is on-line. It refbcts the state ot the cpu_online b~h~ldne signal and exists on all cpu/me",ory boards in the system. This bit always returns 0's on IO boards. This bit is cleared whenever no CPUs are o~line (e.g. a single on-line CPU going off-line or a by a system-wide cold reset that brings all boards off-line). Writing to set (0000008Cx) or clear (0000000Cx) this bit has no effect.
11 First CPU On-line This bit exists on CPU boards only and is set if this board beco",es the first in a system af-29 Dece"ll~er 1995 91 ~9 Polo Software r~ug.~."",ing Guide Stratus (~ompany Confidential ter a cold reset to gain l"astt:,allip by winning an arbitration cycle and being the first to pass through the sync point. This bit is cleared by both cold reset waml reset. This bit always returns 0 on non-CPU boards.
10 Leave Sync Point Jump Switch Wribng to set ant one of these bits (00000086X,00000087X,0000008~x,00000089x, or000000~Ax f.Ss~.ti~ely~ clears the other bits and instructs the board what state it should leave the sync point in. Regardless of the ~tate of these bits, any CPU board which is the first on-line will leave the sync point simplex and clear all these bits. Ex-cept for the first on~ine case, if none ot these bits are set, a board will not leave the sync point.
These bits can be c~eared by writing 00000006x, 00000007x, 0000000~x, 00000009x, or 0000000Ax respectively. Warm reset has no effect on these bits, cold reset clears them. Bits 7 and 8 exist only on boards which can be duplexed and always retums 0s on all other boards, bit 9 exists only on boards with system memory that can be put on-line as just memory boards and always retums 0s on other boards, bit 10 exists only on boards which can ~'jump switch" and always returns 0s on all other boards.
A board reaching the sync point with bit 6 (Leave Sync Point Simplex) set will leave the sync point simplex. Bits 7 and 8 (Leave Sync Point Fast Duplex and Leave Sync Point Full Duplex) functionally are the same. A board will leave the sync point in a du-plexed state only if its partner is also at the sync point with one of these bits set. Two bits exist to provide software a differentiation between a ~fast" duplex (one where two iniffalize but off-line boards are duplexed) and a ~fulr' duplex (one where an off-line board is being s~" ,.,1 ~,~,ni~ed to an on-line board), Bit 9 (Leave Sync Point On-line For Dumping) is for a board to leave the sync point with only its memory on-line, say for a memory dump. Note the Xbus architecture puts the system at risk of crashin~ whenany simplex system memory is utilized. Bit 10 (Leave Sync Point Jump Switch) is to support the jump switch feature wherein the on-line board goes off-line and is re-plaoed by the previously off-line board.
Fore more i"'~ ir n on these bits and there effect on board state see the Board Sync Register on page 93.
9 Leave Sync. point on line for dumping - not implemented in Polo systems.
8 Leave Sync Point Full Duplex 7 Leave Sync Point Fast Duplex 6 Leave Sync. point simplex - not implemented in Polo systems.
5 Duplexed This bit only exists on duplexable boards. As outlined in detail in table 15 on page 72 in the Xbus Functional Speci~ication, when set this bit indir;ates that the board and its partner are in loc~t ~p This bit is set as a result of the boards leaving the sync point in loc~ct-~p The bit is cleared by cold and warm board resets, TIA failure, or by a board or its partner going Off-line or Broken. Writing this bit (0Q000085x or 00000005x) has no effect. This bit always retums 0s on no-duplexable boards.
4 On~ine This bit equals 1 if the board is on-line. As outlined in detail in table 15 on page 72 in the Xbus Functional Spe~,iti~iu-l, an on-line board is fully f~"~,lion,ng from the sys~em pe.~ /e. Among other things, this means the board f~5~ ond~ to all bus transac-tions directed toward it, including paired reads. Writing to set this bit (00000084x) or passing though the sync point (in some cases~ brings a board on-line. Writing to clear this bi~ ~00000004x) and the board breaking bring it off-line. Both warm reset and cold ~9 Dece,.~ 1gg5 92 ~,o .

CA 022575ll l998-l2-03 Polo Sof~ware P~og.~,--,ling Guide ~ratus Company Confidenffal reset clear this bit.
3 Obey Both The Xbus ohal ges obeys based on the source of a transaction, not in ,~:~ponse to enors.
There is no need to set the obey state via software control. To maintain backwards compaUbility with the Xbus, writes to the obey bits have no effect and reads always retum 1.
2 Obey B - not implemented, retums 1 on reads.
Obey A - not implemented, retums 1 on reads.
0 Broken This is a status bit used to indicate a hardware fault which has caused the two sides of the board to behave differently. This bit is set automaUcally by hardware or by the wribng 00000080x to this register. This bit, as well as the diagnosUc broken bit in the Test Control register, are cleared at the end of a cold or warm reset if the two sides are equal or by writing 0000000~x to this register. Resets are the prefened method for unbreaking a board. When this bit is set the board cannot drive any extemal, (Xbus or C conne~,tul)~ signals. The board will sUII respond to writes add.~:.sed to it in l/O
space, (if capable~, and will continue to check data and address inforrnabon which would norrnally be transmmed to the bus. Whenever this bit is set by hardware a dou-ble bus error will be issued to the bus to abort any potentially bad information that may have been driven by this board. The act of clearing this bit ge(ler~les a mainte-nanoe internJpt.

12.3.5 BoardSyncRegis~r Type: Read Only Offset ~ 0l: 7FFFD8 , ,~r~,~t only on some duplexable Xbus boards Cold and warm reset do not directly affect this r~y;st~r C~",~ lllty. exact WARNING: Reading this rag 1~ may retum non-debrministic data and can be different 5 ' de l~ s1dE or boar~to-board; It is important to discard whatever data is r-,tu. .~ed as soon as possible. Data l~tt in registers can be pushed onto stack frames thus causing system crashes if different.
A board reads this register to pass through the sync point. This data retum may be delayed by hardware and may have side effects. Reads ot this register locally will rause the board to wait at the sync point. Reads to this register trom Xbus IO will retum 0s immediately and no sync procedure will take plaoe.
It is possible to perforrn non-local IO ~ ses tolfrom a board waiting at the sync point.
The specific actions resulbng from a board passing through the sync point (i.e. reading this register locally) depend on the state of the board and system involved:
Flrst CPU On-line: The first CPU board in a system to try to read the sync register (or a the first CPU to win arbitration should multipe CPUs simultaneously attempt this) will pass through the sync point (i.e. have data retumed to it when reading this register) and go on-line in a simplex state. Hardware then causes that board to assert thecpu_online backplane signal, thereby nobfying other CPU boards that one CPU
board has passed through the sync point.

Should the cpu_online backpane signal deac~ivate (say by the only on-line CPU
29 De~"lL~( 1995 93 CA 02257Sll l998-l2-03 PCT~US97/09781 Polo Soflware ~ g,~"l"~in g Guide Stratus ~jompany Confidential breaking), another CPU board sitting at the sync point will be released in a similar fashion.
Note that a board passing through the sync point in this fashion will leave the sync point in a simpexed fashion regardless of the state ot the "~eave Sync Point Sim-plex", ~'Leave Sync Point Fast Duplex", ULeave Sync Point Full Duplex", ~Leave Sync Pdnt On-line For Dumping", and ULeave Sync Point Jump Switch" bits in the Bus In-terface State register. Should any of those bits have been set, they are cleared in this case, thereby providing a ,--ecl-a, ~;_m for software to detect that the sync point was passed through as first cpu on-line.
~ I~uplexed: Should a board's ~'Leave Sync Point Full Duplexed" or~Leave Sync Point Fast Duplexed" bit in its Bus Intsrface State register be set, it will leave the sync point in lockstep with its parlner when its partner reaches the sync point (assuming the part-ner's not already there) and a read is pe-~ d of the sync rsgister and that board's ~Leave Sync Point Full Dupiexed" or~Leave Sync Point Fast Duplexed" bits are set. If a board is waiting at the sync point and its partner breaks, it will leave the sync point On~inelSimplexed. (Actually, the duplexsd bit will be set and then cleared i"""e~lidt~
Iy when the hardware detects the partner is not on-line). Any other condition and the board will remain stalled at the sync point.
A pair of boards can pass through the sync point duplexed when both are new to the system (a ~Fast Duplex") or when one board ts a~ready part of the system and a new board is being sy,-ch.o. i~ed to it (a ~Full duplex"). In the former case, the boards were both in the off-line state prior to passing through the sync point (but there was another CPU on-line, allowing the boards to stall at the sync point), in the latter case the ~old" board was in the On-linetU,~,~tetFr~ state and the ~new" board was in the O~f-line/Update/Freeze state.
It is possible to put a board on-line duplexed when a board is already waiting at the sync point. This is done by setting the ~Leave Sync Point Full Duplexed or LeaveSync Point Fast Duplexed" bit on while the board is stalled at the sync point.
~ On-linelSlmplexerJ: Should a board's "Leave Sync Point Simplexed~ bit in its Bus Inter-face State register be set, it will leave the sync point, on~ine but never duplexed. If irs partner is on-line, It will stall until its parlner is off-line. Note, If a board snd its pan-ner both perform a go-on-line-simplex at app uAi",~k~ly the same time they will both go on~ine but not duplexed. Iockstep is not guaranteed after leaving the sync point.
T~his condition is not supported on duplexable boards and Xbus t~dnsd~tiol ,s will con-flict with each other bec~use they will bOth be using the same TRID.
If a board's partner is at the sync point with a the ~'Leave Sync Point Full Duplexed" or "Leave Sync Point Fast Duplexed~' or ~Leave Sync Point Jump Switch" hit set, theboard will pass through the sync point simplexed and not effect its partner in any way.
(The partner will remain at the sync point).
It is possible to put a board on~ine simplexed when a board is already waiting at the sync point. This is done by sefflng the ~Leave Sync Point Si-- ~lF ~ bit on while the board is stalled at the sync point.
~ On-line For Dumping: Should a board's ULeave Sync Point On-line For Dumping~ bit in its Bus Interface State register be set, it will leave the sync point i"""el~idt~:ly with its memory on-line but ~tl,erwise simplex and off-line. It is planned that this mode be used to support memory dumps. A board and its partner can be both on-line for dumping" but Xbus l,~nsac~ions initiated by these boards are not supported 29 Dece--lber 1995 ~L 94 Polo Software B,o5~,a~ ing Guide :jtratus Company Conridential It is possible to put a board On-line For Dumping when a board is already waibng at the sync point. This is done by sefflng the ~Leave Sync Point On-line For Dumping~' bit on while the board is stalled at the sync point.
~ Jump Switch: Should a board's ~Leave Sync Point Jump Switch" bit in its Bus Interfaoe State register be set, it will either leave the sync point on-line and simplex or off-line, depending on whether it was on-line or off~ine prior to passing through the syncpoint. (If it was on-line it leaves off-line and if it was off-line, it leaves on-line.) It is planned that this mode be used to support or~line upgrade of board with different p~cessor speeds or memory sizes.

12.3.6 LEDControl type: n~'llte Otfset 122:~1: 7FFFDO
~Lson~ on al~ Xbus boards Cold and warrn reset affect this regisler as documented below CG~ WIjtY exact This register contains status/control of the LEDs on the board. There are three LEDs (red, yellow, green) a"anged like a stoplight on each board handle. The f~)ll.".~;.lg table indicates their states (disks and dumb devices are included for completeness). For more i, If ul ",ation on the LEDs, their purpose and su~ re model, see the apprup,iaLe sections of the mai"lenance and diag"ostic ~pec;f;~ion.
Table 16. LED States Unit StateRed LEDYellow LEDGreen LED
ServiceDo not PullIn Ope,aI;on No Power OFF OFF OFF
Testing Cycle Cycle Cycle Simplexed OFF ON ON
Broken ON OFF OFF
Duplexed OFF OFF ON
Lamp Test ON ON ON
Disk Drive Inserted OFF OFF BLINK
Disk DriveSpun Down OFF OFF OFF
Dumb device OK OFF NJA ON
Dumb Device Faulted ON N/A OFF
Dumb Device No Powers OFF N/A OFF
31:3 Reserved These bits are reserved for future use. Writes to the register will have no eflect. Reads of this register will retum zeros in these bits.
2 Red LED State The red LED is turned on by ha,~ .e whenever the board is in the broken state. When 29 Dec~"~L,e- 1995 95 PCT~US97/09781 Polo Software r,oy.an",ling Guide Stratus (~ompany Confide,ltial the board is not broken, writing this bit can tum on (write a 1) or off (write a 0) She LED. Note that hardware will not clear the LED when the board Iralls bcns from bro-ken to not broken; that must be done by s ~ c. The red LED is set by warm and cold reset.
2 Yellow LED State The yellow LED on is used to indicate that a board may not be removed. It is transition-driven: on duplexable boards, it is tumed on by hardware when the board ~dnsi~ions from off-line to on-line-simplexed, or from on-line-duplexed to on-line-si...~ ed (due to its partner breaking). It is tumed off by hardware on the ba, s;tion to duplexed or broken; it is not cleared by the on-line->off-line bai sition. On nor~duplexable boards, the L~:D is set when the go on~ine co---",dnd is issued. The yellow LED rnay be set or cleared by soft Nare at any bme. The yellow LED is cleared by warrn and cold re-set.
0 Green LED State The green LED is turned on by ha.~_re when the go on-line collllllànd is issued, and is tumed off when the board goes off-line or broken. These actions are state ~,an~ n driven. Only the ~ ansition causes hardware to set or clear the light. At any time, the s~)fi ~r~ may also set or clear the green led. The green LED is cleared by warm and cold reset.

12.3.7 SlotlD
Type: Read On~y Offset [22:0]~ 8 Pl'~5~.~l on all XbuS boards Cold and Warm Reset have no effect on this register CG~ II llity: exact This register contains the 4 bits that indicate the physical slot position of this board. This is the slot number address that this board will respond to for non~lobal l/O accesses Note that this register is readable locally, when not duplexed (e.g. during diagnosbcs).
The 4 slot id bits are located in bits 3:0 of the slot id register. Bits 31:4 are unused and will return zeros when read.
12.3.8 Read P~ng Interval Type: n~z~JMrrite Of~fsett22:0~ C0 Present on all Xbus based boards that can issue IIO reads Cold reset clears this ~9 ster, wann reset has no effect C~;n~r~til,ility: exact 31:1 Reserved 0 Read Ping Interval The value loaded into this bit determines the amount of time a read will be outstanding be-fore a ~Ping" bus transaction is sent on the Xbus to determine if there is sffll a target that intends to respond to the request. Writing a one sets the read ping inten~al to 31 bus phases, or 2.667 ", ~ osecon~ls. This value is provided for diay"o:.lic and simula-tion use only. Writing a zero sets the interval to 99.416 mi.,,useconcls, or 1,193 bus phases (the aefault)~ Cold reset clears this register; in normal ope, ~n there is no need to touch this register.

This register is not implemented on PCI8 boards ~since they cannot issue l/O reads). When a read ~9 Dect "~l,er 1995 _ l 96 ~ 7 .

Polo Software ~ .~g-a,--"ling Guide ~,uatus Company Confidential request is committed on the Xbus i.e. the request is not terminated with BUSY or ERROR a bme-out counter which was pre-loaded with either 31 or 1 193 is enabled. There is a single counter that times the duration of the oldest request from a board that is outstanding; if a ~t:sponse is not received before this counter reaches terrninal count a ping ~,a,)sac~ion is sent on the bus. Should no achn~ lcd~e to the ping be received the bus i, ll~l Idce retums zeros and an error i"Jiutor to the reqnest~r. This error indieation is a low priority machine check on the CPU board. If an acl~"u; ledge to the ping ope,ation is received the tim~out wunter is reload~d according to the cûntel l~ of this register and the timeout count is ,e~ d. If a fes~onse is retumed before the counter reaches terrninal count data is returned in the usual fashion.
A read of this register provides the most recently written data. Bits 31:1 are unused and will retum zeros when read.

12.3.9 General Purpose Communications [7:0]
Type: Rea.l/~NIile Offset 122:0]~ 8~ 60 At least 4 present on all Xbus based boards, up to 8 on some.
Cold and warm reset affect these registers as docL ~l~e~le~J below Cc,..~ t~ y: exact These registers are provided for any general communication desired by any resource on or off the board. r~ec~se boards may have dfflerent needs that are not defined at this time eight General Purpose Communication reg;,l~,~ are provided in IO space. Not all 8 registers are necessa~ily implemented on all bus ASlCs but each board is required to have at least the first four (3:0).
Table 17. General Purpose Registers Name Offset [22:0~ Cold Reset Warrn Reset GPR0 7FFF60 cleared cleared GPR1 7FFF68 cleared no change GPR2 7FFF70 cleared no change GPR3 7rFF78 cleared no change GPR4:GPR7 7FFF80:7FFF98 cleared no change (unimplemented) 12.3.10 Memory Size/l oC-at~
Type: ReadnNrite Ottset [22:0]: 7FFF68 Pr~s~nt only on Boards with Global System Memory Cold reset clears this re~i~ler, warm reset has no enect CG,~ a~ tY: implementeai tor sottware wll~r)a~ only 31:8 reSeNed Writes have no effect; reads retum zeros.
7:0 These bits are implemented for software CO~ iiity with Jetta only. The bits are ,ead/v.,ite but have no effect on operation.

12.3.11 Test Contr~l Type: P~ead/SeVClear Otfset [22:0]: 7FFF48 29 Deca~ er 1995 ~ S S 97 CA 022~7~11 1998-12-03 Polo Sof~ware Bluyla~ .lg Guide Stratus ~,ompany Con~d~n- ~' l on all Xbus boards Cold reset clears thls reg;ster, warm reset has no effect C-,n".a~ llity: subset This is a ReadJSeVClear register, co"""an,l~ are pe,~u,,-,ed by writing to this register with data bits [4:0] pointing to a bit to set or clear and bit 7 determining whether the bit is set (=1 ) or cleared (=0).
Reading this register retums 32-bits.
31 :7 Reserved.
6 i~side Difference Bit When the D-side Difference Enable is set, reading this bit retums a 1 from the D side hard-ware, and a 0 from the C side. This ~D difference can be used for testing COIllpard-tors intemal to the board. This bit is cleared by cold reset.
5 ~side DiS~erence Enable Setting this bit enables the D-side dilh~rence bit. When it is ena~ed. the D-side difference bit becol-,es a one on the D side bus ASIC, and remains a 0 in the C-side ASIC. This bit is cleared by cold reset.
WARNING: Setting this bit and then reading this register will retum di~f~f~ data on C vs. D sides ot the board (via bit 6 below). This will break you if not read local-IY
4 Diag, ~~ iti~_Broken This bit provided to facilitate self-test. This bit is set whenever a condition that would set the Broken bit of the Bus Int~ ce State register occurs. I l~ ;er, it may be cleared inde-pendenUy of the Broken Bit by writing OOOOOO~x to this register. This permits testing of various board features while the board remains Broken, preventing the testing from pol-luting the Golfbus. This bit is also cleared by cotd reset. Warm reset sets this bit, be-cause it causes the board to go broken for the duraffon of the reset. Wriffng 00000084x (which nullllally would set a bit) have no effect on this register. Clearing Broken (by writ-ing to the Bus Interfaoe State register) does not clear Diagnostic Broken.
Setting this bit enables the D-side ui~fu, ~nce bH. When i~ is enabled, the D-side dmerence bit beco,lles a one on the D side bus ASIC, and remains a 0 in the C-side ASIC. This bit is cleared by cold reset.
3:0 Reserved.

12.3.12 Bus Interface Fault RepG~ g Type: Read~SeVClear Oftset[22:0~ 38 P~;~sent on all Xbus boards Blt 1 set by cold and warm reset;
tor other bits cold reset clears this b~t, warm reset has no effect CD Dlll~re"t; read via unpaired space CG-I~P~;I llity: inconlpatible This register controls the error latching and l~lL.Ig around bus faults. It also is used to rearm the broken feyiSl~ . There are five error registers: Bus Info Error Status, Misc. Error Status, 3us Error Byte Status, Control Bus Error Status, and \~oter Error Trdnscei~er Status. All are latched when an error is del~-,led in the system. This register provides a means of disabling certain faults from causing the "Error Latching" to occur, as a way of allowing new faults to be detected in the p,~sence of known, continuous faults. Ma;n~enance interrupts are genefated on the latching of a tault; if a fault is d --~'E~ from causing error latching, it is also disabled from causing a 29 Dece~ er 1995 6 98 . .

CA 022~7~ll l998-l2-03 Polo Software r~ùg,a",-uing Guide ~tratus Company Confidential maintdnanca interrupt. An overview of the intonmabon supplied and how it is i~ ,te,~ded to be used is p,~,iicled in section 9.4, Error Repo(Lng, on page 98.
Note that this register is both ReadlSeUClear and CD different. A single write to set or clear a bit affects the C and D side bits, thus to set bits 3,11,19, and 27, it is only necessa~y to write 00000083x. It is not possible to write only the C or D side.
These bits indicate whether the various classes of faults cause the Error Registers to record state (~Error Latching") and generale a maintenance interrupt. The default is all zeros, i.e. any fault causes a maint3nance interrupt and all of the Error Registers to Update. Error Latched on Thermal Faults Disabled s~".pr~,sses Error Latching and the resulting maintenance interrupt due to Thenmal faults. I ID~CVer, if another class of fault occurs and a therrnal fault is present, the themmal fault will be present in the updated enor register. Likewise, Error Latched on Incoming 3-Way Vote Fault Disabled, Enor Latched on Control Bus Fault Disabled, and Enor Latched on Clock Faults Disabled determine whether 3-way voter, Control bus enors, or clock faults trigger Enor Latching and maintenanoe interrupts.
~Error Latched on TA Fl.t' " Disabled ~bits 5,21,13,29) is named ~'Error Latched on non-OBEYed Bus Fault Disabled" on Golfbus systems. In Golfbus systems, this bit has two functions: di.,abli.,g - error latching due to errors on the non-obeyed bus, ~n~ due to TA problem. In the Xbus system, only the TA problem functionality is retained, so the name of the bit has been uI,anged for clarity.
31,15 D side Error Latched on Control Bus Faults Disabled 30~14 D side Error Latched on Clock Faults Disabled - CPU only Clock faults only occur on CPU boards.
29,13 D side Enor Latched on TA Problem Disabled' - CPU only TA faults only occur on CPU boards.
2B,12 D side Error Latched on Incoming ~Way Vote Fault Disabled 27,11 D side Error Latched on Thermal Faults Disabled- CPU only Therrnal faults only occur on CPU boards.
26,10 D side Board Logic Maint Int This bit being set indicates that a mainte,)ance interrupt was issued by the logic (excluding the other bus i"t-,.lace) on this board. The ap~u~ on board register(s) should be examined to determine the source of the interrupt. This bit is cleared by cold reset or by writing to clear this bit ~Dooooon~y)~ This bit may be set (and a maintenance inter-rupt gene,dted) by writing OOOOOOR~e Note: ForGambit ASIC, whenever the Gam-bit specific logic sets this bit, bits 18 and 2(C side Board Logic Maint int) also get set.
25,9 D side Bus Interface Maint Int This bit being set indicates that a main~anance interrupt was issued from this bus interfaoe (including the clock chip and board logic rnaint int). Writing to set this bit causes a main~a"an e interrupt to be pulsed on the bus though the read bit in this register re-- mains set until cleared. This bit is cleared by writing 000000~1 x to this register. Sinoe both warm and cold reset break and then unbreak a board (events which genelaIe mainlenance interrupts), either warm or cold reset leave this bit set.

24,8 D side Error Data Reco,ded/Rearm Enor and Broken Registers When this bit =1, the Ic'l~;.,g ,eg;s~e,;, contain new information: Bus Info Error Status, Misc. Enor Status. Bus Error Byte Status, Control Bus Error Status, and Voter Error Tl.,nscc;~er Status. Writing a 0 to this bit clears these ,~s;ste,~ and ~aa""~ them so 29 Deoe",l,er 19g5 _ ~ 99 PCT~US97/09781 Polo Software ~,.,yla"""ing Guide Stratus Company Confid~ntial that they will capture the state of the next applir able error cor,dition. Clearing this bit also f~dr",i the Common Broken Status and ASIC Specific Broken Status rt:9;s~er~Writing a 1 to this bit has no effect.
23,7 C side Error Latched on Control Bus Faults Disabled 22,6 C side Error Latched on Clock Faults Disabled - CPU only 21,5 C side Error Latched on TA Problem Disabled~ - CPU only 20,4 C sirde ~rror i atched on Incoming ~Way Vote Fault Disabled 19,3 C side Error Latched on Therrnal Faults Disabled - CPU only ~8,2 C side Board Logic Maint Int Note: For Gambit ASIC, whenever the Gambit specific logic sets this bit~ bits 26 and 10~D side Board Logic Maint int) also get set.
17,1 C side Bus Interface Maint Int 16,0 C side Error Data Reco,dedlRearm Error and Broken Registers 12.3.13 Cc..~.o~) Broken Status Type: Read Only Offset [22:0~ 30 r-~sent on all Xbus boards Cold reset clears this r~:ster, warm reset has no effect CD DTI~rent; read via unpaired space C~ ilily: inco"".ati~le This register contains status bits from the broken logic collq~a-dlor~ that are cu"""on to all Xbus boards. It is cleared by cold reset (unless the board leaves cold reset broken~. The contents of this register are frozen when a board first goes broken so that the only co" ,~a, a~or~ that caused the broken cor,dition are flagged. This is so that comparators that "~;scGI"~d~e further down the line (aner the error has rippled through the logic) do not obscure the cause of the original failure. This register is re-arrned via the Re-arrn Error ~e~;~le,~ bit in the Bus Interface Fault Reporting register.
3~,15 D-side saw control bus set broken 30,14 D-side saw DLL out-of-lock 29,13 D-side saw drive ~;s~ ua~e error on a or b bus (i.e. A C to D side info out ~";scu",pare) 28,12 ~side saw ~side bus ASIC set broken 27,11 D-side board logic (external to the bus ASIC) gene,aled broken 26,10 D-side slot parity error 25,9 D-side ASIC saw drive ~";sco".pa,~ error on ~way voted line~
24,8 D-side Software set broken by co"",.and 23,7 ~side saw control bus set broken 22,6 ~side saw DLL out-of-lock 21,5 Gside saw drive .n.sco"")are error on A or B bus ~i.e. a C to D side info out ",;s~o"" af~) 20,4 Gside saw D-s~e bus ASIC set broken 19,3 Gside board logic (external to the bus ASIC) gene,dled broken 18,2 C-side slot parity error 17,~ Gside ASIC saw drive ~Il.sco~llpale error on ~way voted line.

29 Dece",ber 1995 ~ ~, 100 . .
. .

Polo Software ~Jylalllllling Guide Stratus Company Con~idenbal lB,0 ~side Software set broken by cG"""and 12.3.14 ASIC Specific Broken Status Type: ReadOnly Onset~ 0]: ~ 28 o~l on all Xbus boards Cold reset clears this register, warTn reset has no effect CD Dl~.~r~-,t; read via ~.~"~a;.~d space CC;..~ ty: IncG.,I~ le This register contains status bits from the broken logic comparators that are specfflc to a particular Xbus ASIC. lt is deared by cold reset (unless the board leaves cold reset broken). The contents of this register are frozen when a board first goes broken, so that the only c~""~.~ -;, that caused the broken condition are flagged. This is so that comparators that ,.,;s~--.~ further down the line (atter the error has rippled through the logic) do not obscure the cause of the original failure.
This register is re-ammed via the Re-arm Error registers bit in the Bus ll~ ce Fault ne~,ti,,9 register.
CPU ASIC (Cyclops) 31,15 Reserved 30,14 ~side Fan broken 29,13 D-side Sabte broken 28,12 D-side Cougar broken 27,11 ~side TA broken 26,10 ~side IM OK broken 25,9 D-side Arbitra~y broken.
24,8 D-sitte Heuristic broken 23,7 Reserved 22,6 ~side Fan broken 21,5 ~side Sable broker, 20,4 ~side Cougar broken 19,3 ~side TA broken 18,2 ~side IM OK broken 17,1 ~side Arbitrary broken.
16,0 ~side Heuristic broken ~CIB ASIC (asmbit):
31,15 Reserved 30,14 ReseNed 29,13 Reserved 28,12 Reserved 27,11 ~side Gambit specific 26,10 ~side Xbus Parity gene~alor fault 25,9 ~side Arbitrary broken.
29 Dece--~ber 1995 v~ ~ ~ 101 CA 022575ll 1998-12-03 PCT/US97/Og781 Polo Software l~ogla~ l;ng Guide Stratus Company Co"fi~ler~al 24,8 ~-side Heuristic broken 23:7 ReseNed.
22:6 Reserved.
21:5 Reserved.
20:4 Reserved.
19,3 C-side Gambit specific 18,2 ~side Xbus Parity ~ene~dtor fault 17,1 C-side Arbitrary broken.
16,0 C-side Heuristic broken 12.3.15 Bus ~nfo Error Status Type: Read Only Offset [22:0~ 20 Pr~s~nt on all Xbus boards Cold reset c~ears this register, warm reset has no effect CD Dill~/ent: read ~/ia unpaired space Compatibility: subset Any bus error signaled by the system, or a voter, control bus, therrnal, or clock error on this board causes this tegister ~and the other Error I atching ,egis~. ~) to freeze with state recorded for the offending info phase. This includes both non-fatal and fatal (i.e. board breaking) errors. All bits in this register co-~espond to the same info phase. Cold reset clears this register.
31,15 ~side some other board drove a the Bus B Error ba~,k~ lane signal and I did not.
30,14 reserved 29,13 D-side The bus error whose info was ,~rded was signaled by a different board 28,t2 reseNed 27,11 D-side ASIC saw parity error on e bus 26,10 D-side ASIC saw parity error on A bus 25,9 D-side ASIC saw loopback fault on the B bus on the info bits the D-side drove.
24,a D-side ASIC saw loophack fault on the A bus on the info bits the D-side drove.
23,7 C-side some other board drove a the Bus A Error ba~,h~,lane signal and I did not 22,6 reserved 21,5 C-side This board was bus master for the info phase te~,ded 20,4 reserved 19,3 C-side ASIC saw parity error on B bus 18,2 C-side ASIC saw parity error on A bus 17,1 C-side ASIC saw loopback fault on the B bus on the info bits the C-side drove.
16,0 C-side ASIB saw loopl,ack fault o~ the A bus on the info bits the C-side drove.
This re~ ~ ter reports the various bus and cG.."~a.e faults sepa-~tely for the C side Bus ASIC and the ~sid~ ASIC.

25 ~e~- ~ Ibe. 1 9g5 ~ ~,~, 102 -., ., .

WO 97/46941 ~CT/US97/09781 Polo Soflware P~ ,a"",ling Guide v.ratus Comp,any Confidenffal This register will not accurately reflect the state of the errors if the board is in the Ufunny" state.
Sinoe boards in the Ufunny'' state do not enter the error sequence, it is not possible to track the errors.
All bits are updated upon the d~,te~Aion of an error and remain at the same state ~ntil the Error Data Recolded/Rearm Error Registers bit of the Bus l"te,lace Fault r~ep~,ling Register is cleared.

12.3.16 Misc.ErrorStatus T)~pe: Read Only Offset ~ 01: 7FFF18 . .~rent on all Xbus boards Cold reset clears all blts except the Thennal faults bits.
Thermat tault Is una~l~l~ by any res~ts.
Warm reset has no effect on this (e~ster.
CD Gll,~rt,)t. read via unpaired space Compatibillty: exact The bit defi.l 'i~ ns for this register are idenffcal to the Golfbus SPB~ f;C ~n definffions with one dfflerence: therrnal and clock faults in Polo only occur on CPU boards so the bits associ~tPd with those faults are CPU only.
28,12 D-side T/A problem (CPU/MEM board ONLY.) 27,11 D-side thermal fault bit 0 - under lelll~Jerdture~cpulMEM board ONLY.) 26,10 D-side clock recovery chip status bit 1 (CPU/MEM board ONLY.) 25,9 D-side clock recovsry chip status bit 0 (CPU/MEM board ONLY.) 24,8 D-side ASIC saw a three way voter failure 23,7 Reserved.
22,6 Reserved.
21,5 Reserved.
20,4 C-side TIA problem (CPU/MEM board ONLY.) 19,3 ~side thermal fault bit 1 - over l~""~,ature(CPU/MEM board ONLY.) 18,2 C-side clock recovery chip status bit 1 (CPU/MEM board ONLY.) t7,1 ~side clock recovery chip status bit 0 (CPU/MEM board ONLY.) 16,0 ~side ASIC saw a three way voter failure There are two clock recovery chips on every board; each recovery chip has two pins which indicate the status of the Dack~,lane tault tolerant clock. The various combinations of these status bits are indicated in table 18 The clock status bits will change off the falling edge of 24 MHz,ll ,er~for~ it is necessary for the Bus Interfaoe ASlC's to latch the status bits on the rising edge of 24 Mhz(phase12_3) to insure 29 Deoel,lLer 1995 103 ~ I

CA 022575ll 1998-12-03 PCTrUS97/09781 Polo Software P~ogldlllllling Guide Stratus ~;ompany Gonfidential proper setup and hold tirne.
Table 18. Clock Status Definition Clk Status<1:0> D~il. ~icn 00 no fault 01 Clock recovery chip de~ ed a CLK A fault Clock recovery chip detected a CLK B tault 11 Clock recovery chip det~c1~d a CLK C fault Any change on a Therrnal fault bit results in a maintenance interrupt (unless disabled) and freezes all rey~ ls. These two bits are from the therrnal sensor on the board.
bit[0] Rep esents the under t~""~-dlure pin(6) on the thermal sensor. (etch run to D ASIC.) bit[1] Rep,esents the over l~n.~)erd~ure pin~7) on the thermal sensor. (etch run to C ASIC.) Table 19. Thermal fault Status Definition Therrnal Fault c1 :0~ De~mlbon 01 Board is within reco,.. ",ended ope,ating te",pe,d~re.
00 Board is warrn above ~~c4--"nen.1ed o,oe,ating t~",~.a~re.
Board is hot customer should conside~ a shutdown.
11 Bad board hardware is causing this condition (illegal).
The T/A problem bit (Cyclops only bit) when set re~"ese"h that there is a nor~fatal problem in the TIA I lar~ . The board or backplane could be at fault. This cor.d ~ic n should be cleared up. If leR
un-fixed REAL T/A problems will not be handled correctly. This bit is masked off by the UError Latched on nor~OBEYed Bus Fault Disabledr bit.

12.3.17 Control Bus Error Status Type: Rea~ Only Offset ~ 0l: 7FFF10 Ent on all Xbus boards Cold reset clears this register, warm reset has no et~ect CD Dlrl~ren(; read ~ia unpaired space CGIII~ llity: New Re3;ste r This register records the control bus error status to aid in fault isolaffon of a system experienc;ng single or multiple bit errors on the control buses. All bits are updated upon any bus error signaled by the system ~ ~ a voter control bus therrnal or clock error on this board and remain at the same state until the Error Data Recorded/Rearm Error Registers bit of the Bus Interface Fault Reporting register is cleared. Cold reset clears these .~g;~,t~ .s.
The bits are set for ~3oth single ~it (co" ~:r ~ble) and double bit (~ t~ causes board broken) errors. Check the Common Broken Status register ~ ~ see if the error caused board broken.
31,15 Reserved 2gDece"lber1995 ~ f~ 2 104 Polo Software Pn~g.~,-,n. ~-g GuideStratus Company Confidential 30,14 Reserved 2g,13 D-side saw error on control_out_p 28,12 ~side saw error on controlin_p 27,11 D-side saw enor on control_out_o 26,10 D-side saw error on control_in_o 25,9 ~side saw error on control_out_n 24,8 ~side saw error on control_in_n 23,7 resel~_d 22,7 ~side arbitration out of sync with ~side.
This bit is set when the C side che.,hing logic sees an unexpected value on the bus_req control line driven by the D-side.
21,5 ~side S9W error on control_out_p 20,4 ~side saw error on control_in_p 19,3 ~side saw error on control_out_o 18,2 ~side saw error on control_in_o 17,1 C-side saw enor on control_out_n 16,0 C-side saw error on control_in_n 12.3.18 Bus Error Byte Status Type: Read Only Offset l22:0~ 08 Pl~s~ on all Xbus boards Cold reset clears thls ~siister, warrn reset has no effect CD Dlll~ l; read via unpaired space C o~r~1n~ility: usage change This register records the bus error status on a per-byte basis to facilitate debug. All bits are updated upon any bus error signaled by the system, or a voter, control bus, themmal, or clock error on this board. and remain at the same state unffl the Error Data Reco.~d/neamm Error Registers bit of the Bus Int~.. Iace Fault Reporti~.g register is cleared. Cold reset clears these registers.
The bits are inte.~ t~.d as follows: If the board was the bus master (indicated in the Bus Info Error Status register) at the hme of the error, this register indicates which bytes had a loq~ k error. lf the board was not a bus master at the ffme of the bus error, these bits will be zero.
31,14 Reserved 30,14 11~5O.~rod 29,13 ~side saw error on trid or func_op 28,12 D-side saw error on parity 27,11 D-side saw error on infol31:24]
26,10 ~side saw error on info~23:16 26,9 ~side saw error on info[15:8]
24,8 D-side saw error on info[7:0]

29 Dece " IL er 1995 ~ 3 105 , _ . .

Polo Software P~ug-a",--ling GuideStratus ~;ompany Confidenbal 23,7 Rese.~_d 22,6 Reserved 21,5 ~side saw enor on trid or func_op 20,4 ~side saw enor on parity 19,3 ~side saw enor on intol31:24]
18,2 ~side saw enor on info[23:16]
17,1 ~side saw error on infol15:8 16,0 ~side saw error on infop:0]

12.3.19 Voter Error Transceiver Status Type: Read Only Oftset 122:0]: r~00 P~ont on all Xbus boards Cold reset clears this r~~~islcr, wann reset has no et~ect CD Dill~r~n~; read via u~.~,ailé~ space Co".patlL~llity: i,~cG,-"~alible These register records the status of the ~way voting logic. All bits are updated upon any bus enor s;y--aled by the system, or a voter, thermal, or clock enor on this board. and remain at the same state until the register is .ea- ",ed by the Error Data Reco, ~dll tearm Enor Registers bit of the 8us Interface Fault nepc rting register. Error latching and mainlendl~cê intemupts due to voter errors can be supp-e~sed by the Error Latched on Incoming ~Way Vote Fault Disabled bit in the Bus Interface Fault Reporting register.
For a list of which signals go through each l.ansconrcr, please see Xbus Voted Signal Pal litioning on page 96.
CPU ASIC (Cyclops):
31,15 Reserved, read as zero.
30,14 Reserved, read as zero.
29,13 ~side saw enor on online_in 28,12 ~side saw error on reset 11, 27 D-side saw enor on sync_in 26,10 ~side saw error on board_not_broken_p 25,9 ~side saw enor on board_not_broken_o 24,8 D-side saw enor on board_not_broken_n 23,7 Reserved, read as zero.
~,6 nesenJ~d, read as zero.
21,5 Gside saw error on online_in 20,4 ~side saw enor on reset 19,3 ~side saw enor on sync_in 18,2 Gside saw enor on board_not_broken_p 17,1 ~side saw enor on board_not_broken_o 29 De~"lber 1995 G 106 Polo Sof~Nare r,u~-a,.--~ling Guide~ atus Company Confidential 1~,0 ~sid~e saw error on board_not_broken_n PCIB ASIC (Gambit):
31,15 Reserved, read as zero.
28,12 ~side saw error on reset_o 27,11 ~side saw error on reset_n 26,10 D-side saw error on board_not_broken_p 25,9 D-side saw enor on board_not_broken_o 24,8 ~side saw error on board_not_broken_n 23,7 n~s3n-ed, read as zero.
22,6 Reserved, read as zero.
21,5 Reserved, read as zero.
20,4 ~side saw error on reset_o 19,3 ~side saw error on reset_n t8,2 ~side saw error on board_not broken_p 17,1 ~side saw error on board_not_broken_o 16,0 ~side saw error on board_not_broken_n 12.3.20 Bus ASIC Chip Revision Type: Read Only Oftset ~22:0]~ t~8 P~s~nl on all Xbus boards Cold and Warrn reset have no effect Compstibility: new values This register retums a revision number of the ASIC. Note: this i"fu,-,-a~ n should also be available from the ID PROM; it is included here to simplify tracking parts in the lab when the ID PF/OM is not stable.
Table 20. Bus ASIC Revision Numbers ~us ASIC VLSI part Board chlp Chip Revision number C~U Cyclops 1st pass 00002??? \/Y12???
PCIB Gambit 1st pass 00002??? VY12???

12.3.21 rE~hr~ance Counter Type~ e Oftset 122:01: r~ d Pr~sent on CPUIMEM boards Cold and warrn reset Clear this re~is~er.
Co"",~ -iflty: exact A read of this register will retum the value of this 32 bit penu,-"al-ce counter. The counter may be set to any value by writing to this register. This counter will i"cre",ent every time the trigger co"d;: ~n (as defined in perf_counter_trig, perf_counter_mask. perf_counter_trig_not_equal and perf_counter_trig_enable) is encountered. This counter will never i"~e" ,e"l past the value of Ot~t l~ ttX. If the counter were set to zero and the trigger condition was set to always 29 Dece"~ber 1995 107 Polo Software F~oy,d"""ing GuideStratus vompany Co,lfi~el,~al increl"~ the counter would reach the value of 0~ x in 358.1 seconds (5.97 minutes).
This register may be read and written while the counter is enabled.

12.3.22 ~. Iur~ ce Counter Trig[1:0~
Type: ReadlWrite Ottset 122:0]: l~tt8:7FFEE0 ~ .I,s~l~t on CPUIMEM boards Cold and warm reset clear these regis~ers.
Comr~t~h~lity: exact This register defines the trigger condition that will i"o,~",enl the pe(lo""anoe counter. A trigger cond,~on is evaluated every 83.381361 ns (one ~12MHZ Xbus phase). All Evaluations occur using data that is Ulined up" with the given Xbus bus ope,~ion. That is function codes physical address busy error ACK (etc.) are all delayed the correct amount so that all the i,lf~.",~ion pleser,~d to the trigger circuitry have meaning for that Xbus ope(alion. Any bit may be masked to a don't care using the perf_counter_mask r~y.itt:,:, (below).
See Perf_Counter_Mask for bit ~efin-- ~ns.

12.3.23 P~. Iorl~ance Counter Mask[1:0]
Type: Read/WriteOffset [22:01: ~tv8:7FFED0 Pl~r~nt on CPUIMEM boards Cold and warrn reset clear these registers.
Com~ ility: exact This register works in conjunction with the perf_counter_trig register above. A bit that is a one '1 ' in this register signifies you "don't care" to evaluate this bit as part of the trigger. A bit that is a zero '0 in this register signifies you wish to c~ ~ lu~t~ this bit as part of the trigger.
Perf_Counter_Trig[0]~ ttO) OR Perf_Counter_Mask[0]: (~ tU0) 31:26 Reserved.
~6 xbus error: Some board drove A or B bus enor for this Xbus opel ~n or this op~,alion was aborted due to a previous bus error. All above data could be wrong/invalid.
~5 my_busy: This board drove busy for this Xbus ope(ttion.
~4 xbus_busy: Some board drove busy tor this Xbus Op6~..ti;)1-.
~3 my_ack: This board dro~Je ACK for this Xbus operation.
~2 xbus_ack: Some board drove ACK for this Xbus ope,abion.
~1 i_drove_bus: This board was ~us master for this Xbus opel Iticn.
~0 first_op: This is the first ope, ticn of a block transfer.
~9 func_op: The info bus contains a valid function code byte enables RC and physical a~
dress else it iS data.
~8:12 trid~6:0): The 7 bit trid used on this bus operation.
11:6 function_code [5:0]: Needs t~ ;fi~ on with func_op (bit 19).

5:4 remote col-6-c~nl bits [1:0]: Needs qualircdtion with func_op (bit 19).
~:0 byte_enables [3:0]: Needs qudlilication with func_op (bit 19).
Perf_Counter_Trig~ tt~) OR Perf_Counter_MaskE1]: ~ U~) 29 Dece"lber 1995 108 Polo Software B~u~,d....ning Guide Stratus Company Confidential 31:0 physicai_add~ s[31:00]: Needs qualification with func_op.

12.3.24 Perfor.,.~..ceCounterControl Type: ReadlSeUClear Offset 122:01: ~tC8 Pr~5~.~l on CPUIMEM boards Cold and warm reset clears thls .eg;sler C~"-~.atiL.ility: exact This is a Read/SeVCtear register cGIl~llarld~ are pe,h"."ed by wriffng to this register with data bits 14:0] pointing to a bit to set or clear and bit 7 deterrnining whether the bit is set (=1 ) or cleared (=0).
Reading this register retums S2-bits.
31:2 Reserved.
Perf_Counter_Trig_(onLNot_Equal This bit retums a 1 if a board's ,c~.f~",~nce counters are ir~-~",enting when the trigger condition is not equal otherwise it retums a 0. Wrfflng to set this bit (00000081 x) will cause the counter trigger circuitry to i"~e,-,~,)t the perf_r;ounter whenever the trigger condition (as defined in perf_counter_trig perf_counter_rnask) is NOT seen. Wrfflng to clear this bit (lJ000001 x) wlll cause the perf_counter to in~.S., ~. n when the trigger condiffon is seen. This bit is cleared on cold and warm reset.
0 Perf_Counter_Enable This bit retums a 1 if a board's pe-h,---.ànce counter is enabled otl,elw:se it retums a 0.
Wriffng to set this bit (000000~0x) will enable the pe,lu,,,,ance counters and the per-I~""~nce counters will inc,e,..e.ll every time the trigger condiffon (as defined in perf_counter_trig. perf_counter_rnask and perf_counter_trig_not_equal) is encoun-tered. Writing to clear this bit (0000000x) disables any incremenffng of the counter.
This bit is cieared on cold and warm reset.
12.3.25 Fault Bit[1:0]
Type: ne~ Offset 122:0]~ tC0 t on all Xbus boards 7FFEB8 Cold reset clears this reyi~t~r, wa~ reset has no effect CC~ r~ r: new (eg~st~r This register is used to gene,dt~ errors on the Xbus or local to the board depending on the setffng of the Error Control register. Each bit in these registers to~ ")or,d~ to a bit on the info bus and the control bus. Setffng that bit in the register will cause the co,-~sponding bit on the bus to be i"co,.~cl (inverted). The ffming of when the bit is ill~--~..1 is based on the Enor Control register setffng.
Fault Bitl0]: (/r~tC0) 31:0 Fault co--~ponding bit on info bus 31:0 for the first half of the phase Fault Bit[11: (~r~tt~8) 24 Fault parity for the tirst half of the phase 23 Fault func_op for the first haif of the phase 22:16 Fault trid[7:0] for the first half of the phase 15:12 Fault co--espon ~ ~ to checkbits 7:4 for second half of the phase 11 Fault unused control[3]
29 Dece"lber 1995 109 .. .. .

Polo Software Bloy,dl,ll,ling GuideStratus Company Confidential 10 Fault busy 9 Fault maint_int 8 FauU ACK
7:4 Fault co,-eaponding to checkbits 7:4 for first half ot the phase 3 Fault bus_err_b 2 Fault bus_err_a Fauit grant_inh 0 Fault bus_req 12.3.26 D~ta Match Type: R~a~'llle Ottset 122:0l: 7FFEB0 P,~ent on all Xbus boards Cold reset clears this register, warm reset has no effect CC~,,,~AIt)llil~r. new register This register is used for the data matching capability of Yhe error generation. This allows the user to set a trigger and the error will not occur until a match is seen on the inwming info bus.
31:0 Data pattem to match against incoming info 31:0 for the first half of the phase 12~3.27 ErrorControl Type: Read/Set Offset 122:01: 7FFEA8 Pr~sent on all Xbus boards Cold reset clears this ~~;sler, warrn reset has no effect C~n,r~i~ llTty: new register Note; The Error Control register is a seUclear register, soflware can set one bit at a time, but cannot ciear any bits. Bits are cleared by hardware once the error has been gene-al~d, also once bit 0 is set no other bits can be changed by software be~se the enor has already been initiated This register is used to control the different types of errors seen on the system. There are three modes of oper~ic n. The first is normal mode, no error will be produced on the system. The second mode is immediate trigger. This causes an error after this register is written to and an access is directed to the Xbus. The last mode is data match. This allows a trigger pattern to be set in the Data Matr;h register. The error will not occur until the pattem is seen on the incoming info bus. The rnatching is done on the first half of the info phase. Note: It is i""~, ~nt to set up the Fault Bitll :~l register and the Data Match register before this register is set. The types of enors that can be produced co,.espond-a direcUy to the cases d~s~i,it,ed in secbon 6.4, Bus Errors, on page 44.
To produce an enor, the first step is to decide which type of error is desired from table 8, Enor Types. Then, if the error is a Data Match enor the Data Match register must be iniffalized along with the ~ault Bit[1:0] register. If the error is i"""~diale then the Fault BH[1:0] register must be initialized. If a transient enor or a bus busy is desired the Data Match register must be iniffalized but the Fault Bit[1:0] register does not have to be initialized. Atter necessaa~y ~eg;~,te.a are initialized the bit in the Error Control register should be set which co,-~spor,~ to the enor the user is looking for. Then bit 0 of the Error Control register is set. If it is an i"."~diale error then as soon as the Xbus is ~rxssed the enor will occur. If the error is a data match error then it will not occur until the data pattenn is seen on the incoming info bus.

29 Dece--ll~r 1995 ~,~ ~ g 110 Polo Software Bluy,~..""ling Guide ~uatus Company Confidential Onoe bit 0 is set and the error has occuned haldw~l~e will clear the bit that indir;ates the error and it will also clear bit 0. The error will be asse"l~d for as long as necessary to produce the particular case. Therefore wme cases require the srror to occur during both CPUTest and lOTest where as other cases will only require the fault to happen for one phase. Only one error is allowed with the r ~ub~tion of the b d- ISiel It fault. There can be a 1, ansient fault on both buses. If more than one bit is set the lowest bit number will have priority. All bits will be cleared after the error is produced.
Table 21. Error Types Bit Typeof Enor Case Category 0 0: normal operation 1: produce enor -- Nomnal CPU Board Faulty Input Circuit- CPU Driving Case 1 Immediate 2 CPU Board FauHy Input Circuit - I/O Board Driving C~e 2 Data Match 3 CPU Board Different Data C-Side and D-side Case 3 I~.,,,,eui~t~.
4 CPU Board Faulty Output Circuil - Buffer to Pad Fault Case 4 lul",6diatc CPU Board Open - CPU Board Driving Case 5 1IIIIII6di. ~
6 CPU Board Open - I/O Board Driving Case 6 Data Match 7 CPU Board Short Case 7 I~"",edid~
8 Dach~A~neOpen Etch Case8 Data Match 9 Da~h~lane Short Case 9 Immediate I/O Board Faulty Input Circuit Case 10 Immediate 11 110 Board Output Circuit Fault- Buffer to Pad Case 11 I~.""e.liate 12 llO Si"~ le sid~ Access - ASIC Parity Gen. Fault Case 12 l~lllllediale 13 I/O Single-side Access - ASIC PCI Data Path Fault Case 13 I"""eLliate 14 I/ONon-single-sideJAccess DfflerentC-DData Case14 II"",édial~
15 I/O Board Open Case 15 I~"",eu'i.~t~
16 llO Board Short Case 16 1~"",6.1iate 17 Transient Fault Bus A Case 17 Data Match 18 TransientFauHBus B Case 17 Data Match 19 Bus Busy Bus Case 18 Data Match The f~ ng lines desc, il~e behavior that might not be expected with some of the error insertion cases.
Case 17: the t~a"sien~ bus fault will not cause the faulting board to freeze its error feg;star;j all other boards will freeze their error registers Case 18: busy will be ass~, led for two phases.

29 Deo~",~r 1995 111 . . ..

Polo Software F~gla~ ng Guide Stratus ~,ompany Confidential 12.4 CPU 5~ ecific Xbus Register Descripti~.ns This section deals with the ASIC specific Xbus/Golfbus registers found in the Polo/HSC2 ASlCs.
All of the re, ~ ~~~s in the board specific space ~nth ~e exception of the l/O Address Map Enor register are nor~privileged It:y;~t~l~. A description of all CPU specific Xbus registers r an be found . 'l~ ng the table.
Table 22. Cyclops/Mirage Specific Register Map Offset ~22:01 Reyister NameType HSC2 7tF830 ASIC Specific Configuration P~ead/SeVClear Not Impl.
7FF828 I/O Address Map Error 2 Read Only Impl.
7FF820 I/O Address Map Enor 1 Read Only Impl.
7F&a8~8 Jiffv Control Read Only Not ~mpl.
7F~810 Master Jiffy Counter Read Only Not Impl.
7F&808 Ouic dime Read Only Not Impl.
7F&800 Time of Day Read Only Not Impl.
a pnvileged space & - F; nan-privileged space. & - E
Cyclops and Cougar (the other large ASIC on the CPU board) share the privileged page 7FF and the non-privileged page 7FE with Cougar reg;ste~ in the lower half of the pages in 000->7F8 and Cyclops fe9;sle~ in the top half in 800 -~ FF8. Table 23 illustrates how the a-hhe~ses for either CPU Xbus specfflc or CPU Xbus co"""on ley;~tels are forrned.
Table 23. CPU Register System Address Formation l~lot ' Ott~et trom table 15 or table 22 (Xbu~) -a. pai~ed space -> - l; uDpaired space -> - O

12.4.1 ASIC Speclfic Configuration Type: ReadlSetlClear Only ~n privileged page 7FF Offset [22:0~ 30 Cold reset clears thls ~e~;sl~r, wann reset has no effect 31:2 reserved 2 IOVA address mode 1 = 40 bit mode, 0 = 32 bit mode, refer to section 5.2.
Fan Fault set broken disabled O Maint~ndnce Internupt disabled for IOVA errors.

12.4.2 IIO A~l~r..ss Map Error 1 Type: Read Only in privileged page 7FF Ot~set ~22:01~ d28 Cold reset clears this ~g;~ter, warm reset has no effect 31:0 IOVA Address - refer to figure 21.

29 De~" lL e~ 1995 ~ ~ 112 Polo Software . .ogramming Guide ;,tratus Company Confidential 12.4.3 IIO Ad~lr~,3~ Map Error 0 Type: Read Only in priYileged page 7FF Oftset 122:0]~ 820 Cold reset clears this regisl~r, waml reset has no effect 31 :10 reserved.
10 IOVA Out of bounds Access Error 9 IIOVA Illegal Slot Error - PCI slot accessi"g (from Xbus TRID) di~ag,ees with PCI slot field in the IOVA map entry.
8 IOVA Checksum Error.
7 IOVA Invalid Map Entry Error.
6:0 IOVA Trid (Xbus trid during 1 st bus cycle of IOVA info phase).

12.4.4 TimeOfDay Type: ne~uwrite in Pr~vileged page 7FF Ottset 122:0]: 7FF800 Read only in non-Privileged page 7FE Ottset ~ 0]: 7FE800 Cold reset clears this register, warm reset has no eflect This register (and the ~L'~ 'i. 19 three) implement the user ?~ C es ~ ~ 'e time of day function on Mercury. These fe9;~ are initialized through Global writes to page 7FF and are read-only in the user visible non-privileged page 7FE.
The time of day is maintained in a single 56 bit master counter that illcfe",eflls at 12MHz. For compatibility with existing Stratus software various slices of this counter are referred to by dfflerent names. The time of day refers to the most significant 32 bits of the master counter: it gives the current time in second~.
Figure 34. Cyclops Master Counter E~it 24 incremenh every second ~it ~ mcrements every 15.258789 usec ~ ~ +
47 39 31 23 ~5 7 ~12M~z I

Timeof Day l reload value ¦

Ouicknme ~ I

Mader Jdfy Counter Jifly rlme This register should only be written when the counter is stopped (Master Counter Enable bit in the Jiffy Control register is off). This register is cleared by cold reset.

29 Dec~",~er 1 995 1 1 3 ~1 Polo Software ~lug-d~ ing Guide Stratus ~,ompany Confidential 12.4.5 Quitl~ e Type: Read Only in Privileged page 7FF Offset 122:0]~ 808 Type: Read Only in non-privileged page 7FE Offset t22:0]: 7FE808 Cold reset clears this register, warrn reset has no effect A read-only view of bits 31 :0 of the master counter. Bit 24 of quicktime in.i,t:",e"~ once per second.

12.4.6 MasterJiffyCounter Type: ResdlU'Ii~e in Privileged page 7FF Offset 122:0]: 7FF810 Read only in non-Privileged page 7FE Offset ~22:0]: 7FE810 Cold reset clears this res~isler, warrn reset has no effect 31:8 Jiffy Counter/Master Counter This ~ield contains bits 23:0 of the master 12MHz counter. Bits 31:16 are often re-ferred to as the Jiffy Counter. A Jiffy interval is 15.258789 usec, or 2-16 seconds.
7:0 Reload Value This is the counter reload value. The least si~nific an~ 8 bits of the master counter get ,.1c?ded with the co"ter,~ of this register when they roll over. The reload value is ~.,uy,d,),,,,able to allow accurate clock operabon when the machine is running with a diflerent speed crystal; i.e. 12MHz is no longer really 12MHz. The reload value should be set to 256 minus the number of 12MHz clock ffcks in 15.258789 usec. Insystems with a S~h~dll~ clock this works out to 49 hex (73 decimal).
This register should only be written when the counter is stopped (Master Counter Enable bit in the Jifty Control register is oft). This register is cleared by cold reset.

12.4.7 Jiffy Control Type: ReadlSetlClear in Prlvileged page 7FF Of ~set [22:0]: 7FF818 Read only in non, .i~lil~e~l page 7FE Oftset 122:0]: 7FE818 Cold reset clears this register, warm reset has no effect 31:1 Reserved 0 Master Counter Enable When this bit is set, the master counter (and thus the rlme of Day, Jiffy, and Quick-time) illcfemer,~ normally. Clearing this bit stops the counter.

29 Dect:" ,~er 1995 114 Polo Software ~ ,an"ning Guide v,ratus Company CGnfidential 12.5 PCIBIHSC2 Specific Xbus Register D~scri~ions These ley;st~l . are the PCIB/HSC2 specific Xbus/Golfbus ,eg;st~ .;. that reside in a PCIB/HSC2 board. Registers des.;, il,ed here are i" ~ l~ "ent~d in the Gambit ASIC on the PCIB board. Refer to secbon 4.5 PCIB MlO/lOBus CompaUble System Address Map for details on the address mapping.
Table 24. PCIB/HSC2 S~ecific Xbus/Golfbus Register Map.

Offset ~22:0] Register Name Type 7FFEC0- nese~ved 7FF080 Gambit Maint. Attention Req. ~31 :0I Read/SeVClear 7FF078- Reserved 7FF008 lOBus Status Register Read/Clear 7FF000- Reserved Table 25 illustrates the fu, ",alion of system l/O ad il~sses for the Xbus specific and Xbus coi ", .,on rey; .t~ .~ found on the PCIB board. Note that the slot number is inverted. Polo PCIB boards do not run dupiexed so they will not respond to paired ar~esses Table 25. MlO Compatible Register System Address Formation l~lot O Otbet from t~ble 15 or tsble 24 a. paired space -> - l; unpaired space -~ - 0. must be n~n-paircd Pelow is a register by register des..i~on of the GambitlMirage specific Xbus/Golfbus registers.
Note that registers are restricted to 32 bits and are spaced at 32 bU boundaries. The actual register width varies according to its d~filN ~ n. Reads to any u"d~ 'ined holes in the address space return U~ ed but deterrninistic data. For writes there is no effect.
The format of this inforrnation is either a bit number or a bit encoding and a functional name depending on the configuration of the register followed by a desc,i~ion of the funcbon of that bit or bit encoding.
All regist~,;, are reset to zero unless olhel~:;e noted.

12.5.1 Gambit Mai--l6,~ ce Attention Request [31:0]
Type: ReadlSeVClear- CID ~ erenl Ottset l22:0]~ 080 Cold reset clears this legist~r.
Warrn reset and going through sync point have no effect.
This CD different register indicates which PCI slot (including the Gambit and Mirage host bridges) caused a ",a;.,!~ nance interrupt. Note: In Polo, this is a CID dill~z-~nt register, but it differs from other CID dlll~:le~lt ~oy;Slt:~s that bits 8, 9, 10, 11 and 12 are cleared by writing 29 D~c~ul~l 1995 1 15 CA 02257~11 1998-12-03 Polo Software R~ug~ ing Guide Stratu~ _ompany Confidential 3-~'0000000~, 32'hOOOOOOO9, 32 I~OOOOOOQ~"3> hOQOOOOG~ and 32 I~OOOOOOOC r~spe ~ ly.
31,15 Reserved 30,14 Reserved 29,13 Reserved 28,12 Host_bridge (D-side) 27,11 PCI slot 3 26,10 PCI slot2 25,9 PCI slot 1 24,8 PCI slot O
23,7 neserved ~,6 Reserved 21,5 Reserved 20,4 Host_bridge (C-side) - Polo Only 19,3 PCI slot 7 - Polo Only 18,2 PCI slot 6 - Polo Only 17,1 PCI slot 5 - Polo Only 16,0 PCI slot 4 - Polo Only 12.5.2 lOBus Status Type: ReadlClear O~fset[22:0]~ 008 Cold reset clears this register.
Wsnn reset and going through sync point have no effect.

31:22 Reserved ~1 Xbus/Golfbus Maintenenace Interrupt asselt~3d due to PCIB/HSC2 Attention Reques~
See the SAM Maintenance Attenbon Request Register for deta~ls.
20:0 Reserved 29 Dec~"lber 1995 ~ 116 .

Polo Sof~vare ~.vgramrning Guide ~,~.atus Company Confidential 12.6 SAM Int~rface R~-st~r DescrFiptions These registers are the l/O registers that reside on a SAM board from a s~ft~ pe,:~i,ecti~c. For Polo, these registers exist either in the Cyclops ASIC on the CPU board or in the Gambit ASIC on the PCIB board. For HSC2, these registers exist in the Mirage ASIC located on the HSC2 board.
Refer to section 4.5, PCIB MlOllOBus Compatible System Address Map, for details on the address mapping.
The SAM ~,..~,dti-Jle ASlC's register space is divided as follows. The 16k space is divided into four 4k pages to simplify the decode of this region in the Cyclops Polo ASIC and Gambit Polo ASIC.
Polo impiements a single Map RAM in the CPU, four sets of per bus registers (one set for each of four PCI buses) and 16 sets of per slot registers (one set per each of 16 possible PCI slots). HSC2 implernents a single Map RAM in the Mirage ASIC, one set of per bus ,~g;~.t~,~ and 4 sets of per slot fegi~ib,~ (one set per each of 4 possible PCI slots).
All Ar~sses alias to the app,opfiate physical register, although so': - ~..e locking is required to prevent conflicts. See section 5.5 for software lock leabi~tions on these registers.
Table 26. SAM Compatible ASIC Register Map Offset [13:12] Page Polo HSC2 2'h3 System Busslotregisters Gambit Mirage 2'h2 Map RAMregisters Cyclops Mirage 2'h1 System Bus registers Gambit Mirage 2'hO PCI slot registers Gambit Mirage Note that ~y;~ are re~hi~;ted to 32 bits and are spaced at 64 bit boundaries. The actual register wiFith varies according to its definition. The byte significance of these registers is big-endian. Reads to any undefined holes in the address space return zeros. For writes, there is no effect. The f "~ ing table illustrates the formation of a SAM compatible l/O register space address on the system bus.
All SAM registers are a~essed lh.o~yh Xbus slots 2 and 3 in Polo systems. Acc~3s~s to re~isters associated wi~ PCI slot numbers greater than seven ~ll not be ~6 ~ond~d to In Polo systems.
Table 27. SAM Compatible Register Systern Address Formation 1 o 9 8 7 6 5 4 3 2 1 O g B 7 6 5 4 3 2 1 0 9 8 7 6 5 4 3 2 1 0 1 1 1 1 sy t m ~ 1 o PClSlot 1 ' re~isterof~settrombble29,table bu~ slot numbor 30, or tabb 31, Inv~rl d a. pair~d space -~ - I; ullpai~ed space -> - O
b. p~e offset fwm lable 26 A bit by bit cies~ h,on follows (bit 31 being most significant and bit zero being tine least sig~ an bits). The forrnat of this iuf(""- t ~ n is either a bit number or a bit encociing, depending on the 29 Dec~,nL~r 1995 /\ ~ 117 Polo Software P~uy~d~ ning Guide Stratus ~,ompany Confidenbal configuration of the register and a funcbonal narne I.'lc .:ed by a des~ on of the function of that bit (bit encoding~. All registers are reset to zero unless uU,enAise noted.

29 De~"lL~er 1995 ~ ~ 118 Polo Sof~ware ~ g~a~ ning Guide ~.~ratus Company Confidential 12.6.1 SAM Interface Registers - Page 3 These register reside in the Gambit and Mirage ASIC. For this group of registers there is only a single register for the entire Polo and HSC2 system to which all 16 PCI (Polo) or 4 PCI (HSC2) slot a~J~esses map.
Table 2B. SAM C~",paiil,le l/O Space Map - Page 3 Oflset l11:0] Register Name Type HSC2 FF8 Disk LED Control nea~'~ite Supported FF0 Disk Status (Polo is CD different) Readl\Nrite Supported FE8 Power supply LED ReadlSeVClear Not Supported FE0 Power supply status Read only Not Supported FD8 Fan speed control ReadlSeVClear Not Supported Note that unlike a Polo/Gambit the C-side and D-side Mirage's bridge a single PCI bus to the system. Therefore a HSC2 system does not have support tor multiple disk shelves (SCSI port 0 &
1 ) and will react to Aoe.~ 5eS to SCSI Port 1 (analogous to Gambit ~side) only. Accesses to SCSI
port 0(analogous to Gambit C-side) are ignored.
12.6.1.1 Disk LED Control Type: ne21N~IUe Offset ~ 0]: FF8 Cold reset as documented below, warm reset has no effect This register is used to control the amber LEDs on the front of the Polo disks via the Stwa9eworhs specified "Disk fault bus." Although this register is read/write it should be treated as write-only by software. The reason is that the PClBs in Xbus slots 2 and 3 share the fault bus lines. Writes to this register in either of the PClBs set the disk LEDs but only the most reoently written PCIB will retum a value from this register that is gua, al ,teed to ~r.~:~pond with the actual state of the disk LEDs.
31 :6 reserved.
5 Disk shelf select bit. When writing this register and this bit is a 0 SCSI port 0 is selected (C side Gambit). When writing this register and this bit is a 1 SCSI port 1 is selected (D side Gambit). This bit has no effect when reading this register.
Note: This bit is not supported on HSC2.
4 LED on bit. This bit is set to a 1 to tum on the amber device fault LED for the device whose SCSI TID is specified by bits 2:0 or all devices when bit 3 is set to a one. Se~ting this bit to 0 turns off the LED(s).
3 LED address test bU. This bit is used in conjunction with bit 4 to test the amber devioe fault LEDs on all disks in the self. Setting bit 3 to a one overrides the TID set by bits 2:0.
2:0 These bits define the SCSI devioe Target IDeu~fi~lion (TID).
12.6.1.2 Disk Status Type: ReadlSetlClear Oftset ~11:0l: FF0 Cold reset as documented below, wann reset has no effect 29 Dece,lll~er 1995 119 Polo S~ f~.Y,~c..e F~uylall l".ing Guide ~tratus ~,ompany Confidential CD Dill~rent (Polo Only): read via unpaired space This register displays information based on the St~rage~rolhs disk subsystem SHEU_OK and SWAP_L signals. There are 4 SCSI ports available... C side LOCAL, C side REMOTE, D side LOCAL and 1:~ side REMOTE.
31:15 reserved 30:14 reserved 29:13 ,t,ael~QJ
28:12 ~ 3d ~7,11 D side REMOTE Swap Interrupt Disable - Setting this bit by writing 0000008Rx suppress-es the maintenance intenupt that no").ally is issued when a REMOTE swap on the Dside occurs. Disabling the interrupt does not disable the sefflng and clearing of the REMOTE Swap has occurred bit. This bit is cleared by cold reset, or by wribng 0000000Bx.
~6,10 D side REMOTE Swap has Occurred. This bit is set when a disk has been inserted or re-moved trom the D side disk shelf. It stays set until explicitly cleared by writing 0000000Ax, or until it is cleared by cold reset. Attempting to set this bit by writing 0000008~x has no effect. When this bit is set, a maintenance interrupt is issued un-less masked by D side REMOTE Swap Interrupt Disable bit.
25,9 D side Swap Internupt Disable - Setting this bit by writing 00000089x su~ sses the main-tenance interrupt that norrnally is issued when a swap on the D side occurs. Dis-abling the interrupt does not disable the setting and clearing of the Swap has occurred bit. This bit is cleared by cold reset, or by writing 00000009x.
24,8 ~ side Swap has Occuned. This bit is set when a disk has been inserted or removed trom the D side disk shelf. It stays set until explicitly cleared by writing 00000008x, or unffl it is cleared by cold reset. Alt~ll.pting to set this bit by writing 00000088x has no ef-fec~ When this bit is set, a l"a;u~enance interrupt is issued unless masked by D side Swap Interrupt Disable bit.
23,7 reserved 22,6 reserved 21,5 reserved 20,4 reserved 19,3 C side REMOTE Swap Internupt Disable - Setting this bit by writing 0000008~ suppress-es the maintenance interrupt that norrnally is issued when a REMOTE swap on the C
side occurs. Disabling the interrupt does not disable the sefflng and clearing ot the REMOTE Swap has occurred bit. This bit is cleared by cold reset, or by writing 00000003x.
This bit is not implemented in a HSC2 system.
18,2 C side REMOTE Swap has Occurred. This bit is set when a disk has been inserted or r~
.moved from the C side disk shelf. It stays set unbl expliciby cleared by writing 00000002x, or unbl it is cleared by cold reset. Attempting to set this bit by wribng 00000082x has no effect. When this bit is set, a maintenance interrupt is issued un-less masked by the C side REMOTE Swap Interrupt Disable bit.
This bit is not i"",le."e"l~d in a HSC2 system.
17,1 C side Swap Interrupt Disable - Setting this bit by writing 00000Q81x s~lpp~sses the main-tenance interrupt that normally is issued when a swap on the C side occurs. Dis-~9 Dec~"ll~r 1995 ~ ~ 120 , Polo Software ~._.Jram~ling Guide ~ atus Company Confidential abling the interrupt does not disable the setting and clearing of the Swap has occurred bit. This bit is cleared by cold reset, or by writing 00000001x.
This bit is not implemented in a I ISC2 system.
~6,0 C side Swap has Occurred. This bit is set when a disk has been inserted or removed from the C side disk shelf. It stays set until explicitly cleared by writing 00000000x, or until it is deared by cold reset. AU~Ill,.Jti,l9 to set this bit by wrihng 00000080x has no ef-fect. When this bit is set, a maintenance interrupt is issued unless masked by the C
side Swap Interrupt Disable bit.
This bit is not implemented in a HSC2 system.

12.6.1.3 PowerSupplyLED
Type: n~litc Offset [11:03: FE8 Warrn and Cold reset have no enect on this register.
Polo Only Regi-cle~
~1:3 Reserved.
2 Red LED bit. Writing a 1 to this bit position turns the Red LED on. Writing a 0 tums the Red LED off. This bit will also be set if a iocal power supply fault is detected.
Yellow LED bit. Wribng a 1 to this bit posibon turns the Yellow LED on. Writing a 0 tums the Yellow LED off. This bit will also be cleared if a power supply fault is detected on either the local or remote power supplies.
~ Green LED bit. Writing a 0 to this bit position turns the Green LED on. Writing a 0 tums the Green LED off. This bit will also be cleared if a local power supply fault is detected.
~2.6.1.4 Power Supply Stat~ anguard powe supply) Type: Read Only Offset [11:0~: FE0 Warm and Cold reset have no effect on this regist~r.
Polo Only Register ~1:10 reserved.
~ Local power supply fault bit. This bit is set if the local power supply reports a fault.
~ Remote power supply fault bit. This bit is set if the remote power supply reports a fault.
7:0 Local power supply ID.

12.6.1.5 Fan Speed Control Type: Write Only Offset [1~ :0]: FD8 Cold reset clears this ,e~aisl~r.
Polo Only ne~ ster This register is used to control the fan speed. On power on, this register is cleared, there by forcing the fan to run at normal speed. Wrihng oxh80 to this register sets this register, there by forcing the fan to nun at higher speed. Wribng oxhO0 to this register clears this register, there by forcing the fan to run at norrnal speed.

29 Dece~-lber 1995 ~ ,~ 9 121 PCTrUS97/09781 Polo SomNare ~Oyla~ ing Guide Stratu~ _ompany Confi.le.,~al 12.6.2 SAM Inte.face Registers Page2 These registers reside in the Cyclops ASIC in Poto systems, hD~ erthey are ad~ssed through Xbus slot 2 or 3. In a HSC2 system, these ~eyiste,~ reside in the Mirage ASIC and should be ad-h~ssed through the normal Golbus slot.
In a Polo system, there is only a single set of ley;.,t~ to which all 16/4 PCI slot ad.~ sses map.
In a HSC2 system, there is a set of registers for each PCI bus (Dune).
Table 29. SAM Compatible l/O Space Map - Page 2 Offset [11:01 Register Name Type FF8 Address Table Co,-.. -.a"d/lOVA ReadNVrite FF0 Address Table Data 2 ReadlWrite FE8 Address Table Data 1 ReadlWrite FE0 Address Ta~le Data 0 Read/Write 12.6.2.1 Address Table Co~ .J and IOVA Register ~ype: RearJN~Iite Otfset [11:01: FF8 Cold reset clears this register, wann reset has no effect.
This register, combined with the three Address Table Data reg;~ , is used to access the rr~p rams for IO~/A llanslat\on. See sec~on 5. for a complete de~c-i~.tion of the address mapping funcbon. To write the map rams, fi~st load up the Address Table Data registers with the desired write value then write the Address Table Co"""ar~d and IOVA Register with the proper device index and page and bit 31 equal to 1. In order to read a map ram simply write to this register with the proper device index and page and bit 31 set to zero. Follow this initial write with a read of any of the four leg;~t~,~ to retrieve the data.
31 Co--,---dnd 0 = read 1 s write 3~28 Reserved 27-20 Device Index 1~14 Reserved 1~12 Pags 11-0 Resemed t2.6.2.2 Address Table Data 2 Register Type: nea~lM~l ile Oftset ~ 01: FF0 Cold reset clears this reyister, wann reset has no ettect.
Refer to section 12.6.2.1, Address Table Command and IO\IA Register, for a desc,i~,~on of this register.
31 - 30 Reserved 29 Dece"tl~er 1995 ,~ 122 ,~ O

, Polo Software F~ug~ ing Guide ~"atus Company CG"fiden~
~9 -26 PCI Slot Number Note: In HSC2 the most siy"ificanl bit of the PCI slot number (bit [29]) is not used due to space limitations in the Mirage Map RAM. Since a HSC2 only supports 4 PCIdevices this should not be conside,~:d an issue.
25 5 Reserved 4 Option bit- Xbus/Gollbus Incohe(e,l~ Memory Access 3 Opbon bit - Swap 16 or ~ bit Endian 2 Option bit - Xbus/Golfbus Lock Cycle Option bU- Data Pre-rsad 0 Entry vaîid bit 12.6.2.3 Address Table Data 1 R~gis~r ~ype: Rea~'llle Offset [11:0l: FE8 Cold reset clears this fegisler, waml reset has no effect.
Refer to section 12.6.2.1 Address Table C~"--.,.and and IOVA Register, for a descripbon of this registen 31:16 Block Xbus/Gotfbus Virtual Index 15:12 Reserved 11:2 Block Xbus/Golfbus Ending Physical Address 1 :0 n~ d 12.6.2.4 A~ ss Table Data O Re~ister ~ype: nez~uwr;te Offset [11:0]: FE0 Cold reset clears this register, warm reset has no effect.
Refer to section 12.6.2.1 Address Table Co"""and and IOVA Register for a des~ ,i~on of this register.
~1-2 Block Xbus/Golfbus Starting Physical Address 31-12 Physical Cache Tag 11-2 Line Oflset ~:0 Reserved.

29 Dece"ll~r 1995 123 PCTrUS97/097~1 Polo Sof~ware F'l~g,~.,.,.ling Guide Stratus t,~mpany Confidential 12.6.3 SAM l~t~rf.,ce Registers- Page 1 Each GambiVMirage ASIC has one ot each of these l~y;Stt;lS for the one PCI bus (up to four controllers per bus) it controls.
Table 30. SAM Compatible Common lO Space Map Page 1 Offset [11:0] Register Name Type FF8 PCI config_addr Re21 7FO PCI config_data nEadN\'~it~
FE8 Test Control Read/SeVClear FEO PCI ErrorRegister ReadlSeUClear FD8 PCI IOVA Error Register Read only FDO SAM Status Read/SetlClear FC8 - FB8 Reserved FBO Arl,ib, ~ n Freeze Counter Max. Value naad~ itd FA8 Host Request FIFO Timeout Value Register Read~Write 12.6.3.1 PCI config_addr Type: n~lM'Ii~e Otfset [11:0~: FF8 Cold reset clears this l~gisler, wann reset has no effect.
IO reads and writes to this register cause no PCI cycles. IO reads and writes to the 'PCI
config_data register' use the illf~JIl--a~iol~ in the 'PCI config_addr register' to perfomn a configuration cycle on the PCI bus except for device zero i.e. the bridge which is accessed intemal to GambiVMirage using this same ~--echauis,.,. Thus configuration reads and writes to the host bridge~s configuration space will not cause PCI cycles but will instead be handled intemal to the host bridge.
Special cycles will be gen~ldl~d when the 'Bus Number~ equals zero i.e. the bridges secor,d PCI bus and the 'Device Number' is all ones and the 'Function Number' is all ones and the 'Register Number' is all zeros.
31 Enable config_data - As per the PCI spec contig_data is disabled it this bit is a 0.
3~24 F~eserved - read only return 2eros 2~16 Bus Number - e"cocbd value to select 1 Of 256 PCI buses in a system.
15-11 Device Number - encoded value to select 1 of 32 device on a given bus.
10-8 ~unction Number - encod~d value to select 1 of 8 functions on a specific device.
7-2 PCI Configuration Register DWORD Index - specific bytes from BEs.
1-0 2'bOO - Read only The configuration type will be added in by hardware based on the 'Bus Number' - type 00 config to attached bus. type 01 config passed to 'Bus Number'. Refer to the PCI Local Bus Speu;fi,~d~iQl- for more illfOIlllaliull on type 00 and type 01 cGIlllndlnls.

29 Dece,nl~er 1995 ~ , 124 , .

CA 022575ll l998-l2-03 WO 97/46941 PCT/US97/097Xl Polo Software Proyrammlng GuideS~ us Company Confidential 12.6.3.2 PClconfig_data Type: ReadlWrite Onset [11:03: 7F0 Cold reset clears thiS register, warm reset has no etfec~.
IO Reads and Writes to this register result in PCI configuration cycles on the PCI bus as defined by the address in 'config_addr' except for device zero i.e. the bridge configuration space is ac~ssed intemal to Gambit/Mirage using this sarne Illecha,~ . The PCI configuration data is litlle-endian. The Xbus and GoUbus are big-endian. Reads and writes to the PCI configuration space of either the host bridge or the mated PCI adapter are always byto s~-:apped to provide 'byte address consistency' rather than 'MSByte consi;,~n~'. Where siy";fi~ance of bytes is ,ea"anged by the swawing Ille~h;l~l;5~ software will have to re orderto restore significance. Referto section 5.6 for a discussir!n of byte ordering (big endian versus little endian) on configuration cycles.
Sefflng bit 31 in the 'PCI config_addr register' will cause ~ccesses to the this register to be enabled. If bit 31 of 'PCI config_addr register' is 0 writes will be ignored, and reads will return 0's and a failed_op status.
31-0 Configuration data 12.6.3.3 TestControl Type: ReadlSeVClear Ottset ~ 0]: FF8 Cold reset clear th~s re~;s~er.
31:16 n~.v~d 15 Disable Protocol enor signal generabon This bit is set by soflware to disable Protocol error signal generated by PCI core logic.
When this bit is set, the PCI core logic does not assen protocol_error signal to rest of GambiVMirage logic. When this bit is set, bit 5 of Polo PCI Error register will not be set. This bit is cleared by cold reset, or by writing 0000000Fx. It is set by wribng 0000008Fx.
14 Disable Peer to Peer error signal generabon This bit is set by sul. a.~: to disable Peer-to-Peer error signal gene,ated by PCI core logic.
When this bit is set, the PCI core logic does not assert peer_to_peer_enor signal to rest of GambiVMirage logic. When this bit is set, bit 6 of Polo PCI Error register will not be set. This bit is cleared by cold reset, or by writing 0000000Ex. It is set by writ-ing 0000008F~t 13 PCI read retum ahead enable (Polo Only) This bit is implemented in Rev. 1 Gambit only. When this bit is set, the Gambit sends read retum data to the PCI bus as soon as data is available on post_qual bus. This will im-prove the PCI bus read pe.~o""ance for cacle line read. If this bit set to to zero, the Gambit waits till last peace of data is available on post_qual bus for cache line read before it sends data to the PCI core. This bit is cleared by cold reset, or by writing 0000000Dx. It is set by wriffng 000000~Dx.
12 Disable PCI drivc~l ,eck error signal generation This bit is set by sof~ware to disable PCI drive/check error signal generated by PCI core logic. When this bit is set, the PCI core logic does not assert drive_check_error signal to rest of GambiVMirage logic. When this bit is set, bit 9 of the Polo PCI Error register will not be set. This bit is ~eared by cold reset, or by writing 0000000Cx. It is set by writing 0000008Cx.
11 Disable System error signal generation 29 Dec~l "L.er 1995 ,~ 125 CA 022~7~ll l998-l2-03 PCT/US97tO9781 Polo Software Ploylallllll;ng Guide Stratus ~;ompany Confidential This bit is set by s~f~ a.e to disable system error signal gen~,.~d by PCI core logic.
When this bit is set, the PCI core Ingic does not assert system_error signal to rest of GambiUMirage logic. When this bit is set, bit 28 of Polo PCI Error register will not t)e set. This bit is deared by cold reset, or by writing OOOOOOOBx. It is set by wribng 0OOOOO8R'IC
10 Disable Parity error signal generation This bit is set by software to disable parity error signal ~enbfi~t~d by PCI core ~ogic. When this bit is set, the PCI core logic does not assert parity_error signal to rest of the GambiVMirage logic. When this bit is set, bit 27 of Polo PCI Error register will not be set.This bit is cleared by cold reset, or by writing OOOOOOOAx. It is set by writing 0000008Ax.
g n~.~
8 Flush Mirage Write Buffer Map Entry Caches (HSC2 Only) Writing to set this bit will Invalidate the two write buffer entry caches. This will insure that Mirage will go to the Map RAM for the next PCI write access and allows software the ability to gurantee con~i~tdl~l data. Refer to the Mirage Specification for more details.
Wfiting to clear this bit has no affect. This t)it will always return a zero when read.
7 Inhibit Broken SAM on SERR
This bit is set by software to inhibit PCI error logic from removing a SAM from servir,e when an error is intentionally caused for diag..~s~i~ testing purposes.This bit is cleared by cold reset, or by writing 00000007x. It is set by writing 00000087x.
6 Error Data lltco~ d/Rearm Error and Broken neg;ste.~
This bit is set by hardware when an error condition that requires logging in any of the error f~gist~. i occurs. Writing a O to this bit clears the error state of the chip and ~re-arrns" them so they will capture the state of the next applicable error condition.This bit is cleared by cold reset, or by writing 00000006x. This bit can not be set It is set by writing 00000086x.
5 Force PCI data parity enor This ~U is set by software to i,~t~ ionally cause a PCI data parity enor for diagno~lic test-ing purposes.This bit is cleared by cold reset, or by writing 00000005Y It is set by writing 000000~5x.
4 Foroe PCI address parity error This bit is set by sofhNare to intentionally cause a PCI address parity enor for diagno~Lc testing purposes.This bit is cleared by cold reset, or by writing 00~00004x. It is set by writing OOOOOOB~x 3:0 Roserved 12.6.3.4 PClError Type: Read Only Offset [11:0]: FE0 Cold reset clears this register, wann reset has no effect. IO Reads and Writes to this register cause no PCI cycles.
This register is valid if Sta~s[16] is set (PCI Enor d~ c~J) ~1-29 PCI Grant Slot#(~O) indicating n~asld.~l~ip during .ey; .tered error. This in~ ation is en-coded in the i~"c.~;ng manner:
~9 D6~-lll~dl 1995 ~ ~ 126 . . .

POIOS ' Yar~ ram".lngGuide ~atusCompanyConfider,t~al 0: Slot 0 1 : Slot 1 2: Slot 2 3: Slot 3 4: Host 28 Pridge detected SERR# on PCI Bus.
27 Bridge d~tav~!d PERR# on PCI Bus 26 Bridge detected Master Initiated Terrnination 25 Bridge d~ ~ ~ed Target Initiated Terrnination 24 n~ 0d.
24 Host request FIFO timeout error.
Note: When this e~or occurs, other info in this .~ist~!r may or may not repre-sent correct into about PCI bus t~ _ s ~ ~ L on that caused this error. Refer to sec-tion 9.6.2.3 ~or details.
23:20 Reserved.
19-16 PCI Co"""and.
15-12 ll~cr~od 11 IDLE timeout after GNT#
PCI master failed to start an access within 16 clocks atter GNT# asse- tt:d. Note that ~e PCI co,-,--,and field and IOVA values are not applicable.
10 rlme Out Error (see PCI Common Logic Spec.) 9 Drive Check Error (see PCI Common Lo~ic Spec.) 8 Retry Count Error (see PCI Common Logic Spec.) 7 D;s~l-ne-;i Count Error (see PCI Common Logic Spec.) 6 Peer to Peer Error (see PCI Common Logic Spec.) 5 Protocol Error (see PCI Common Logic Spec.) 40 Reserved 12.6.3.5 P CllO VA ErrDr Type: Read Only Offset ~01: FD8 Cold reset clears this ~e~ ler, warm reset has no ettect.
10 Reads and Writes to this r~lsler do not cause PCI cycles.
This register is valid if Status[16] is set (PCI Error d~ J).

31-0 Xbus/Gbus IOVA for the PCI ~dnsa- tiun which got an error.
12.6.3.6 S A M Status Type: Read only Offset 111:0]: FD0 Cold reset clears this register, warm reset has no ettect.
State Reduction (Oft-line, ~l~t nQ?dy):

29 Deoe- ~ Il~r 1995 , 127 ~ /~j Polo Sof~ware ~oy.a",l,l ,g Guide Stratus ~,npany Confidential The contents of the state reduction are frozen when the SAM tirst goes off-line or broken so that the w"~,~,s that caused the state reduction are flagged. This is so that co,,~a~alur~ that l~ ~;S~ further down the line (after the error has rippled through the logic) do not obscure the cause of the original failure.
Writing to clear any of the state reduction bits will have no effect. The state reduction bits are re-armed via the Rearm Enor register bit in the Test Control register.
31:17 Reserved.
~6 SAM Maint~nallce Interrupt issued due to PCI ErrorlAbont condition.
Refer to the PCI Error register and the PCI Configuraffon Space Status regls~,s for details.
15:14 Reserved ~3 Software SetOff-line Co"""~nd.
This bit is set by writing 00000081x to the page 0 Bus Interface State register.~2 Map Error (HSC2 Only) This bit is set when an l/O Address Map Error is detected. Refer to the CPU Specific Xbus Register Desc, i~tions on page 112 for further details about the error.
~1 Failed Op (HSC2 Only) When set this bit indicates that the return for a ~c,-sa.;tio" has timed-out and zeros were retumed to prevent the device from hanging. Status~10:9] indicate which slot (Saml was fesponsible for issueing the read request. Note that this Sam's state will have ~;I ,anged from online to online - not ready.
10-0 P~eseved 12.6.3.7 Arbitration Freeze Count Max. Value Type: ne~.lM.'I,1e Otfset ~11:0]: FB0 Cold reset clears this reyist~r~ warm reset has no effect on.
After oertain types of PCI enors. arbitration logic is ~frozen" and a PCl_reset is issued to the affected PCI slot. Writing an 8 bit value to this register determines how many 24Mhz clocks the arbitration logic will remain ~frozen" after PCl_reset is de-assei led.
12.6.3.8 Host Request FIFO Timeout Value Register Type: ReadlWrlte Ottset ~11:0]: ~A8 Cold reset sets this register tO 32'hO100_0000; warm reset has no eftect.
This register contains Tlmeout value for host request FIFO. This register is i,-i~ali~ed to 32'hO100_00 on power on. After a host request for PCI bus access is sent if the request is not completed by number of 12MHz clock ticks ~pecif;ed in this register the PCIB or HSC2 is broken and host request FIFO timeout enor bit in the PCI error register is set. Sefflng this re~ister to 32'hO000_0000 disables the host request FIFO timeout error.

29 Dec~" lber 1995 128 ~- g ~J

Polo Soflware, .ogranlming Guide ~,~ratus Company Confidenbal 12.6.4 SAMI~Ito~hteRegisters-Page0 There is a copy of each of these registers for each slot that a given ASIC controls. Thus the Gambit and Mirage have four copies of each register.
Table 31. SAM Compatible l/O Space Map - Page O
Offset [11;0~ Register Name Type FB8 BoardReset Read/SeVClear FB0 LED Control Read/SeVC~ear FA8 not implemented FA0 SAM Host Interrupt Bit Register Read/Write F98 SAM interrupt Mask Register Read/Write F90 SAM Interrupt Souroe Register Read Only F88 SAM Host Interrupt Address Pointer nead/\~'lit~
F80 SAM Host Interrupt Table Pointer#2 Read~Write F78 SAM Host Internupt Table Pointer#1 neadN~Iite F70 SAM Configuration Read/Write F68 PCI IO Space Offset Read/Write F60 PCI Memory Offset Read~Nrite F68 Bus Interfaoe State Read/SeVClear 12.6.4.1 Board Reset Type: n~ eVClear Otfset ~ ~l: FB8 Cold and wsrm reset affect this regi~ler as documented below This register behaves almost identically to the Xbus Board Reset register (or Golfbus Board Reset register) listed in section 12.3.3. except that PCI reset is also activated through this register. PCI
reset is asse~ldd by an explicit PCI reset access to this register, or by a power-up reset. A software initiated cold or warrn reset also asserts PCI reset. NOTE: A PCIB ha,~lu- .~ or software cold reset (as indicated in the Xbus Board Reset register) has no effect on this register.
31:7 These read bits/write bits are reserved for additional board specific resets.
6 Clear SAM entry cache valid bit (HSC2 only) Wfiting to set this bit gene-d~ a pulse which will invalidate the entry cache in the reader and flush any read data in the read RAM. This will insure that the readsr will go to the Map RAM for the next PCI access and allows s.~fi~rafe the ability to guaran-tee eonsi;.b- ,l data when using the pre-read option.
Note that the wite buffer caches are unaffected by this ope,ation. Write buffer caches can be invalidated through the Test Control Register (page 1).
Writing to clear this bit has no effect. Reading this bit will always retum a 0 Software Note: Clearing the entry cache during a PCI initiated read bansa~,tion can cause unpre-- dictable results.
Software iniffated PCI Reset These read bits/write bits are reserved for additional board specific resets.hen read-29 Dece--lLer 1995 129 Polo Software ~luglallllll ~9 Guide Stratu~ ~,ompany Confidenffal ing this register, this bit =1 indicates that the board hss received a software generat-ed PCI reset since this bit was last cleared. Writing a 000û0085x to this register will freeze PCI allJit~tion logic, ~e,-er~te a PCI reset pulse on the board and set this bit.
Writing 000û0005~ to this register clears this bit. The acbvation of either of the cold reset bits also dears this bit. For SAM, PCI Reset clears the internal PCI macro, in-cluding the configuration space and state machines. It also causes the asse, lion of a PCI reset to the attached PCI adapter ~onwsre Note: On Mirage, PCI resets also cause the the Reader hardware to reset similar to a wamn_reset. Software inibated PCI resets during a PCI initiated read tlansa.:tion can cause unpredictable results.
4 Software initiated Warrn Reset When reading this register, this bit =1 indicates that the board has received a soft-ware gene-a~d Warrn Reset since this bit was last cleared. Wribng a 00000084X tothis register gene-d~ a Warrn Reset reset pulse on the board and sets this bit. Writ-ing O00û0001x to this register clears this bit. The acbYation of either of the cold reset bits also clears this bit. For SAM, Warrn Reset Clears all bus state machines, includ-ing PCI state machines, clears all non-pe,~ist~:n~ error condi~ons and causes a PCI
Reset.. All pending bus activibes are cleared. The effects of Warm Reset on individu-al .t:g;~ is spe~ ed in the heading of each register.
3:2 Reserved Software inibated Cold Reset When reading this register, this bit =1 indicates that the board has received a soft-ware ge"eral~d Cold Reset since this bit was last cleared. Writing a Oû000081X to this register gere.d~:s a Cold Reset reset pulse on the board and sets this bit. Writ-ing 00000ûO1x to this register clears this bit. The ac~;a~i~n ot the Power-up cold re-set bit also clears this bit. For SAM, a Software Cold Reset clears all error reporting ,ey;st~-~ in addition to clearing everything effected by a Warm Reset and PCI Reset.
The effects of Cold Reset on individual .~ rs is specified in the heading of each register.
0 Cold Reset due to Power up When reading this register, this bit = 1 indicates that the board has received a power-up ge,,e,c~l~d Cold Reset sinoe this bit was last cleared. This bit is cleared by writing a 00000000x to this register or by a software initiated Cold Reset. When this bit is set by a power-up reset, all other bits in this register are cleared. This bit is NOT cleared by Warm Reset.
12.6.4.2 L~ D C ontrol Type: nez~ e Offset [11:ûl: FBO
Cold and warm reset affect this r~;s~er as documented below P, u Jide(J there is a PCI card present in the specified slot, this register functions idel l~ally to the register des~iLed in section 12.3.3. with the addition that PCI reset will set the red LED and clear the yellow and green LEDs. Reter to section 12.6.4.1 for more irlfu,.,,d~iùn on PCI reset. If no PCI
card is present, all LEDs will remain off.
12.6.4.3 S A M H ost Interrupt Bit Type: Re~ lle Offset [11:0~: FAO
Cold reset clears this register. Warm reset has no effect.
Bits 4:0 of this register points to the bit to be written in the host interrupt register, EIR and the least 29 De~-,lL~r 1995 130 . . .

Polo Soflware ~..)gran.ming Guide _~atus Company Confidential si~,~ilios-n byte is also the pattem to be written to the table in host memory.
31:6 Reserved.
5 1, this is the Usef~ bit for the read/~U~,bar register 4:0 Point to the bit location to be set in the host interrupt register.

12.6.4.4 SAM Interrupt Mask Type: nc~ e Offset 111:01: F98 Cold reset clears this ~e~;~ter. Warm reset has no effect.
These bits are used to individually mask or enable ths four intern~pt source si~nals, INTA# - INTD#
All four interrupt lines are tied together in hardware and must be masked/unmasked as a group.
31:1 ne5~ d.
0 interrupt mask - (1=enabled, 0=masked~
12.6.4.5 SAMlnterruptSource Type: Read Only Offset l11:0l: F90 Cold reset clears this register. Warm reset has no enect.
This bit is used to indicate if any of the four interrupt source signals, INTA# - INTD#, are active with interrupts pending. Most single function PCI devices only use INTA#. All four interrupt lines are tied together in hardware and can only be checked as a group.
31:1 I te~r~ed.
O interrupt status - (1=active, O.inactive) 12.6.4.6 SAM Host lnterrupt Address Point~r ~pe: ReaclN~'Iite Offset ~ 0~: F88 Cold reset clears thls register. Warrn reset has no enect.
This register contains the IO address of the EIR register in one or all cougars.31:2 Bits 31:2 point to the location of the interrupt register in host memory.
1:0 Reserved.

12.6.4.7 SAM Host Interrupt Table Pointer #2 Type: ne~ ne Offset 111:0]: F80 Cold reset clears this register. Wsrm reset has no effect.
These bits are the upper 16 bits ot the address of ~e interrupt table entry in host memory.
31:16 n2se~,d.
15:0 Bits 47:32 ot the table base address pointer.
12.6.4.8 SAM Host Interrupt Table Pointer #1 Type: Read/Wrne Onset 111:0]: F78 Cold reset clears this feg~ter. Warrn reset has no enect.

29 De~,-lL~~ 1995 131 ~~

_, . _ ,, CA 022~7~11 1998-12-03 Polo So~ware rr~g.a..,.,)ing Guide Stratus ~,ompany Confidential These bits are the lower 32 bits of the address of the interrupt table entry in host memory.
31:0 Bits 31:0 of me table base address pointer.

12.6.4.9 SAM Con~lguration Type: ReadlWrlte Offset ~11:01: F70 Cold reset clears this reglster, wann reset has no effect.
31-0 Reserved.

12.6.4.10 PCI IO Space Offs~t n~pe: ReadNVrlte Offset ~ 01: F68 Cold reset clears this rdgister, warm reset has no effect.
IO reads and wfites to this register cause no PCI cycles. IO reads and writes to the 1 6KB window known as the SAM PCI/IO space window use the information in the PCI IO Offset register to map the base address of the 1 6KB IO space window. Refer to section secbon 4.5.1 for further information on the use of this register. This register is big endian.
~1-14 Upper 18 bits of 32-bit PCI address for ~rS~esses to the 16KB PCI IO space window. PCl only defines use of the first 64KB of IO space, but the il~lellace supports IO space ac-cesses over the enbre 32-bit address.
1~0 Reserved ~2.6.4.t1 PCI Memory Offset Type: ReadJWrite Onset ~1t:0]: F60 Cold reset clears this register, warm reset has no effect.
IO reads and writes to this register cause no PCI cycles. Memory reads and writes to the 64KB
window known as the SAM Memory space window use the informaffon in the PCI Memory Offset register to map the base address of the 64KB memory window. Refer to secffon 4.5.1 for further i"for"lalion on the use of this register. This register is big-endian.
~1-16 Upper 16 bits of 32-bit PCI address for ~c~ses to the 64KB PCI mernory window. A
64KB window to memory can be put anywhere in the 32-bit PCI memory range. The PCI base address register in the configuration register space of the PCI adapter must be set to align with this window.
~~0 Reserved ~2.6.4.12 Buslnt~.f.,ceState Type: Rea~~0UClear ~ Offset ~11:0]: F58 Cold reset clears this le~isl~r, wann reset effect is documented below.
All blts remain cleared If there is not a PCI card in the specined slot.
These bits reflect the current state of the SAM board~
31 :3 Reserved 2 Off-Line- Not Ready ~oard capable of recei\l:ng and ~eapon~ g to Xbus/Golfbus requests, but is incapa-ble of performing Xbus/Golfbus initiated PCl requests. There is non-zero error state in the Gambit.

29 D~ 19Q5 ~ 132 CA 022575ll l998-l2-03 Polo SofhNare . rogr~ ,lming Guide itratus Company Confidenbal This status bit is used to indicate the occunence of a non Xbus/Golfbus related hard-ware fault which was d~ d by the cheching logic. This bit is set automatically by ha,~_.~. Wribng a 0000008~Y to set, or a 00000002x to clear will have no effect.This bit is cleared by a cold reset, or by writing the Test control register with a Rearrn Error Co--""dnd.
Off-Line- Ready Board capable of receiv;. ,9 and responding to Xbus/Golfbus requests but is incapable of perforrning Xbus initiated PCI re~ es~ No error con~ s are present.
This state is entered automabcally after a cold or warrn reset (provided there are no errors d~ h ~d by the Gambit or Mirage ) or when the error state of the chip is cleared via a Rearm Error Coll"l~i1d. Writin3 a 0000081x to set, causes SAM Sta-tusl13] to get set and a Attenbon Maintenance Interrupt to be gene,~d. Clearing this bit has no effect. (Note if the SAM is broken you won't be able to read this.) 0 On~ine Board capable of r~cei J;n9 and responding to Xbus/Golfbus re~ estc Board capable of perforrning Xbus initiated PCI re~l ~estc PCI adapter enabled to post requests.
Wribng a 00000080x to set, bit causes the PCI arbiter to be enabled provided there is a non-zero error state. Wribng a 00000000x to clear causes the GambiVMirage to go Off-line - Ready but a Attenbon Main~enance Interrupt is not gene,dt~,d. Note if the SAM is broken you won't be able to read this.) 29 Decei"ber 1995 133 rhis document contains c~l~fid~ntial and p~u~ie~,y inforrna~on of Stratus Computer nc. and any reprod~m;o~ dis~s~e or use in whole or in part is eA~,essly ~rohibi~d exoept as may be specifically authorized by prior written ag, ~" ,e, l~ or ~"";O:.ion of S~atus.

Xbus Functional Specification "If You Want Fault Tolerance, Buy A Jetta"

Wlll Leavitt (editor) Conrad Clemson Jeffrey Somers John Cl~d~es David EarLera 21 July 1995 Revision: 1.3 ( c) S tratus Computer, Inc .

Appendix II

~ ~f?

CA 022~7~11 1998-12-03 Xbus Functional Specification Stratus vompany ConfiJen~al Table of Contents 1. Introduction.......... ................................................................ 8 1.1 Applicable Documents .. 8 1.1.1 Stratus Documents 8 1.1.2 Other Documen~ . 8 12 Revision History , g 2. Functional Overview........... ~..................... 10 2.1 Intrs~u( Dn. 10 2~ Architectural Altematives 1~
22.1 New Bus - 11 2 ~.2 PCI Based Bus 11 22.3 Ibus based Bus 12 22.4 Golfbus Based Bus. 12 2.3 Gol~ous Logical Overview 12 2.4 Golfbus Physical Summary . ...... . 13 2.5 Polo XBus Logical Overview. .. 14 2.6 Polo XBus Physical Overview. . . . ~ --14 3. Detailed Functional Desc,iption.... ~................ ~5 3.1 Bus Naming Convention . . . 15 3.1.1 Data Bus Naming Convention 15 3.1.2 Control Bus Naming Convention 16 3.2 Bit Nu~ g - . 17 3.3 Terrninology 17 4. Xbus Signal Description ............................ 21 4.1 Signal Description- ~ ~1 4.2 TheInfo Bus ~ t 4.3 TheControl Bus " ~
4.4 The Voted Sign~s .. . - . - ~5 4.5 O~er Control Signals . ~7 5. Xbus IIO Interface Re~;sle,~ ....................... . 28 6. Xbus Protocol ................. .. .............. . . .............................. 29 6.1 Overview 62 Bus O~)eld~iOl~
6.3 Bus P~u~es - 30 6.4 Bus Errors ~ 30 6.4.1 Bus Error Broken Conditions - 32 6.4.2 Xbus Fault Analysis 33 6.4.3 Fault Conditions 34 6.4.3.1 CPU Board Faulty Input Circuit- CPU Drivin~ 34 6.4.3~ CPU Board Faulty Input Circuit- I/O Board Driving 34 6.4.3.3 CPU Board Different Data GSide and D-side 34 6.4.3.4 CPU Board Faulty Output Circuit - Buffer to Pad Fa~'t . 34 6.4.3.5 CPU Board Open - CPU Board Driving 35 6.4.3.6 CPU Board Open - I/O Board Driving- 35 6.4.3.7 CPU Board Short 35 21 July 1995 ~ 9 3 2 Xbus Functiona- _,~ci~ tion ~ atus Company Confidential 6.4.3.8 n~Ck~ -neOpen Etch .. 36 6.4.3.9 I t~ - h~ ne Short 36 6.4.3.10 I/O Board Faulty Input Circuit 36 6.4.3.11 I/O Board Output Circuit Fault - Buffer to Pad 36 6.4.3.12 I/O Sin~'~ s~de Access - AS~C Parity Gen. Fault. 36 6.4.3.13 I/O Single side Access - ASIC PCI Data Path Fault.. 37 6.4.3.14 I/O Non-singl~side Access, Dmerent C-D Data 37 6.4.3.15 I/O Board Open 37 6.4.3.16 IIO Board Short 37 6.4.3.17 Transient Fault 38 6.4.3.18 Slow Driver Fault 38 6.4.3.19 Board Not Broken Generation 38 6.4.320 ArbitraryBreaWng 38 6.5 Summary BusStateDiagram 39 6.6 Arbitration 40 6.6.1 Bus Mastership Upon Bus Grant 43 6.6.2 Special Arbitration Rules for Non-Paired Read Retums 43 6.7 Block Transfers- ... 45 6.7.1 Bus Errors . - 45 6.72 Global Writes . 48 6.7.3 Board Breaking, . 48 6.7.4 Bus Busys- ............ 49 6.8 Peer to Peer ~lars~ . . 49 6.8.1 Bus Errors ......... 51 6.9 Major Tlai~sa~ - 53 6.9.1 Write Tlansactiùns - 536.9.2 Read Transactiorl~ ,, ,, 54 6.9.~ Flush and Purge T,a"sa~lic ~s . 56 6.9.4 LoadlClearTra"s, tior-s 58 6.9.5 Ping Transaction - 59 6.9.5.1 The Need for PlNGs 59 6.9.52 DetailedRules During PlNGOper~tions. . - 59 6.9.6 busy, ack and error combinations 61 6.10 Xbus Signal Formats 62 6.10.1 trid[6:0~ and func_op_ Bus Format .. 62 6.10.2 Info Bus Format - 63 6.10.2.1 Function Code (rc and byte enable) Transfer 65 6.10.22 Remote/Coherent bits (rc[1:0l) - 69 6.102.3 Byte Enables (byte_en[3:01). 69 6.11 Board Sy,n;h,uni~d~ion/Board States 70 6.11.1 CPU/Memory Bus Interface Modes - 71 6.11.2 Simplexed l/O Board Bus Interface ~ 73 6.11.3 ~ oopl~h mode 75 6.12 Board Breahi-lg/Boal.l Removal 75 6.12.1 Un-l,-aal.' ,9 a Boar~ - 75 7. Xbus Routing and Xbus Interface Clocking -.................. 76 7.1 On-board Clock Generation 76 7.1.1 Clock Phases 76 7.2 Signal Routing, , 77 7.3 Clock Fault Tc'e anc~ Issues. Outbound Signals. . 78 8. Board Insertion and Removal.................... .......... 79 21 July 1995 Xbus Functional SpecificationStratu~ ~,ompany Confidenbal 8.1 Hot Plugging , ~ , 79 8.2 Xbus In~llace Testing at Board Insertion~ - , , ....... ,.. 79 9. Xbus Fault Tolerance.............................................. 80 9.1 Info Bus Protecbon ~~
9.1.1 Parity . . 80 9.1.2 1 ~~op~r k cl~ecl~ing . 82 9.1.2.1 I nopt~k on CD different ~sses- ,,83 9.1.22 1 nop~-k on Singl~side accq~ses 83 9.1.3 Loop_ck_ops 83 92 Control BusProtection . - 84 9.3 TIII~E . ~ Voted Signal Protection ,~ 84 9.4 Enor Reporffng 84 9.4.1 Producing Errors ~5 9.4.2 Main~enanoe Interrupts 87 9.4.2.1 Broken E~lents 87 9.4.22 Error Events . 87 9.4.2.3 Soth~ are Control - 88 9.4.2.4 Deterrnining the Source of a Maint~"ance Internupt.......... 88 9.5 BoardBreakingrlming. . 88 9.5.1 Board breaking and inforrnation latching, . - -,88 9.52 Board Breaking Timing on Info I ~or~ k Error 90 9.5.3 Board Breaking Timing on Heurisffc or Arbitrary Broken, 90 10. Reset. .~. - ................ ~ .... ~ . .. . ~ . .. .~. .. ... 91 10.1 Xbus Resets . 91 10.2 Power System Gene,-lldd Reset, 92 10.3 Fault Tolerant issues: 95 10,4 ASIC Pins Required For Reset Functions. - 95 10,5 DUMB FET's .-. 95 11. Xbus ~hysical Partitionin~ .. ~. ............ ~ .~..... ........... ~ .. 96 11.1 Info Bus Partitionin~ 96 11,2 Control Signal Partitioning 96 11,3 Xbus \/oted Signal Partffioning .- 96 11.3.1 ID PROM Par~tioning 98 12. Xbus InteTface Block Diag, a,ns .. - .= .. .. .. ..... ... ....... 100 12.1 Xbus Top Le~/el Block Diagram, 100 12,1,1 Port I iSt ~ 100 12,1,2 Functional D~sc~i~lion -. 100 12.2 Inbound Pipe Section - , , 101 12.2,1 Port l ist 12~,2 Funcbonal Description 108 12.3 PING Section 110 - 12,3,1 Port i L5t 111 12,3,2 Funcbonal Desc,i~tion 113 12.4 Xbus Outbound Data and ~ontrol Unit 114 12.4.1 Port I ist . , 115 12.4.2 Funcffonal Desc,i~,tion 118 12.5 Arbitration Logic. . . 126 12,5.1 Portl ist, , - 127 12,5.2 Functional Desc,i~bon 127 21 July 1g95 4 Xbus Functiona. Jpeci~ ~ffon. .,atus Company Con~ide"~al 12.6 lO Register Section 130 12.6.1 Port Lis~ . - - .. 131 12.6.2 Functional Description 135 12.7 Error Checking and Reu;sl~,i,,g Logic , . . 136 12 7 1 Port I is~ 137 12 7;2 Functional Des~ ,tion - 149 13. Xbus/Golfbus Ditrerel~ces................ . . ....... 151 21 ~luly 1995 5 PCT~US97/09781 Xbus Functional Specificaffon Stratus ~,ompany Co, If ida"~ia List of Figures hgure 1. Xbus inte~connect 11 Figure 2. Xbus data Ruses . . 15 Figure 3. Xbus Control Buses . 16 Figure 4. Xbus Bit Nu"ll~,ing Convention . 17 Figure ~. Terrninology - Simple Word Read T,ansa,.tion 18 Figurs 6. Terrninology - Simple Line Write Tlallsia~Lrn 19 Figure 7. Basic Xbus Cycle ~9 Figure 8. Basic Bus Busy Operabon 30 Figure 9. Basic Bus Error O,oeration- 31 Flgure 10. Bus Enor Operation Flow Chart 32 Figure 11. Xbus Intel~onne"1 .............................. 33 Figure 12. BASIC Bus O~ue,_ ' ~ Flow Char~ . . 39 Figure 13. Inter-CPU Bus Arbitrabon Network . 40 Figure 14. Arbitrabon After Being Burio~/ . . . 43 Figure 15. High Level Block Transfer 45 Figure 16. Error During First Operabon of a Quad Block Transfer 46 Figure 17. Error During Second Operation of a Ouad Block Transfer 46 Figure 18. Error During Third Operabon of a Quad Block Transter 47 Figure 19. Enor During Fourth Ope,ation of a Quad Block Transfer - 47 Figure 20. Enor causing entire block transfer to be canceled 48 Figure 21. Peer to Peer Cycle Initiated by a Simplexed CPU - 51 Figure 22. Peer to Peer Non-paired Read from a Duplexed CPU 51 Figure 23. Store and Forward with Error on 2nd Ope,ation of Quad Block Transfer...... 52 Figure 24. 32 Bit Write T,ans.,ctior, ~ 54 Figure 25. 256 Bit Block Write Transaction . ............ 54 Figure 26. 32 Bit Read T.ai1sa~:~n 55 Figure 27. Slave Resoonse to a 32 Bit Read 55 Figure 28. 256 Bit Read Transaction 56 Figure 29. Slave Response to a 256 Bit Read T,ansa~ tion 56 Figure 30. Flush with Modified Data T,ansdc~ol ,. 57 Figure 31. Slave Response to a Flush with Modified Data Transaction~7 Figure 32. Flush (no modifbd data) or Purge Transaction 58 Figure 33. Slave Res,~onse to â Flush (no IllGdifi~d data~ or Purge Transaction 58 Figure 34. PING T,snsa"tion 60 Figure 35. PING Transactions Boundary Condition 1. .. 60 Figure 36. PING Transactions Boundary Condition 2. 61 Figure 37. tridl6:0] and func_op_ Forrnat for the ~st Cycle of an Info Phase . 62 Figure 38. trid[6:4] Forrnat for a PCIB for the 1 st Cycle of an Info Pha~ie 63 Figure 39. trid[6:0] and func_op_ Forrnat for the 2nd cycle of an Info phase 63 Figure 40. Infol31 :0l Forrnat of CPU Address FUNC_OP . 64 Figure 41. Info~31 :0] Forrnat of IOVA Address Bus FUNC_OP - 64 Figure 42. Info[31 :01 Forrnat During Data Transfers . ..... 65 Figure 43. byte_enl3:0] d~fi-l ti~n - 70 Figure44. CPU/Memory Boardstatetransitions. 73 21 July 1995 ,~ 6 .

CA 022~7~11 1998-12-03 Xbus Function~ pec ~ on .ratus Company Cc"l~iden~i~

Figure 45. Simplex l/O Board State T~dll ~ns . . 75 Figure 46. Routing of info lines 77 Figure 47. Routing of 3way voted lines 78 Figure 48. Self Checking Parity Logic. ....... 81 Figure 49. 1 oopt~rk Co"ne.~l/ity. -82 Figure 50. Board ~ .ahing rlming and latching (arrived at phase_12_1 ~ 89 Figure 51. Board Breaking Timing and latching. (arrived at phase_12_2) .89 Figure 52. Board Breaking Timing on Info I ~Qpha~ k Error 90 Figure 53. Board Breaking Timing on Heuristic or Arbitrary Broken go Figure 54. Reset Signal Tlming. 91 Figure 55. Power Reset State Machine 93 Figure 56. Board resets. 94 Figure 57. Info Bus Paltitiull lg 96 Figure 58. board_not_broken_ routing 97 Figure 59. reset_ roubng 97 Figure 60. sync_ roubng . 98 Figure 61. online_ routing 98 Figure 62. ID PROM Impementabon 99 Figure 63. Inbound Pipe Block Diagram 102 Figure 64. Post-Qual and Pre~ual Bus Timing............... 109 Figure 65. data_valid and last_info Tlming . 109 Figure 66. PING Control Block Diagram 110 Figure 67. Outbound Data and Control Unit . 114 Figure 68. Outbound Timing Diagram 120 Figure 69. Ou1bound Master Control . 121 Figure 70. Outbound Slave Control 122 Figure 71. Outbound Peer-t~Peer Timing .. 123 Figure 72. Echo Peer-to-Peer rlming 124 Figure 73. Outbound Peer-t~Peer Con~rol................... 125 Figure 74. Arbitration Block Diagram . 126 Figure 75. Arbitrabon State Machine 128 Figure 76. Arbitrabon Timing Diagram 129 Figure 77. IO Register Bloc~ Diagram . 130 Figure78. RegisterWriteTiming 135 Figure 79. Register Read Timing 135 Figure 80. Error Checking Block Diagram 136 Figure 81. Bus Error O~dtion Flow Chart ................ 150 21 July 1995 7 I ~g WO 97/46941 ~CT/US97109781 Xbus Functional Specification Stratus (,~,mpany Confidenbal lro.l~ction This dowrnent provides a detailed ~s." i~,tion of the Polo Xbus. Contained in this document are the signal descli,~;ons, tunctional protocol desc,i~)tions, functional timing ~liag,~",s, and physical bus cl~sc,i~tions, including pinouts and clocking Il l~:th~ ~ ~ h~ JJ~. The Xbus protocol is ~irectly based on the Stratus Gbus, and this specification is itself derived from the Gbus Functional Specific~ffon.
1.1 Applicable Documents 1.1.1 Stratus Documents Cougar Specification Rev 12, 919192, or later /auto/je~aldoclCPU/cougar/...
Polo rloglalll~lel~ Guide Rev 3.0 or later /auto/polo/doclfunc_spec/...
Gofer Specific~bon Rev 2.0 or later /autoljetWdoclCPU/goferl. . .
GolfBus Specificaffon Rev 3.1 or later lauto/jettaldoc/gbus Mercury Functional Speci~ir*tion /autoljeU~cJ~i~ u/boardl...
Polo Cyclops Specifir~bon Rev 1.0 or later /auto/polo/doc/CPU/cydops/. ..
Polo Gacl~lane Specffl~tion Rev 1.0 or later lauto/polo/lJ~,clL.ac~' -ne/...
Polo Clocking Speci~tio Rev 1.0 or later /autolpololdoc/clGcl~l.. .
1.t.2 OtherDocu.-,e.ns IEEE Standard 1149.1: IEEE Standard Test Access Port and Boundary Scan Architecture February 15, 1990 Test Te~l~n~1Q~ Technical Committee of the IEEE Computer Society The Institute of Electrical and Ele-illo" ~ En~ineers, Inc.
345 East 47th Street, NY, NY 10017-2394 21 ~luly 1995 ~ 3 8 W O97/46941 PCT~US97/09781 Xbus Func~onal ~,,,ecih~,ation ~ ~tus Company Confidenbal 1.2 Revislon History Ite.;~'~r 1.0:
~ Changes to almost every section in order to complete document.

21 July 1995 ~ 9 ~,, , , .

WO g7/46941 PCTrUS97/09781 Xbus Functional Specification Stratus ~,. mpany Confidenbal .

2. Functional Overview 2.1 In~oduction The Xbus pel~ulllla a number of functions:
~ It supplies a means to transfer info,..._~ ~n between Polo CPU buards and PCI Bridgs cards.
~ It supplies a means to transfer information between two Polo CPU boards.
It provides fault d~t ,ction and isolation for the bus and the boards on the bus. I lo/~cvcr, it is only fault detectins, not fault tolerant.
This Xbus must have a number of functional characteristics. First, it must provide enough throughput so that it is not the system bottle neck. Second, it must be ine,-~nsive to implement.
Third, it must be able to support both PCI a~ssses 32 bits of address and multiple words of data, and PA-RISC ~c~*s,ses 48 bits of sddress and multiple words of data. It must also be able to support Cougar/PA-RlSC functionality such as regurgitated infos, tlushes, and purges.
There are a number of possible i"le,co"-,e ;~ struc,tures that could be used for this bus. The two that were carefully investigated are a fully ints~cû~ eL.ted bus, such as in Jetta or StrataBus based systems, and a series of 4 point to point buses as shown in figure 1. In order to delt:l ,n;"e which of these inte,~nnecba would best meet the Polo imple" ,e, lt - n requirements, several key areas were examined.
An in,~nal)t part of any Stratus design is live insertion. In order to accomplish live insertion on a fully connected bus, it is necessary to insert a board into the system without perturbing any activity.
This is accomplished through the use of special bus t~ansoe.Jers. The bus ~dnscei~ers are expensive, power hungry parts. Further, these b ~ ca;Jers have some strict requirements ~c~oc~ d with them. ~irst, they require a bias voltage of 4 to 5 volts. This voltage must be applied before the signals make connection with the ba~ ,lane. This places constraints on the power architech re. Con~ide-dtion must be given to where the bias voltage is gsne.d~d, and how it is delivered? Second, the BTL l,d,.sco,Jers require a terminabon voltage of 2.1 volts. This voltage must be very tightly CGI~t~ 'kd It could be p~obably be generdlt:d by the suitcase s~ ;B5 but that would i".,~ase the cost of each supply by over $400. Fven though the slJit~ses can provide a source for the power for the termination voltage, in order to regulate it to the ffght constraints of the ~TL bal1s~i~ers, it would~e neoessary to add bac~p~s.
In a poj~on, buses are not used whe~ both units~fe not in the system~ not broken~
anp~OO% funcb~ .fThis eliminates the need for liJ~ ~as we know it today. ~y removing thi~ require~nerl~, the need tor the BTL b ~nsce; ~ers is eliminated, thereby removing the need for bias~e and 2.1 volt gene _tic n.
Also, with the point to point solution, terrnination bec~" ,es much simpler. Instead of parallel terminaffon series terminaUon is used. This change in terminaffon ren.o ~es the need for gene-~ti,1g a termination voltage and the need for b~-.h~,anel power supplies.
There are some secor,dary con:.;Jelations such as bus isolation, and complexity of imple"~u~tion, but the live insei tion issue is the real driver for the ~ch-- ~ choice.

21 ~uly 1995 ~ 10 ~ I

_ Xbus Functional .,pe ,caticn _.. atus Company Confidenbal Figure 1. Xbus i,~t~rL;~nnect CPU Board O ~ ~ CPU Board 1 PCI 8ridgeA ~ PCI Bridge B

KEY
~_~ Pidirecbonal, point-to-point data bus.
Unidirectional, point-to-point control bus.

2.2 Archltectural Alternatives There are 4 fur,dd" ,e"~al choices for implementabon of this bus. The bus could be Golfbus based, IBus based, PCI based, or totally new. These possibilities are explored in the next sections.
2.2.1 New Bus In order to justify inventing a new bus, there must be a good reason. If pe"~"",ance, functionality, or cost goals cannot be met by an existing structure, then it is necessary to invent one. This is not the rase for Polo. Therefore, this possibility is disca,ded.
2.2.2 PCI Based Bus A PCI based bus is one possible imple",e"~tio" altemative. It meets pel h)""ance and cost goals.
It could be modified to meet the addlessing and Cougar funcbonality. The PCI is ~ fobaLly not quite optimal from a pe, fo" "ance stand point due to the fact that it is not split transaction. This is ~early a negdb-c point for a PCI based solution. This design probably simplifies the Gambit ASIC at the 21 July 1~9~ 2 Xbus Functional Specific~ffon Stratus ~,ompany Conf;derltial expense of the Cyclops ASIC.
If a PCI based bus is used, the PCI protocol will have to be modified to support the virtual index of the PA chip. This is required to support CPU to CPU tldllalel~. The PCI protocol will also have to be modified to support the fault detecffng nature of the il ~ ,onneo~. Finally, some new funcbon codes will have to be added to the bus in order to allow it to Support the sync functiona~ity of ths CPU boards.
For these reasons, a PCI based solution is cl;~.~r~led.
2.2.3 Ibus based Bus The Ibus basr d imple",e"tdtion has quHe a few adval~tages over the previous two possibilibes.
The Ibus is already defined and has a pel ~---ance and r,ost point that meet the needs tor the Xbus. It already supports PA virtual indexes, and a Split bànsact;on protocol. It will require ",0. cf,~l;.,n to support PCI based non virtual index 1, an~ h.~, but there are spare bits in the virtual index transfer phase that can be used for this purpose. So inventing that cycle should not be dmicult.
There are some disad~,dnlages to an Ibus based solution. First, an error protocol will have to be invented that can support the fault deba~,ti"g needs of this bus. This will require ,~ vrl~i, ~9 some of the central state machines of the Ibus. Second, although the Ibus has some s;,.,ila,i~ies to the Golfbus, there are enough di~ferences that it is unlikely that the RTL recycling inside of Gofer will be large. The tie between the Ibus and the Golfbus inside of Gofer is tight, tl ,erefore, removing the Golfbus may require a sul~ tial ~e.~-JIl~ing of the Gofer internal int~,. Iaces.
For these reasons, although the Ibus based solubon is a ~easondble one, it is not consi~ered to be as good as the next possibility.
2.2.4 Golfbus Based Bus Although at first glance a Golfbus based solution for the Xbus may seem like overkill, it has a number of sig~ ant advântages eYen over the Ibus based solubon. The Goltbus l~Z..I~i"s all of the Ibus ar~antages. However, the Golfbus also has the advantage that it already includes a robust enor handling protocol. It has an interface that is well integrated into the existing Gofer RTL
logic. Hopefully, a Goltbus based solution will allow for a rnaxirnal sharing of RTL code between Gofer and Cyclops. Addibonally, the Golfblus logic has already been used in a co"""on logic fashion. This may allow for good sharing of logic b - ~eh the Cyclops ASIC and the Gambit ASIC.
- ~fortunateiv ~e Golfbus requires a large number of external l, ansce;;ers and parallel ba~ plane termination, which do not meet the cost ~b~ ~tiJCS of the polo project. For these reasons, the Polo Xbus will use a Golfbus-based protocol running on a newly desig"ed, point to point inter~nne~,l.
2.3 Golfbus Logical Overvlew The Golfbus is a single logical split bdnsa~ol) multiplexed addressldata bus. It is duplicated to provide fault tolerance in a manner similar to the S~atabus. Major features of the bus include the ability to support 32 and 64 bit bus illtel(ace widths, co"" '~ 'y s~",ch,ono~s operation, cache consislen.i~l support, and a single logical ASIC intb"ace.
In its initial imple",e"t~liol" the bus support~
~ 2 logical (4 physical) CPU/memory boards, with a 64 bit bus.
21 July 199~ '2 c 12 .

CA 022575ll l998-l2-03 Xbus Functional _,ec~ on ~ .us Company Collliden~i. ' ~ 10 physical IO boards, with a 1'32 bit" bus.
Peak supported bandwidths are 1~8~'~ytc~Jsec tatv.~,en CPU/Mem boards (assuming all line llansf~la) and 7~ ytes/sec tolfrom IO boards. All iufo~"~atioll transfer auoss the GoHbus occurs s~",..l"~,~ously to/from the bus I,ansceivers at -24MHz.
A single logical bus is used for both address and data transfer. The bus contains up to 64 bits of information (data or address and function code), 7 bits of tag, and a single bit which indicates whether the informabon lines are carrying data or address and funcbon code. The IO boards in the inibal implementation will only connect to 32 bits of the inforrnation bus. (Special funcbon codes will be used to interface to and from the IO boards.) The i,,fu,,,ldtion bus, tag, and func_op bit are covered by 8 or 16 bits of parity forl~32 bir~ interfaces and~64 bir i~ "aces ,t:spectiJdy.
Control signals and arbitration lines are protected by a three-way voting algorithm. This provides the ability to tolerate a single control signal failure within each three-way voted control line.
The GoHbus interface logic provides full cl,echi-lg tet ~_en the ~side and D-side of each Golfbus board via ll~opl~ok, thereby providing board level fault deter,~ion. The Golfbus interface is able to isolate itseH from the Golfbus, even in the event of clock failures, thereby providing board level fault isolabon.
e of the spit nature of the bus, all boards in the system must be able to arbitrate for the bus and initiate a bus opelttion.
2.4 Gol~ous Physical Summaly The Golfbus has an e,.l-t:"~ly efficient physical implell,t:ntdtiol). Support of its il~fo~ dffùn transfer and fault detectionrlsolaffon capabilibes require (in addition to clock generation circuitry) only:
~ Conne..to,~
~ T,~.n"ce;vr~rs ~ 1 logical (2 physical) ASlCs ~ 1 logical (2 physical) MSI 2~pin register coll,,uonellt Support of board ID PROMs and board indicator circuitry (card edge LEDs) requires some addibonal MSI circui~y.
The 32-bit version of the bus interface requires fewer than 160 b~ lane signal oonne~ ~na and the 64 bit version of the bus interface requires fewer than 250 ba~,l~lane signal c<"~ne ~ )s. 64-bit and 32-bit interfaces coexist in the same bach~Jlai ,e, though b~ -'.p' ~ne slots are wired to support one or the other. The AMP SL-1 û0 con,-e~,t~, is used for the Golfbus bacl~Jldne cooneL,th~ns.
The l,dns~;Jer used on the Golfbus is the FB2033, a Stratus s~.;fiad bicmos device provided by Signeffcs Corp. and Texas Instruments Inc. This is a Futurebus compaffble device with cor,h." ~ d edge rates, capable of delivering 100mA on the bus side. These octal devices are implemented as a three port device, i.e. each bit has a board side input port, a board side output port, and a bus side bidi,. bunal port. The bdnsceiver is used in a clocked mode for both inbound and outbound data flow. A testabi!ity feature, which allows intemal loopb~r~ cl,eck,ng, i.e. a path that includes the enbre part exoept the bus side output driver and input buffer, is also utilized to fadlitate tesbng.
The l"tnsceiver supports live insertion and removal.

21 Ju~y 1995 13 ~1 Xbus Functional Specification Stratus ~,~mpany Confidential U64-bit" Golfbus interfaoes can be supported via 2 391-pin ASlCs and "32-biY' Golfbus int~ laces can be supported via 2 304-pin ASlCs, though for the first implemen~d~on the former will be used for both.
The Golfbus interface requires three docks to transfer i"'~ ",dlion across the bach~lane, a -24MHz transmit cbck, a -24MHz receive clock, and a ~1 2MHz phase inforrnation ctock quaiifier.
In addition, first gel~e-dtion Xbus interfaces will require 2 -24MHz clocks (really OE signals) ~or l,.ln~h,.i"g i"~ ,~.ation from the bus bansceivers to the bus ASlCs.
2.5 Polo XBus Logical Oven~iew Polo implements a subset of the of the Golfbus protocol. The f~ .ng discusc~on outlines key Golfbus features not supported in the Polo imple~e~ltdtiol1.
Bus widths are limited to 32 bits. Since there are no CPU boards with remote memory on the Polo Xbus, there is no need to use the optional 64 bit widths of the Golfbus.
The configurabons are limited to 4 devices. Two CPU boards and two PCI bridge cards.
Although the physical imple,--entut;on uses 4 point to point buses instead of 2 fully connecled buses, the protocol is still implemsnted with a single bus view. On any given cycle, there is no more than one t~ansdction on the i"te,connecl. This l,a,)saction may be on all 4 buses for the case of a duplexed opelation or on only 2 of the buses for a simplexed operation. It is the les,uons;bility of the arbitration network to ensure this functionality.
Like the Jetta implementation, the signals are divided into two calegories, b dnsdction based bused signals and control signals. ~ ID~ cr, due to the point to point nature of the signals, the implen,e,l~ion is very different. The bused signals are implemented as 4 bidirectional pointto point buses. These are shown as the heavy lines in figure 1. The control signals are i",~le."er,led as sepala~ unidirectional buses. These are shown as the lighter lines in figure 1. The bused signals are ~ tected through the map ram, parity, and looph~- 4 checking. The control signals pro~ected through ECC and lonpl~. k chechi.lg.
One fundanlenl~lly new feature of the Xbus is the error pro~c c ~ l Unlike the Golfbus, the Xbus dlt~ ts not only to detect bus enors, but also to diagnose the source. This aspect of the Xbus is covered in a later secUon in detail.
A second fu~l~l-~ll~lly different feature is the ,-,ecllan;_.-- by which two CPUs communicate. A
CPU to CPU ~ans~cLon is aocomplished by means of a peer-to-peer ope-dtion in which the bd,.sa thnisbrokenintotwosepardle~ansa~tions.lnthefirst~thecputransmitsthelldllsac'jcn from the CPU to the PCIB. In the second, the PCIB transmits the information from the PCIB to the CPU. This is explained in detail in the section on peer-to-peer l.~nsa~,tions.
2.6 Polo XBus Physical Overview The Polo XBus physical imple,n~,,ta~iun rnakes use of a direct ASIC to ASIC connecffon tech,.~ . The bus interface is accomplished through the use of 2 391 pin PGA package ASlCs.
The Xbus uses two different styles of connecto,s. The CPU boards (s~it~ees) use the SL-100 oonne~.tol from AMP. This connector was chosen for its ruggedl ,ess, built in stiffeners, and grounding scheme. The PClBs use the multiple sourced futurebus connect~r. These connecto,~, while less rigid, provide good signal integrity, and are i"c~,~ens;~e. A single 24Mh~ clock is used for clocking data and control onto and off of the bus.

21 July 1995 ~ 14 W O 97/46941 PCT~US97/09781 Xbus Functiona. ,p~ ~ti~n .atus Company Confidenhal 3. Detailed Functional Description 3.1 Bus Naming C~ ..lion llle Xbus actually consists of 4 data buses and 12 control buses. With so many buses in the system, it is .,,I~.La,)l to have a concise intuitive naming convention.
3.1.1 Data Bus Naming Convention Figure 2 below, shows a block diagram of the 4 data buses in the Xbus system.
Figure 2. Xbus data Buses Xbus Slot O Xbus Slot 1 Bus AO Bus BO \/ Bus B1 Bus A1 PCI Bridge O PCI Bridge 1 Xbus Slot 2 Xbus Slot 3 The number for the bus is taken from the CPU slot number; U lerdfui~ data buses connect~3d to CPU O end in O and data buses oonnecLd to CPU 1 end in 1. The letter for the bus is deterrnined by whether or not the bus is a c, i~sc, uss bus (i.e. co""e~,~ an even slot number to an odd slot number) or a straight bus ~i.e. conne.,h an even to an even or an odd to an odd slot). Based on this convention, CPU O has co- ,. ,e~,tions to data bus AO and BO. CPU 1 has c~n"e~.tions to data bus B1 and A1. PCI Bridge O has conne.Aions to data bus AO and B1. PCI Bridge 1 has co,-neutions to BO and A1. Note that unlike traditional Stratus boards, the PCI bridge cards do not run in lock-step.

21 July 1995 Xbus Functional Spec~fication Stratus ~,_.npany Confidential 3.1.2 Control Bus Naming ConYention Figure 3 below shows a ~lock diagram of the control buses in the Polo system.
Figure 3. Xbus Control Buses cPU o CPu 1 Xbus Slot 0 o, Xbus Slot 1 control_1_0 ~ ~l in_p ~ 'S~I O
control_0_1 ~ - ' /~

control_3 2 ~I ~'control_2_3 ~

PCI adaptor 0 PCI adaptor 1 8 Xbus Slot 2 Xbus Slot 3 The control naming convention for the bach~Jlane signals uses the signal narne tollowed by the source of the signal and then the destination of the signal. The nam~ng convention for the associated ASIC pins uses the signal name, the direr,tion (in or out), and the c~nl,~on (n for neighbor, o for op~c~it~!, and p for peer). Examples of this naming convention are shown in Figure 3. Table 1 lists all o~ the control bus names.
Table 1. Control Bus Names Control BusControlBusControl8us ASICDrivingASIC nec~ ;n9 Source D~tillation Name Pin Name Pin Name CPU 0 CPU 1 control_0_1 control_out_pcontrol_in_p CPU 0 PCIB 0 control_0_2 control_out_ncontrol~n_n CPU 0 PCI8 1 control_0_3 control_out_ocontrol_in_o CPU 1 CPU O control_1_0 control_out_pcontrol_ln_p 21 Jul;y 1g95 ~ 16 .... ,, _, . , .. _ ., CA 022575ll l998-l2-03 Xbus Functiona. _pecitlcationatus Company Confidenbal Table 1. Control Bus Names Controi BusControl ~3usControl ~usASIC DrivingASIC Reoeiving Source Destination Name Pin Name Pin Name CPU 1 PCIB 0 control_1_2controt_out_o control_in_o CPU 1 PCIB 1 control_1_3control_out_n control_in_n PCIB 0 CPU 0 control_2_0control_out_n control_in_n PCIB0 CPU 1 control_2_1control_out_o control_~n_o PCIB0 PCIB 1 control_2_3control_out_p control_in_p PCIB 1 CPU 0 control_3_0control_out_o control_in_o PCIB 1 CPU 1 control_3_1control_out_n cornrol_in_n PCIB 1 PCIB0 control_3_2control_out o control_in_p 3.2 BitNumbering The Xbus and HP-PA7100 use different bit nu,lIL~,ing r;onventions. Refer to Figure 4 for a cles~ tion of the bit nu"lLen"g scheme used on the Xbus. Refer to the Mercury Funcbonal Spec f~ on for a description of the HP-PA7100 bit null.be,ing scheme. Refer to the MlO/Polo PCI p,oylal-ll,ling s~"fic~tion forthe PCI bit m"l~ ing ~heme.
Figure 4. Xbus Bit Numbering Convention incr~a~ing o~r Xbu~ data bi~s _ ~1 2 23 16 15 7 Big-Endian byteO by~e 1 byte2 byte3 (Xbus) 3~3 Tenninology bus cycle--the 24MHz (-41.67 ns) building block from which all Xbus operations are built. A bus cycle is the time which a valid logic level driven by one board on the ba~ lane is seen by all other boards. Two bus cycles cG,--~,ose a bus phase 4 bus phases co,.-pose a bus ope,a~on and one or more bus operations co"",ose a bus transaction.
bus phase--the 1 2MHz (-83.3ns 2 bus cyde) building block from which all bus operations are constructed. There are logically 11 types of bus phases on the Xbus; ~Arb~ Info" ~Post1" and ~Post2" ars the phases that occur during norrnal ope,a~on. When an error is detected the special phases "Error1" 4CPU tesr' ~CPU Posrl IO Tesr IO Post1~ "IO Post2'1 and "Error2" are inserted in the protocol for fault isolation. (The error phases are sG",e~i",es c~''e ~lely referred to as ~Post3"). During each bus phase it is possible to transmit two se~s of i"~ ",alion on a physical set of ba~ ~,la,-e lines (i.e. I'double pumping") though this is not done for all bus phases and/or signals.
bus operation--A bus operation is the basic unit of address and data trans"~;ssion and checking on the Xbus. It is generally co""~osed of at least 4 phases; an Arb phase followed by Info Post1 21 ~uly 1995 ~ 17 Xbus Functional Specificabon Stratus ~,ompany Confidenffal and Post2 phases. Bus errors cause the inser~on of the error phases after Post2 and illweasê the number of phases required to complete a bus Op~ h n. Bus operations may consists of multiple info ~a,~ ,sions in the case of a block transfer. A bus operation can be thought of as a full one way transfer on the Xbus.
bus sub-operabion--A bus wb~Dpêldlion is an operation inibated by a bus master during s~6equent phases of a block transfer. Sub-operations always carry data and BUSYs are ignored.
It ~s generally uJI~sed of 4 phases: an Arb phase during which grant inhibit is used, I~" Dwe d by Info, Postl and Post2 phases. Bus errors may increase the number of phases required to complete a bus sub-operation. A sub-operabon is diff~,lenb~d from an opefabcn in that a bus operation for a block transfer consists of 1he first transfer plus a number of sub-operations ~n,;_~.lg of multipe data banslb.~.
bus transacffon--a complete high level eA-;llange of infonnabon on the Xbus. Examples include reads and writes. A read llansa ~)n b elwoen CPU and PCIB is cul"posed of a minimum ot two bus operabons; one operation provides the address and function code, and one or more operations provide the retum data. A write ttansscbon to a PCIB is cG",~osed of a minimum of two opel ~ns, one opel~on provides address and funcbon code, and one or more additional operabons provide the data.
The ~ ;ng diagrams illustrate the terrninology surrounding a simple word read and line write transacbon.
Figure ~. Terminology - Simple Word Read T.a,~s~lion A 24MHz (-41 .67ns) bus cyde 11'~
/~
~ 7j~
A 12MHz (-83.3ns) j~
bus phase J of 2 bus cydes) A four phase bus operation. In the word A four phase bus ~pe~ l ~ n. In the word read transaction shown here this first read l,~" Isac,tion shown here this first operabon is used to transmit the read operabon issued to transmit the data.
address and funcbon code.
A complête bus llall a ~icn. This example simple word read h dl 15d~1iOn iS ~SII "~osed of two ope,a~ions, one to transmit address and one to transmit data.

21 July 1995 18 WO 97/46941 rCT/US97/09781 Xbus Functiona, .~peci1 ~l on _.. atus Company Confidential Figure 6. Terminology - Simple Line Write Transaction A 24MHz (-41 .67ns) bus cycle '~ ~

A 1 2MHZ (-83.3ns) bus phase (co,-,l,osed of 2 bus cyclss) A four-phase bus ope, ~ic n A four-phase bus sub-operation A complete bus t,dnsac~ion. The simple word write transaction shown is co",posed of two over-lapping operations.

Bus Master--A board that has won arbitration. This board drives the info lines in the info phase of the bus ope,alion. A bus master can be a transacffon master or a ~dnsaclion slave.
Bus Slave--A board that has determined the info lines carry information that it must receive. A
bus slave can be a bansa ban master or a transaction slave.
CD di~felel-t read--A read in which the C and D side ASlCs each provide half the data, e.g.
when reading error status 1~9;Jt~ i. Loopb~u~ cheching of the bytes driven by the other side is suppressed.
Cyclops - The Xbus to Ibus interface ASIC on the CPU board in Polo systems.
echo ba~.sa~t;on--The second half of a peer-to-peer bus bdn a ticn between CPUs. Send and echo transacffons are not split; grant inhibit is used to ensure that no other lldl I ja_'ic ns occur between the send and echo. This is to prevent re-ordering.
EFQ - Eviction-Flush Queue. This ajueue exists only on CPU boards. Refer to the Cyclops (Bus Interface) .Specifi~ffon for details.
EF~Fr~ State--Set via bit 19 of the Bus l,lt~,late State Register. This mode only exists on CPU boards. When in this mode, a board will busy all ac~sses directed to its EFQ except those from its partner unless it is a data retum of any size.
Fsst ~L ~ ing - Fast duplexing is a fQrm of duplexing in which no memory is ~ atPd This is done when both boards are coming up for the first time as opl~sed to updating a new board from a nunning OS.
Free~State- Set via bit 14 of the Bus lln~ ce State Register. This mode only exists on boards that can be duplexed. When in this mode, a board will busy al~ accesses directed to its RWQ
21 July 1995 ~( ~ 19 CA 022S7Sll 1998-12-03 PCTrUS97/09781 Xbus Functional Spe T~tion Stratus ~;ompany Confidential except those from its partner.
Gsmbit- The Xbus to PCI ASIC on the PCI Bridge card. IT interface to the Xbus and the PCI bus.
110 virtual address (IOVA) - An IOVA is an address gener.~toJ by a PCI card. This address is l~arss"~ilted across the Xbus to the cyclops ASIC. Inside the cyclops ASIC the address is t,an~ S~ d into a valid system address. The IOVA is used to provide fault tolerance. It gualcrllees that a PCI card will gene,ale a correct address range.
Io D~ba: ch~ ~ Tn~ This is when an ASIC checks that the value it sees on a pin is equal to the value it thinks should be on the pin during norrnal operation.
Ioopcheck o,~ .~on--This is when the bus ASlCs drive 55/Ms as part of the error protocol in order to determine the site of a fault.
peer bD peer tran~ n--A two part bansaction b t~- ~3n two CPUs. The Xbus does not have fully in~e, ~nnected data buses and b culafers ~ etween the two CPI I boards must occur in two steps: first a send ~et~-. een the CPU and the PClB(s), then an echo from the PClB(s) to all of the CPU(s). The reques~ng CPU drives a complete ~anad~;on on its A and B buses. The PClBs look at the address and deterrnines whether the l,ansa tiun is directed to the CPUs. If it is they buffer the l,dnsaction in order to repeat the llallsa ~ ~ on their A & B buses once Post2 of the last info phase has passed with no bus errors. ln this way, both CPUs see the h d"~ n at the same brne.
Peer to peer transactions ~at - een CPU boards require a minimum of four ope,ations.
RWQ - Read/\Nrite Queue. This queue exists only on CPU boards. Refer to the Cyclops (Bus Interface) SrE '~a~i~n for details.
Regurgitated Into - A regurgitated info is a cougar gen~,aled cycle used during the update process. It is generated by the update-on-line board and transmitted to the update off-line board. It is unique bec~ce an update off-line board accepts info cycles even if the base address does not match the base address of the board.
send t~a-~sa~.tion--The tirst half of a peer-to-peer bus transaction between CPUs.
single side operation--An operation with data s~ - ~e~ entirely by either the C or D ASIC; e.g.
a read from a PCI card. I o~-pl~c 1~ cl,ecl~i"g is pe,fo""ed only by the side supplying the data.
TRID--T,ansaution idenfffbr. This a unique binary number used to tie together two bus operabons during a split transacbon and to identify the source for write transacbons (~RlDs on write transactions are strictly for debug). A TRID is unique only while a given transaction is still outstanding and will be re-used for later transactions by a banss~io~master. Note thattrid bits 0 - 00 are used for the slot number of the transaction-master and brid bits 06 - ~4 are gen~,aled by an on-board master - thus allowing a board to have 8 unique masters with transactions outstanding.Trid bi~ 03 is a new bit for Polo that indicates the forrnat of the aW~s a zero indicates a Jetta style system address is being transmitted and a one indicates an IOVA (I/O
Vlrtual Address) format is on the bach~,lane. The IOVA address needs to ~e translated into a system address via the map RAM. Refer to section 6.10.1 for a complete description of the trid field.
Tr~.,s 2 L ~ n Msster--The specific resouroe on a board that generated a llal ,~a ~m The t~ahsa~iu-l master is ,~:sponsiL)le for generating the TRID of the l,an:,a~.~on.
T.~ Lcn Slave--The board on which a ll ansaction was directed towards. i.e. a board in which the function code and address of a bus ope, alion has dec4cl~d to.

21 July 1995 ~ 20 Xbus Functiona, ~.pecinoation ~ atus Company Co"fi~ential 4. Xbus Signal Description The h " ~ :ng section desc~ as in detail the various signals that cc, "~J, iae the Xbus. As there are 4 buses in the Polo imple"~,l'~t ~ n, a new naming convention has been dc~.vloped for the buses.
This naming convenbon is desv-ibed in section 3.1, Bus Naming Convention, on page 15.
A tunctional descripbon is included for each type of signal and not for every individual signal (for example: there is a TRID field for all 4 buses, however there is only one descripffon that covers bothJ. Power contrvl and .JTAG signals are not dv-sl,,ibed here.The signal names are desiyl,ed to be cG",~atil~le with the Verilog coding slanda.da so that the specification will match the code exactly. The I 'l~ 9 rules were used in creating the signal names:
~ all lower-case ol)ald .t~la ~ the " " is a delimiter when it is not at the end of a signal name ~ the ~t_" at the end ot the signal name indicates that the signal is low-true ~ _a indicates A_bus signals ~ _b indicates B_bus signals ~ _x_, _y_, and _z_ are used for the three low-true signals on a triplicated net~ ln:m] is used to describe a mulU-bit field 4.1 Signal Description As deav~ibed earlier, the Polo Xbus implements the control and bused signals very dfflerently.
The bused signals are implemented as 4 point-to-point bidirectional signals. The ability to drive these signals is co~ through the a,Lit~dtian network desv-iL.ed later. These signals are pr~tb~.hd by a single parity bit that ac~l"pa(.ie s the bus. The buses are interconne.A~:d so that each CPU is conne~ d to both PCI bridge boards and each PCI bridge board is connect~d to both CPUs. Error recovery is accomplished through the XBus error protocol.
The control signals are Glgdrli~ed in a set of point to point unidirectional buses. Each of these buses is ECC ~ d. These buses carry control signals which are not govemed through a.l,i~,dlion. Unlike the bused signals, there is a control bus in each direction between every board.
This is nec.essa~y in order to ensure the single bus view of the system. For example, if one PCI
bridge card sees a bus error, that in'.~ ",alion must be transmined to all three other boards in order for the boards to all perform the error sequence.
The bused signals and control signals are double pumped at 24MHz each cycle. That is, they carry different data during the first and second 24Mhz bus cycles that make up a single phase. All buses and control signals are active high on the back~lane.
A small number of reset and broken related control signals are buffered by the 26S10 ~,dnsceiver and replicated for three way voting. These signals are active low on the bach~lane.
4.2 The Info Bus The f 'h .. :. ,9 signals are collectively refened to as the info bus. Although there are actually 4 sets of these signals (aO,al ,bO,bl ), for simplicity's sake only the aO version is listed. For example, when the TRID field is desc, il,ed, it should be u, Id~l SIOOd that there are actually 4 TRlD buses:

2~ July 1995 '~ l ~ 21 Xbus Functional Spe~ficdtion Stratus Company Confidenbal trid_aO, trid_bO, trid_a1, and trid_b1.
Table 2. Xbus Bidirectional Buses signal width dsscripbon - info_aOI31 :OI 128 Xbus into bus- Into is driven during the info phase of a bus operation (32x4) by the current bus master. This field may contain either an address (physical address or virtual index with funcbon code) or data.
depending on the func_op control line.
trid_aO~6:0] 28 Xbus tran ~ n id (TRID) - The trid lines carry the TRID
(7x4) Itransaction ID) during the first cycle of a phase. During the second cycle, it carries the number ot phases rernaining, the first_op bit, and cache cvl,~r~"niy bits.
func_op_aO_ 1 Xbus tunc_op - This line carries the func_op_ signal which indicates that the wrrent inforrnation on the info bus contains a function code that should be decoded This signa~ is low true so that an idle bus will indicate that a function needs to be ~l~c~cle~ and thus a no-op function. This bit is valid during an info phase and is p,utt:..W by parity along with the lower half of the info field. During the second cycle, X is unused (driven to logic O on the bach~lane).
parity_aO 1 Xbus parity - This parity signal covers all of the bidirecffonal signals on a bus: info, trid, func_op_.

4.3 The Control Bus This section desr ,ii~vs the control signals. There are ac~ally 12 control buses, but again only one is deav,ibed here. The narnes of the tweive control buses are listed in table 1. For simplicity of docl,"~"~tion, the control bus is identified by bit m",li~,ing, similarto the trid field. I lo..v~cr, since the meaning of the control bus bits is very significant, each one is desuii ed in detail here.

21 July 1995 2 ~ ~, 22 Xbus Function~. Sp e ~ fion ~,tratus Company Co, liide"tial The control buses are protected by a single bit co"tz~;tion, double bit detection ECC code.
TabJe 3. Control Bus Signals signal width descripbon control10] 12 bus_req and ack - During the first half of the phase, this bit is used for (12x1 ) bus_req. During the second half of the phase, this bit is used for ack.
During the first half of the phase, this bit is driven by a board when it is requesting the bus. As desc,ibed in later secbons, the bus uses a distributed arbitrabon model loosely based on the Golfbus. Each board in the system drives the bus_req and tests all of other boards bus_req signals to determine who will drive the info signals during the next phase.
During the second cycle of the phase, this bit is used to a..l~ Ige a bus odnsa~on. Ack is asse, led in Post2 by the target board of the transaction. This signal is the result of an addre~ decode, so it is only valid in Post2 of an ope,alion that is ban~ft:" i"g an address. Ack provides an indicabon of whether or not a bansa.,lion is p,uy,ess;ng.
This is relevant in a Polo system, since a PCI card may go away resulting in a no ack for a ping ope, dlion.
Acks in a Polo system also let a PCI Bridge know that a CPU's map RAM has mapped a PCI inibated access to a valid CPU address. If a PClB's read or write is not ACKed, the PCI slot that initiated the access may or may not be set off-line depending on bits in the Gambit's configuraUon register, des~,.il,ed in section 12.6.4.9, SAM
Configurabon, on page 127 in the Polo P~ùg~ uing Guide.
Writes from the CPU to the PCIB are acked to facilitate debug, but are vU,en~ e unused. A peer to peer CPU write is not ACKed by the PClBs, but the echoed nl~r~ on is acked by both CPUs; again, this is only for assisti"g debug.
Ack is ignored for the send portion of a peer-to-peer op~,ation. The CPU initiator of a peer-to-peer operabon must track the entire operabon to see whether the cycle is ack"o.~ledged.

21 July 1995 q I ~ ~ 23 .. .. ~ . . . ...

PCTrUS97/09781 Xbus Functional Spe~ i~c ~tion Stratus ~ mpany Confidenbal Table 3. Controt Bus Signals signal width descripbon controlll] ~2 ~rant_~nh and maint_int - During the first half ot the phase, this bit is (12x1 ) used for grant_inh, during the second half of the phase, this bit is used for maint_int.
During the first cycle, the grant inhibit control bit is driven by the current bus rnaster to extend the info cycles when the bus rnaster is moving a block of data. The arbitraffon logic will not issue a grant to any other board when a board is driving this signal. This ensures that the current bus master will retain ownership of the bus for another cycle.
During the second half of the cycle, this bit is used to signal a maintenanoe interrupt. Maintenance Interrupt indicates that some board in the system is requesting dtt~n : n from the software maintenance process. Any board in the system may drive this signal during the second half of a bus phase regardless of bus ",ast~ ip. All boards in the system will sample ~,-a;nt~na"ce interrupt and use it to resettheir a,L ~;~Lon priority.
controll2~ 12 bus_enr_a and busy - Assertion of this signal during the first half of (12x1 ) Post2 signals that a bus error was detected on the info bus a%sociated wiith this particular control bus (n,o,p). Any opera~ic ~ in Arb, Info, or Postl will be aborted. Operabons in Post2 are suspended while the error protocol runs, and then will retum to the Info phase. See section 6.4 on page 30 for more into""dti~." on bus_err.
The CPU inibator of a peer-to-peer ope,dtion must track the entire operabon to see whether the cycle is errored in either the send or echo ponion of the bdllsdclion.
Assertion of this signal during the second half of Post2 indicates to the bus master that the ope,aLon should be aborted and r~tried at a later Ume.
Busy is ignored for the send ponion of a peer-to-peer operabon. The CPU initiator of a peer-to-peer operabon must track the enbre oper_ ~ n to see whether the cycle is busied.

21 ~uly 1995 ~ _ 24 . ., W O97/46941 PCT~US97/09781 Xbus Functiona, ~ c~tionatus Company Confidential Table 3. Control Bus Signals signal wirJth descripbon co,m. ~] 12 bus_err_b and funny_stat~ Asserbon of this signal during the first (12x1 ) half of Post2 signals that a bus error was d~ d on the B bus connected to this board. Any ope,.Jt,on in Arb Info or Post1 will be aborted. Operations in Post2 are su~nded while the error protocol runs, and then will retum to the Info phase. See section 6.4 on page 30 for more informabon on bus_err.

The CPU initiator of a peer-to-peer ope,ation musttrack the entire ope~'ic n to see whether the cycle is errored in either the send or echo portion ot the bdns~. Lon.

During the second half of the cycle this bit is used to signal that a board has just gone unbroken. The board will continue to assert this signal unbl it has seen eight phases without a bus error occurring. At that point the board will stop asserbng this signal. Then all other boards - will treat this board as an active responding board. This prevents a board that is going unbroken from responding to bus errors in the middle of an error sequence that is already under\~.ay.
control[7:4148 che~ The top 4 bits of the control bus are cl)t,chbi~ generated ~12x4) from the lower 4 bits of control signals.
The cl-e~ hl,il algo,i~"" is:
controll4] =control[0]~control[1]~control[21;
control[5] = conrtol[0~co"bu1~1]Arontrol[3~;
control[6] = control~O]~co"t~ Acontroll3~;

control~71 = controll1]Acontrol[2]~co"~ol[3];

4.4 The Voted Signals The voted signals are the only signals driven through 26S10 Dansceivers. These signals are point to point and terrninated at each end so that insertion of an unpo/.e.~d board does not disturb the 21 July 1 9g5 25 'Zl ~

Xbus Functional Specification Stratus ~;ompany Confidential termination (and timing3 of a net in use. Only the _x_ versions are listed; there are _y_ and _z_ Table 4. 3-Way Voted Signals signal total descripffon rese~_0_1_x_18 reset - There are sepA~g~ 3-way voted triplets reset_0_2_x~(2x3 from each CPU to the other three boards in the reset_0_3_x_ x3) system. When a RECC needs to reset the reset_1_û_x_ system, all lines go active. When a CPU wants to reset_1_2_x_ reset another board, only the lines going to that reset_1_3_x_ board are active.

board_not_broken_0_1_x 36 broken status - This three way voted signal is board_not_broken_0_2_x (4x3 driven from each board to each other board. It is ,,~, board_not_broken_0_3_x x3) driven when a board is alive and not broken in the , board_not_broken_1_0_x system and is used to determine which buses are board_not_broken_1_2_x active. The Gside ASlCs drive the signals and board_not_broken_1_3_x the ~side ASlCs drive the output enables for the board_not_broken_2_0_x 26S10s.Thiso,yal~ ionguaranteesthat k~) Yboard_not_broken_2_1_x board_not_broken is deasse, ~d whenever either ~ ~
board_not_broken_2_3_x side of the board thinks that the board is broken,. ~l board_not_broken_3_0_x The (~cci~ring board votes the x, y, and z J
board_not_broken_3_1_x signals.CPU0 in slotO drives board_not_broken_3_2_x board_not_broken[0], etc.

sync_x_ 3 sync status - These signals are on the CPU only (1 x3) and are used when s~" ,chror,i~i,)g a pair of CPUs to enter the duplexed state.
even_online_x~ odd_online_x_ 6 o~line status - These signals are on the CPU
(2x3) only and are used by the CPU boards to communicate to each other which CPU board(s) are in the on-line state. Even_online_ is ass~ d when the CPU in slot 0 is on-line; odd_online_ is asse, ted when CPU in slot 1 is on-line.

signals maWng up the triple~

21 July 1995 26 ~(~

WO 97/46941 PCT~US97/09781 Xbus Functiona~ onalus Company Corlfidential 4.5 Other Control Signals Table 5. Mi~c~ r~eous Control Signals slot_ida 2 slot id - The slot ID signals are hard wired on the bach~lane forslot_idb each slot. There is one duplicated slot id. Slnce the slots are dedicated in Polo it is only necessary to determine if a board is in an even or an odd slot. These two bits will be r~gic.le,ed and checked by each ASIC at reset and will not be sampled again. If an error is detected at reset the board will break and hence will never be capable of being brought on-line.
xb_clk8 1 system clock - This is the system clock received by the Sentry clock chip and used to gener~t~ the board clocks. It is gen~dt~d by the bach~Jlane clock os~ . This clock runs at 8MHz so every board in the system will be in sync with each other and there will be no need for additional syln;l"or,iL~tion clochs to be passed along the bach~lane. The clock is pulse width mod~ ~lated so that 4Mhz can be gene,dted.
slotO_ta Id 4 ta signals - These Uta signals are only present on CPU boards andslotO_ta_c_ are sent t~a~rreEn a duplexed board pair for early det~on of the slotl_ta_d boards going out of lochstep. These explicitly named t~ch~Jlane slot1_ta_c_ signals are used in place of the Golfbus pair_comml7:4] signals which carry the ta i"~o""alion in Golfbus systems.

21 July 1995 27 ~1 ~

CA 022S7Sll 1998-12-03 W O97/46941 rCTAUS97/09781 Xbus Functional Specification Strahls ~,ompany Confidenffal 5. Xbus 110 Interface Registers A complete ~ ol, of the Polo register set can be found in the Polo P~ ""uing Guide.

21 ~luly 1995 q, ~ 28 Xbus Functiona- ~.pecincation_~atus Company Coufiderltial 6. Xbus~,otocol 6.1 oveNiew The Xbus is a point-to-point s~"lcl,.onous, pipelined, multiplexed ad~lless/l~ald, error det~,~,til ,9 bus with split-t~ransaction capabilities. Function-code, address and data are parity and loop-back checked. Control lines are ECC plut~led and loo~back cllechêd. Three-way vohng is implemented on the reset, clock, and broken indicator lines.
The bus supports word ~rr~esses and has a block transfer capability for support of cache line ce~ses The Xbus has a logical 32-bit wide daWadd~eas bus.
6.2 BusOperation The basic r~",~ne,lt of al~ Xbus ban,a ti~ la is the operaUon. An operaUon is cu,,,posed of four phases as illustrated in figure 7: arb, info, post1, and post2. Two information lldllafer;~ can occur on the bus during each phase; this is refened to as ~double pumping". The double pump frequency is app,oAi",ately 24MHz. The figure below illustrates the logical activity on the bus. All information is direcUy registered in the ASlCs without any external bulfers.
Figure 7. Basic Xbus Cycle 83.3ns Arb I Irrfo , Po~st1 , Po6t2 ~7~ l ~ '\~, ~ ' ' ~ ' ' ' ' '~ ' f ~
Whereinfo is one of ~
!
~- virtual index, function code, remotr~cohe,e, It bits, and byte enables or l/O
The phases are used as follows:
Arb phase: boards drive their arbitration request lines during the first half (cycle) of the arbitration phase. During the second half they determine whether they won arbitration and prepare for the info phase.
Info phase: For non-lOVA address lldn:,fela, boards drive Uhe virtual index, funcbon code, remote/~l,e.~.lt bits, and byte enables during the first half of the info phase and Uhe physical address during the second half. For IOVA address banafe.~ (IOVA bit in the trid is true), boards drive the IOVA during the first half of the info phase and deterrninistic data with good parity during the second half; the physical address is gotten from the l/O address map RAM
look-up. For data ballsfela, data is driven during both the first and second halves of the cycle.
Note that non-cache consiste"l address ballSIel:~ need not supply a virtual index though the driven informabon must be detemninisffc and parity will be computed across it.
21 ~uly 1995 ~," 29 CA 022~7~11 1998-12-03 Wo 97/46941 PCTlUS97/0978 Xbus Functional Specification Stratus ~,ompany C(",f,der~al Post1 phase: During this phase, boards are determining whether any error condibons existed with the info phase and whether there is need to BUSY the operation. CPU boards map the device index portion of the IOVA to obtain the full physical address and virtual index of an l/O
board's transfer for IOVA address l-ansfe.s.
Pos~ phase: Any board detecting an error with the info phase drives the error lines during the first haN. If a board does get errored, it next goes to the error sequence phases to deterrnine the source of the error. Any board detecting the need to BUSY an addresslfunction code driven during the info phase drives BUSY during the second cycle of this phase. It is also during this phase that aG~e~ses are a-,~--u..~ledged (this is desc-iLIed in greater detail in su.,.~tho~;.ection).
6.3 Bus Busies Figure 8 illustrates the effect ot bus BUSYs on the basic bus cycle. As shown in the figure, BUSY
has no effect on a bus Op~OIl during any phase except for post2; a BUSY during post2 will canoel the bus operation. Note that busys for multiple cycle bus operations, such as block l.ansle. s, have special rules and are des~" il~d in section 6.7.4 on page 49.
Should a cyde be both BUSYed and ERRORed, the ERROR takes p,~cede"oe.
Figure 8. Basic Bus Busy Operation Busy during Pos~2 of an address/function code info phase causes the access to be cautsl~i Busy during Post1 has no ef~ect, though the info may not be used as ~liscussed in section 6.7, Block Transfers ,~ ' '' ' , ' Busy during Infd has no effect, though the info may not be used as discu~sed in section 6.7, Block Transfers, j~ , ' Busy during A.Lii~Uon has no effect, though the info may not be used as rliscussed in section 6.7, Block Transfers, 6.4 Bus ErrorsC~r~C~I
This section covers bus errors on the info bus, for inforrnation on the control bus see Section 8.2 on page 84.
Figure 9 shows the effect of bus errors on the basic bus cycle.

21 July 1995 30 CA 022575ll l998-l2-03 Xbus Funcbona. ,ecification ItUs Company Confidenbal Figure 9. Basic Bus Error Operation Error during Post:! causes the address to L r~,L ~I)sluLd the cycle l~ ng the error protocol. ;~4~, ErrorduringPost~ aborlsthereques~

Error during Info aborts the request.
', ~.
, Error during Arb aborts the request. Info is not driven.
1' ~

The board tha~ was transmltting during the error automaffcally gets the first available info cycle ng execuffon of the error protocol. The arbitration is ignored in the previous cycle.
Since the Xbus has no ~an~iv,ers, the loor~l,e..h phase of the Golfbus error protocol (Post4) has been modified to allow each board an opportunity to verify its transmit and rer eive capabilities.
This has resulted in new states being added to the bus error ope,a~on. These states are des~ ibed below:
En-1: The Errl state is entered on the cycle after a bus error is detected This state is used to allow for ffme to turn off the info bus before the loop~rk checks are p~- fo" "ed. A board that is in its info phase during Err1 will disable its output enables half way through the phase.
CPUTest: The CPUTest state is used to test the CPU's ability to drive panerns on the Xbus. On the first cycle of the phase the CPU will drive 55 on the info bus, 55 on the trid bus, 1 on the parity line and 0 on the func_op line. On the second cycle of the phase the CPU will drive AA on the info bus, 2A on the trid bus, 0 on the parity line and 1 on the func_op line.
CPUPost: The CPUPost state is used to turn the bus around b~t~r~een the CPU's loopbac,lc check and the UO boards looph~c 4 check. This phase is also used as a Postl cycle for the CPU's l~,pl~.~ pattem.
lOTest: The lOTest state is used to test the ItO board's ability to drive pattems on the Xbus. On the first cycle of the phase the llO board will drive 55 on the info bus, 55 on the trid bus, 1 on the parity line and 0 on the func_op line. On the second cycle of the phase the llO board will drive AA
on the info bus, 2A on the trid bus, 0 on the panty line and 1 on the func_op line. Bus errors from the CPlJTest phase are ~e~, ted during this phase. This information is used to ovqlu~te the bus, CPU, and llO board at the end of the error sequence. The last l/O ASIC to drive the data bus drives the bus during the lOTest phase.
j lOPost: The lOPost1 state is used to evaluate the lOTest data.
21 July 1995 Xbus Functional Specification Stratus ~_~"npany CGUf;denbdl lOPO5t2: The lOPost2 state is used to transmit any bus errors from the lOPost1 state. This illh ".~on will be used to make an intelligent deasion about how to deal with the error.
En2: The Err2 state is used to evaluate the inforrrlaUon from the loopb~,k checks. Bus errors from CPUPost and lOPost2 as well as information shared betuwn the C and D sides of each board are used to deterrnine what course of action to take. This set of actions will be desuiL.ed later in this section.
Figure 10 shows the basic state rnachine and state transitions forthe bus error handler.
Figure 10. Bus 7ror Op~l~lion Flow Chart IDLE ~

(OPost2) (CPUTes~3 ~OPostl2 fCPUPos~

~ lOTest) The key challenge for the bus error algo~ .. on the Xbus is to di~--ose errors so that system operation can continue. Unlike previous systems that use duplicated buses to allow all functional units a guaranteed path for communications, when the Xbus removes a bus from service, it must also removs one of ~e ~J~d~ hed to that bus. In some cases, the right thing to do is obvious. In other ca~DL~e t ~ " > :ng sections analyze various faults, how they are handlea and how they mann~ e...sel~es.
6.4.1 Bus Error Broken Condltions At this point it would be helpful to classify the different types of conditions that cause a board to go broken when a bus is d~ 4~ bad. For a full discu~ion of the broken logic, see Section 9. on page 80.

Z1 July 1995 ~ 3 32 WO 97/46g41 PCT/US97/09781 Xbus Function~ ,p e ~ r ~bon atus Company Conlicle- ,~al ~ Ioophark on con~ol - C and D AStCs must always agree on what to drive on the control lines, including whether or not to assert bus error. If one ASIC asserts bus enor and the other side does not, the board breaks.
~ 1~Qp~l7 ' on data - C and D ASlCs must always agree on whatto drive on the ~inrli~Pt~d info lines. When driving CD same data, ASlCs co"",a,e the data they drive with the data they receive. An ASIC asserts bus enor on parity errors when r~ .,g data, and parity errors and lou~ k enors when driving data. I oc~l ll~rk cl1ecl~i"9 is disabled when a board drives UCD
dfflerenr' ctata, such as the ~nt~"t~ of enor f~pOItillg registers or data from PCI cards. The board breaks if the two sides ~ti3~g~e on which bytes have or do not have enors,.
~ arbttrary - Break the designated board in Err2 when there are bus enors signaled during CPUtest and lOtest and no board has broken by the end of lOpost2. This is called an arbitrary shoot bec~se the fault is most likely on the b~ck~lane, so it is arbitrary as to which of the two boards connected to the faulty bus to break. Typically, the CPU is set broken, so that the system can conbnue with all of its l/O available, but if bit 21 of the Bus Interface State register (Section12.3.40npage87inthePolo~o~..".lingGuide)issetthenthePClBboardwillbe the designated board.
~ heuristlc - a board breaks itself during Err2 if there is a bus enor when it drives, but no bus enor when the other board drives, and the other board did not break by the end of IOPOST2.
6.4.2 Xbus Fault Analysis In order to un~-a~nd various faults and what they can mean, it is il,lpul~ t to present a detailed block diagram of the Xbus intefl;on' le~;L Figure 11 show the interconnect for a typical Xbus line.
Figure 11. Xbus Interconnect D-Side CPU ASIC C-Side CPU ASIC
r , 2 2 5 }~ ~e ~ ,-~) 6/, ~ S

~ r? ~ ~ ~

~~ -.r . ~ 6 D-Side l/O ASIC C-Side ItO ASIC
'~ fault site number - see section 6.4.3 The black dots in figure 11 feplesell~ the co... 16.,tOI:~ to the bachplane. For fault tul E~ a"ce and fault isolation reasons, it is i,.,~, ~. l~ that the boards should be routed so that the etch between the D-21 July 1g95 C~,L~ 33 PCT~US97/09781 Xbus Functional Specification Stratus ~Jmpany Con~i~le"tial side and the C-side runs through the connector conne~,~ion. This limits the amount of etch on each board that cannot be isolated ~ to a minimum. On the CPU board, one ASIC both drives and receives a given net while the other ASIC only receives that net. On the l/O board, each ASIC can pote,lLa'1y drive every net. The CPU ASlCs are always in lockstep and ll,~,elo,e each ASIC is capable ot sharing the data out load. I loJveJer on the l/O board, each ASIC connec~ to a different PCI bus so a signal ASIC may need to drive the entire Xbus. There are cases in norrnal oper t ~n when only one CPU ASIC will drive the enbre bus.
6.4.3 FaultConditions The ~"ov ;ng secbons identify all known fault condibons and deso,ilJe their handling. Refsr to figure 11 to deterrnine the locabon of the fault site indicated.
6.4.3.1 CPU Board ~aul~ Input Circuit- CPU Driving fault site 1 ~ break via IGvptJ~Ck on control fault This fault deals with a fault in the input secbon of one of the CPU ASlCs. In this case, the fault occurred during or just before a cycle in which the CPU droYe the info bus. The error is d~v~ed when the CPU drives the bus. The ASIC with the faulty circuit will signal a bus error during the Post2 phase of the cycle and the other side ASIC will not. The board will go broken and drive bus error during Err1. The error sequence will be e~ecuted. and the operation will be retried by the partner CPU with no error.
6.4.3.2 CPU Board Faulty Input Circuit- I/O Board Driving ~ tault site 1 ~ break via loopback on control fault This fault deals with a fault in the input secbon ot one of the CPU ASlCs. in this case the fault occurred during or just before a cycle in which the llO board drove the info bus. If the error is a multi-bit error that evades the parity logic, the error will be caught intemal to the CPU board and the CPU board will go broken. If the error is a single bit error the faulty ASIC will detect a bus error during the Postl phase of the transfer. The ASIC will drive bus error during Post2 of the transfer and the otherside ot the board will not. The board will break with a loopl~ 4 on control failure in the next phase. After the enor sequence, the Opeldtiull will be retried by the partner CPU with no error.
6.4.3.3 CPU Board Diftor~.~t Data GSide and D-side ~ fault site 2 ~ break via lool~l,acl~ on data fault This fault deals with an intemal CPU fault that results in different data being driven out of each ASIC. The enor is detected when the CPU drives the bus. The C and D ASlCs will trade enor status and diOag-~e on where the error is during Post1; both C and D sides will see an error on bytes the other side drives but no error on the bytes it drives. The board will go broken and drive bus error during Post2. The error sequence will be P~ecut~d and the operation will be retried by the ,oartner CPU with no error.
6.4.3.4 CPU Board Faulty Output Circuit Buffer to Pad Fault ~ fault site 3 21 July 1995 34 2(25 .. . . . ..

CA 022575ll l998-l2-03 W O97146941 PCT~US97/09781 Xbus Functiona, -Jpe~ ation ~tus Company Confi~ential ~ break via heuristic broken This is a fault in the output section of the CPU ASIC resulffng trom an output driver cirwit fault that blows in a manner that causes an internal open beh~l~en the output driver and the ASIC pad, while not disnupting the functionality of the input reoeiver. All ASlCs on the bus will detect a bus error during the Post1 phase of the transfer. The ASlCs will drive bus error during Post2 of the transfer.
All ASlCs will detect a bus error during the CPUPost phase. No bus errors are detected during the lOpost1 phase. The CPU board will go broken during Err2 based on the fact fhat it detected a bus error when it drove the bus, no bus enor when the UO board drove the bus and the llO board did not go broken after its error sequence.
6.4.3.5 CPU Board Open - CPU E~o~rd Driving fauK site 4 ~ break via lonplJacl on data fault This fault results from an open due to either a broken etch or a lifted pin on the CPU board.The routing is very ~~ JG- hnl for this class of fault. The etch t e l 3n the ~side and the D-side should be routed throu~h the conn~,t,~ pin. This limits the possibility that an open on the CPU board is mistaken to be an open on the bach~ ~a,-e. In this case the fault occurred during or just before a cycle in which the CPU drove the info bus. The error is d~c~ -d when the CPU drives the bus.
During the Postl phase, the driving ASIC will not see an enor but the checking ASIC will signal a co"-l.a,e error. During Post2 the board will go broken and drive bus error. The ope,ation will be retried by the partner CPU with no error after the error sequence.
6.4.3.6 CPU Board Open- 1/0 Board Driving ~ fault site 4 ~ break via lool~b~-k on r,ontrol This fault results from an open due to either a broken etch or a lifted pin on the CPU board. The routing is very i"",ollant for this class of fault. The etch between the C-side and the D-side should be routed through the connector pin. This limits the possibility that an open on the CPU board is mistaken to be an open on the bacl~.lane. In this case the fault occurred during or just before a cycle in which the l/O board drove the info bus. The ASIC with the open between it and the cun"e~Aur will detect a bus error during Post1. During Post2 one ASIC will assen bus error and the other will not, r ausing the board to break. The operation will be retried by the partner CPU without any error after the error sequence.
6.4.3.7 CPU Board Short ~ fault site 4 break via arbitrary broken This fault deals with a short on the CPU board. When the fault occurred and who was driving the ~nfo bus during the fault are not relevant to this class of fault. During Postl of the transfer all ASlCs on the bus will detect a bus error. The ASlCs will drive bus error during Post2 of the transfer. Both ASlCs on the CPU will detect a bop~l~,k error during the CPUPost phase and the ASlCs on the l/
O board will signal a bus error during lOTest. The ASlCs will detect a bus error during the lOpost1 phase and signal a bus error during lOPost2. During Err2 the designated board will go broken based on the fact that it has deter,ted bus errors during the error sequence and no other boards went broken after the error sequence.

21 July 1995 ,;~ 35 ,, -PCT~US97/09781 Xbus Functional SpeU~i ~inn Stratus ~.ompany Confidenffal 6.4.3.8 R~c~pl~ne O pen Etch ~ fault site 5 ~ break via arbitrary broken When the fault occuned and who was driving the info bus during the fault are not relevant issues for this class of fault. During Post1 of the transfer some ASlCs on the bus will detect a bus enor and drive bus enor during Post". Both ASlCs on the ItO board will detect a bus enor during CPUPost. The CPU ASlCs will detect a bus enor during lOpost1. During Err2, the designated board will go broken based on the fact that it has detected bus enors during the error sequenoe and no other board went broken during the enor sequence.
6.4.3.9 Bachq~lane S hort ~ fault site 5 ~ break via arbitrary broken When the fault occuned and who was driving the info bus during the fault are not relevant issues for this dass of faulL All ASlCs on the bus will deter,t a bus enor during Postl and drive bus error during Post2. All ASlCs on the bus will detect a bus error during CPUPost and lOpost1. During Err2, the desig,-~ted board will go broken based on the factthat It has d~ ~e~ bus enors during the enor sequence and no other board went broken during the error sequenoe.
6.4.3.10 UO Board Faulty Input Circuit fault site 6 ~ break via loopl,~c k on control This fault deals with a fault in the input section of one of the l/O Board ASlCs. For this parbcular fault, it is irrelevant who was driving the backplane when the fault was detected. The faulty ASIC
will detect a bus error during Post1. During Post2 of the transfer, the faulty ASIC will drive bus error and the other ASIC will not, causing the board to go broken. If it was a CPU initiated request the ope,dtion will be retried by the CPU with no enor after the error sequence. If it was a request initiated by the l/O board, then the request will be dropped.
6.4.3.11 IIO Board Output Circuit Fault ~ Bu1fer to Pad ~ fault site 7 ~ break via heuristic broken This is a fault in the output section of the l/O board ASIC. This class of fault results from an output driver circuit fault that blows in a manner that causes an intemal open between the output driver and the ASIC pad, while not disnupbng the funcbonality of the input receiver. All ASlCs on the bus will detect a bus error during Post1 of the transfer and drive bus error during Post2. No error is detected during the CPUPost phase. The llO ASlCs will detect a bus enor during lOpost1. During Err2 the UO board will go broken based on the fact that it has detected a bus error when it drives the bus, no bus error when the CPU board drove the bus and the CPU board did not go broken after its error sequence.
6.4.3.12 U O Single~side A ccess - A SIC P arity G en. Fault ~ lault site 8 ~ break via intemal parity gene,~or broken A fault in the parity genel ~Lon logic during single side accesses (data driven entirely by either the 21 ~luly 1995 2 ~ 36 CA 022575ll l998-l2-03 W O 97/46941 PCTrUS97/09781 Xbus Functional _"eciflcation ~ .tus Company Co"fidenti~' C or D ASIC) could cause the system to bus error forever. The ~:: ,e,~ide ASIC doesn't know the data and has no way of cl-ecki-lg the parity. For this reason the info bus parity generabon is duplicated and selfch6ching inside of the Gambit ASlCs. See section 9.1.1 Parity, on page 80 for details.
6.4.3.13 UO Single-side Access - ASIC PCI Data Path Fautt ~ fault site 8 ~ break each PCI slot due to checksum and map RAM errors.
This is a fault within the ASlC's PCI data path. This ha,~:a,e is not running in lockstep with the other side of the board and U,e,~hre is not self-.;hechi"g. Eventually bad add~esses produoed by the PCIB will cause map RAM errors and/or data checksum errors in the CPU ASIC. The CPU
ASIC noACKs A~esses that cause map RAM errors, and the PCIB sets off-line (breaks?) the PCI
slot that originated the noACKed cycle based on an option bit in the Configuration register in page zero of SAM compatible IIO spaoe. Eventually all PCI slots handled by the ~ c ASIC will be set broken. In the meanbme corrupted IIO data is ~ cl-~ by checksums and handled by higher level plu~ocols.
6.4.3.14 llO No~single-side t~ces~, Dit~rl_.,t C-D Data ~ fault site 8 ~ break via bo~hark on data This fault deals with an intemal IJO board fault that results in different data being driven out of each Gambit ASIC during a regular (not single-side) read retum. Each ASIC will di~ag,~e with the data driven by the other side and the board will break with a lou~l ack on data error. The error sequence will be executed and the ope,alion will be retried by the CPU and get noACKed.
6.4.3.15 UO Board Open ~ fault site 9 ~ break via loopb~rh on control This fault deals with an open either from a broken etch or a lifted pin on the IIO board. The routing is very i""~,la"~ for this class of fault. The etch b~h~een the C-side and the D-side should be routed through the co"ne. ~o- pin. This limits the possibility that an open on the l/O board is mistaken to be an open on the bach,ulane. The driving ASIC sees no errors and the uthe~ide ASIC asserts bus error; the board breaks the f ~ ing phase with a loo~ k on control fault.
After the enor sequence the ope,alion will be retried with no error and get noACKed.
6.4.3.16 UO Board Short ~ fault site 9 ~ break via arbitrary broken This fault deals with a short on the l/O board. When the fault occurred and who was driving the info bus during the fault are not relevant issues for this class of faul~. During Post1 of the transfer all ASlCs on the bus will detect a bus error and then drive bus error during Post2. All ASlCs on the bus signal a bus error during CPUpost and lOpost1. During Err2 the designaled board will go broken based on the fact that it has cle~P~ed bus errors during the error sequence and the other board did not go broken after its error sequence.

21 July 1995 ~ ~ 3 PCT/US97/~9781 Xbus Functional S,~e.;~ n Stratus _~mpany Ce"fi~lentidl 6.4.3.17 Tr~ ientFault ~ fault site an,~l,e,e ~ action depending on fault site and timing This fault deals with a t,~nsienl fault. A ~lansiell~ tault is defined as a fault that is detected, but annot be reproduced by the error sequence. It the fault is on the driving board and it is caught by loopl~ check, that board will go broken. If this is not the case then no board will break until Err2.
then the de~igr,~d board will go broken.

6.4.3.18 Slow Driver Fault ~ fault site 4,9 ~ possibly break the wrong board with a loo~t~cl; on rontrol fault When an ASIC wi~ a rnarginally slow driver drives the bus, four ASlCs clock in data from the net while the net is changing. Due to differences in speed between ASlCs from different lots, its possible that some of the ASlCs will detect a bus error, and some won't, resulting in a loopb~ k on control fault. This may result in one or both of the boards on the bus going broken.

6.4.3.19 Board Not Broken Generation The board_not_broken_out signal is generdled by all boards. If a board is going to break the norrnal operation is for the board to assert bus error. All boards will enter the error seguence and the board that was going to break will de-assert board_not_broken_out during the error sequence.
It is possible that a board could break without asserting bus error. This could only happen if there is a problem wffl asse, lilly bus error on the d-side gate array (for instance the clock line going to the flip-fbp that produces the control signal (bus error) opens). Please refer to figure 60t Board Dl ~ahil~g Timing and latching ~arrived at phase_12_1 ) on page 89 for detailed broken timing liag,d-"s.

6.4.3.20 Arbitrary Breaking There sre a series of csses mentioned before that error logic cannot determine where the fault is.
For these r;ases the goal is to not use the bus that the two boards are connected to. The default is to break the CPU board (de~ignalad board). This makes sense ber~ e there is a partner running in lock-step so no conneLti~ity is lost. It is possible that the CPU board will break but the fault really lies with the PCIB. The only way this will pose a problem is if the tailure in the PCIB effects the other bus that is still conne.,t~d and is being used. Norrnally it is the CPU board that is the designa~ed board, but if bit 21 of the Bus Interface State register is set then the PCIB board will be the designated board. THIS IS ONLY INTENDED FOR LAB DEBUG. UNDER NORMAL
OPERATION THE DESIGNATED BOARD MUST BE THE CPU BOARD. IF THIS IS NOT
ADHERED TO IT IS POSSIBLE TO HAVE BOTH BOARDS BREAK OR NO BOARDS BREAK.

21 July 1995 ~ q 38 , Xbus Functiou~.. Specmcation ,tratus Company C~nfid~n '~' 6.5 Summary Bus State Diagram A flow chart for the basic bus operation des~i~ed in the previous sections is shown in figure 12.
The thinner state transition lines ~ 5enl ~nu,,,,al op~r..Uon and the thicker lines ,t:p,ea~nl e~ n c~nditions (for the sake of ~is diagram, losing arbitra~bon is c~ns;cbr~d an e .c ~ ' ~ n case).
Figure 12. BASIC Bus Operation Flow Chart ( IDLE ~1 Irans~clion !grantlgrant_inh (ERR1 ~ ~ng ~p, " ~3 ~ , Er~

grant &!grant_inh IOTEST) \ Error ~1 (oPoST3 (POST2) ~--( INFO ) (OPOST~) ~ Error ~/
(ERR2) (POST1) 21 July 1995 39 '2,3~

.

Xbus Functional SpsrifirAffon Stratus ~ ,mpany Cor,fi~r.tial 6.6 A~l~ilrdlion The~us a ~itration ~ ~ ~~;1 ~ar-;~ on PQIO 1~ the key to enforcing the sv. .~A ~.~" ,i~dffon Qf the four buses in t~ ~ Polo a,chitt:..ture. Th~albiLdtion algorithr~maint ins t~ e ao~ea,~noe of one functional aJs. This is accomplished through a senes of point tc poin. request lines that are part of the contro buses. Figure 13 shows the inte,conne~,l for the arb tration network.
Figure 13. Inter-CPU Bus Arbitration Network bus req, grant inh ~

Slot O bus_req, grant_inh Slot 1 s ~

~bus req, grant inh PCI Bridge O PCI Bridge 1 Slot2 bus req, grant inh ~ Slot3 This arbitraffon network is a break from previous Stratus systems. Unlike the binary search arbitraffon ne~Nork used on Jetta and Stratabus based systems, Polo uses a distributed arbitration b sed on the request signals that each board drives.
As in Jetta, the determination of a bus grant depend~ on more than the request lines. Bus Grant can be inhibited by any board about to enter the data phase of a block transfer, or by an enor on the bus. Thus any arbitration phase where grant inhibit or a bus error is asse. I~d will not result in a grant for any board. The grant inhibit in~.eonr,e~ is also shown in Figure 13. Grant inhibit is also part of the control bus. A board driving grant inhibit will drive all three sets of its grant inhibit lines.
This gua-a-i~e es that all boards see the same grant inhibit and will make a conect arbitration decision.The gmt_inh signal is the ~--e.;l ,ar is. " used to guarantee that a block transfer will occur in wc~s~l~Je clock ticks. gmt_inh is also use as the ,,,e-;l,an;v,,, to enforce a dead cycle on the 21 July 1995 ~3 ~ 40 .

CA 022~7~11 1998-12-03 Xbus Funcffon~--F ~ c ~ t;- n . atus Company Co, If,derltial Xbus to tum the bus around. A bus master will always drive grnt_inh when it is acbvely driving the bus. This guaraulees that no device will be granted the Xbus the cycle after so",eone has driven info or data on the Xbus. A funcbonal ffming diagram which illustrates the use of gmt_inh is shown in figure 1~ High Level Block Transfer on page 45.
Table 6. Arbitration Prionty Rotation Inibal And Maintenance Interrupt Arbitration Value Table 6 shows the sequence of arbitrabon addlesses that a board will follow when rotabng it's address priority. Arbitraffon value zero has the highest priority and arbitration value 3 has the lowest priority. The sequence is such that adjac~nl boards are always assiyned partner add~sses and that all boards will have each priority level at one bme or another. For example in a four slot system where all boards rotate arbitrabon priority together every board will have the highest priority 114th of the bme the lowest priority 1/4th of the time and each priority in between for (a total of) 214ths of the time in every cycle of address rotation.
The system maint~nance interrupt signal is used to reset the rotating albillalion address on each board to the board's slot id. For exampte when a new board is inserted into the system all boards musthavetheira,Lib~banadd~essesresettobetheirslotadd~:sses sincethenewboardhasnomeans of knowing what its a, b it,dl;on address must be other than its own slot address. When the p(og,ession through the add~esses shown in Table 6 resumes all boards will conbnue to arbitrate with unique arbitration addlesses. Note that the asse,i ~n of ",ai"~nance interrupt has no effect on the actual a, l,ib dlion period II ,e,~ fo,e a node's priority at the beginning of the arb period may be different than at the end due to a main~"al1ce interrupt. I loJ~ er eYery node will always modify their arbitration priority either from a ",.A;.,lenance intenupt reset or an end or arb period e~eul s~nol"onously on the same clock edge. Since a ",ai"tenance intern~pt is issued when a board leaves reset it is guaranteed that no board will try to arbitrate before it's arbitrabon address has been sy,l~l"or,i~ed with the re",~.;nder of the system.
Note: There are other rotabon sequences that could be argued to be ~more fair" in that the priority will jump from a high priority to a low priority instead of pluyles~i, ,9 within a grouping of either high or low values but in an effort to maintain simplicity this rotabon scheme has been chosen.
The duplicate slot id signals on each board are latched at reset inside each ASIC and verified that they are idenhcal. A n,;.,co" ~fe between the two signals will break the board while it is in reset.
The actual arbitrabon address may be implemented with a simple two bit up-counter that is XORed with the slot id. This counter will i.,..,t:"~nl at the end of each arbitrabon period on the bus. An a,bibalion period is the hme when at least one board has a bus_req held active. A board can only win the bus once during any arbitration period. To clarify: some number of boards are arbitrating for the bus with one board winning. The losing boards re-arbitrate with one of those winning. This process repeats itself until all boards win arbitrabon. During this whole time at least one bus_req is asse, tt:d. Any board that wins a, b;t~ ation cannot arbitrate again unbl all bus_req signals have been de-asse, led. Any boards that haven't arbitrated during this period may join the remaining t~oards. If all bus_req signals have been negated then all boards in the system haYe had the 21 July 1g9~ z,~, ~ 41 .. . . .... . .. .

CA 022575ll l998-l2-03 Xbus Functional Specifl~bon Stratus mpany Confidential chance to be bus master once during that arbitraffon period. The arbitration period is now over and arbitration priority rotates.
A board always a~;~a~ -s against the Omer three boards in the system. If the CPU boards are duplexed, the duplexed pair arbitrates simultaneously. If either of the CPU boards wins the arb, both act as though they have won it, and both drive their info on the bus, This means that the duplexed CPUs will win half roughly of the arb cycles, and each PCI8 will win one quarter of the arb cycles. When the CPUs are simplexed, each board will win roughly one quarter of the cycles.
During a bus error operation, as des-.-iLed in section 6.4, Bus Errors, on page 30, all nodes will de-assert their request and grant inhibit control signals. This will insure that the priority remains in tact during Xbus error r~ e y.
Figure 14 illustrates a case where arb_req prevents a board from re-arbitrating until each board that is requesting the bus gets a chance to be the bus master. Assume here that there are only three boards in this system. In this case board t has the highest priority, board 2 has the next highest, and board 3 has the lowest priority. All three boards begin to request the bus in the same cycle (atthough it should be noted that this is not a requirement for this type of situation to occur).
Here board 1 wins the a- I,i~ . t on and its access is BUSYed, thus it must r~arbitrate for the bus but cannot unbl the current arbitration period has ended. Notice that at least one bus_req is held active until Board3 has been granted the bus, which gua,ant~es that ~oard3 will get a chance to be bwl"a~ler before any other board retums to arbitrate.

21 Ju~y 1995 ~ 3~ 42 Xbus ~unctions. ~oecincation~atus Company Confidential Figure 14. A,l,itldlion After Being Busied Grant_inhibit Board1 req ~\ ' ! ~ , , , ' , Board2req _~
Board3req Busy ' ' , ~\
' Board1 winsarbitration, Board , ' , Board 1 may nowre-, ~ 2and3continuetoarbitrate. ' , ' arbitrateforthebus.

Board 1 e Board2 winsarbitration, Board3 l continues to arbitrate.

Board 2 ', , , ', , ~ ', , , ~2 Board~ins arbltr~ori l and .~,leases arb_req.
Board 3 ' e ~' Note that there is atways a dead cycle ~t~:cen two different boards' info cycles. It is neces~a-y to allow one cycle to turn the bus around to avoid unwanted bus fight.

6.6.1 Bus Mast~rship Upon Bus Grant If a pair of duplexed boards arbitrate for the Xbus then when the higher priority peer wins the grant, both boards will assume Ill~s~el .llip ot the Xbus. I lo.~e~er, only the higher priority board will actually drive the Xbus because only the Xbus lines driven by the board winning the grant are watched. This is due to the fact that a bus reoeiver cannot know whether or not the requesters are duplexed so it will not know if valid data will be present on the two buses sourced by the non-winning board. Therefore these two buses will be left tristated.
6.6.2 Special Arbitration Rules for Non Pai.~:J Read Retums A duplexed board whose partner gets a non-paired register read request must go through the motions of retuming the data in order to remain lock s~~p~ I lol~e~er the non-target duplexed board in a non-paired read retum must not arbitrate for the bus and instead wait for its partner to win the bus. According to Xbus only the buses al~cl~ed to a board that won an arbitration phase are looked at. Only the target board of a non-paired access will drive bus request in order to guarantee that the correct data is looked at for a non-paired read retum. The non-target board will 21 ~uly 1995 ~3l~ 43 ........

CA 022575ll l998-l2-03 Xbus Functional Spe.ii~r~tion Stratus _~mpany Confidential continue to go through the motions on the read retum but it will not drive ~e bus since only the board winning arbitration needs to drive.

21 July 1995 44 W O 97/46941 PCTrUS97/09781 Xbus Funcbon ~ c~on lratus CompanyConfidential 6.7 BlockTransf~rs The Xbus perrnits boards to hold bus .-ldste,ship for a block transfer via the grant inhibit (gmt_lnh) signal. This does not allow tor multiple block lldl lafela. A board wanbng to perform a block transfer asserts the gmt_lnh signal the arbitration cycle after it first wins arbitrabon just as if it was doing a single cycle transfer. Then the signal is activated for as many additional arbitrabon phases as needed for the block transfer. This signal gives the board highest priority regardless of other arbitrabon activity, thereby perrnitting it to retain bus Illaslc~ ihip for addibonal bdnsl 71a. Note that the ~mt_inh signal is used for all cycles, not just block t dn:~fal~.
Figure 15 below provides a high level conceptual diagram of a block transfer which retains bus rr~stershlp for 4 bus operations.
Figure 15. High I evel Block T,ans~er FlrstOperabon ~ ' of Block ransfer ~ ~ , , Grant Inhibit ' v~ ' , of i310ck ~ransfer , } ~
, ,~ l , , , i ' , TBjrd kOTpe dtifon of Block ~ransfer I / ~=~

First Operaffon /, ¦ , ~j~j~
of NextTrans, - ~ n , / /, Gmt_in 1 (a~s~-t~d by the block tran r master) prevents other devices from winning the s~ hseguent arb phases.

6.7.1 ~3us Errors The Xbus block transfer enor recovery ~ - ~echdn; .. ~, is architected so that thie same ~, ~cl-an;~.., . is applicab e to block ~al Isfel ;~ of any length. The burden of enor recovery iâ placed on the iniffator of the transfer. All bus nodes will be .es~onsi~ie for de-asserffng their request and grant inlrilbit control lines to insure that the arbitration priority remains in tact during the Xbus error Opts~dl~ion.
Figure 16 below illustrates whiat thiis means for a bus error during the first operation of a four opel ~ ~n block transfer.
~ the master must ruhc~.,s.uit any data ~cso~iated with the block transfer maintaining conect order 21 ~uly 1995 ~ ~ 45 PCTrUS97/09781 Xbus Functional Specificauon Stratus ~ .npany Cor~iden~al ~ The grt_inh signal must be reactivated in time to keep the entire block transfer contiguous As ~ cussed later in this section, there are also situabons wherein a bus error causes a block transfer to be canceled, requiring it to later be res,~,t~d from scratch, i.e. requiring it to It:ali~Jit~dt~
for the bus.
Figure 16. Error During First Operation of a Quad Block Transfer ~ ? ~

Operatio~n~ ~ ' Q '~~ ~' '<~ ~K2~C~2X

transfer , ' , ~j ~ ' ' ' '~s~nX
~\' '' ' '' ' ~
Gr~nt Inhibit ' I I I ~ 'Gran~.lnhibit (The enor phases are shown comlensecl to ft on the page) Similar,y, errors during the second, third, and fourth ope,~t;on for a quad block transfer are illustrated in the figures that follow.
Figure 17. Error During Second Operation of a Quad Block Transfer '_~ ~, , , , I , , I I ' , os~d~ons ~ 3~ " o O ' o?~
block , , ,C2 transter I I ~ I I ~ I I I I I ' ,' I I ' ', ,' ~'~' ' ~
~ I ' Grant In~ibit I I ' I l ' ' I Grlant Inhibit ' 21 July 19g5 46 ~~3~

_ . .

CA 022~7~11 1998-12-03 Xbus Functiona~ ~ecification ~ .tus Company Confidential Figure 18. Error During Third Operation of a Quad Block T,~nsfer ~' I
'~i O~laliOnS I ~2~ ' ' ' ' ' ' ' with the~
transfer I ' I ' t ~'' i i ', ~' l ' ~i Ope,ration ' ~ ' Gr~ntlnh¦bit , , , , I ~ , I ' , Gra~tlnhibit not assoc ' ' , ' , '~
with the block , , ' I ' lThisoperationgetsca~ led~
transfer I ' ! ' ' ! must rearb due to the bus error Figure 19. Error During Fourth Operation of a Quad Block Transfer ~ ' 1I

Pd with the~ 3~F~2 block , , , , /,\ ,~/ , I ~OI O, , _, 5y, transfer ~

! ' ! !, ' ?

block ' l ' I ' , l~;;~ ,, , , , transfer I ' I ' I ' ,\ , , ' ~ ' I , , These Op~ef~ 5 must , rearbduetothebuserror The four cases in the preceding diay,a",s illustrated error recovery when an information cycle a~sGc:d~,d with a particular block transfer is ERRORed. Should a bus error occur which causes all the ~CC~s.5es ~~ d with a block transfer to be aborted. that bîock transfer is effe~,tiJ_ly can ~ 1 The initiator must, ea, ~ib d~ for the bus and start from scratch. The net effect is that at any point in time, only one block transfer is using the error recovery p,otc ~ ~ ' 21 July 19~5 ~ 3~

Xbus Functional Specificaffon Stratus ~ mpany CGnfi~en~al This is illustrated in figure 20 below. Two 2-operabon block tt~n~f~., are shown. The second infommaffon transfer of the first transfer gets ERRORed. This block transfer still completes using the recovery ~,-e-;l-an;.,.--s just ~1iscu~sed All cydes ~soc-c~-~d with the second block transfer on the other hand are completely aborted as a result ot the error. This block transfer is then completely canc o le d and its initiator at some later bme must rearbitrate for the bus and restart the transfer. t lad the second block transfer been long enough to have grant inhibit assetled at the time the first block transfer is ERRORed, it d~asse.t~ ~rt_inh upon seeing the bus ERROR.
Figure 20. Error causing entire block lr~nsfer to be canceled -firsts~pe,~ bo~ blo -k ~an ~,fer com~r tt Secofr d ~,-jopier,~tior~lbloclk canee.~ bythRernr:n~ I ~ ~ , , , , scrax l ~

Grant Inhibit 6.7.2 GlobalWrites Global write operations ars a variabon of pser-to-peer transactions. During the echo bal ,saction of a global write, the PCIB clocks in data at the same it is sending data to the CPU. In this way, all boards in the system see the global write s~,n., hfOIIOUSIy. It also means that only a CPU can issue global writes, 6.7.3 Board BreaWng It is possible for a simplexed board to be perfomming a block transfer and then break part of the way through the transfer, resulting in a never to be completed partial transfer. For this reason, all boards targeted by a block transfer must watch the TRlDs of the entire block transfer. Should the TRID change before a block transfer was s~ Ipp~sed to have co,n, 1~ ~1, the bus ASIC needs to pad the block with 0's to replace the misslng data.
Though padding the block with 0's to replace the missing data may at first appear undesi~al~le (no data is better than bad data), a few factors must be kept in mind:
1) This is only an issue for simplex l/O souroes such as disk and eU7emet. The so~h.dle for these boards musttolerate the board ~ pe~ir,-y~ and hence must have recovery ",e.;l,an;s."s for incomplete blocks (0's in the memory are no worse than whatever random information would have been there prior to being over~ ~n by the l/O board).
2) The Stratabus based systems exhibU the same characteristic.
3) To not do this has a potential pe"o.-.,al7ce impac2: bus ASlCs may have to vU,e~v;-~e hold the entire block prior to passing onto the Doard rather than passing good data as it iS received.

2~.)u~y~9~75 ~3 ~ 48 Xbus Functiona. ,Jecification ~ ltUS Company Cor,~ide,l ~' 4) This situaUon should only occur on CPU initiated ~Idll~f~,~. On the CPU, an LPMC is retumed cDincidel It with the read data.
Individual bus interface designs may choose to provide a notifi~d~ion lllkchan;~ll should this occur.
It should be noted that the breaking board will drive a double bus error when it breaks. In the case desctibed above, the errored information should get re-transmitted, but never will because the board has broken.
6.7.4 Bus Busys A block transfer may only be BUSYed during the first phase of the transfer, i.e. BUSY may be as~e, t~d during the Post2 phase a~socia~Pd with the first i, l' I "aLon transfer. This single BUSY
has the effect of canceling the enUre block transfer. In practice, it is too late to prohibit the transfer of the s~hse~ ent sub-ope-aliuns on the Xbus so the i"lurl"alion in these sub-operations is simply ignored. The transfer iniUator must re-arbitrate for the bus and repeat the block transfer.
Note that bus error takes p,ecedence over busy. The only reason a s~ ~bse~l)ent cycle will ever have BUSY asse, t~d is if a board misi"lel ~J, utcd the infommaUon within the cycle or is about to break. In both these cases, ERROR will be asse, It:d concurrently with BUSY and should take p~ecedence.
6.8 PeertoPeerl-d--~a_lions Peer to peer oansactions t : - ec,1 the two CPUs are based on a simple store and forward protocol. A CPU sends a transaction on any of its two buses attached to a non-broken PCIB. Once any enors have been f~sol~e~ (the PosP phase of the last operabon has passed without error), the PClB(s) echo the transacUon back up to the CPUs. The send and echo l,ansaction are not split; the driving CPU holds grant_inhibit until the CGIIl, ' tc send-echo pair is complete. This is to prevent re-ordering p.~1e ~-s arising from eviction/flush queue (EFQ) and ~ead/~ ,ite queue (RWQ) ope,alions passing each other. Read bansa~tions are split be~een the request and the data retum. An i~ldi~ ;' le send-echo carries the read request. Other ope,dliol1s may intervene while the read is pel~ù-,lled, then the data is retumed in an indivisible send-echo data retum. The CPU initiator of the peer-to-peer ope,c,~ol,s must track the entire ope-dlion to see whether the cycle is achl I ~wledged, busied, or enored.
Ach..o,~rled5~e and Busy are ignored for the first half (send portion) of a peer-to-peer h~nsa..tion. A
busy .~;sponse on the echo will terminate the bd-~sa ~ and it will be retried at a later time beginning with the send portion. The send cannot be busied. The echoing board needs to track the enor line during the echo so that an error on the echo portion will cause the normal seven phase enor protocol l ~ .. cd by a re-echo of the l- al1sa~ion starting from the first phase that was errored and then proceeding forward. The send portion is not ,~epeated in the event of an echo enor.
The initiating CPU needs to keep track of whether a cycle will be peer-to-peer or not and indicate it on the Xbus via bit 4 of the second half TRID. Table 7 lists the Xbus ope,_tk ~s that the CPU will mark as peer-to-peer by using bit 4 of the second half TRID. The cycles listed under non peer-to-peer ope,_~ s will not be echoed. At a minimum the peer-to-peer initiator must drive grant inhibit from the first info phase of the send until one phase prior to the first info phase of the echo. The board performing the echo must handle grant inhibit from the first phase of the echo until the completion of the cycle. The PCIB will only echo opefi,t;~,ns marked as peer-to-peer via the TRID

21 July 1995 49 ~( O

...... .

PCT~US97/09781 Xbus Functional Specification Stratus ,mpany Confidenbal blt. The peer-to-peer bit in the TRID is only set for the send, the echo sets it back to zero since a Table 7. Peer to Peer and Non Peer-t~Peer Operations Peer-to-Peer Operations Peer-to-Peer Operations peer-to-peer mode~ ~non p6 peer-to-peeroU,en,~
non paired llO ope,~ons to all load clear ope~atiol)s non paired llO operaffons to slot 0 or slot 1 slot 2 or slot 3 all memory reads and all special cache line paired l/O ope, dlions to slot 2 memory wr,ites operations or slot 3 all global llO operaffons paired l/O operations to slot 0 data retums to slot 2 or slot 3 or slot 1 all global CPU b o~l~sl l/O PlNGs resulbng from non operabons peer-to-peer l/O reads.
data retums to slot 0 or slot 1 PlNGs resulbng from peer-to-peer l/O reads.
a. bit 22 of thc Bus l~tcrface State Register cyde with this bit set will not be lesponded to.
Only CPU's can issue peer to peer transacbons, ber~ ~se only PClBs have the bufYering to perform a send and echo. PCI peer to peer bansa~ions are not supported, and all traffic betv:san PCI cards must occur via main memory.
Note; the l~"~w;~-g diag-.""s show A0, B0, A1, B1 info buses, not an i.ba' ed overlaying of ~p~ ~ns on a single bus (as in the preceding~.

21 ~uly 1995 ~ 1 50 Xbus Functional ~cif.~tion ~ ~us Company Confidenbal Figure 21. Peer to Peer Cycle Initiated by a Simplexed CPU
Errors on send ~ ~ I Error on echo would appear here j j j would appear here ' ,re~r~er-r~err3~e~4\ ' i i ,~e~ 'err2,~e'rr~'e'rr~
AO~

Boj~

A1, ' , ' ' ' j , ~ , gmt ~\W/~ ' ' ' ' ' _inh j j j CP.J drh~eS g~,nt_ir~ j ' j j j ' ~ ' i gmt i i ' ' i ' ' ' '/~\
_inh, , , j j j j , ~CIB,drive,s grr~_inhj Figure 22. Peer to Peer Non-paired Read from a Duplexed CPU
~Non-pairedreadof requestis , split , CPUsarbin j PClBs CPU1sentfrom j 'lechoed 'j t~nsc.ction j unison;CPU1 , echodata iduplexedCPUs ~ drivesdata j retum AO~ , ~3 lI ~ I j BoZ~3 ' ~ ' '' 11 ' ' grnt_in~
~,PUjdriv~s grr~t_inh~ PU ~rive$ gm~_inh grrllt_inh ~ , ~ '~ ' ' ' , ll ' ' ~ ~ ' ~ ' '/~
PCIB drives gmt_inh PCIB drives gmt_inh 6.8.1 Bus Errors A bus error during the send portion of a peer-to-peer cycle is handle identically to any other cyde.
Figure 23 illustrates this. A bus enor in the echo portion will result in eYecutinn of the error sequence toHowed by the l~lll ~der of the echo ~ans",,ssion.

21 July 1g95 ~~t '2 51 CA 022~7~11 Isss-l2-o~
WO 97t46941 XbusFunctionalSpecific~ion Stratus npanyCvn~iclen ~' Figure 23. Store and ~orward with Error on 2nd Operation of Quad Biock Transfer Error protocol driving 55/AAs - j se~d ~ ' ' rer~ain~er of jsend' , ' ech~

Ao~3~) j 'j !
l l l l l l l l l l l l l l l Bo~3j ji~) i i i~
l l l l l l l l l l l l l l l l l Ati ~ ! ~ ~ i ~; i, i i i i ~
l l l l l l l l l l l l l l l l l l l l l l l l ~ ", , , , , , , ~, ,~
B1 i j j j j j _ _ J; j j , i i i , i g~/',~y~j/\ i' 'i i ' ' j' ; CpUd¢ives~mt Inh grr~t_inll ' ' ' ' 'j '; i ' " i ' ' ' ' j PCI~ dri~es g~ht_inh 21 ~iuly 1995 52 'Zk~
.. . ..

Xbus Functio\ ~pe~ ~a~on ~ratus Company Confidential 6.9 Major T.~.,sa.,tions There are three types of transactions that can be pe,fo-",ed on the Xbus, (non-split) memory writes, (split) memory reads and (split) flushlpurge type transactions.
Non-split transactions are when the bus master does not require any data or .t"ponae from the slave device to complete the hdnssction. These bdnsa..tions supply an address in the first operation and the data in the sU~sertuerlt sub~pe,dlion(s). All data write transactions are non-split and use this protocol.
A split transaction is 'split' into two sel~a, dte bus ope, ~t ~ ns, a request I 'lowed by the ~ espùnse. A
block read is a good example of this. The master initiates this banasction by a read request for X
bits of data for a given address (the function code, virtual and physir,al address are all s~ ,~ F'ie d in the info phase). The slave who has dec~ded the function code and address then pelfi,,.lls the request on-board. When it gets the data, it retums the data via another bus ope. t~ion. A
transaction id (TRID) is used to match requests wi~ les~nses and is unique to all transactions pending.
Flush and purge b al ~s a ~ns are split. They are unique only bec~l~e they do not need to see a bus op~.dtion retuming data to complete the bdnsa..~on. Any flush or purge transmitted from Cougar to Cyclops is transmitted on the Xbus. It is normally just looped back on the Xbus.
~lo..~evcr, if the board is in update mode, the b.lnsa~,Lon is pe(~o,---ed as a peer-to-peer l.ar.sac~ ~n. They will always see an INFO RETURN FUNC_OP (or INFO RETURN 256 when retuming data) as the retum on this bd.. a: ~ 1. The virtual index and physical address is included in this type of retum.
Notes:
~ The figures in this section contain a holi~ulltdl line that indicates either a tri-sbte or don't care condition.
~ All signals are drawn active high so they are 'Hl' when signals are in their TRUE state.
~ bus_req and gmt_inh are driven on both cycles of the ARB phase. Note they are clocked in on the first half of the ARB phase and evaluated tor the grant during the second half of the ARB
phase.
~ When VADR is seen in these figures, it actually rt:p~sen~ the tunction code and virtual index.
See section 6.10.2 on page 63 more details and bit defini'ians.
~ When the note 'OPIN' appears on the TRID signals, it ~ep,esent~ operation infommation and transaction defined inforrnation that will be passed with the bus ope-dlion. see section 6.10.1 on page 62 for more details and bit definitions.
~ The Xbus control bus bits have been spilt out for readability. For example, BUS_REQ and ACK are drawn as two separate signals while in reality they are both on control[0]. Ukewise BUSY, ERROR, and GRANT_INH are broken out from the control bus (maint_int is not shown in these ~lias~ldllla).

6.9.1 Write Transactions A write transaction is completed in one bus ope~ dtion with from one to four su~operations for the data transfer. The infflal bus op~, a~ion supplies the function code, the virtual address. and the physical address in the info phase. The acco"",d,.ying sub operation~s) are used to supply the dataneededforthebdr,sa ~icn.
Figure 24 Illustrates a 32 bit write transaction. Note it is up to the bus master to assert GRANT_INH (after recc ;~;. -g a grant) to reserve the info phases for the data llan ~fel (s).

21 ~u~y l995 ~ 53 wo 97/46941 Xbus Functional Specification Stratus .npany Confidenbal Figure 24. 32 Bit Write Transaction SUB-OPERA;TlON IDLEARB_INH INFO POST1 POST2 IDLE IDLE IDUE
SUB-OPERA;TION IDLEIDLEIDLE IDLE IDUE IDLE IDLF IDLE
SUB-OPERA:TION IDLEIDLEIDUE IDLE IDLE IDLE IDLE IDLE
SUB-OPERAnONIDLE IDLEIDLEIDLE IDLE IDLE IDLE IDUE
IINBIBIT ~
GRANT_INH ~ INIIIBIS INIIIBIT
BUS_REO --~>
FUNC_OP
.--TRID,~.00] ~ ss~J
INFO,~001 ;~;
susr ERROR
ACK
~Ses notes on page 53.
A block write ~ s~ion is much iike a 32 bit write operabon. The only ~lif~ .S:nce is that the master must assert GRNT_INH for additional phases to reserve the extra info phases needed to transfer the extra data. Figure 25 illustrates a block wTite t~ ~, ,sa~ ic n.
Figure 25. 256 Bit Block Write Transaction OPERAnON STATE ARB INFO POSTl POST2IDLEIDLE IDLE IDLE IDLE
SUB OPERA:llON IDUEARB_INHINFO POST1POSt2 IDLE IDLE IDLE IDLE
SUB'OPERATION IDUE IDUEARB_INHINFOPOST1POST2 IDLE IDUE IDLE
SUB~OPERAnONIDUEIDLE IDUEARB_INHINFOPOSTlPOST2 IDUE IDUE
SU~OPERA:llON IDUE IDLE IDLE IDLEARB_INH INFO POST1 POST2 IDLE
IINU~BIS INilIBIT INIIIBIS INNIBIT INlIIBIS
GRANT_INH
BUS_REt~ {~ >
FUNC_OP
TRID
INFO~1:31.0tll ~3 ~usr ~_6 ERROR ~
ACK /f~\
~See notes on page 53.

6.9.2 ReadTra..~.t;7..s A read transaction requires two bus operd~olls; a request ope, ~ ~n followed by the ,esponse operation. These two bus ope~ ns will always have the same TRID. The request ope- ,lion does not have any accom~anying sub-operations but the rasponse operabons will require su~
21 July 1995 2 ~ S 54 Xbus Function~ ,peri~ ~ion atus Company CGn~ide"'i~' opelaffcns if the amount ot data reqller~~d cannot be bansfellèd in one info phase. Figure 26 illustrates a basic read transaction with no sub cFe t ~ns.
Figure 26. 32 Bit Read Transaction OPERA;TION STATE ARB INFO POST1 POST2 IDLE
SUBOPERATION IDLE IDLE IDLE IDLE IDLE
GRANT_INH IS~IIIIBIS SN SIJIS
BUS_REQ {~3 FUNC_OP
TRID~ 00]
INFO,r:31:00 BUSY
ERROR ~, ACK ~CI
'See notes on page 53.
The slave then re~ponds with the data as follows:
Figure 27. Slave Response to a 32 Bit Read SUB-OPERATION IDLE- IDLEIDLE IDLE IDLE IDLE
SUB-OPERATION IDLE IDLEIDLE IDLE IDU- IDLE
SUB-OPERATION IDLE IDLEIDLE IDLE IDLE IDLE
SUB-OPERATION IDLE IDLEIDLE IDLE IDLE IDLE
GMNT_INHIINIISIIIS SN SIIST
BUS_REQ --~3 FUNC_OP ~_ TRID106:00] G~;
INFO,~ 00]
BUSY
ERROR ~, ACK
~See notes on page 53.
Block read b al)~..Lon~ are ll ~r,~a ~s that requires su~operabons on the data retum operation.
Figure 28 illustrates this type of transacffon.

21 July 1995 ~ ~ ~, 55 CA 022~7~ l l 1998 - 12 - 03 Xbus Functional ~ ?tion StTatus _.,?mpany Confidential Figure 28. 256 Bit Read Transaction OPERAnON STATE ARB INFO POST1 POST2 IDLE
SUB-OPER~TION IOLE IDLE IDLE IDLE IDLE
SUB OPEMTION IDLE IDLE IDLE IDLE IDLE
SU8-oPERATlON IDLE IDLE IDLE IDLE IDLE
SUB-OPERATION IDLE IDLE IDLE IDLE IDLE
GRANT_INH IINIIIII~T INHII~IT
BUS h. .? _~
FUNC_OP
TRlDlOt?:OO~
?'' INFOt31:00J ~30 BUSY
ERROR \~f ACK
~See notes on page 63.
The slave then ~spc-?rld~ with the i~''D~.;.I9.
Figure 29. Slave Response to a 256 Bit P~ead Transaction OPEMnON STATE ARB INFO POST1POST2IDLEIDLE IDLE IDLE
SUB-OPEM:tlON IDLEARB_INHINFOPOST1POST2 IDLE IDLE IDLE
SUB OPERAnON IDLE IDLEAR8_1NHINFOPOST1 POST2 IDLE IDLE
SU8 0PEMnON IDLE IDLE IOLEAR8_1NH INFO POST1 POST2 IDLE
SU8-OPERA;llON IDLE IDLE IDLE IDLEIDLEIDLE IDLE IDLE
GR~NT_INH IINllI?3I'r ~ dl ~T ~ 'IT
8US_REO {~3 FUNC_OP ~_ 'IRID106:00]

Susy ~
ERROR
ACK
~See notes on page 53.

6.9.3 Flush and Purge Transact~ons Flush and purge operations look just like read llal lia Yc ,s with the e~ J~;on that the response may not have data to retum. The retum for these operations will always have FUNC_OP true with an INFORMATION RETURN (or INFORMATION RETURN ~56) tuncbon code. The INFORMATION RETURN will have atleast one bit of b~",6a~Yan defined i,.fo,."dtion to pass back to the requesting device~ The-inforrnation that is passed here will be hitlmiss i"'~ ",alion on the other board's cache. This informabon is passed during the second cycle of the info phase and uses the tridt6:0l bus lines (OPIN) (see section 6.t 0.1 on page 62). If returning data, it will use an INFORMATION RETURN 256 function code. Note that the virtual index and physical address are 21 ~luly 1995 2 '- ~ 56 XbusFunctional ~cifiodtion ' usCompanyConfidential always passed back on the retum operation for these har,sa tic ~s.
figure 30, Flush with Modified Data Tlansac~ion on page 57 and figure 31, Slave Re~uonse to a Flush wffl Mcdifi~d Data Tl~ a L~n on pa~e 57 are examples of a flush op~,d~ion with a mod-data return. Note this retum will always be a c;ache line and that the retum looks just like a cache-line write operaffon in that it has FUNC_OP true and supplies the virtual and physical address.
Figure 30. Flush with Modified Data Transaction OPERATION STATE ARB INFO POSTl POST2IDLE
SUB-OPERAnON IDLE IDUE IDLE IDLEIDLE
SUB-OPERAnON IDLE IDLE IDLE IDLEID E
SUB-OPERA:llON IDLE IDLE IDLE IDLEIDLE
SUB OPERAllON IDLE IDLE IDLE IDLEIDLE

GRANT_INH
BUS_REQ ~3 FUNC_OP
TRID~06:W] O_ INF0131:00]
BUSY ,~
ERROR
ACK
~See notes on page 53.
The slave then ~espor,Js with the following. Note the state of the FUNC_OP signal.
Figure 31. Slave Response to a Flush with Modified Data Transaction OPERATION STATEARB INFO POST1 POST2 IDLE IDLE IDLE IDL_ IDLE
SUB-OPERATIONIDLEARB_INH INFO pOST1POST2 IDLE IDLE IDLE ID~ F
SUI~OPERAnONIDLEIDLEARB_INHINFO POST1ROST2 IDLE IDLE IDLE
SUB-OPERAnONIDLE IDLE IDLEARB_INHINFOPOST1POST2 IDLE IDLE
SUB~OPERAnONIDLEIDLE IDLE IDLE ARB_INH INFO POST1 POST2 IDLE
GRANT_INH II1111585T ~85T
BUS_REO {~>
FUNC_OP --~
TRIDr,1)6:00] ~_ I-mlllN 15~
INF0131:00 BUSY
ERROR
ACK ~j~
~See notes on page 53.

Figure 32 illusSTates a flush transaction with no modified data to retum. Note that the retum for this transaction uses the info retum FUNC_OP as a f~sponse. The TLB purge, data purge and instruction flush transactions look exactly like this transaction.

21 July 1995 ,z ~ 57 CA 022~7~11 1998-12-03 Wo 97t4C941 Xbus Functional Specifica~ion Stratus ,npany Confidential Figure 32. Flush (no modified data) or Pur~e Transaction SUB~OPERATION IDLE IDLE IDLE IDLE IDLE
SUB-OPERATION IDLE IDLE IDLE IDLE IDLE
SUBOPERATION IDLE IDLE IDLE IDLE IDLE
- SU~OPERATION IDLE IDLE IDLE IDLE IDLE
~INITII~TT ~
GR,~NT_INH IINIIT~IT~
BUS_F~EO ~3 FUNC_oP ~~
TRlD[06:00j INFO~ 00] ~S~
BUSY iJ
ERROR ~r ACK
~See notes on page 53.
The slave then l~a,~lOIl~ with the ICY~;n9~
Figure 33. Slave Response to a ~lush (no modified data) or Purge Transaction OPEMTION STATE ARBINFO POS~l POST2 IDLE IDLE
SUB~OPEMTWN IDLEIDLE IOLE IDLE IDLE IDLE
SUB OPERATION IDI F IDLE IDLE IDLE IDLE IDLE
SUB OPEMTION IDLEl~LE IDLE IDLE IDLE IDLE
SUB-OPERATION IDLEIDLE IDLE IDLE IDLE IDLE
GR~NT_IN~ I~N~ T
~US_REO ~3 FUNC_OP
~~
INFO,r:11:00 BUSY
ERROR
ACK
~See notes on page 53.
6.9.4 LoadlClearT~ io,~s Load/Clear Xbus llai Sa~bOh5 are hardware supported lock operations. A lock is won by performing a 32-bit load ~nsacbon in which the data received is non-zero. A zero result indicates that the lock is not won. A lock is cleared by issuing a clear transaction or by simply writing non-zero data into the rnemory lock location.

Load/clear ~a~ sa 'i~ns look exactly like a 32 bit read transaction. The only diflerence other than the function code is that the byte enables are not used. For more inforrnation on load/clear tra" ~ s, refer to the Mercury Functional Sper,ifi~stion.
21 ~uly 1995 ~ 58 CA 022~7~11 1998-12-03 Wo 97/46941 PCT/US97/0s781 Xbus Funcffona- ~pecitlcation .ratus Company Co,Aidel,tial 6.9.5 PingTral.saction A PING transaction is a single bus operation ba"sa~L n. Only the TRID is needed in this l,ansac~on. The only reason a PING 0dnsadiùn is done is to see if so",eone drives the ACK
signal in POST2 of the bus operaffon.
6.9.5.1 The Need for PlNGs Re~ lce the Xbus uses split transactions, there is a need for the ability to handle cases where simplexed boards break and take U.e..l~.el~es off-line while there is still a ballsa~,Lon pending on them. This can result in a device waiffng forever for a TRID retum that will never come. This condition cannot occur on write operaffons bec~se they are not spnt. See section 6.7.3 on page 48 for infonnabon on boards breaWng during block transfers.
The system will crash when an on-line simpexed CPU/...~...ofy breaks so there is no concem with transactions that can only be directed to on-line CPUh-,e,-.ùry boards. When CPU boards are off-line tesbng, there is a need to do l/O reads to them. This is a case where the tesffng CPWI,~"-uly can break and not cause a system crash but sUII leave transacffons incomplete.
I/O boards can be simplexed while on-line or off-line and can break at any bme causing the same conditions. The only split transaction directed to l/O boards are l/O reads.
Therefore, the list of l,a"sa ~ ~s that has the potenUal of causing this condition is limited to l/O
reads. I/O reads will require a test operaffon (called a PING) to gain inforrnaUon on whether or not a given TRID is still being worked on, i.e. if a board has broken or not.
6.9.5.2 Detailed Rules During PING Operations.
Aner the transacbon master sends an IIO read operabon it will start a bmer. If the timer times-out before the TRID is retumed, the transaction master will send a PING ope,d~ion. This PING
operaffon only contains a valid TRID (the info lines must be determinisUc but are conside~d a don't care). The transaction slave device that is cunently active working on this TRID will ACK the PING operation. The transaction master will then set the timer again and conbnue waiting for the TP~ID retum. If the PING is not ACKed, the transaction master will drop the lldl ,sa..~on and assume the transaction has failed and/or the board has broken.
The slave devioe must keep a copy of all the l/O read TRlDs outstanding on the board. This is so that it may ~Il~Ja~t: the TRID of all PING operations and ACK the PING when the TRID "~ih;lles.
~ The transaction master must ignore results of PlNGs if the matching TRID was marked comp~ete. This can occur if the PING and TRID retum were both on the bus at the same time (or ff due to BUSYs that passed each other on the bus) ~ The transaction master should marh TRlDs complete on the phase after POST2 and no-error of the TRID retum. If the TRID was compete at the exact instant the timer ffmeo-out, the PING
~p~ _ticn can be sent, but if it is sent, the results of the PING must be ignored.
~ The b a, s~ th 1 master must not start any additional bus transacffons on the same b an _ L
master if a PING transaction (operation) is pending. This is so that an old PING is not using the same TRID as a new transaction. This has the p!otential of occurring when a TRID retum was seen before the PING completed it's bus operabon.
~ The slave must marh a TRID ~.~ rhing- and stan ACKing PlNGs on the Xbus before the transacbon master timer times-out for the first bme. There is ample bme but the slave should do this as soon as possible. The cycle after POST2 and no-error of the request is the earlies~
this could be done.
21 July 1995 59 ~ S ~

Xbus Functional Specification Stratus ~,~,tnpany Ccrlfid6r,~1 ~ The slave should invalidate a 'working TRID' the phase after POST2 and no-erTor of a TRID
retum. It could be done later but this only adds complexity. Note that in theory there may be only tour phases bet -leen POST2 of the retum and POST2 of a new request with the same TRID.
Figure 34. PING Transaction GRANT_INH JINU~-~? IIIRII~S
AR8_REa {~3 FUNC_OP
TRID~ 00 INF0~ 00]
BUSY , ERROR
ACK
ThB t. 1( ~l~ wlll ~cKO,nOtAoKthbph~ ~See notes on page 53.
~.
n ~CK ~n tn~, th~ t r wlll ~b~d th pn~ htor~l oount r.
n ~CK w ~ , th~ t~ m~t r wlll bort th~ L nd tonnul~
~ r hlm d O ~ Wnh o blbd op Figure 35. PING Transactions, Boundary Condition 1 PING TRANsAcnoN ARB INFO POST1 POST2 IDLE IDLE IDLE IDLE
DATA RETURN IDLE ARB A~B INFO POST1 POST2 IOLE IDLE
GRANT INH ~~ ~~
-- IINulus INUIBIS :INIIIIIIS INIIIBIT
BUS_REO ~3~C33 FUNC_OP i TRIDt06:001 ~ ~;~
INFOr~ 00] ~OE ~X3 ERROR ~ RR, ACK

Tho L
ACK t]l~ ph~
TbB b m_t~r ~lll r b~d thB pln~ ht~l count r hor~
Th~ I ~ mo-t r ~lll dl~obb th pho h~l oounbr IL~ m-rk ~See notes on page 53.

21 July 1995 ~ ~ ) 60 Xbus Functional _ "ecification~ tus Company Confidential Figure 36. PING Trdnsa~1ions Boundary Condition 2.

PING TRANSACTnON IDLE ARB ARB INFO POST1 POST2 IDLE IDLE
DATA RETU~NIARB INFO POST1POST2 IDLE IDLE IDLE IDLE
GR~NT INH ~ ~
IIN8IBJT IN81BIS IINIIIB~TIN8IBJT
BUS_REO ~3 FU~IC_OP _ --TRID~;:001 ~;~ .-?

BUSY
ERROR
~CK

Th i J m~-t r '' compbb h~
T~l~ ~lll ACK thh pll~.
Tl ' ~I~ 1ll111 -l o hv lld~ th- TRID.
Thel L I~ t~ ~111~ thl~
L ~ compbb ~nd do notl~h~ Thb 1~ n n pln~ not ~CK-d.
~See notes on page 53.

6.9.6 busy, ack and error combi,-..ll~..s There are eight possible combinations of busy, erTor and ack on the xbus. The table below shows the outcome of the bus operation for the given combinations of busy, ack and error.
Table 8. busy ack and error co",binations.
error busy ack o~ ~ - n results o O O Non acked operation. Wr~tes are dropped on the floor. CPU
reads retum O and cause an LPMC, if enabled. PCIB reads cause the initiating PCI slot to go off-line.
O 0 1 Norrnal complete acked operaffon.
o 1 0 Busied ope.~i~n, retry.
o 1 1 Busiedoperation,retry.
0 0 ElTored op~,ation, follow error protocol.
0 Errored ope,~ion, follow error protocol.

0 1 ErTored ope,~ion, follow error protocol.
- Errored o~,dtion, follow error protocol.

21 ~uly 1995 z ~ ,~ 61 , ~ . .

PCTrUS97/09781 Xbus ~unctional Specifi~tion Stratus .npany Confidenbal 6.10 XbusSignal~r,.,..ls -6.10.1 trid~6:01 and func_op_ Bus Format A master that has won a,l,i~tion will drive these lines along with the Info lines. Below is a list of signals intemal to the ASIC that are multiplexed and ll~- ,a~r- ed on the trid and func_op_ Xbus signals.
trld[6:01 These are the transaction identification bits.
taihd_op This bit is valid during the second bus cycle of an info phase. It is only true during a peer-to-peer echo resulffng from a failed peer-to-peer send. The info for phases that were transmitted without error are sent normally. The info for the phases that were never sent is set to zero and the failed_op bit is set indicating that this data was never transmitted so it has been zero-filled.
first op This bit is only true during the first bus operation of a transfer of infomnation, i.e. it is only false during sub-operation and idle phases. Note it must be true during the first bus operation of a data retum.
func_op_ This bit when true signifies that the info lines carry a valid function code, address, etc. on this phase.
HiVMiss This bit ~~pr~se, l~ the hiUmiss status on special cache operaLions. It is zero on all other l-a,)sa,,l ~ ns.
IOVA This bit indicates that the address is an l/O Virtual Address and must be b ~n~ldt~d.
peer-to peer This signal indicates that ~e CPU bus master wants this current cycle to be driven peer-t~peer and it will control the grant-inhibit line to allow for it. This bit only has meaning when dnven by a CPU and should o~erwise be zero.
remain[2:0~ These bi'6 ~ ae, n a binary number that indicates how many bus su~
opera~ons remain to complete this transfer of informabon (at~er this one). This field is zero for ~ana~ that do not have su~ope,dlions.
The figure below d~sc-il,es the format of the TRID bits (tridt6:0]) and the func_op bit on ~e first cycle of any INFO phase.
Figure 37. tridl6:0] and func_op_ Format for the 1 st Cycle of an Info Phase func_op_ trid [6:0 ~ene.dted by_ ~ O O
umque on~board func_op_ T. .hS~ tion Master ~ ~
g_. e.dt~ by onlineloffline state for duplexable boards - slot_idtOl for non-d~ le boards 21Julyl995 z ~ l~ 62 W O 97/46g41 PCTrUS97109781 Xbus Functional ~ ~Jffon ' us Company Confidenbal Figure 38 shows how the unique TRID filed is gene(al~d on the PCIB board in order to be able to check the PCI slot field in the llO map RAM look-up (refer to Polo r~ug-a~ ing Guide, secbon 5.4.1, System Address Generabon, for details on this cl .ecl ing).
Figure 38. trid~6:4] Format for a PCIB for the 1 st Cycle of an Info Phase trid [6:4 PCI slot n.~ b. l side generating genefating transa_tion tr~ n slot O - 00 d side - O
slot 1 - 01 c side- 1 slot 2 - 10 slot 3 -11 The figure below des~"ibes the forrnat of the TRID bits (trid[6:0]) and the func_op bit on the second cycle of any Info phase. The unused bits should be set to zero.
Figure 39. trid[6:0] and func_op_ Forrnat for the 2nd cycle of an Info phase func_op_ ~ trid 16:0 07 1 06 1 0~ 1 04 1 03 1 02 1 01 1 oO
~ ~ ~ remaint2:0l _ failed op nu-sed ~ UMiss peef '~ ~eer f rst_op 6.10.2 Info Bus Format The figure below des~iL~s the forrnat of the 32 bit info bus (infol31:00]) during the INFO phase of a func~on code ope-dlion (FUNC_OP). This is an illustrabon of an operabon using a CPU address 21 Ju~ 1995 '~ S ~ 63 Xbus Func~onal SF~ ~ffQn Stratu~ ,ompany Confiden~al Figure 40. Info[31:0~ Format of CPU Address FUNC_OP
first cycle:
infol31:0~ ~

¦31¦3~ 26¦~5¦~ ?~¦?~1?1¦20¦19¦18¦17¦16¦15¦14~13¦12¦tl¦10¦09¦08¦07¦Q~IQ'¦~ 3¦02¦01¦00¦

vindex[1 9:0l func[5:01 rc~1:0] ~¦~
byte_enl3:0]
secG..J cycle:
intol31 :0]

¦31¦3C~ P<l;~tl;~6l~5l24l23l22l21l2ol19l18l17l16ll5l14l13l12l1ll1olo9lo8lu~ lQ:l~qln3lQ~ 1lool ph, ;cr' add-~s The figure below d~sc-ibes the format of the info bus for the INFO phase durtng a func~on code operabon (FUNC_OP) when using an IOVA (I/O Virtual Address). Note ~at Ule RC bit field is zero;
the C bit is stored in the map RAM and looked up during the IOVA to CPU address bdl. ' ~ n, the R bit is always O since there is no remote CPU board.
Figure 41. Infol31:0] Format o~ IOVA Address Bus FUNC_OP
first cycle:
intol31:0]

3~¦~J~ R~ 6¦;~5~ 1¦7~¦?~¦71¦20¦19¦18¦~7¦16¦15¦14¦131~2¦11¦10¦09¦08¦07¦~6¦~E¦~¦Q3¦02¦01 loo¦
oo~ ~ I loo~
chunk device index chectcsum page tuncl5:01 byte_enl3:0 second cycle:
info~31:0]

31¦~ q¦,~A¦,~¦26¦~5¦24¦23¦22¦21¦20¦19¦18¦17¦16¦15¦14¦13¦12¦11¦10¦09¦08¦07¦06¦05¦0~ 3¦~¦01¦00¦

,ese~ ~ed ~ ~ line offset 21 Juty 1995 ~ 64 WO 97/46941 PCT~US97/09781 Xbus Funcbona ,~l~cation ltUs Company Confidenbal The figure below descl iL,es the forrnat of the info bus on the first and second cyde of a of an INFO
phase when the FUNC_OP signal is false (i.e. data~. It is an example of an o~r~tion retumin~ 64 bits of data. If the opal~tion was retuming 32 bits of data the sarne data would be driven first and second cyde.
Figure 42. Info[31 :~l Format During Data Transfers first cycle:
intol31:0]

¦3~ A~ 6l~5~ l77l~1l2ol19l18l17l16l1sl14l13l12~ olo9lo8lo7lo~l~El~'4lQ3ln~l~1 data[31:0]

second cycle:
into[31 :0]

¦3t¦30~ ;26¦;~5¦24¦23~ 21¦20¦19l18~17~16l15l14l13l12l11l10l09l08l07~ C5l~ql~3ln~lo1lool dab~31 :0]
(next word) 6.10.2.1 F~u.,ction Code (rc and byte enable) T~ansfer the function code bits are used to des~ ibe the Xbus o,oera'i ~n andlor request. A funcbon code is not passed on a norrnal data retum operabon.
The function code bits are t,dnste-,ed on the tirst cycle of the Info phase. These bits are valid only when the FUNC_OP signal is true. tunc[5:0] use bus lines into[11 :B] ~e~ ly. Below is a list of the basic opelalions that can be pe,f,""ed on the Xbus. These are defined within tuncl5:33.
Note that tunc[2:0] are re-defined for each basic operaffon however, the meaning of these bits were kept cons,~ten~ where possible.
Note that table 9 through table 14 are subsets ot the Golfbus tunction code detinitions.
32 bit read operations hnc[5:3] = 000 Included in this group are all the read operaffons/requests that only require a single 32 bit retum.

21July1995 ~S~ 65 W O 97/46941 PCTrUS97/09781 Xbus Funcbonal Sp~ ;on Stratus ,mpany Co"fi~"~dl func[2:0l These bits ,e~,ese,lt a specific 32 bit read operabon.
Table 9. 32 BIT READ OPERATIONS/REQUESTS

func[5:0] OPERATION DESCRIPTION hnfs~e5 transled oooooo IDLE BUS ~ ~
ooooo1 resen~ed 000010 PING (used to test TRlDs) 1 0 000101 32 BIT l/O READ 1 ~32 000110 reserved - 000111 reserved 32 bit write operations func[5:3] = 001 Included in this group are all ~e 32 bit write operationst~ansa-,1ioos.
funcl2:0] These bits ~ep,esenl a specific 32 bit write operation.
Table 10. 32 BIT WRITE OPERATIONS/TRANSACTIONS

funcl5:0~ OPERATION DESCRIPTION info data blts 001 000 reserved 001 001 reserved oo1010 reserved 00101 1 reserved 001100 ~ BIT MEMORY WRITE 2 ~32 001101 ~ BIT l/O WRITE 2 ~32 00111 0 le5erv3d 00111 1 reserved Special Cache Line Ope. dlions funcl5:3~ = 010 This group of operations perforrn special tasks on cache lines.

21 Ju~ 1995 ~ 5 ~ 66 Xbus Functional 3r,j~ ~*on ~ tusCompany Confidential tunc[2:0] These bits uniquely idenbfy a bus operabon Table 11. SPECIAL CACHE LINE OPERATIONS

func[5:01 OPERATION DESCRIPTION info bits phasestransfer 010000 FLUSH CACHE LINE (data) (may ~.~iteback) 1 0 or256 010001 FLUSH CACHE LINE (instnuction) (never a v..it~ bach) 1 0 010010 TLB PURGE (data) 1 o 010011 TLB PURGE (instruction) 1 0 010100 reserved 010101 PURGE (data) 1 0 010111 INFO P~ETURN 256 5 256 Read Retum Funcffon Codes func[5:31 = 011 This group of funcbon codes is used only internal to the bus ASlCs. These funcbon codes will never actually get l-dn~ ,tled on the Xbus.
func[2:01 These bits re~ se"l how many data words are being returned.
Table 12. READ RETURN FUNCTION CODES

funcl5 0l OPERATION DESCRIPTION info bits ~ phasestransfer 011000 32 bit retum on a 32 bit bus 1 32 011001 128 bit retum on a 32 bit bus 2 128 011010 256 bit retum on a 32 bit bus 4 256 01101 1 reserved 011 100 l~bselv_d 011 101 reserved 011110 256 bit retum on a 64 bit bus8 4 256 01111 1 reserved althougb the Xbus is 32 bits wide this hmctior~ code is supported ror b~.. ' - , ' "~ with tbe Caugar ASIC.
Block Memory Read Ol,e-d-ionslne~ stc hJncl5:31 = 10 This group of ope,a~ious are used to read blocks of data from memory.
tunc[21 This bit is always zero 21 ~luly 1995 z _,~ 67 Xbus Functional Specification Stratus ~Jompany Confidential ~unc[1:0] These bits le, -~sent the number oS info phases required to transfer the data on this bus ~ o~ L~ ~.
Table 13. BLOCK MEMORY READ OPERATlONStREQUESTS

func{5:0] OPERATION DESCRIPTION phases request 100000 reserved 100001 READ 128 BIT on a 32 bit bus 1 128 100010 READ 256 BIT on a 32 bit bus 1 256 100011 reserved 100100 ressrved 100101 REA~ 256 BIT on a 64 bit busa 1 256 100110 reserved 100111 reserved a althougb ~e Xbus is 32 bics wide this fuuctia~ code is supponed for b~l~. A.J~ ~ u .~, ~il .J;l~ with the Cougar ASIC
Block Memory Write Operaffons/T a,~a~,tions tuncl5:3] = 101 ~his group of operations are used to write bbcks of data to memory.
~unc~2J This bit is always zero.
~unc[1 :01 These bits ~eprese- Il the number of info phases required to transfer the data on this bus transaction.
Table 14. Bl OCK MEMORY WR~E OPERATIONS/REQUESTS

func[5:0l OPERATJON DESC~IPTION data bits phases request 101000 reserved 101001 WRITE 128 BIT on a 32 bit bus 3 128 101010 WRITE 256 BIT on a 32 bit bus 5 256 101011 reserved 101100 reserved 101101 WRITE 256 BIT on a 64 bit busa 5 256 101110 .~,sol~,d 101111 f~se~ d a. al~ougb the Xbus is 32 bits wide this ~CtioD code is supponed for b~c~ witb the Cougar ASIC.
n~ 6d ~uncl5:0] -11~

21 July 1995 ~Z S ~ 68 .

Xbus FunctionL "oecincabon atus Company Confidenbal 6.10.2.2 Remote/Coherent bits (rc[1 :~l) These bits were dc~,~1c,~ ed to support a MESI CPU caching scheme and the Golf CPU/memory a~ re. They give the tbxibility to opti",i~e pe-l~""ance and to maintain cache cohefen~,y between all boards in the system. The meaning ot these bits for a given basic opelalion is shown below.
For all memory read operabons (block or byte specific), Irc[1:0] I~ esel)l the f~" ~ ..i.,g.
00 lI~-col~ Don't snoop, don't set remote tags. This will be used by CPU code fetches and reads to l~ ns where cache coherency is co"0Uled by software.
01 Icohe.~Jlt - Snoop but don't set remote tag. This will be used by boards without a cache when reading cacheable locdtions.
Iremotelshared - Snoop, set remote tag. An example is a CPU reading shared data.
11 Iremotelexclusive - Snoop, set remote tag. Informabon will be passed to the CPU
that exclusive rights are required on this cache line. A CPU (a board with a write into cache) will be the only board to initiate this type of operabon.
For all memory write OpelaliOIlS block or byte specific), rc~:0] ,~ en~ the i;~ ,9.
00 li.,coh~ t - Don't snoop this, write this to main memory. This will be used on writes to locdlions where cache cohe~ncy is conb-'led by soflware.
01 Icol~o-~n~ - Snoop, Invalidate and writeback any Illodif;ed data, Don't change remote tag status. (Note: If a writeback is gene,dled from a snoop of this type of write, The . .it~back will use the remote/exclusive protocol and that witl clear the remote tag.) This will be used by l/O board writes that need to maintain cache cohe,en~,y.
INVALID.
11 Iremotelexclusive - Snoop, clearthe remote tag. This will be used on CPU cache line writebacks.
For all other operations, rc[1:0] have no meaning and will be = 0. This includes the ~.1h :.:ng.
I/O WRITE OPERATIONS.
I/O READ OPERATIONS.
PURGE OPERATIONS.
FLUSH OPERATIONS.
BOTH INFO RETURN OPERATIONS.
6.10.2.3 Byte Enables (byte_ent3:0~).
The byte enables are valid on 32 bit read, 32 bit write, I/O read and l/O write operdlions. On all other ope,a~ons, these bits will be false (z0). When these bits are set, It signifies that the ope,c ~icn is requesting/modifying the indicated bytes. Figure 43 identifies each byte and its ~o"e~po"ding byte enable bit.

21 July 1995 69 ~ G ~

,, ~ . . .. . ..

WO 97146g41 PCT~US97/09781 Xbus Functional Specific~don Stratus npany Confidential Figure 43. byte_en[3:0] definition l~-cre..si.,~ byte a.~.lt~ss 0 ¦ 1 ¦ 2 ¦ 3 31 24 23 , 16 15 ~ 08 07 00 b~te_en[3l b~te_enl2~ byte_en[11 byte_en~0l Points to this byte points to this byte Points to this byte Points to this byte ~y asserffng 1 2 3 or all the bytes enables all 8.16 24 and 32 bit ope, dlhns are possible.
This is shown below.
8 bit operations 1000 Requestormodifydata3124 (~-hh~ss1:0=0) 0100 Requestormodifydata'23:16 (add~ss1:0=1) 0010 Request or modify data 15:08 (address '1 :0 =2) 0001 Request or modify data '07:00 ~adulr~ss 1 :0; ~3) 16 bit operahons 1100 Request or modify data '31 :16 (address~1 :0' =0 0110 Request or modify d:~t:l 73:Q8 (a~less 1 :0 =1 0011 Request or modify data.15:00 (add~t:ss 1 :0 =2 24 bit operaffons.

1110 Requestormodifydata~31:081(ad~1~t,ssl1:01eO) 0111 Requestormodify dat~p3:~0] (add~essl1:0le1) 32 bit operahon 1111 Request or modify datal31.00l(address[1:0]-0) 6.11 Board Sy-.cl-r~nizationlBoard States Board syl ,~ ,iLation is covered in more detail in the ASlC and board s~ e ~ ~ E tions for the desired board. This secffon in the Xbus ~ if;c~li~n is limited to the board states necessary to perform the task of putting a board on-line simplexed or on-line duplexed. lt gives a detailed desc,iption of the type of Xbus lla,)sa~;~on~ that are po~ible in each state.

In Golfbus based systems a mode known as simplexed mode was i,,,ula.,,ented for certain debug situations. This mode was never used or tested and it has been removed on Polo.

21 July 1995 70 2~ 1 WO 97146941 PCTtUS97/09781 Xbus Functional _"ecincaffon ~ ~tus Company Co~ dential 6.11.1 CPU/Memory Bus l"l~ ce Modes Below is a description of the board states needed to sync two CPUtmernory boards (i. e. bringing a CPU,'~"~",o~y board on-line-duplexed). This is con~ide,~:d a super-set of all the modes necessa~ for booting andtor bringing a CPU/~"e",ory board on-line-simplexed.
The f~'louing si3nals are required on this type of board.
Broken, On line, Dupexed, Update, Freeze-state (BODUF) (simplexed_mode bit false) BODMUF
1 XXXX Broken: Board is invisible to the Xbus. The board will ignore all Xbus o~, dtions.
The board must be reset through the dedicated reset lines before it can be Aoc~s,,ed at all.
00000 Olff line: The board will respond to non-paired l/O read cycles and will perform non-paired and Global write transactions. It will ignore all other bus operations.
The board will arbitrate for the bus to retum l/O read data. The board is capable of initiating bus l,ansactions but conceptually, the board should not generate bar,sa L~ns. Software should enforce this where needed.
00010 Otf-linelUpdate: The board will respond to non-paired l/O read cycles and will perforrn all l/O write llansactions (Global, Paired and non-paired). The board will NOT respond to paired l/O read l,ansa.,~ions. The board will also perform all memory write b ansa~ 7S that index into its memory array. The board is capable of inWating bus transaction and will gene,ale only odd TRlDs regardless ot what slot it is in.
01000 On-line: This is the norrnal state of a board that is on-line simplexed. The board will listen to and perform ALL bus han6a b~ ~s directed to him. The board is capable of initiating bus bans~ tic n and will gene-at~ only even TRlDs regardless of what slot the board is in. In this mode, the board will respond to and drive the Xbus during a retum of l/O paired read data.
01010 On-line/Update: This is the state of a simplexed on-line board that wilt broadcast all local memory writes to the Xbus. The board will listen to ALL bus ~ansactions directed toward it. The board is capable of initiating bus ba,lsaction and will gent:ldte only even TRlDs regardless of what slot the board is in. In this mode, the board will respond to and drive the Xbus during a retum of l/O paired read data.
01011 On-NnelUpdak~/rr~3Le. This is the state of an on-line simplexed board when it is doing the final copy of state over to it's partner board. The board will ONLY listen to l-dnsac~ons that have the same TRID as its slot number or its partners slot number. (i.e. odd or even TRlDs). The board will BUSY all other new transacUons directed towards him. The board may iniUate l,ansa~,tions and will use even TRlDs.
00011 Otf-linelUpdatelFr~. This is the state of an off-line board that is getting state coped in via IIO spar;e. The board will listen ONLY to Dansautions that have thesame TRID as it's slot number or i~s partners slot number. (i.e. odd or even TRlDs). The board will BUSY all other new transactions directed towards him.
The board may initiate ~ansa-,~ons and wlll use odd TRlDs.
01100 Duplexed: Entered only after the board went through the Sync point and it was a duplex request. Both boards enter this mode simultaneously and are ~ons;clered duplexed and running in lock-step. The 'update~ and 'freeze-state~ bits are cleared upon entering this rnode. Odd and even slot boards will generdl~ even TRlDs. On paired l/O reads, the board will perform the read of the register in lock step with irs partner, and both boards will drive the Xbus during the retum of the data.This mode is cleared in the result of a T/A failure, or by this board or its partner going Broken.

21 July 1995 71 Xbus Functional Sp~-;fi~ n Stratus mpany Confidential Online for Dumping loo~back mode: These modes are unsupported on the Xbus.
Note: States not listed are consi~,~d illegal Table 15 below gives greater detail on op~,~ticns p~,f-""ed.
Table 15. CPU/Memory Board Statesa z ~ODE ~ Q 3 ~ 3 ~ ~ z z ~
6 ~ ~ ~ _. O ~~ ~

~ ~ ~ ~ c ~ rr o ~ L' ~ Z ~ m r O ~s O ~ ~ O LLI W O J ~ ~L cc U
m O cl ~ ~ z ~ z ~ L Z c~
~ 1 X X X X I I I I I I I I I I I I I N
on lh O O O O O P I P I P I I I I I I I P O
on: ~u O O O 1 0 P I P P P I I P P I I P P O
On~h 0 1 0 0 0 P P P P P I P P P P P P P E
c r U,#~ O 1 0 1 0 P P P P P I P P P P P P P E
o~ r. O 1 0 1 1 B B B B B I B B B B B B P E
onr u, r. O O 0 1 1 B I B B B I I B B I I B P O
Dupb~ O 1 1 0 0 P P P P P M P P P P P P P E
r r o~ - O 1 1 1 0 P P P P P M P P P P P P P E
r !~ c. 0 1 1 1 1 B B B B B B B B B B B B P E
L KEY
State bit true.
O State bit f~llse.
X State bit tn~e or false P The boa~ will perfo~m this Xbus 1. - ~ fully.
M The board will go thra~gh the motions of performing this Xbus h ~ but will DOt drive tbe Xbus in the info phsse of the data ~etum for tbis i lbe board will ignore tbis operation. (i.e. will not drive ACJC, BUSY or retum data.
B Tbe board will busy this I . . . ~. I ;. ~A urlless the TR~ of the I matches its slot number or its pattne~s slot Dumber. I~ote: the board will busy the l ~ until it is released from this mode.
EFQ Freeze State busies cycbs sent to the EFQ ~e%cept data retums), and Freeze Stau busies cycles seDt to the RWQ. Refer to secttan 3.3 for more details on freeze states. the EFQ. and the RWQ.
1~1 The board wiD never initiau a i in this mode.
O lhe board can initiate all I in this mode and will only use odd l'RlDs.
E The board can uutiate all I in tbis mode and will anly use even TR~s.
b. Refers to either EFQ F~eeze State or Freeze Stau c. CPU VO wriu refers to an ~/O wriu that is broadcast to aU CPUs.

21 July 1995 72 2~ 3 Xbus Functional ~ ,~cific~tion 5~ .us Company Co"fic~al Figure 44 shows the ,~o,..,..anded state b~ns.t~ons for CPU/~"e",ory boards. Not all possible transitions are shown here, only the ones necessaly for software to put a board on-line simplexed or on-line duplexed. A board can transition to the broken state trom any state but this is not shown in the diagram to improve readability. The dotted line is the fast duplex path.
Figure 44. CPUlMemory Board state transitions ( Broken ) ( Off-line I
On-line ~ I \
Simplex J
~\ I \
\I
On-line ~ ~\ ~ Off~ine ~
Update J I \ ~ Update J

\ _ On-line ~ Off-line Update/Freez~ I ~ Update/Freez~,J
,J
Online ~ Dupex J

6.11.2 Simpiexed l/O Board Bus l~ rf.,ce Modes Below is a description of the board states needed to bring a llO board on-line that has no capability of duplexing. Note, a simplexed l/O board will never respond to (or ACK) paired ItO read ~nsactons.
The Ic'lo~ g signals are required on this type of board.
8roken On-line (BO) BO
1 X Broken The board is invisible to the Xbus. The board will only perform l/O writes when possible. If the board is busy performing an l/O banssclion, the board willattemptto busy e L ~ nal l/O t~a"sa ~ns. Re~se the board is broken it cannot drive the busy Xbus signal. This means that any l/O w ites that occur when a broken board is busy fall to the bit bucket and are not pe, ~ ,ed. The write will also not occur If def~:c~,c hafd/.a,e precludes the write from occurring.

21 July 1995 73 CA 02257~11 1998-12-03 Xbus Functional Specification Stratu~ ~mpany Confidential oo Otl-llne The board only res~or,d~ to no~paired 1/0 read snd 1/0 write transactions. The board will arbitrate for the bus to retum 110 read data. This board will never respond to ~or ACK) any paired 1/0 read l, dnsacLons. The boardis capable of initiating ~us transacbons but conoeptually, the board should not generate ll ai ~ac ~s. Soflware should enforce this where needed.
01 On-LinelSimplexed This is the normal state of a board that is on-line simplexed.
The board will listen to ALL bus ~ai~sa~ ns. The board is capable of ~nitiabng bus ba"s~on and will ge"~,dle TRlDs that are equal to its slot ID. This board will never respond to (or ACK) paired 1/0 read bansa~tions.
loo~b~h mode This mode is unsupported on the Xbus.

Table 16. Simplexed l/O Board slalesA

a board state ~-- 3 ~ ~ ~ ~
Cl rr C~ ~ o ~

O J, Z C: Z ~ O ~ Z
~ o z ~ z ~ ~
Broken 1 X I I T I T I N
Off-line 0 0 P I P I P I Y
On-line 0 1 P I P I P I Y
a ICE~Y:
St te bit true.
O State bit false.
X St te bit truc or false T Wlll perf~m if pc6sible (i.e. if na busy perfo~ming one just before it) P l~e bosrd will perfo~m this Xbus I ~ fully.
The boald will iguore tbis operatio~ (i.e. it ~ill not drive ACK. BUSY or retum ~
~1 lbe board will never irutiate a t.~ in this mode.
Y lhe board can irutiate all i in this mode aDd will use the TR~ of its slo 21 ~lUly 1995 Xbus Funcffon~ e ~ ~ ~tionatus Company Confidenbal Figure 45. Simplex l/O Board State Transitions ( Broken )~ Olf-line On-line ~ Simplex J

6.11.3 Loopbackmode The Xbus does not support Golfbus style loopb~ k testing.
6.12 Board Breaking/Board Removal 6.12.1 U~breaking a Board Boards Should generally be ~unbroken" by resetting them rather than directiy clearing the broken bit in the Bus Interface Status register (see the FTIO / PCIB Pl'uyldllll,,at'~ Guide). There is at least one (unlikely) s~nalio where simply clearing the broken bit in the Bus Interface Status register can cause a board to receive mulffple read retums, specific~lly:
~ A board ack"o~Jledges an Xbus read requesttargeted to it and then breaks. While it is broken, the original reql~estor PlNGs the read. Since the board is broken, the PING will not be ACKed and the reqllestor will then retum 0's to its on-board ,c,ocessou If software then simply clears the broken bit in the Bus Int~ldte Status register, the original board may then retum the read to the reques~r (assuming it took awhile to process the read), thereby giving the requestor a second ,~e:,ponse on a read it has already passed up 0's to the p,~ cessor for. Clearing broken via reset in this case would have ensured that the broken board clears the read res,oonse out of its state rnachines prior to unbi, -~ irlg.
Also, there should be some delay L ~t ~en the ffme a board breaks and when it un-breaks and starts performing Xbus a~ss~s to other boards. The l~ wing (exb ~- "ely unlikely) danger ~,er~ s~exists:
~ A board pe. h", . .s a read request, then breaks, and then gets u, Ibr~ ' en via a reset. At that time, it immediately gen~ t~s another read request and happens to reuse the same trid. irs possible that the read data from the first (pre-broken) read (which may be to a completely IJI ll~JI.,~d. register) then comes back and gets grabbed for the second (post-broken) read.
Irs difficult to quantify how large of a gap is neceasaly between l)n,zh ,9 and issuing the first read access after unbll -' ing, but even without precautions, this problem is very unlikely. Nommal onboard diagl 1~ ' ' which are executed as part of the ~" ,b~caking code are p,~bably long enough in duration so as make the above scenario il l lr ~- S 5'~ '~ -21 July 1995 ,~ ~,G 7 Xbus Functional Specificabon Stratus- empany Conhdential 7. Xbus Routing and Xbus l.-t~l tdce Clocking All information ~a--sf~--ed across the XBus is s~" ,d"l~nously clocked onto and off of the bus at 24MHz. The bused, control and TA signals are driven and received directly by the bus ASlCs. The voted signals are buffered by 26S10 ~ansce;~c,s. Clock are gene-d~d from a single souroe located on the ~ ne and distributed through ECL clock lines.
7.1 On-board Clock Generation The Polo project reuses much of the Je~ta System clock design. The Sentry ASIC receives an 8 MHz clock from the backplane, and generates a 48MHz clock and 4MHz phasor for the bus ASlCs. The Jetta 48MHz 0.8 micron DLL (delay locked loop) and acffve dock spine are used for distributing a single 48MHz clock on the ASlCs.
7.1.1 Clock Phases All flops in the Gambit and Cyclops ASlCs are clocked by a single 48MHz clock spine. However, it is often neces~r to have PCI and Xbus related logic b~ehav8 as if clocked at 24MHz or 12MHz.
To accomplish this, we define a group of pseudo-clocks that Nn at 24 and 12MHz, and have rising edges at all of the useful times.
,ons 20ns 40ns 60ns, clk48 1 U _ dk48_24 --I n ~ n clk48_24_not I n n I n n I

dk48_12_0 .
dk48_12_1 1 I
clk48_12_2 I n I n clk48_12_3 I n I n_ phase_12_0 phase_12_1 I n I n phase_12_2 I n I n ph~se_12_3 I n I n 21 July 1995 q~ ~ 76 WO97/46941 PCTrUS97/09781 Xt)us Function~ ~pec ncaffon ;atus Company Confidential There is really only one clock in the ASlCs, clk48. 1 lo. ~,~or, by controlling flops with the phase signals on the clock enables, it is possibls to make it appear that the design has all of the clocks shown above. Phase_12_0 is used to create dk48_12_0, etc. The flops ll,e",_elJes are a D flop with a mux on the input, where the mux selects between new data and Q output. The phasors are conne.led to Ule sslect line on the mux.
7.2 Signal Routing The bidirectional info buses are routed point to point b. :eon boards. On each board, sepa,dt~
traoes run from the co"ne~,tur to each ASIC; this allows the error protocol to determine if a fault is in the ba.k~,lane or in the board. Series l~ai~tùl~ are used to control signal quality, and CMOS
levels are used for i"~ eased noise margin and reduced suscepbbility to cross-talk.
Figure 46. Routing of info lines oen c~ ~

1 R" connectur The three way voted signals are routed as separate point to point cGn"ec~ons. The reset_x_y_,z_ signals are driven by the CPU boards, wire~Red on the ba~hue, and reoeived by all boards.
The board_not_broken_ signal is driven by a single board and received by all boards. These signals are driven by norrnal 4rnA drivers of ~e ASIC, and buffered by 26S10 ~,anscc,~r~rs. The voted signals are temninated with a pull-up and isolation diode on both tho reoeiver and driver side.
This means that it is possible for there to be one or two pull ups on the line. I I ~wt,~cr, whenever the line is being used for active signalling, there are exactly two terrninators, one at either end of the line.

21 July 1995 .. . . ... .

Xbus Functional Spe~fi~;on Stratus vompany Confidenffal Figure 47. Routing of 3way voted ~ines 'pt6012 ._________________ ~
__________________~ \ ,________________ 4" ~ -4"

~_3" 1 1 3~_~

~__________________ ~_________________ 7.3 Clock Fault Tolerance l~sues, Outbound Signals.
The Polo system must protect against failures of a clock chip on a board. There are two physical broken lines on a board one with all clocking from the ~side clock chip and another with all clocking from the D~side clock chip.
The Xbus has all clocWng of signals on snd off the bus pe,f~,r",ed within the bus ASlCs. If a side of the board loses its clock the other side will detect data hop~k errors and break the board. The three way voted signals are buffered by 26S10 h~nsc~:J~..;. and are not susceptible to clocking failures.
In Polo the clock distribution is not triplicated. An analysis on clock line distribubon was pe,f~"",ed and it was calculated that due to the small number of dock loads system reliability was lowered by adding extra drivers to the system to perform the clock roubng triplicabon.
See the Polo Clock Distribution Spe~if~ ~I;un and the Sentry Clock Recovery Device Specific~bon for a ~fisc~ss~ of clock fault toleranoe at a system level.

21 July 1995 ~ ~, 78 .

Xbus Functional _, ~cificadon S us Company Confidenbal 8. Board Insertion and Removal 8.1 Hot Plugging Polo is the first Stratus produce to interface CMOS ASlCs directly to the l~chpldne. As such, it is the first to address live inse, L ~, n of CMOS parts.
If a CMOS part is supplied significant current through its l/O pins before its power grid has come up, the part will ~latchup'l. That is, p~la~tic bipolar lldnsi~tul ~ at the l/Os behave like an SCR
(Silicon Control Rectifier), and the part beco-"es stuck in a non-functional state. The ;3rr~l~t ~hle limit is one diode drop; as a part powers up, the voltage on any l/O pin must not be more than 0.6v above the VCC inputs. To prevent this, all ba.,h~Jlane signals conne..ti"g to the Polo bus ASlCs must be tristate or driven low while a Polo board is powering up. Whenever the possibility exists that a board is u,-~ u~,.ed ~as d~ ~ ch~d via the board_not_broken sign~ Is), the other boards must drive any signals connected to that board low. This should be done by riving low for one cyde, then bis~li.-g the lines. All nets directly c~-.ne-,led to snother board's .9 SIC should be equipped with pull down ~~s,sto.-~, instead of the pul~ups used on Jetta. signals ~uffered by an i"~. ~-~diale parl, such as the 26S1 0s, require no special tl~alm~
8.2 Xbus l"t~. f..ce Testing at Board lnsertion The Xbus interface does not include Golfbus style hardware sup~ort for bus interface testing.
Instead, Xbus systems will rely heavily on scan testing in manufa,cturing to detect faults in the error d~t~ution logic.

, / / ~'/ ,'~
~ I ~ ~1 ~J

21 July1995 z ~ 79 . .

CA 022575ll l998-l2-03 Xbus Functional ~pe~ o-7 Stratus t _.npany ConfirJential 9. Xbus Fault Tolerance The goal for fault tolerance on the Xbus is for no single point of failure to cause a system crash or allow a transaction to complete without t,ans".i~li"g the correct data. A single point of failure could be any co",~on~,lt any signal or any pin on a cG".~,onenl.
The Xbus fault toleranoe scheme assumes that one of the cu,, ".onent~ conne..l~d to the bus is always .espol-sible for hard bus errors. To this end the design has been simplified around the assumpbon that when an enor occurs an offending Field F~-~.p'~oe~l lle Unit (FRU) will be removed from service. This extends to include all buses that the FRU is connected. Thus when an Xbus fault occurs the faulted bus servioe is removed taking at least one FRU with it Both sides of the CPU board drive signals to the bus and both sides of the board perforrn various checks to ensure that the board is functioning normally. This is identical to the Golfbus met h ~ ~ ~ 3~. The PCIB board behaves differenUy; both sides of the board drive and check the bus when driving data from Xbus related lO r~;st~ . ;" and when driving data originating on a simplexed PCI card only one ASIC drives and checks the entire width ot the bus.
The Xbus signals are divided into three classes: info bus signals control bus signals and three-way voted signals. The fault i ~ 'e a,)ce n ,~U ,~ d~ ~ gies are different for each of these classes.
9.1 Into Bus ProtecUon The info bus is protected by parity and loo,.~l~aoh cl)ecl~i"g. I lo~;evcr tt7ere is no A So B bus cross cl,ecl~ing by anyone on the bus other than the bus master. When a PCIB is bus master only one of the two buses ~Uacl,ed to a given CPU module will be active becal~se the PClBs do not duplex and U,e,~f~" the bus from the PCIB that did not win arbitration must be idle. Lik_. ~ E the PClBs cannot know if two duplexed CPU modules are driving the bus or a single simplexed CPU has c.~r.,e,sl7ip (two si~ ed CPUs could have started a,biL~ion at the same instant). Thus receivers on the Xbus will only listen to the bus that is driven by the module that won a~ n.
9.1.1 Pariq~
Parity is gent~ t~d and O~ns",iUed across each of the physical buses. The 32 bit bus trid field and func_op bits are all covered by a single parity bit. The purpose of the parity check is to protect against etches that open bet~t en the transmmer and receiver. 80ards in the system check the l,dns",i~i~ d parity that they receive with parity that they g~ner~t~ n,~hre3.
A bus error is signaled on any parity error.The bus_err signals are also effected by other error ch6cking that is desc, ibed later in this document. In an effort to simplify these descriptions the reader may assume that the actual error signals are a logical OR ing of the various outputs from diflerent ch~ blocks.
Both the C and the ~sides of the boards will perform the check and ~""~,e results individually and decide on any action that must be taken. In the event that the two sides of a board do not agree upon the status of a bus cycle the board will break in the 1~ ~ ing cycle in one of two ways.
~ The ~side of the board tJe ~ .cl.:d an error and the ~side did not. In this case the ~side will drive a bus error and the ~side will di;,a~, ee with what the D-side has done and break the board. Here the IICII _ a ~ n is errored and re~ad~ed but the broken board should no longer be checki,)g and the bansa~ lion will complete.
~ The Gside of the board .le:tecled an errc- and the D-side did not. Here the Gside will once again break the board but this time be~l ~se it expected to see an error on the bus and didn't.

21 July 1995 80 ~1 .

Xbus functional _,Jecification ~ .tus Company Confidenbal The t\ansactiun will complete normally and the bad board will be removed from service.
CPU boards compute panty in both C and D ASlCs and break U a defect arises in one side's parity gene,d~-~. Most of the l.a"sfe.~ from a PCIB board, however, originate from a PCI bus co"ne~,ted to a single Gambit ASIC. When the Gambit drives this single sided data, it drives aU the bits on the Xbus, and an error in the simplexed parity ~ent:,dt~,r would hang the system. Fortunately, there is no need to duplicate the entire Gambit ASIC, only the parity gene,dlion section.
Figure 48. Self Checking Parity Logic., 55 ~
A~ ~ info 2 ~ V

3 0 ~ parity info_out 1 {~ ~ ~

~parity generator ~ ~ --fault broken 5~ ~
A~ ~ info oopcheck ~P data loopback mis-compare ~ A fault at site 1 causes erroneous data to be tJdl 51 lli~led with r orrect parity. This will be detected by higher level checksums.
~ A bult at site 2 will cause the board to break with a data loopba~ 4 fault.
~ A tault at site 3 will cause the board to break with a parity gen~,ator fault ~ A bult at site 4 or 5 will break the board with either a data loop~)~rk fault or a parity gen~,dtor tault.

21 ~luly 1995 81 Xbus Functional Specification Stratus (,v,npany Confidenffal g.1.2 1 o~r~)~ck cl~c~ g Figure 49. 1 o~ k Connectivity ~side ASIC Gside ASIC
c_~Rw_comp_en D16bib I c_~Rw_drwe_err D1 bit~ I
D 16 bit~ I D 16 bils I
d_~aw_oomp~
D 3Z,~d9 I d_~sw_dnve_~ D 32 bib I

1~--32~ 16~ 32 ~ . . _ X-Bus Physical Bus On norrnal A~r~e-sses ~non-CD different, non-single sided), each side of the board drives half of each field to the bus and receives all of both busses. Thus there are two checks that each ASIC
can do on the data retuming from the bus.
1: Does the retuming data match the data that was driven from this ASIC. This will be known as the 16u~ 4 drive check.
2: Does the returning data match the data that was driven by the other ASIC. This will be known as the Ic,oph~r4 con,pa,e check.
Two enor signals are gen~,dled inside of each ASIC for each physical bus that it is driving. In order to know the true state of the board each side of the board must know the results of the co- ",~ ons on the other side as well as the co" "~ari~on~ it has done. Thus two signals are passed in each direction, as shown in figure 49 above, so that the each side may correctly determine the state of the board.
Each ASIC will determine wether the data on the bus is faulted or not, as well as if this board is broken or not. Table 17 below shows what combinations of the loopba~k signals for each bus indicate that the board is broken.
Table 17. Board Broken Matrix for a Single Bus drive_err from comp_err from my drive_enmy comp_err the other side of the other side of brd_broken the board the board O O O O o 21 July 1995 ~ 82 .

Xbus Functiona. ,~ n husCompany Confidential Table 17. Board Broken Matrix for a Single Bus drive_err from comp_err from my drive_err my comp_err the other side of the otherside of brd_broken the board the board O

A double bus error will be asse-t~d and the not_broken signal de-asselted if a condition exists that should break the board. In addibon a bus error will be asst,-lud for any condibon where a co""~ar~
error of any type has occuned and the board has not broken. The bus enor will only be on the bus while the error is d~t~. ~cl 9.1.2.1 Loopback on CD dil~_n,.)l Access~s When an ASIC drives data from a CD different register on the bus, it pe.lu,,,,s loopback checking only on the trid, func_op_, and parity lines. It asserts bus error if it sees an error, and will go broken according to table 17 9.1.2.2 Loopback on Sing~side ~tce ~c ~
When a Gambit ASIC drives single-side data from the PCI it pe-f ,""s loorl~a. 4 ~)e.,ki"g on all of the bits; it does this only for the purpose of asser~ing bus error, not to see if it should go broken. It will not break according to table 17, bec~lse the cmp_err and drive_errsignals from the o~,e-~,de are invalid. ~lo. a-cr, if the driving ASIC sees a bus error, and the o~e,~ide ASIC does not (parity reoeived good), the board will break due to a loopbac k on control fault.
9.1.3 ~ oop_ck_ops Loop_ck_ops are used during the error sequence to make intelligent choices about the state of various boards and the buses that they are attached to. A loop check op is a cycle in which a board drives an al~",~ ,9 pattern of AA~s and 5~s on the backplane. The board eYaluates the pattem and similar pattems driven by boards on the other end of the bus. Based on this i,,fu,,,~dlion, bus state is altered.
21 July 1995 83 'Z ~ ~

CA 02257~11 1998-12-03 Xbus Functional Specification Stratus _~mpany Confidenbal 9.2 Control ~us Fs.ot~clio.l The Xbus collects control signals into a series of point-to-point unidirectional signals protected by an ECC code. Each board drives a separate control bus to each of the other boards.
There are two completely independenl types of checks pe, h" ",ed.
1: FYery ASIC corrects for any single point failures (shorts, opens, dead drivers or receivers) with a single error correcting. double enor detecting ECC code. Single bit failures cause a maintenanoe interrupt and are logged, but otherwise have no effect.
2: lf a driving board detects a double bit error on the outgoing control signals it will attempt to assert bus error and then it will break in the error sequence (it will break regardless of its ability to drive bus error). If a r~cel~;ng board detects a double bit error on any incoming control signals it will attempt to assert bus error then it will break in the error sequence. If the error was actually on one board and only one board saw the error then the correct board will break. If the error was such that both boards saw it men both boards will break. This is to limit the possibility that data gets corrupted.
9.3 Thre~way Voted Signal Pl~te.,lion Some broken and reset related control signals on the Xbus are triplicated and voted. The reset and board_not_broken_out signals are buffered through tldnsceivws bec~use they must be valid during power up or power down events. The CPU signal sync_out is buffered through transoeivers to reduce ASIC pin count. The CPU signal my_online_out is sourced directly by the ASIC as a cost reduction.
The D-side ASIC drives signals to the ~al-s~3.~rs and to the C-side ASIC. The ~side of the board checks what was driven by me ~side with what it thinks should have been driven. It the C-side disagrees with the D-side the board breaks.
Each ASIC that receives a three-way voted signal will perform a majority voting ? 'g ~ ithl l l on the three signals to determine the value. In normal operation all three lines will be the same. If there is a fault on one of the three lines (either on the ba~,k~,lane or on the board), two of the three lines still will agree.
When a voting enor occurs, the device detecting the fault should send a maint3nance interrupt and log which Odns~;~cr has the non-unanimous vote. The logged i,~fo,~ n is stored in the Voter Error T,a"sce.~rer Status rsy;stt~.(s) and will not be over-written until the status register is explicitly re-enabled by software.
9.4 Error Reporting A number of errors can be d~ c~eJ by a board's bus interface logic:
1: A loopback error that breaks the board. This specifically refers to luopb~rk errors on the info, parity, TRID, and func_op signals which are seen by one side of the board and not the other during bus cycles in which the board drove the bus.
2: A di_29~ee..~-,1 b~tva,cn the two-sides ot a board on whether a three-way voted signal should be driven.
3: A parity error on the bus, either on a cycle where the local board was driving the bus or another board was driving the bus.

4: A loopl,~ h error on the bus.
2t July 1995 -CA 022~7~11 1998-12-03 Xbus Functional ecihwtion ~ tus Company Confidenbal 5: A three way voter error in which not all three signals are in ag-ee,l,e,n.
6: Any bus error signaled by any board in the system 7~ A thermal fault (high te~ lalure detected on the board) Some of the above errors may break a board te.g. #1 and 2), some may cause a bus error to be genelat~d (e.g. # 3, 4 and 6), and some have no effect on nomnal machine ope,dlion (e.g. #7). The Xbus interface contains two classes of status registers to record infommation sunounding the above errors: Broken Status and Error Status. Broken Status registers are only acti~d when a board goes broken, and are de~igned to pinpoint the logic that set broken. Error Status registers are for repol ti"g non-fatal errors such as faults on the bach~Jldne. All of the Enor Status registers are updated simultaneously on any bus error signaled by the system, or a voter or thermal error on the board. Speci~ically, when all error reporting is enabled, any of the above errors cause:
1: A rnaintenance interrupt to be generated 2: The type of error condibon to be stored.
In the at)ove case, suhsenuent error state recording and the resulbng maintenance interrupts are disabled until explicitly re-enabled.
It is not always optimal that all the above errors trigger the error leg;~le.;. and geneld~e a maintenance interrupt. The reporbng of new errors can be effectively blocked by an t:)~CeS5i~E
number of reports on a known error. Hence, it is possible to turn off error latching due to different sources:
1: Perform enor latching and main~nance interrupts on all errors 2: Inhibit error latching and main~enance interrupts of incoming three-way voter enors 3: Inhibit error latching and maintenance interrupts on thermal failure related errors The error latching disable bits inhibit those faults from t~igge, ing the error registers. I lo..cJer, if a disabled error is present and non~ d error occurs, both errors will be lecorcJed in the error registers.
Details on the operaffon of this error re~llil)g circuitry are provided in the detailed lO register definihon section of the Polo Rl og, ~" " "er'~ Guide. The specffic registers that implement the above funchonality are the Bus lu~elldce Fault Reporting Registers, the Bus Info Error Status Registers, the Misc Error Status Registers, the Pus Error Tlansceiver Status[1:0] Registers, and the Voter Error Trdnsceiver Status Registers.
9.4.1 Producing Errors The Cyclops and Gambit gate arrays have specific registers which allow software to produce various errors on the Xbus. The le9;~ are desclil-ed in detail in Polo P~ogldllllllt~ Guide.
These registers allow the error logic to produce all of the error cases d~ ed in section 6.4.3, Fault Conditions.
The basic logic to produce an error consists of XOR gates on all the info, trirJ, func_op, parity, control lines going out and all the same lines coming in. One input on the XOR is the data line the other input is the error line and the output is the resultant data. If the enor line is high the data coming out will be in error. If the error line is low then the data coming out will be the correct data.
If data is errored only on the input or only on the output then the error will be in the form of a loopba~ k error. If the data is errored on the input and the output then the error will be in the torm of a bus error since it will not be a loopback error.

21 July 1995 85 '~G

PCT~US97/09781 Xbus Functional Specification Stratus npany Confidenbal Table 18 on page 86 shows all the enor cases and the necessary settings from a hardware point of view to produce the error. 'receive error' means the XOR gate on the receive side is inverting.
llan~ t enor means the XOR gate on the transmitside is imertin~. The category des~ibes if the enor occurs only on a particular pattem match or if the enor occurs just aner the register is written.
For many cases it would be possible to produce an error with different cases msn those desclil ed below. The goal is to only produoe the enor in one manner.
Table 18. Error Settings Type of Error ¦ Category ¦ ~Side ¦ ~Side CPU Board Error CPU Board faulty Input Circuit - CPU Driving hlln~lididt~ receive enor no enor CPU Board Faulty Input Circuit - I/O Board Data Match receive enor no error Driving CPU Board Different Data C-Side and l:)-side Im...ediale transmit error no error/receive error CPU Board Faulty Out,out Circuit - Buffer to l~lllllediale transmit error no enor Pad Fault CPU Board Open - CPU Board Driving Illlllledidlv no enor receive enor ~ CPU Board Open - I/O Board Driving Data Match receive enor no enor CPU Board Short l~"n-6diat~ transmit enor no error Rarkp'~ne Enor n~cl~p' ,- ,e Open Etch Data Match CPU and lO CPU and lO
board board transmit enor receive error Ireceive enor Oa~l~lane Short hn",6diat~ CPU and lO CPU and lO
board board transmit error no enor lO Board Enor 1/0 Board Faulty Input Circuit h~---le-lid~ receive error no enor 1/0 Board Output Circuit Fault- Buffer to Pad l-~ e,Ji..~ transmit error no enor 110 Singl~side Access - ASIC Parity Gen. I-""-eJ;dt~ parity gen. no enor ~ault fault l/O Singl~side Access - ASIC PCI Data Psth l~ e~lidt~ bad address no enor FauH
1/0Non-single-sideAccess l~ elltC-D h-,...ediate transmitenor noenor Data /receive enror l/O Board Open l~ edidle ' no enor receive error 1/0 Board Short li.. l-.edi.. I. transmit enor no enor 21 .~uly 1995 86 -CA 022~7~11 1998-12-03 W O 97/46941 PCTrUS97109781 Xbus Functiona~ n ~tusCompanyConfidential Type ot Error ¦ Category ¦ D-Side ~ C-Side Transient Error Trsnsient Fault This causes a bus error, but when the error sequence is PYecu~ed there will be no enor.
This fault will not produce a maint. int. from this board, but all the other boards will assert maint. int.
Bus Busy Bus Busy ¦ This causes a bus busy.

9.4.2 Maintenance Interrupts Each Xbus board gene,dtes maintananoe interrupts whenever an event occurs that would be of interest to the M~D software. These events fall into three classes: broken/unbroken events, error events, and software co. ""~r,d~. As desc,il.ed in the i " ~w;.~g section, broken/u, Ibr~ ' - n events include board resets, board illse,tions, and board removals.
9.4.2.1 Broken Events Boards ~ansitioning to or from the broken state gene,~te a maintenance interrupt. A board can break due to a detected ha.dv:~fe failure, a cold or wamm reset, a board removal, or a software set broken cu,,,,,~nd. A board can unbreak due to a software co",."and, a reset, or an insertion.
Cold or warm lesetlil ,9 a board causes that board to go broken at the start of the reset and unbreak itself at the end of the reset, thereby gene,aling two main~al)anca interrupts assuming the board was not initially broken (one maintènance interrupt if the board was inibally broken).
Inserting a board into the system generdtas a single maintenance interrupt when that board unbreaks itself.
9.4.2.2 ErrorEvents A number of enor events may cause maintenance interrupts. These are reported in the aus Info Error Status and Misc Error Status registers, defined in the Polo F~Oy~a"""e~'~ Guide, and controlled via the Bus Interface Repo,ling Register, also defined in the Polo ~lu91allll"e~ 's Guide.
The error events are:
1: A bus enor on a bach~Jlane bus 2: A voter enor on a three way voted backplane signal 3: A thennal fault 4: A failure detected by the clock recovery chip 5: A board specffic fault/failure. Note that unlike Stratabus systems, main memory single bit ECC errors do not gene,~le maintenance interrupts on the first generabon Polo CPU/
Memory board.
As desc,ibed in section 9.4 on page 84, it is possible to tum off maintenance interrupts from certain sources so that a known hard failure does not flood the system with maintenance interrup~. Specifically a voter error, a clock error, or a thermal fault may be disabled via the Bus 21 July 199~ 87 ~g CA 022s7sll l998-l2-03 PCT~US97/09781 Xbus Functional Specification Stratus ,mpany Cullfi~l,~l Interface Fault Reporting register.

9~4.2.3 Sof~Nare Control Sûftware may expliciliy activate maintenance interrupe by wribng bit one of the Bus Interface Fault Repo.ting register.
9.4.2.4 D~rmining the Source of a Maintenance Interrupt Upon seeing a maintenance internupt in a system, the ( "~ g process can be used to determine the source of the intenupt.
If a board that was in the system is no longer visible to software then that board generutdcl a maintenance internupt. To get more detail on what ha~,pened to the board, it can be reset and its Common Broken Status and ASIC Specific Broken Status r~y;~te, ~ examined. If multiple reset atle""~ts have no effect, the board was either removed or is too broken to be read.
Boards that have not broken or not been removed from the system will have either the ~Bus Interface Maint Inr' bit or the ~Board Logic Maint Inr bit set in the their Bus Interface I lepo,~iny register if they activated maintenance intenupt.
The ~Board Logic Maint Int" is board specific and informabon on further identifying mainlenance interrupts with this bit set is diccussed in individual board specs.
If the "Bus Interface Maint Int" bit is set, then Board Reset register may be read to see if a maintenanoe intenupt was due to a board reset. Similarly, the Bus Info Error Status and Misc Error status registers may be read to deterrnine if the rnaintenance interrupt was generated by any non-fatal errors (the board would be broken oll ,arw;se). If no bits are set in either of these two f~;S~Ia. the maintenance interrupt was software gent~ Ld.
9.5 Board Breal~ing ~ming Note in the figures below, InSo_0's shows the time the info bus is driven to zero assuming that the bus is not atready tristated from this side. xcever_disable shows the time the drivers from the broken board are disabled from the bus. Bus error is also shown when valid on the Xbus.
9.5.1 Board breaking and i~f~r~atiG~ latching.
There are several status bits in the Broken Status register on each board to determine the reason a board went broken. These bits are frozen aner a broken con ~. n occurs and will remain frozen until after a cold reset. All bits in the ~------on register are straight forward with the ~cep1ion of board specUic_set_broken. Board spedfic logic may have several reasons to assert this signal and may also have a similar register for the reasons it has asse. tt:d this signal. If board_specific_broken is set in the co,----,on register, the board specific broken register will contain the real reason the board asse-led the board_spe~;fic_set_broken signal. If a board breaks for a reason other than board specific broken, the board_specif ic_lbroken bit will not be set in the broken status register. I Iv. a~er, board_sp~ci~ic_set_broken may have been ass~ d after the original broken condibon. This would have caused the board specifk broken status register to latch and hold a reason it had set broken. It is i- n~ n~ to ~,ndt" ~nd that because the boarri_specffic_broken bit is not set in the cu------~n broken status reglster, this was not the reason the board went broken. The reason the board went broken is logged in the broken status register and the board specific status register may be ignored.
In the timing diagram below, soa.~one_set_broken is the logical OR of the f ~ lt ~ ;ng signals;

21 July 1995 88 2~

.. . . .....

Xbus Functional ,ecih~;ation ~ .lus Company Co~tide,l~ial cold_reset, warm_reset, slot_parity_error_set_broken, control_or_3way_vote_sig_error, ssic_specific_set_broken, ecc_dbl_enor_in. In addiffon Gambit contains the ~ \;ng signals:
break_pclb, xb_psrity_gen_set_broken. The board wiil break if any broken signal is ass~rl~d on any phase (i.e. any 48mhz edge). my_set_broken_reg is the equivalent of otherside_set_broken_reg on the other ASIC. my_set_broken_reg is ORed with otherside_set_broken_reg to create set_broken_sync.
The ~ on broken status register is frozen when latched_broken_status is true. Note, All status signals for the 1/0 register have the same timing as broken.
Note the bus enor driven on the bus at the time the board went broken. This bming is illl~i~ulldll~.
Figure 50. Board Breaking Tming and lalching (arrived at phase_12_1) 24_MHe ~ -12_MH
SOMEONE_SET_eRCkEN I 1~ ! I I I ~ I I
MY_SET BROKEN I ~ I n MY_SET_BROKEN_R~G I I n SET_BROKEN_SYNCI
DouBLE_BUs_ERROh BOARD_NOT_BROKEN
INFO_OE_ I j I I , CONTROL_OE_ I . I
BUS_ERROR_FREE~

Figure 51. Board Breaking Timing and latching. (arrived at phase_12_2) 24_MH
12_MH
sO~ONE-sET-BRaKEN I I n MY_SET_BROKEN I ! ! n ~
MY_SET_BROKEN_R~G ~ I n SET_BROKEN_SYNCI , I ~ , I l ! I
DOUBLE_BUS_ERRO~ I
BOARD_NOT_BROK~N
INFO_OE_ CONTROL_OE_ BUS_ERROR_FREE7f 21 ~UIy 19g5 89 ~ C~

PCT~US97/09781 Xbus Functional Specification Stratus ~ mpany Confidentia1 9.5.2 Board Breaking Timing on Info Loopback Error If a l~pba~ error occurs that will break the board, bus_error will be as~,ted in POST2 and the board will break in ERR1. Both ASlCs will know of the error beca~se of there are four lines for each info bus that communicate loo~ status b 911rocn the ASlCs (cmp_err_in, cmp_err_out, drive_err_in, drive_err_out).
Figure 52. Board Brea~dng Timing on Info Loop~ack Error OPERATIONSTATE i ARB I INFO I POST1 j POST2 I ERR1 ~CPUTESTiCPLlPOSTI IOTEST jlOPOST1 j BUS_ERROR I I I ¦~ I I I ~ I
BOARD_NOT_BROK~I I j j INFO_OE_ CONTROL_OE_ 13US_ERROR_FREEZE

9.5.3 Board Breaking Timing on Heu~istic or Arbitrary Broken.
~oards thaS break due to a Heuristic or an Arbitrary broken will pa, ticipa~ in the enffre enor sequence and then deas~ll board_not_broken in ERR2.
Figure 53. Board Breaking Tming on Heuristic or Arbitrary Broken OPERATION STATE I POST2 I ERR1 ~CPUTEST~CPUPOSTI IOTEST I IOPOST1~ IOPOST2 I ERR2 1 INFO
24_MH2 12_M~Z _ =
EIVS_ERROR ~ I I I I I I I I
BOARD_NOT_BROKE~N
INFO_OE_ CONTROI _OE_ 8US_ERROR ~FREE

21 July 1995 ~ go ( .

CA 022575ll l998-l2-03 WO 97/46g41 PCT/US97109781 Xbus Functiona\ acib~don ~ ~us Company Confide,. -1 O. Reset In the Polo systsm there are three kinds of reset. A cold reset is gene,alt,d through a softwarec~"",and and implemented as a set of dedicated lines on the Xbus. a warm reset uses the same method. Finally there is a power up/down reset. These resets are de5.;, il,ed in this secUon.
10.1 Xbus nr~,ets Xbus resets are handled differently on Polo than they are on Golfbus systems. In Polo an out of service bus cannot be used for ~al~sl";lting informabon, like llO writes to cause warm or cold resets. Instead Polo implements a set of dedicated lines for this funcUon. The operaUon of these lines is similar in protocol to the ReCC reset on ~etta.
The three-way voted C/D-side c.""~.anson ,-,e..l,an;s." just des.;,il,ed is such that if a board mistakenly drives a U "~.e vr_,~ voted signal due to a hardware failure other boards will see that signal before the driving board breaks. For most of the control signals this does not adversely impact system integrity. An exception to this general nule are the warm and cold resets. Should a board mistakenly assert a reset line. it will eventually break itself but not before potentially resetting one board or the entire system. This section desc,iLes how the Xbus ad~llesses the above issues.
Drivers of reset are required to acbvate reset for 2 4MHz periods (12 24MHz bus phases) in order to cause a warm reset and 3 4MHz periods in order to cause a cold reset. An example of a warm reset is shown below.
Figure 54. Reset Signal rming 12_0 12_0 1 '_0 12_0 12_0 12_0 12_0 ~41Vl nZ ~
12MHz 4MHz Rst Out on Brd on Bokpln /
B~t on Brd /

~ \ /
Wor-- c-~ Ion~t l~t 'Norrn r P_ot ~rUononb cl~ n 11 - on P_ t ml~t~ nll~ ~d b~pl~
1 At 1-t 12_3, th- oppo Ib ~Id Cyclop~ A~IC ch~- th- on bo rd r~t ~lonnl 2 At th n~t 41~ clocl~ (12_0~, t~ oppo lt cld- C~clop drboc ~ ~t b~n ~loncl to th- bu- ASIC
3 At th rwtt 48~ cbc~ 112_1), thn bu~ AUC r~h~ th ~t brok n ~lond 4 Thl- 12 1 cbcl~ ~doo b th~ l~t thc bu~ A~IC c n r~o th- ~ct brol~on ~l~n-l ~nd rn~t th- Illu-tr~d tl~nhl~ ior ~ 1 _ th- ~Igncl on th- bu~

5 At thb 13 3 ~, tho bu i drb~ Ob to ~ll th- b ' ~o ~ to _, I , 1~ d~rt th~
~bn-b It b drblnll on t~ bu~
6 ~u~ A~IC ~nd o~rn-l r~l-tcr db bb b ~1~ n t~Ot r~ct ~111 b db~bbd) 21 July 1995 91 .. . . . . .

Xbus Functional Specification Stratus ~ompany Confidenbal 10.2 Power :;yst~.n r~ rdt~l Reset Board reset, from power related events, is gene,~t~d by logic outside of the s~dnd~d clockina ot the ASlCs. This is necessaly due to the fact that ASIC DLLs have specific reset requirements that must be met. There are three signals associated with the reset sequence. The power supply drives a signal called power_fault_. This signal is asst"~d when the power supply voltage is outof specificaffon. cc_lock_c_ and cc_lock_d_ are driven by the Sentry clock recovery chips when their clocks are out of specification. When all of these signals are d~asserl.3d, logic high, the board level reset sequenr,e is initiated. This consists of the phases ~sc~i~,ed below.
power_fault_ cc_tock_c_ and cc_lock_d_ are ANDed together to forrn alert_. alert_ is used to release dumb logic, reset_n_c~ reset_n_d_, and trst_. The deassertion of reset_n(c/dL causes the ASIC DLLs to begin their infflalization sequence. This can take 4096 48Mhz clock bcks. At the end of this phase, good_power is as~e, t~d signalling that the DLLs should be settled. At that point, the bus ASlCs can conclude the reset seguence and ~asse, l broken.
On power dom, as soon as power_fautt, cc_bck_c_, or cc_lock_d_ is de-ass~-t~d, alert_ is asst:,ted and the power down se4uence begins. good_power is il-U-ledi~UIy de-asss~,led. The bus ASIC must immediately assert broken and reset. 1 4Mhz clock later, trst_ and reset_n(c/d)_ is ass~;,t~d and the power down sequence is complete. A state diagram for this seguence is shown in the figure below.

21 Ju~ 1g95 ~ 3 .
.

Xbus Function~ ,XCJ ~ffon atus CompanyConfidenbal Figure 5~. Power Reset State Machine lalert_ C
~Power Down~
_State ~ ~ J~ lalert_ alert_ ~Bad Power~ ~ Reset Coun~
State State alert_ & 4095 tick~ ~ J

alert_ & 14095 ticlcs ~Good Powe~\
!alert_ ~ State J

alert_ alert_ = power_fault_ & cc_iock_c _ 8 cc_iock_d_ 21 July 1995 93 ~1 Xbus Functional Specification Stratus ~,ompany Confidential Figure 56. Board resets.
, ~,, or t ll bck chip tailu~
VCC@4.W ' r~
PowER-FLT
CC_LOCKtClD)_ RESET_N(Cn~)_ I I I
TRST_ Ai~ERT_ POWER_OK
004CJD)~
COLD_~el~.10~_1 ~1 1 WARM_~t:~tll~1 BROKEN
Pow~r d~n ~
r~t count ritab 40~6 48MHZ TICKS (ma gxd pow~rdate~
--I bad power~tate~
po~ r do~o_ VCC @4.8v As)n ,d~,onous signal internal to power supply. Re,~fesef,ts that the power supply is at 4.8 volts or greater.
POWER_FLT_ Asyl l~l "onous signal driven from power supply to reset logic only.
rep,ese"ts that the power is within specification.
POWER_OK Sy"~l,rùnous signal derived from the alert_ and driven to only the bus ASIC.
CC_LOCK~C/D)_ Asy, I~A " onùus signal from cun ""on clock chip driven to the reset logic.
RESET N(C/D)_ Asy". I"~nous signal trom the reseS logic and driven to all ASlCs with a DLL. This signal must be r leased before the ASlC's can lock their clocks.
At this point it looks like this signal can be the equivalent ot the power supply's DUMB signal.
OOL(C/D)_ Intemal ASIC signal. Means the clock has achieved lock.
~OLD RESET(C/D/) Syncl,-u"ous signal driven from the bus ASIC and received by all other ASlCs. This signal iniffalizes most intemal logic.
~ARM RESET(CID)_ Syn~ ,nous signal driven from the bus ASIC and received by all other ASlCs. This signal initializes all intemal logic.
TRST_ Asy"ch,unous signal derived from DLL_RESET(C/D). Needed on all ASlCs to iniffalize JTAG.
BROKEN S~ I"unous signal internal to the bus ASIC. Means the board is broken.

21 July 199~ ~ 5 94 Xbus Functiona "ec . ~tion ~tus Company Confi~le, -' 10.3 FaultTolerantlssues:
POWER OK is a s~"cl,lonous signal driven from as~ ;l"unous ALERT_. It is not possible to remove a single signal run here. I loJ~ cr, If POWER_OK is stuck false, the board should never come out of cold reset and never clear broken. If POWER_OK is stuck true, broken will be guaranteed set by the OOL_ signal. When broken is set, it needs a reset complete signal to occur before the board will clear broken. This cannot occur t~ec~ce reset is not active.
The power supply design cunently asserts ALERT when CC_LOCK is lost. RESET_N_ should not be driven i"""edi..~ after power or CC_LOCK_ is lost as this will cause even more clocks to go out of sync. RESET_N_ may be driven however after the board breaks.
An out of lock (OOL) cor.~;L ~ n on bus ASlC's will break the board but for other ASlC's we will rely on the out_of_lock condibon to break the board for other C and D dilfa.eno2s. Therefore a failure of reset_n_c_ or reset n_d_ to a bus ASIC is not a single point of failure. I 'D~ ~r, a failure of reset_n_(cJd)_ to non-bus ASlC's will not cause an immediate broken condibon. On boards that do not have any other bus interfaoes (CPU boards), the OOL condibon will eventually break the board when bad data is bcln~fe,.ed ~hreen ASlC's. If the back-end ASIC loses its lock on (I/O) boards that have buses in addibon to the Xbus, the board may still cu,,,~,e data correctly but not be able to talk to the bus in qu~ ~ L ~ n due to timing viol~tions. This should be taken under oonsidera~bn on a board by board basis. If it is determined that broken should be set on an OOL condit,ol~ then the boards may do so via the board specific_set_broken signal.
10.4 ASIC Pins Required For Reset Functions.
The fo11~ ..9 pns are required for each ASIC.
resetn_ The Dll reset in signal trst_ JTAG reset.
warm_reset_ Wamm reset out cold_reset_ Cold reset out 10.5 DUMB FErs The Polo DUMB circuitry is a variation on the Stratabus boards' dumblalert daughter card. This card provides dumb clamping when the board is in the power down and reset_count states.

21 July 1995 9!i, Xbus Functional Spedfication Stratus _ompany Confidential 11. Xbus Physical Partitioning This section gives a high level overview of the physical partitioning for the interfaoe ASlCs to the Xbus. It des~il,es the c~n".onent~ necessaly to inter~ace to the Xbus as well as a functional desoli~on of how the interface is i"tel,ded to operate. The fault tolerant strategy is also desc.l iL ed.
11.1 Info Bus Partitioning The C and D ASlCs on the CPU board each drive half of the bits on the info bus when they are bus master. The C and D ASlCs on the PCIB drive either half or all of the bits, depending on the type of transter. During normal ~sses they drive half the bits as shown; during single-side ac~sses they drive all of the bits, including the parity bit.
Figure ~7. Info Bus Partitioning 6 0 0 31 ~4 23 16 15 8 7 0 0 I [1 ~ ~ I 0 trid fuDc opIDfo Bus Panty Bits a~ as~igned sSarting from th~ least ~i~, '~ bit.

[~ Driven by Ihe D~ide ASIC O Drh~en by tbe C~side ASIC
during oormsl acce:sses, aDd duriDg Dormai 9a:~9ses, aDd bl~ Ibe driving ~SIC dumlg by Ibe driving ASIC during ~ ' IlOCeS~ 9a~SeS.
Parity is driven by tbe D-side on the CPU.
It is driven by the D-side of the PCm durinO nonDa1 ~ccesses, and by tbe driving ASIC during SinOle Side a~:esses.

11.2 ControlSignalPartitioning The D-side ASlCs drive tne control buses, and both C and D-side ASlCs check.
11.3 Xbus Voted Signal Partlffoning The 3-way voted signals in a Polo system have varying t F o'c ~ ~ depending on function. In general, each ASIC on a board drives half of the ~way voted signals originaUng on a board, and the other side ASIC sits at the end ot the net and checks what is driven.
In the f ''c. i.,g ~liay-a",s, each t,dnsc~ er section (shown as a NAND gate or inverter) of a 3 way voted triplet is in a sepa,.~te p3c~9e, to prevent a single lailure from disturbing more than one bit of the triplet.

21 July 1995 96 CA 022~7~11 1998-12-03 Xbus Functiol ir~c ~cn ratusCompanyConfidential Figure 58. board_not_broken_ routing ~D~ X cD
D ~p Y G~ ~ D

~D~ Z CD
C ' C
~0 ~0 , etc.
~ each board receives --LJ 3 dedicated board_not_broken lines from the three other boards ~v etc.

~--~~ ~ ok ,~

Figure 59. reset_ routing C~p x cD
D D~ Y cD , D

C)~ Z CD
C ~ C
D~
etc. The PClBs receive a voted D~ triplet from each CPU board.
The CPUs receive one triplet from the other CPU.
The same reset lines are used D~ ' etc. for sofh~are reset and RECC
~systemt reset. When RECC
D~ reset is asse,t~d, all reset lines go active.

21 July 1995 97 PCTnJS97/09781 Xbus Functional Specification Stratus _~mpany Confidential ~igure 60. sync_ routing D '' ~ D
~ y ~
C ~ ~ ~ ~ C
The sync signals are the only bidirectional 3way voted signals.
Figure 61. online_ rou~ing ~ D~ X cD
D --D~ Y cD ~I D

D~ Z ~D
C > C
The even_online and odd_online signals are unidirectionat b.h~een the CPUs.

11.3.1 ID PROM Pallll;~.~i..~
GolWetta/Polo systems implement an IDPROM architecture based around a 2Kx8 EEPROM
A~cessed via a JTAG scannable buffer. The Xbus perrnits ID PROM access under system s~"tv~_,t, control as with the current Golfbus, but also supports:
1 ) IDPROM access as part of the board level boundary scan chain 2) IDPROM access on an ur~pu~.e~bd freestanding board These later two ~-lhanc~--,ents ease manufacturing's utiliation of the IDPROM.
The IDPROM is JTAG 1149.1 ~Gcesc;l~le. JTAG is an industry s~ncla,d 4 pin serial bus usedtor testing the inte.~onne.,l between ~""~on~u~c on Polo boards. Initially the test bus will be driven from IBM compatible p~;,;,onal computers e~uir~ped with JTAG software/hardware acc~r"~"~;~ ,g all Polo debug stations.
A diagram of the ID PROM implernentation is provided in figure 62 below. Nommally, the illustrated jumper is always present. The jumper block must be removed If manufacturing wants to access the ID PROM wi~out pO~3~-119 Up the board (or before the board is tested or tully populated).
Once ~e jumper is removed, the ID PROM can be ~c~ssed via the illustrated co~-nectur which provides both the JTAG signals and the power and ground to the ID PROM logic.
In normal ope, _tion (with the jumper present) the board comes out of the reset state with the IDPROM in the board's scan chain, i.e. the MUX is selecting the A inputs. Once softwa,e makes any access to the ID PROM logic through the bus ASlCs ID PROM registers, the MUX B inputs are selected, thereby removing the ID PROM logic from the board~s normal scan chain and 21 July 1995 98 , Xbus Function, ,pecl,lcation atus Company Confidential conne~,~. ,9 it to the bus ASIC instead.
Figure 62. ID PROM Implementation X2816C 2Kx8 EEPROM a ~ 8 ~
, ~ , ABT18245 c~
m m o ~

~J Manufacturing " ~ Free Standing v . . ~ Access Jum~er Rlock I I Connector Isolates IDPROM from board for manufacturing Board Vcc access when board is free standing Board Gnd C-Side ~ 1 l D-Side Bus Quad Bus ASIC ~ /A B A B A - A \ 2-to-1 ASIC
id_prom out[0] id_prom_out[0] ¦
¦ (MUX cntl on C-side) (TCK on D side) id_prom_out[1] L id_prom_out[1]
~TMS on C-side) ~TDO on D-side) id_prom in] ~ ~ ~ id_prom in[2 ¦ (TDI on C~D-sides) (TDI on C&D-sides) ¦

--TCK ~ ~Normal TMS ~ Board TDO ~ Scan TDI Chain When MUX is in the "A" posibon, the IDPROM logic is part of the normal board scari chain. When MUX is in the UB" posWon, the IDPROM logic is r~"w~ed from the board scan chain and is ~rcessil~le from the bus ASlCs.
Warm or Cold reset place MUX in the A position; any bus ASIC access places it in the B position until the next reset.

21 July 1995 99 Xbus Functional Spe~ifi~bon Stratus _Jmpany Confidenbal 12. Xbus Illt~l ~ace Block Diay~,..s The Xbus interface portion of the ASlCs that are on each board in the Polo system is referred to as the Xbus cu""-,on logic. A single set of source files is maintained and the logic is cu~tu",i~ed by Verilog ifdefs and induded intû the Gambit and Cyclops designs.
12.1 Xbus Top Level Block Diagram AS~C ASIC ~clc outbound specific m~ound ASIC
logic ~ lO regs logic specfflc loglc _ _ _ _ _ _ , Xbus CGIl~-n~,~
e~reck re~ t rb~ock logic ping outbound inbound logic pipe cobn~o~ p~pe + ~ ~

12.1.1 Port List The Xbus ~,.,...on logic corlne- t~ to the ASIC specific logic within the bus ASlCs and to the Xbus side ASIC pins. Also included in the ~m...on logic (but not shown above) are miscellaneous support modules: the clock phasor gene,~tur~ reset gene.~t~ r C)LL active clock spine etc.
Port lists for signals entering and leaving the Xbus com---on logic are ~iscussed in the sections specific to each module.
12.1.2 Functionsl Description The Xbus w,-----on logic provides all Xbus error cheching and board state ~--anagemenl (broken duplexed on~ine etc.). It contalns all control for IO register fl~sses although the leg;;,l~l~
Ule..lsell~s are split ~e~,een the c~-.-- - -on logic for bus related regi~st~ . ;, and the ASIC specific logic for board specific reg;_~. s. I D~-Iowr the IO register contr~DI handshakes with and passes read data through ffhe ASIC specific ouff~ound control unit (OCU); ffhis is because ffhe OCU
requirements vary greaffy be~reen different board types and is best implemented as ASIC specific logic.
The inbound pipe provides a raw ~before bus error and busy are available) version of the Xbus 21 July 1995 100 Xbus Function~ pecil,~ation ~tusCompany Confidential data to the lO le9;~ .a, ASIC specific inbound pipe, and error logic for ack, busy, and error generaffon. Two (or more) phases later, it delivers qualified, known good data to the lO registers and ASIC specffic inbound pipe for p-ucessi"g. On CPU boards, the inbound pipe also pe.~"-"a the IOVA address r~, l PF ~9-The outbound pipe pa~ L~ges data for b~i1s".;~ion on the Xbus. It does not buffer the data beforeass~:"lLly into a bus hal~6a~ ~n; rather, it sends mux selects up into the ASIC specific logi c as needed to construct the transaction. It stores a copy of the transacbon in a recirculation pipe in case the data needs to be transmitted again as a result of a bus error.
t2.2 Inbound Pipe Section The inbound pipe watches the Xbus bus cycles and ackn ~ !ed~ea, busies, or errors bus cycles based on intormation contained within the cycle. The inbound pipe also p-oiides timed versions of the Xbus for use intemal to the ASIC. Figure 63 contains a block diagram of the logic contained within the inbound pipe.

21 ~uly 1995 q~

CA 02257~11 1998-12-03 Xbus Func~onal Speuficauon S~atus npany Confidential Figure 63. Inbound Pipe Block Diagram _ indKates i . ~ toblock KEY: ~A-info a reg,trid a r~g,func op a reg ~B~info b_~eg,trid b Teg,func op b reg b ~g_ -rite dds[31:0]
bnd sd~ t ble b d ~dtress ta ~ e d ta 0 bsd atdr~s ta~ e dst~ 1 losd sot~s t~o e dst~ 2 resd ood~s t~o~e 1~' add Yalit ~d ~c~ t~ble data _ data vslid~
resd acd~s t ble d~ta ~cb bus ~r I l~ternal ast mfo ~cb_busY ~ Control ~ failed op .... . .
110 Map~gislers ",~
P~"~ ~ pre qual trid optiODS/ ~L -- --PCI slol ~
A lK~cll ; ~ 3 & msp er~r .. _ _ nh~s start ~ ~
A map~eg out~31:0]
~ . . 1 : . :. . . . . . . .'-' ' A
~irt ' ~
ph~rsend pre qual bus post qualbus ~ ' ' _ __ use a ~ru~t n ~ ~cb no ack ~nt~ b oo ack trid In~ert ~elect ~cb no_ack si~e Return own~nt ping_no_ack ~ IControl ~
piDg Do sck tr~d~ ¦

I

~cb,pin~ no ack done,inpipe bus~ r Info Postl Pod2 I Posl2~1 pb Je ¦ phase ¦ phase ¦ phase 21 ~luly 19~5 10Z

.

CA 022575ll l998-l2-03 Xbus Functiona. "ecincaUon atus Company Conriden 12.2.1 Port List Table 1g. Common Logic Inbound Pipe Port List signalname width l/O descripbon addr_40_rnode 1 1 (Cyclops only). This signal tells the map ram logic to gene~te 40 bit style virtual address fields on the pre_qual bus (only valid for map RAM IOVA
cycles).
board_not_broken in_n 1 I neg;t~d not broken signal from ne;ghl,o~
board_not_broken_in_o 1 I ne9;~t~lbd not broken signal from opposite bus_oe 1 1 (Gambit only) This signal is used for failed op insert on no-ack cycles and indicates that the wrrent read request was driven by this ASIC.
clk48 1 1 48MHz clock input cold_reset 1 1 (Cyclops only) cold reset signal - used for reselti. ,9 the map registers.
err_in_p ug~ss 1 I Signal from error block true when currently in an error sequence phase.
func_op_a_reg 1 I Xbus A bus func op from input reg-;.te,~.
func_op_b_reg 1 I Xbus B bus func_op from input registers.
ibi_efq_avail 1 1 (Cyclops only) Tttis IBus inbound pipe signal indicates that the EFQ pipe can accept an insert retum cycle.
info_a_reg l31:0] I Xbus A bus info from input r~g;~t~
info_b_reg ~31:0] I Xbus B bus info from input ,~:g.;.tt, ,.
io_reg_u r~te data ~31:0] 1 (Cyclops only) From the register control block the data to be written into a given register on a load_address_x)o~ signal being asse-t~d.
Ioad_address_table_ 1 1 (Cyclops only) indicates that the address table ce-""~nd co"".~nd register is being written.
load_address_table_ ~1:0~ 1 (Cyclops only) indicates that the address table data data register 2, 1 or 0 is being written.
online 1 1 (Cyclops only) Board online signal from board states logic.
phase_12_0 1 1 12 MHz phase clock 1st half of cycle 0 of phase phase_12_1 1 1 12 MHz phase clock 2nd half of cycte 0 of phase phase_12_2 1 1 12 MHz phase clock 1 st half of cycle 1 of phase phase_12_3 1 1 (Cyclops only) 12 MHz phase clock 2nd half of cycle 1 of phase 21 July l995 '2 ~ 103 Xbus Func~onal Specification Stratus ~mpany Contidential Table 19. Common Logic Inbound Pipe Port Lis~
signal name width IIO description phase_24 1 1 24 MHz phase clock ping no_ack 1 1 (Cyclops only) This signal from the ping module a PING it sent out on the bus was not acl~n~le~ed.
ping no_ack_t~d l6:4] 1 ~Cyclops only) This signal from the outbound pipe indicates the TRlDl6:4] of a ping_no_sck cyr~e.
pu dside 1 1 (Gambit only) From an ASîC inpllt pin high - D
sideASIC low- Cside read_address_table_ 1 1 (Cyclops only) indicates that the address table cG.. ~nd oor -- )ancl register should be placed on the map_reg_out bus for reading read_sddress_tab~e_ 1 1 ~Cyclops only~ indicates thatthe address table data data register 2 1 or O should be placed on the map_reg_out bus for reading read io_map_enor 1 1 (Cyclopsonly) indicatesthatthe l/Omaperror register should be placed on the map_reg out bus for reading rearm_error_regs 1 1 (Cyclops only) slot_id_latched 1 1 (~3ambit on~y) one bit slot id registered off input pin at reset trid_a_reg [6:0] I Xbus A bus trid from input registers.
trid_b_reg ~6:0l I Xbus B bus trid from input registers.
update 1 1 (Cyclops only) Board update mode signal from board states logic.
warm_reset 1 I s~ onous warm reset to block xb_bus_en 1 I Xbus bus error A or B registered - created from 1hree input control ~uses and one output control bus.
xb busy 1 I Xbus bus busy A or B registered - created fromthree input control buses and one output control bus xb_grant_neighbor 1 I Xbus grant signal from a ~ ticn block valid one phase prior to ARa phase at 12_3.
xb_grant_opposite 1 I Xbus grant signal from arbitration block valid one phase prior to ARB phase at 12_3.
xb~rant_peer 1 1 (Cyclops on~y) Xbus grant signal from arbitration - block valid one phase prior to ARB phase at 12_3.

21 Ju~ 1995 ~ 9 S 104 CA 022~7~11 1998-12-03 wo 97/46941 PCT/U$97/09781 Xbus Functiona~ ,,eu,.vation ItUS Company Co, lfiden~dl Table 19. COI~ Gl~ Lo~ic Inbound Pipe Port List signalname width llO descripffon xb_no_ack 1 l This signal from the outbound pipe indicates a read (l/O or memory) it sent out on the bus was not ;i..hl ~uwledged.
xb_no_ack_size 1 l (Cyclops only) This signal trom the outbound pipe indicates the size (high = 256, low = 32) of an xb_no_ack cycle.
xb_no_ack_trid l6:4] l This signal from the outbound pipe indicates the TRlD16;4] of an xb_no_ack cycle.
xb_own_grant 1 l Xbus grant signal from arbitration block, valid one phase prior to ARB phase at 12_3.
early_xb_p2p_1ast info 1 O (Gambit only) This signal indicates that the last xb_p2p_valid is coming.
failed_op 1 O When true, this signifies this is the retum for a t.a"iacba n that has timed-out (i.e. ping not ACKed) or that a simplexed bus master has 'broken' t~fore completing a transter involving sub-operations, or that a read issued on the Xbus has not been ACKed. Wfites do not cause failed_ops. Note for this condition, the address portion (or first data in the case of a data return) of the transfer completed un-errored and un-busied but the last data has not yet been I,dnsfe"~d. This condition cannot occur on bus ope, dlions that do not have any sub-oper ~ions. If failed_op is true, add_valid, data_valid, retum_valid and last_info may also be true. These signals will emulate the completion of the sub-op~,dliuns. failed_op will remain true until all the su~operdtions assoc ~ with the b.lnia~ticm have been emulated. When failed_op is true, The post_qual_bus is gud,d,l~aed to be all O's. This signal is clocked by dk48 and is qualified by phase_12_0.
illegal iova 1 O (Cydops only) This signal indicates that the cunent pre_qual bus contains a cyde with an lOVA fault and therefor should be ignored).
inpipe_busy 1 O This signal indicates that the cunent Xbus cycle should be busied bec3~se the inpipe is busy servicing an insert retum.
insert_retum_need_efq_ 1 0 This signal illJi~ ~es that we are currently in post 2 post2 of an insert retum and the retum will start with the post_qual bus in post 3.

21 July 1995 105 -CA 02257~11 1998-12-03 Xbus Functional Specification Stratus _ompany Confidential Table 19. Common Logic Inbound Pipe Port List signal nams width IIO description map_error 1 O (Cyclops only) This signal indicates that a map look-up enor occurred on an IOVA Xbus access.
This signal is valid in the first half of Post2 and is sent to the co"""on register logic where it will be stored in the ASIC specific broken status register.
This signal is also used intemal to the block to ~nhibit a. h,ov.~ed~,.,6,l~ of the cycle. Map_errors do not cause bus errors.
map_rea out l31:0] O (Cyclops only) This 32 bit bus contains the value from the last map register intemal to this block that was read.
pinp no_ack_done 1 O (Cyclops only) This signal indicates that the insert return resulting from a ping_no_ack has completed and the trid no longer needs to be held valid.
post_qual_func_op 1 O This is the post_qual bus func_op.
post_qual_hit_status 1 O (Cyclops only) This signal is valid when an info cycle is on the post_qual bus and is equal to the hit status found in the TRID.
post_qual_info 131:0] O This is a pos~+1 (i.e. one phase after post2) version of the info lines on the bus in use. A valid address is presentwhen add_valid is asse,led and valid data when data_valid is assei ted. This illfolll~dth~ll has been qualified with busy and error and is fit to be passed deeper into the system.
post_qual_retum_256_32 1 0 (Cyclops only) This signal is valid when a data retum is on the post_qual bus and indi~;ales if the retum length is 256 (high) or 32 bits (low).
post_qual_trid [6.0] O This is the post_qual bus trid.
pre_qual_ben l31 :~l O This is a post1 version of the byte enable lines on the bus in use. Informabon on these lines should be used for generabng busy and ack and then di~ a-ded. The byte enables are valid during the second half of post 1.
pre_qual_func_op 1 O This is the pre_qual bus func_op.
pre_qual_func l3t :0] O This is a post1 version of the function code lines on the bus in use. Inforrnabon on these lines should be used for generating busy and ack and then di~ carded. The function code is valid during the second half of post 1.
pre_qual_info [31:0~ O This is a post1 version of the info lines on the bus in use. Informabon on these lines should be used for gene,.~ g busy and ack and then discarded.

21 July 1995 ~ 106 . .

CA 022~751l l998-l2-03 WO 97/46941 PCTrUS97/09781 XbusFunction~ peci, 4tion atusCompanyCo, ~er,i Table 19. Common Logic Inbound Pipe Port List signal name width l/O descripffon pre_qual_ping 1 0 (Cyclops only) This signal is valid when a ping is on the post_qual bus.
pre_qual_rc 131:0] 0 This is a post1 version of the remote/col)e,~nl lines on the bus in use. Inforrnation on these lines should be used for gene,ati.~g busy and ack and then disca(ded. The r/c is valid during the second half of post 1.
pre_qual swap 1 0 (Cyclopsonly)Thissignalindicatesthatthe pre_qual bus contains a cycie which should be loaded into the swap table if accepted.
pre_qual_trid 1 0 This is the pre_qual bus trid.
xb_add_valid 1 0 When true this signifies there is a valid address phase on the post_qual_bus. This is a func_op phase and it is not an idle phase. It has not been busied or errored by anyone on the bus. It does not mean the address is neoessa, ily to my space. This signal is clocked by cik48 and is qualified by phase_1 2_2.
xb_data_valid 1 0 When true this signifies that there is vaiid data on the post_qual_bus. This data is assoc;~d with a previous add_valid and has not been enored by anyone on the bus. This signal is not true during data retums. This signal is clocked by clk48 and is qualified by phase_12_2.
xb_iast_info 1 0 When true this signifies that this is the last phase of i"fu-",alion ~ssoc~ d with this bus o~e,alion (It may be the only i, If o,,,,ation needed). This signal is valid for a signal phase. If this is the first piece of illh "-dtion it has not been busied or enored by anyone on the bus. If this is not the first piece of intormation it has not been errored by anyone on the bus. Note xb_add_valid xb data_valid or xb_return_valid can be tnue while last_info is tnue.
This signal is clocked by clk48 and is qualified by phase_12_2.
xb_next_add_valid 1 0 This is the xb_add_valid signal prior to being It:y;_~led at 12_2 xb_next_data_valid 1 0 This is the xb_data valid signal prior to being .egistè,ed at 12_2 xb_next_retum_valid 1 0 This is the xb_return_valid signal prior to being ,~:gist~ ed at 12_2 21 ~uly 1995 107 ~.9''~

PCT~US97/09781 Xbus Functional Specification Stratus ~,~mpany Co"' ~ al Table 19. Common Logic Inbound Pipe Port List signal name width l/0 description xb_no_ack_done 1 O This signal indicates that the insert retum resulting from a xb_no_ack has completed and the trid no Ionger needs to be held valid.
xb_p2p_func_op 1 O (Gambit only) The peer-t~peer send post qual func_op bus sent to the X~0.
xb_p2p_incomplete_send 1 0 (Gambit only) This signal indicates that the p2p send in p,uy-ass has been dropped and should be di~-~d.
xb_~2p_info 1 0 (Gambit only) The peer-to-peer send post qual info bus sent to the XB0.
xb~2p_1ast info 1 O (Gambit only) This signal indicates that this is the last xb_p2p_valid and the p2p echo can begin.
xb_~2p_trid 1 0 (¢ambit only) The peer-to-peer send post qual trid bus sent to the XBO.
xb~2p_valid 1 O (Gambit only) The peer-to-peer send valid sent to the XBO. This signal indicates thst the xb_p2p_info, trid, and func_op are valid xb_retum_valid 1 O When true, this signffies that there is a valid data retum on the post_qual_bus. The lower 4 bits of the trid ",atcl ,es this slot address (and duplex/
simplexed state). This signal also indicates that the data has not been busied or errored on the bus.
This signal is clocked by clk48 and is qualified by phase_12_2.

12.~2 Functional Descrl,ution The Xbus inbound pipe handles all issues relaUng to the inbound Xbus. This block generd~es bus error and the acknowledge (with some help from board specific logic). It also gene.dtes busy with input from four different sources: the l/O register block, the ping logic, the board specific logic, and the rnap RAM logic intemal to this block.
The info and trid buses are registered and made available to the rest of the logic on the post_qual and pre_qual buses. The post and pre_qual trid, func_op, and info buses are simply referred to as the post_qual or pre_qual bus in figure 63 in order inc,ease readability. The l/O address rnap resides within this block so that IOVA Xbus add~esses can be converted to system ad ilt:sses before putting them on the pre-qual bus. A map error of any sort will inhibit the bus ach~o~le~e and assert the signal map_error. The timing of the post and pre-qual buses relative to and Xbus cycle is shown in figure 64.

21 ~luly 1995 ~ Cl C~ 108 ..... . . . . .

Xbus Function~ ,pecl"cationatus Company Confidential Figure 64. P~st-Qual and Pre-Qual Bus Timing ' info ' post1 ' post2 info_a_in ~3<
info_a_reg ' }d~ ' prequal postqual ' ' ' '~3<
add_valid I I , ;/ ~dd~ v~lld \_, ~ . ..
IOVA ~ Virtual ~ddress PAD - Physic~l address The valid signals indicate when the various buses can be used for address or data. Figure 66 iilustrates the timing of data valid and last_info for a 2~6 bit transfer on the Xbus. The timing for retum valid is identical to the timing for data_valid.
Figure 65. data_valid and last_info Timing infbo , IPn~fot1 ' ppOSst1 ' post2, , arb , Info , post1, post2, , arb , Info , post1, post2, info_a_in ~3~3 info_a_reg prequal postqual ' ' ' ' K~d= ' data valid I I , . . . . . .
-last_info ' ' ' ' ' ' ' / ' ~
~ . . . . . .

The page 2 SAM rey;ste,s (IAM ASIC and Cyclops ASIC Map RAM Registers) are also located in this section within the Map RAM Control block (CPU only). These are the registers that control the l/O address table (i.e. Address Table colllllldnd~ Address Table Data 2,1, and O registers). These registers can be written directly from the post_qual bus or read with the data being placed on the map_reQout bus two bus phases after Post2 and kept there until the next read. It is also possible to read and write the map RAMs via these registers with that control also placed in the map RAM
control block. Refer to section 12.6.2 on page 118 in the Polo R~uyl~l""ling Guide for details on the l~:g;~t~ intemal to this block.

21 ~uly 199~ 10 3~

.. . .

Xbus Funcffonal Spe~i ~ffon Stratu ~mpany Confidential 12.3 PlNGSection Figure 66. PING Control Block Diagr~", address v~lid, 0Dtrol_in post_qu~l bus pre_qu~ bus pi4,_~ck_in re~d_pin;~iDterval returD v~lid r----~ ~----------------------------------_ _ _ _ _ _ _ _ IL~ _ 3 _ l b~d ~ 2 _ s~te ~ piDg oouDter ¦ ¦
0ntn~l ~ _ , e m~chiDe 0 _ ping Ynd : -trid table ping ~clc ~ ' du m~ne?
PING MODULE
L _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _l piD D-O hll busy ping Jlck out piDg on rdy pinLtrid for bus error The PING block cor~rols generation of PlNGs tor l/O reads that are waiting for re~ponses and acl~u. Ied~,..e,~t of PlNGs coming from other boards.

21 July 1995 . 1 10 3c-~ ) Xbus Functiona~ ~C~ n tus Company Co,hiden- -' 12.3.1 Port List Table 20. PING Logic Port List signal name width l/O ~sc,i~Jlion addr_valid 1 1 output from inbound pipe, used to geneldl~ ack for incoming PlNGs. Indicates that the post_qual bus has a valid address cycle presenL Tied to xb_add_valid of the XBI.
clk48 1 1 48MHz input clock inc_invalidate 1 1 (Gambit only) Indication to invalidate a ping table fifo entry.
inc_trid_to invalidate 1 1 (Gambit only) Trid of the entry that is to be invalidated via inc_invalidate.
inc_nodrv_npaired_rd, 1 1 (Cyclops only) These signals come from the inbound co,~,-,.on_nodrv_ 1 pipe and l/O register blocks and are loaded into the npaired_rd ping table with each outstanding IO request. When true they indicate the cunent ItO read being loaded into the table is a non-paired request to partner so this ASIC should go through the moffons but should not drive the Xbus.
inc_take_cycle 1 I These signals come form the inbound pioe and l/O
ioreg_take_cycle 1 register blocks and indicate to the ping logic to load the bdnsa..tion on the post_qual bus into the pin table ioreg_cd_dmerent 1 I This comes from the IIO register block and indicates that the l/O register read that is being placed in the trid table is CD differenL
ioreg_read_ping_ 1 1 (Cyclops only) This signal comes from the co,.. "on interval register block (bit 0 of the read ping intelval register).
Lo - ping after 100 usec; Hi - ping after 2.667 usec.
online 1 1 (Cyclops only) Board online signal from board states logic.
u~t;,~i.le_ping_trid_ 1 1 (Gambit only) Indicates PING TRID Illatclles other-matchi side ASIC
phase_12_0 1 1 12 MHz phase clock for 1st half of cycle 0 of phase phase_12_1 1 1 12 MHz phase clock tor 2nd half of cycle 0 of phase phase_12_2 1 1 12 Mltz phase clock for 1st half of cycle 2 of phase phase_12_3 1 1 12 MHz phase clock for 2nd half of cyde 2 of phase ping_no_ack_done 1 1 (Cyclops only) Indicabon from inbound pipe that no ack p,ucassing is complete (insert retum finished) post_qual_bus ~ 6] I The post_qual (after Xbus .e:.~onse) info bus from the XBI.

21 July 1995 111 3~1 PCT~US97109781 Xbus Functional Specificadon Stratus .npany Confidential Table 20. PING Logic Port List signal name width l/O des~ on post_qual_return_valid 1 1 (Cyclops only) Output from inbound pipe, used to clear entries from the PING table. Indicates that the post_qual bus has a valid data retum cyde presen~
rled to xb_retum_valid of the XBI.
post_qual_t~id 16:0] I The post_qual TRID bus from the XBI.
pre_ping 1 I The pre_qual PIN~3 indication from the XBI.
pre_trid 16:01 I The pre_qual TRID bus from the XBI.
warm_reset 1 I s),ll~l.lonous warrn reset to block xbus_ack 1 1 (Cyclops only) This is the Xbus ac~ O~ e signal used to determine whether the PING was ACKed xbus_op_done_ack 1 1 (Cyclops only) This is the Xbus op done from the XBO. It is used to push ou~ing I~O reads onto the PING ~IFO.
xbus_op done 1 I This signal comes form the outbound pipe and indicates the final piece of illfulllld~ion went across the bus without a busy.
xbus_op_done_trid 16:0] I This signal comes form the outbound pipe and indicates the trid of the completed transaction xbus_op_done_func [5:01 I This signal cornes form the outbound pipe and indicates the function of the completed transacbon xbus_p2p_issue 1 1 (Cyclops only~ This signal comes form the outbound pipe and indicates that the l/O read being issued is peer-to-peer so any PlNGs will have to be also.
xbus_trid_for_bus [6:0] I This signal comes form the OCU and indicates the trid of the ~ansac~Jn in plugless.
nodrv_npaired_rd_out 1 O This signal comes from the trid table fifo and indicates that the current read retum is a non-paired read. This is to be used by the che~ ing logic as well as the arbitration.
cd_dlfferent_out 1 O This signal comes from the trid table FIFO and indicates that the cunent read retum that we are perforrning is CD different and the che~ i,lg logic needs to be aware of that, error_no_ack 1 O This signal indicates that we had a no-ACKed png error_trid [6:01 0 This is the trid of the no-ACKed ping myside_ping_trid_ .1 O (Gambit only) Indicates PING TRIO "lal~hes this matcho ASIC
ping_ack_out 1 O sent to the ack loglc, indicates ping on bus should be ach~ lEdged, ~.alid 12_0 of post2.

21 ~uly 1995 3 112 ~ O

Xbus Functiona. ecinudtion ~ tus Company Confidential Tabie 20. PING Logic Port Llst signal name width l/O dss~,i,~ion ping_fifo_full_busy 1 0 This signal goes to the busy block in the inbound pipe and indicates that this cycle should be busied.
ping_op_ready 1 0 This signal indicates to the OCU that there is a ping ready to go.
ping_trid_for_busl6:0] 0 Trid to be used for ping cycle - to the OCU
ping_p2p_issue_for_ 1 0 This signal is sent to the OCU as an indication to bus whether the ping is a peer-to-peer b~fisa ian 12.3.2 Functional Description This module has two functional pieces: the send sub-module and the ack sub-module.
The send sub-module keeps track of the outbound read requests and their trids. This block also pings l/O read requests that have bmed-out. The ping frequency is detemmined by the input read_ping interval from the col"",on register block. During normal ope~lion the interval will be 100 usec however the ping interval can be set to 2.667 usec for testing purposes. The send module has an intemal ping count used to keep track of when a cycle should be pinged.
The ach sub-module trachs incoming read requests and their status. It also achno.vledges any incoming pings that hit a trid cu,-esponding to an active read ~espùnse being worked on. A trid table in the ping ack module keeps track of inforrnation regarding a read cycle that has been 'e~ by this ASlC. lt maintains this inf " " ~n for a~ h ,owledging pings and for use during the read retum. Information specific to the read retum such as the trid and whether a given cycle is CD
dfflerent is stored in the trid table. The table can hold up to seven entries; once it is full it sends out the signal ping_fifo_full_busy indicating that the current b~n:,a~ tions to be busied on the Xbus.

21 ~luly l 995 113 3~

wo 97146941 Xbus Func~onal Sp~ ~don Stratus Inpany Confid~nbal 12.4 Xbus Outbound Data and Control Unit Figure 67. Outbound Data and Control Unit - XBUS OB Decode ~UiU:j Vl:S ~ilaVe (~4) 1 - ~U~ ~no ~la~e (~4) XBUS OB Master XBUS OB Peer-to-Peer Control Echo Master Control (lO devices onlv) INFO Bus Post-Qual Bus ~ 32 ~ 32 64 Bit INFO Re~ . s64 Bit ECHO Re~,lsl~. s (XS) (10 devices onlv) (X5) I

A's S's O's /out_mux_sel ~ ~ Cycle info_out -- -- -- -- _ _ _ _ _ 21 July 1995 114 3~

Xbus Functional a-~ ,jlti~r~ tus Company Confidenbal 12.4.1 Port List Table 21 des~ ,es the port list to the outbound data path.:
Table 21. Common Lo~ic Outbound Pipe Port List signalname width l/O descripbon go_arb 1 O This signal is driven from the master state machine to the arbitration logic. It is used to initiate an arbitration cycle for the Xbus/
xbus_op_done 1 O This signal is passed to the OCU. It tells the OCU
that the last ope, ~ic n has been sent across the bus (without being busied) and that the infonmation may be overwritten. This signal is qualified with phase_12_1 and thus may clocked by tbe OCU on phase_12_2.
drv_gmt_inh 1 O This signal indicates to drive Xbus grant inhibit controt signal.
Iower_into_sel l3:0] O These lines choose from the 10 possible pieces of information that could be l,a"al~"ed across the lower 32 bits of the Xbus infommation bus. The encodings are as follows:
1000 - Virtual address (with function code) 1001 - Physical Address 0000 - data word 00 0001 - data word 04 0010 - data word 08 0011 - data word Oc 0100 - data word 10 0101 - data word 14 0110 - data word 18 0111 - data word 1c drv_xbus 1 0 This signal indicates to xb_en_block to drive the Xbus.
info_out l31:0] O This signal is the info output It is driven from the data out mux to the t,anscciJers.
trid_out [7:0l O This signal is the trid output as well as the func_op.
It is driven from the data out mux to the lla,~sc~ rs~
xb_no_ack 1 O This signal is driven from the master state machine to the in-pipe for failed_op insertion.
xb_no_ack_trid 12:0I O This signal is driven from the master state machine to the in-pipe for failed_op inser~ n. These three bits correlate to bits l6:4] of the 1 st cycle trid.

21 July 1995 115 3 ~

CA 02257~11 1998-12-03 Xbus Functional Specification Stratus _~mpany Confid~n~al Table 21. Common Logic Outbound Pipe Port Llst signalname width IIO description xb_no_ack_ske 1 O This signal is driven from the master state machine to the in-pipe for failed_op insertion.
0 - 32 bit operation 1 - 256 bit op~, dtiOl~
retrans_op 1 0 Indication to the OCU that transaction has been busied by the bus and offers an opportunity tor switching to a higher priority queue (EFQ) en_in~h~,~s 1 0 From error t)lock indicaffng that an Xbus error sequence is in p,og,~ss.
xact_single_side 1 O This signal is driven from the master state machine to the enor chechi-lg Iogic to signal that loop back c I ,~..hing should be pe, Iu, " ,ed on the co"~ e bus width of the current cyde. This signal is used in conjunction with drv_xbus by the enor block to deterrnine when and how to do loopback cl ,e~ hing xbus_op_done_ack 1 0 Indication to Ping module that the l,.~nsa~i~n has completed with an ach-,o~lc;lge xbus_op_done_trid 1 O Indication to Ping module of the l,dnsa~ic ) id for the completed h~nsacl;ûn.
xbus op_done_func 1 O Indicaffon to Ping module of the func_op for the completed ~an&al tion.
op_ready 1 O This signal is passed to the master control logic. It i"~ es that all the i,,fu,,,,a~:on needed for a bus l,clnsaction is available and ready to go out to the Xbus. Sync. to 12_0 or 12_1 for one phase.
newest_cd_read_return 1 0 CD read retum indicaffon qual ed on a transacffon basis cd_different_out 1 I CD read retum indicaffon from Ping module newest_nodrv_rd_retum 1 1 Indicaffon from Ping module that transaction is a non-paired read retum.
read_retum_32 1 1 Indication from OCU that the sut)se~ ent transaction will be a rd_ret_32.
single_sid~e 1 I This signal from the an IO devir e s OCU when asse~led indicates that the current b ansac~ol, is single-sided transaction.
bus_oe 1 I This signal from the an IO device's OCU is used to gate the asse, lion of drv_xbus for single sided ~ccesses info_for_bus [31:0] I From OCU to outbound new data. This is the info bus for the next 1- dnsaution.

21 ~luly 1995 116 3~

CA 022~7~11 1998-12-03 Xbus Functiol ;pe~, "~tion .ratus Company Confi~el,tial Table 21. Common Logic Outbound Pipe Port List signal name width l/O de~ripbon trid_for_bus 16:0] I From OCU to the outbound new data. This is the trid for the next ~,ans&~,tion.
xb_own_grant 1 I From xbus arb to master state machine indicating that this node has been granted the Xbus.
grant_peer 1 I From xbus arb to master state machine indicating that this node~s peer h~ been granted the Xbus.
xbus_err 1 I registered clean version of bus error, used by master state machine.
xbus_ack 1 I registered clean version of bus_ack, used by master state machine.
xbus_busy 1 I registered clean version of bus busy, used by master state machine.
xb_err_drv [1:0] 1 ' From error block, indicates when and what to drive during error sequenoe.
00- Normal operabon 01 - Drive O's 10 - Drive 5's,A's 11 - Drive O's no_ack_done 1 1 Indication from in-pipe that failed-op insertion has been completed.
p2p_mode_enable 1 I From register module indiacabng the status of p2p_mode.
ping_p2p_issue_for_bus 1 I From module, indication whether ping llansa.. tion is to be peer-to-peer'ed.
arb_for_cpu_online 1 I From board states module, arb for 1 st cpu online.
cpu_online_gmt_inh 1 I From board states module, issue grant inhibit if won 1stcpuonline.
xb_p2p_info 131:0] I Piped version of the info on the Xbus, needed for Peer-to-Peer echo.
xb_p2p_trid l6:0] I Piped version of the trid on the Xbus, needed for Peer-to-Peer echo.
xb_p2p_func_op 1 I Piped version of the func_op on the Xbus, needed for Peer-to-Peer echo.
xb_p2p_1ast info 1 1 Indication from in-pipe that the last phase of the tlallsa 'icr~ is currently posted on the peer-to-peer post_qual bus.

xb_p2p_incomplete_send 1 1 Indication from in-pipe that the current peer-to-peer operaffon will not complete.

21 July 1995 3~% 117 PCTrUS97/09781 Xbus Functional Specihcaffon Stratus ,mpany Confidential Table 21. Common Logic Ou~ound Pipe Port List signal name width llO descripbon xb_p2p_valid 1 1 Indication from in-pipe that the information on the p2p_post_qual bus is valid with either an info, data or a retum cycle~
xb_add_valid 1 1 Indication from in-pipe that the inforrnation on the post_qual bus is valid with either an add~ess~hid swap_table_load 1 1 Indication from in-pipe that an IOVA map RAM
lookup has yielded a byte swap hit on a read.
post_qual_trid_4 1 I Delayed version of trid trom Xbus used for post_qual_trid_O 1 I swsp_tabie load. Bit 4 deterrnines C/D side. bit 0 determines PCIBO or PCIB1 update 1 I From board states module, update state.
duplexed 1 I This signal indicates whether or not the board is duplexed.
hit_rniss 1 I From OCU, to be appen~d into bit 5 of second half of trid.
p2p_issue 1 I From OCU, to be append~d into bit 4 of second half of trid.

12.4.2 Functional Desc,~tion The Xbus outbound data and control unit issues all outbound ~nsa.,tions onto the Xbus including the scho of Peer-to-Peer 11 al ,~a~gc ns. The XBO will load all of the necessary ir,~ r. IldliOn trom the OCU upon the asst:,tion of op_ready by the OCU. The XBO then issues the Xbus tl~"sa L 1 onto the X~us f~ ;ng the Xbus p,-~.c ~ ~ The XBO is co",p,ised of the f~ D~\ ng functional blocks as illustrated in figure 67:
~ XBO Decode - This block decotlec the 1unction supplied by the OCU.
~ INFO Registers - Fh/e registers, each holding one info phase worth of data.
~ XBO Control - Five control sequencers, one master and four slaves. Each sequencer controls one info cycle ot the transaction.
~ ECHO Registers - Five registers, each holding one echo phase worth of data for Peer-to-Peer echo.
~ XBO Echo Control - Five control sequencers, one master and four slaves. ~ach sequencer controls one info cycle of the Peer-to-Peer echo transaction.
~ Outbound Mux - This mulffplexer ~ ",.els the ap~op(id~a data onto the into_out bus which is ultimately sent onto the Xbus.
~ Swap Table - A four entry table, one entry for each PCI, which stores IOVA read byte swap inforrnation for outstanding read rerll-estc This table is loaded I~ . :.,9 an IOVA Map RAM
lookup for a read requesL The XBO uses this i".~.,.,a1ioo when issuing the data retum.
Figure 68 illustrates an example of XBO timing and its handshake wi~h the OCU block. This particular illustration is that of a 128 bit write. Figure 69 and figure 70 illustrate the control flows tor the master and slave sequencers, res~c~ y. The master sequencer controls the first info phase to be posted on the Xbus, each of the four slave sequencers control the other possible Xbus info 21 July 1995 118 3(,~ .
-Xbus Functional ~c~ on ~ tus Company Confidenffal cycles Figure 71 illustrates the issuing of a peer-to-peer transaction onto the Xbus. Note that the issuing co,lb.'ler remains involved in the b~naa..lion until the h~n6~ :n is echoed by its ne;!Jhbor. The issuing controller checks for error on the info cycles that it transmits but it does not check for busy since it is only valid on the echo. Therefore the issuing controller checks for busy on the echo portion of the transaction and ret.ans"lits the complete transacbon in that occurrence.
Figure 72 illustrates the echo peer-to-peer bming. The XBO loads the post-qual info into its echo info registers when it receives a peer-to-peer indicaffon in the post qual_trid. It then transmits the received data onto the Xbus, .;I,ecki"g for bus error and driving grnt inh. In the case of an error during the echo, it I~L d~ its the info. Figure 73 illustrates the echo master and slave control sequencers. Similar to the outbound sequencers, the master controls the first info phase and the slaves each control the other possible info phases of the echo.

21 July 1995 119 ~ l~

WO 97/46941 PCTtUS97tO9781 XbusFunc~onalSp~ ?tion Sbatus~mpanyConfidenbal Figure 68. Outbound Tming Diagram ~ ~ _ _ _ _ _ _ _ _ _ _ _ _~ ~ _ _ J ~ r , _ ~] ~ _ ~ ~ l _ _ J _ b ~

J
J
, ~
,..
~ q ~~
, , ~ ~ Y ~' .

.~ N X D D ~ I ~ ~ ~ c ~ C 2 21 July 199~ 3 l \

CA 022575ll l998-l2-03 Xbus Functiona~ ~ci~. _Jontus Company Confiden~al Figure 69. Outbound Master Control -err ~ any pstl err_in_~PrDB
Q err_ln_Pr~B + err ( P2P S W T ~ err p (P2P SE W T ~

op re dy ~ -err ~ -dD; p~ -err_in prog ro ~ -grant ~ crr ( C N Tl ~ y I~FO S~

\ P2P E E W T
graDt ~ -err-err ~ crr in_prog ( CNT2 ~err iD prog err r ~ -err \ \
I~F O Jerr ~ err in_prog \ -crr ( C N T3 ) ~ ~ P2P E E W ) \ -err err err PSTl J
aoy_~pstI ~ -err ~ ~ PPSTt J
_err p2p issue P2P S W T -err ~ -Hoy pstl ~ -err ~ ~
1 PST2 ) p2p issue ~ C N Tl ( ~ bU5V ~ L OA~D
/ rr ~ \ -busy ~
/ err -p2p usue ~ L OA~D \ (-rd r q I ack) / -busy ~ ~ IDLE
( PST3 ~ ( rd_req ~ ~ck) Il)LE
-busy ~ rd req ~ -~ck ~busy ~ rd req ~--ack err in_prog \~ Q no ack done l NACK WT I
ERR WT p rNFO ~ ~
-err in prog no nck done - rDLE
MASTER CONTROL

21 July 1995 3 ~ ,~ 121 Wo 97/46941 PCT/US97/09781 Xbus Funcional Spocificaion Stratus ,npany Confidonbal Figur~ 70. Outbound Slave Control -op lengtb +
~ -pst~te_reg IDLE J
op length ~
p~te lo~d lU~Ct busy (- nst~te Info ~ ~ct bu y) ~ ~ (-err ~ ~act busy) LO~D ) ps~tc_info -err ~
busy ~ct busy ~-pslate_iale ~ LO~D
. ~rr ~ -~ct bu y '' pst~ e ~ ERR_WT

-err ~ ~ct busy ~sct_busy err ~ act bu-y ps~te_idle ~ LO~D
~, ~~r ~ ~ct bus!~
PSTl ~ ~ pstate_i~e ~ ERR_WT
-err ~ -err ~ -~ct busy ~a busy err ~ ~ct busy ~ err in prog ( ERR WT ~ FO
-err m prog SLAVE CON~OL

21 ~lu~y 1~95 1Z

Xbus Functiona, 3ci~ "ion tus Company Confiden~al Figure 71. Outbound Peer-t~eer Timing J ~1 o~ C

-- _l [ L

L l ' E
c-- L J
~~ L~ L ~ .
- ~-' l L~ L
c-- L J
L l ~ c ~ ~ r _ LJ l~

L~
L
- L~l L~ r~

L l - _ ~
-- L J
o L
s s cn a~ E ~ ~
m I I D~ ~ ~D 2 ~ ~ Ei; ~l ~l ~! 2 x 2 ~n cn 21 July 1995 123 '?~

Xbus Functional Specific.. lon Stratus lpany Cunfide,l~ial F,gure 72. Echo Peer-t~Peer Timing c- l J
~ l ~ .
J e _ J
~' 0- ~ J
u~ L~ L

[ ~ , ~ Y
l] l J J ~ ~ ~< , ~

o~ ~J lJ ~ ~ a ,, ~1 o - l l ' _ ~ 1 ~

o l l ~ --J :

~r x " O ~ ~ ~E ' ~ el 21 July 1995 ~?~ 1 5 124 .

Xbus Funcbonal 3cih,,bon tus Company Confidenbal Figure 73. Outbound Peer-t~Peer Control -p2p valid+-lsst info IDLE ) p2p v81id ~err ~ Isst info p2p valid ~ ~err ~ I~st info INFO ) err ~ ERR WT ( IDLE ~p pst~te info ~-e%oct busy ~r ~ -remain zero ~ ~err (err ~ -pstste idle) e~n bus~
PSTl ) err f ERR WT INFO ) ~IDLE
~\ err ~ pstste idle crr ~rr \~ sct busy ~ERR WT
~-eusct busy (err ~ -pstste idle) PST2 ~ -err ~ II)LE () I e~csct busy IDLE
~~ err ~ pstste idle err trr \~-e%act_busv ~ ERR WT
~-e~ct busy PST3 ) -err ~ IDLE (PST2 ~ rr ~ e%scl busy ~ ~LE
err ~-evsct busy err in prog - -( ERR WT )(;~ ~ INFO ~ )Q err_in_prog -err in prog -err in p~g ~51~ CONI~OL ~A~8NTRoL

21 Ju~
~1 ~

CA 02257~11 1998-12-03 WO 97/4694]
PC~/USg7/09781 @' o ~

.______________________________.
O N
aa ,~

~
P~
o ~6 O
r' z O
o ,~

CO

n 'S tD '~ I
N ~) Xbus Functiona edt~ n tusCompany Confidentiai 12.5.1 Port List Table 22. Arbitrat~on Functional Block Port List signal name width 1/0 description xb_req_p 1 I Xbus request control input bus from peer.
xb_rea_n 1 I Xbus request control input bus from nei.Jlll,or.
xb_req_o 1 I Xbus request control input bus from opposite.
xb_rnaint_int 1 I Xbus maintenance interrupt control input bus drv_maint_int 1 I This signal is an input from the error block indicating that the maintenance interrupt control line will be driven in l ~ ; ;. ,9 clock phase.
xb_grant_inh 1 I Xbus grant inhibit control input bus.
drv_grant_inh 1 I This signal is an input from the outbound pipe block indicabng that the grant inhibit control line will be driven in 1~ ;ng clock phase.
go_arb 1 I This signal comes from the outbound pipe. It is asse.lt:d when the outbound pipe wants to arbitrate for the Xbus.
own_gmt 1 O This signal goes to the outbound pipe to notify that a-Li~ aLion for the Xbus was s~ Irr*scfl ll drv_req 1 O This signal is used to issue the Xbus request on the )wing clock phase.
slot_id[1 :0l 2 I The slot ID signals are hard wired on the ba-.h~ iane for each slot. The slot ID is used to reset the initial arbitration value.

12.5.2 h --cl,~ al Description Polo uses a distributed all~iLIdtion system. Each CPU and each PCIB has its own arbitration controller. Each board also has connecffons to the other three boards' control signals and is aware of the arbitraffon rules. This means each board can keep track of who is arbitrating and who should win. For more detail please refer to section 6.6 on page 40.
The main function of this logical block is to determine when the requesting node captures the Xbus. To do this it maintains a priority address which consistent with that of each of the other three nodes. The dlL ibdLon logic also deterrnines when to block the node's request from being issued onto the Xbus. The arbitraffon control block is co",p,ised of the 1 11o~;ng functional blocks:
~ PRIO_ADD_CNT - This block maintains the priority address for the resident node this value is determined by the slot_id. The priority address is initialized to a value identical to that of its slot_id whenever a maintenance interrupt occurs.
~ ARB_PRIORITY - This block determines whether or not the resident node should be granted the XBUS based on the outstanding requests as well as the current priority address.
~ ARB_COMROL - This block maintains the necessary state in'.""dLion to delineatearbitration cycles. This block also initiates the updating of the priority address in between 21 July 1995 127 3~ '~

, .

PCTrUS97/09781 Xbus Functional Spedfic~aon Stratus npany Confidenoal arbitration cycles.
Figure 7~. Arbitration State Machine -any_reg --C?
IDLE ~
~ ~any req * ~own_gmt own~m~

any_req ~ (any_req * ~own~nt) REQ CYC'Q own gmt ~REO CYC~
CL~ D~ OPEN

~any~ ~eq * ~O ~ _gmt IDLE

ARBITRATION CONTROL

21 July t995 128 3 1(~

CA 022~7~11 1998-12-03 Xbus ~unc~on~ ,~c~ on atus Company Co,~ en~al Figure 76. Arbitration l~ming Diagram L J
J
L L -O ~ l ? ~ _ L J 1 ~ --, I ,~

L J à~ _ ~
l, J
L ~

N L

_ ~ l ,, _ L

- L ~
- L ] ,~ ,~ _ o ~J l N _ C ~, C) 2 21 July 1995 129 ~2~

Xbus Func~onal Specifi~-lon Stratus npany Confiden~al 12.6 lO n~ist~-r Section Figure 77.10 Register Block Diagram lo_~Lr d_d~
to~ 'b~ ' I
control (OCU) b~ d_mu~

~b lo_~p_out F 1 2L, to ~lC- ~ ri~c rq~r control to~I ~l ~ a (Cychps) ~ O
o ~

Q' Q' ~ o o _ O ,~ ~ "
(Cycbps) ~ O

21 July 1995 3 130 CA 022~7~ll l998-l2-03 Xbus Functiona eri~ ~t Jn tus Company Conlidenbal ~2.6.1 Port List Table 23. Common Logic Register Control Port List signalname width l/O descripbon any_cpu_online 1 1 (Cyclops only) from board states logic indicates at least one CPU in the system is on-line asic_broken_status 3:0 I The ASIC specific broken status register asic~o_regs_out 31:0 I The ASIC specific register read output board_local_bus 31:0 1 (Cyclops only) The local IIO register read/write address/data bus from the IBO.
board_local_byte_enable 3:0 1 (Cyclops only) The local l/O register ,t:ad/~.ite byte enable bus from the IBO.
board_local_io rd 1 1 (Cyclops only) Indicates that the board local bus contains an l/O write board_local io_wr 1 1 (Cyclopsonly) Indicatesthattheboard local bus contains an l/O read board_local_trid 6:0 1 (Cyclops only) The local l/O register read/write trid bus from the IBO.
board_reset 4:0 I The board reset register broken 1 I Broken signal from board states logic clk48 1 1 48MHz clockinput cold_reset 1 I cold reset signal ~",~,lon_broken_status 7:0 I The ~"""on broken status register diagnosbc_broken 1 I bit 4 of the test control register duplexed 1 1 (Cyclops only) Duplexed signal from board states logic drv_xbus 1 1 Indicates that this ASIC drove the Xbus for the wrrent cycle, used by the pe-~u""a,)ce monitoring registers ef~_freeze_state_sta 1 1 (Cyclops only) EFO Freeze state stable signal from the IBI
freeze_error_regs 1 I Bit 0 of the bus fault report register freeze_state 1 1 (Cyclopsonly) Freeze_statesignalfrom board states logic map_regs_out 31:0 1 (Cyclops only) The map ram register read output misc_err_status 4:0 I The misc. error status register id_prom_in 1 I serial input from the ID PROM

21 July 1995 3~ 2_~ 131 Xbus Functional Specification Stratus ~mpany Cor~fiJer,tial Table 23. Common Logic Register Control Port List signal name width IIO description io reg_xbus_op_done 1 I Fromtheocu,indicatesthattheregisterread retum started by io_rea xbus_op_ready has completed.
io_reQlocal_op_done 1 1 (Cyclops only) From the IBI, indicates that the local register read retum started by io_reg_local_op_ready has completed.
led_control 2:0 I The board led control register online 1 I Board on-line signal from board states logic.
own_ack 1 1 indicates the assertion of ACK on the Xbus by this cu"t\ ~
own_busy 1 1 indicates the asse, lion of BUSY on the Xbus by this con~"er.
own_main_int 1 1 Indicates that this ASIC issued a maintenance interrupt on the Xbus phase_12_0 1 1 12 MHz phase clock 1st half of cycle 0 of phase phase_12_1 1 1 (Cyclops only) 12 MHz phase clock 2nd half of cycle 0 of phase phase_12_2 1 1 12 MHz phase clock 1st half of cycle 1 of phase phase_12_3 1 1 12 MHz phase clock 2nd half of cycle 1 of phase phase_24 1 1 24 MHz phase clock post_qual_func_op 1 I Post qual func_op bus post_qual_info [31:0] I Post qual (atter Xbus (esponse) info bus post_qual_trid l6:o] I Post qual trid bus pre_qual_tunc 5:0 I Pre qual tunction code bus pre_qual_func_op 1 I P-e qual tunc_op bus pre_qual_into [31:0] I Pre qual (before Xbus ~esponse) into bus pre_qual_rc 1:0 I Pre qual RC bus pre_qual_trid ~6:ol I Pre qual trid bus pu_dside 1 I hi = D-side ASIC, low ~ ~side ASIC
set_first_cpu_online 1 1 (Cyclops only) Sets bit 11 ot the bus int~, ~dce state register set_xbus_int 1 1 (Gambit on~y) bit 2 of the bus fault repon register sync_complete_dlyd 1 1 (Cyclops only) From the board states logic,I indicates that a sync register read has co",,: '~ t~d 21 July 1995 3 2 3 132 , Xbus Function. ,~ on atus Company Confidenbal Tsble 23. Common Logic Regist~ r Control Port List signal name width l/O descripbon update 1 1 (Cyclops only) Board update mode signal from board states logic.
update 1 1 (Cyclops only) Board update mode signal from board states logic.
warm_reset 1 I sy,,cl,.unous warm reset to block xb_ack 1 1 indicates the asst" t n of ACK by one or more Xbus controllers xb_add valid 1 1 indicates that the post qual bus contains a valid address (valid from 12_2 of post3 Ull 12_2 of post4) xb_bus_err 1 1 indicates the asseltion of Bus Error by one or more Xbus controllers on either the A or B bus xb busy 1 1 indicates the asseft;on of Busy by one or more Xbus contr. ers xb_data_valid 1 1 indicates that the post qual bus cc-I~;"s valid data (valid from 12_2 of post3 Ull 12_2 of post4) xb_io_reg_failed_op 1 I ned to failed_op signal from XBI. When true indicates that the data signaled valid by xb_data_valid is the result of a failed_op.
asic_specific_broken_ 1 0 The ASIC Specific Broken Configurabon registerconfig board_local_data_sel 1 O Tells the IBO to place data on the board local bus for writing to the register currenffy indicated on the board local bus board_local_done 1 O (Cyclops only) Indicates to the IBO that the current local l/O access has completed break_even_board_on_ta 1 O Bit 18 of the Bus Interface State register break_pcib_on_err 1 O Bit 21 of the Bus Interface State register bus_fault_report 1 O The bus fault report register clr_broken_colll'',dnd 1 0 Un-break the board clr_diag_broken 1 O Clear bit 7 ot the Test Control register clr_efq_freeze_state 1 0 (Cyclops only) Clears EFO Freeze State dr_freeze_state 1 O (Cyclops only) Clears Freeze State clr_online 1 O Clears On~ine mode clr_udpate 1 O (Cyclops only) Clears Updatemode id_prom_out [1:0] O Output to the extemal ID PROM

21 July 1995 133 ~2~~

PCTrUS97/09781 Xbus Functional SF e ~ ~ ~ Stratus .1pany Confidential Table 23. Common Logic n~y;sl~r Control Port ~ist signal name width llOdescripffon io_reg_cd_diff 1 0 Indicates that this register access is CD different io_reg_i_drive 1 0 (Gambit only~ Indicates that the Gambit will drive the Xbus on this l/O register access.
io_reg_local_op_ready 1 0 (Cyclops only) Indicates to the IBI that a local l/O
register read retum is ready for the IBus io_reg_write_data 131:01 0 The data to be written into a given register on a load_address_xxx signal being assel ~d.
io_reg_read_trid l6:0~ 0 The TRID tor the Xbus register read retum signalled by io_rec xbus_op_ready io_reg_single_side 1 0 ~aambit only) Indicates that this register access is a single sided retum.
io_reg_take_cycle 1 0 Tells the PING module to the wrrent Xbus read is being respond to so the PING table has to be loaded io_reg_xbus_op_ready 1 0 Indicates to the OCU that a register read return is ready for the Xbus io_reg_write_data ~31:0] 0 The data to be written into a given register on a load_address_xxx signal being asst'l t~d.
leave_sync_fast_duplex 1 0 (Cyclops only) Bit 7 of the Bus Interface State register leave_sync_full_duplex 1 0 (Cyclops only) Bit 8 of the Bus I~ ,lace State register leave_sync_pointjump_ 1 0 (Cyclops on~y) Bit 10 o1 the Bus Inlelldce State switch register load_xxx 1 0 Load register xxx (one for each writable bus AS~C
register not wntained with the register control block) p2p_mode_enable 1 0 (Cyclops only) Bit ~ of the Bus Interface State register read_xxx 1 0 Read register xxx (one for each writable bus ASIC
register not contained with the register control block) rearm_error_regs 1 0 Set by wribng bit zero of the Bus Fault Report register set_broken_~l-,---a-,d 1 0 Break the board set_efq_freeze_state 1 0 (Cyclops only) Sets EFO Freeze State set_freeze_state 1 0 (Cyclops only) Sets Freeze State 2~ July 1~9s 3 ~ s ~34 ~, ~ .

Xbus Functiona ,eci ~tion ,tus Company Con~i~en -' Table 23. Common Logic Re~ist~r Control Port List signal name width llO dea.;,i~on set_online 1 O Sets On-line mode set_udpate 1 O ~Cyclopsonly) Sets Update mode software_set_maint_int 1 O Set by writing bit one of the Bus Fault Report register xb_io_reg_ack 1 O Xbus ACK from the register response logic xb_io_reg_busy 1 O Xbus BUSY from the register res~nse logic 12.6.2 Functional Description This block contains a sub-block which controls the reading and writing of all registers within the bus ASlCs. In addition, it contains the a sub-block for the control of the ID-PROM ,eg.;,t~,a, a sub-block containing several of the ~"",-on registers, and a sub-block that is used to mux all of the register read sources tog~U,el. The timing for a register write is shown in figure 78 and the timing for an Xbus register read is shown in figure 79 Figure 7B. Register Write rlming info ' post1 ' post2 ; post3 ' post4 load_reg_xxx io_reg_write_data, , I , ~
reg_xxx ' ' ' ' ~VAL12 ~ , , Figure 79. Re~ister Read Timing . I.nO; pO n ;pO~ ;p~u ;pO~
read_reQxxx ' ' , ' ~ , ,~~~, , , L

asic_io regs out , , I I ~VAIJUI I --- I I I IX
io_reg_xbus_op_rdy ~ ' ' 'J ' ' ~~~ ' , ' ~
io_reg_xbus_op_done ~ ' ' ' ' ' ' ' ,l , L
io_reQread_data , , ~ ' ' X VALID ' --- ' ' ' ' X

32ç~

CA 02257~11 1998-12-03 Xbus Func~onalSpecih~a~on S~atu~ ,mpany Confiden~al 12.7 Error Cllecl~iny and Registering Logic Figure 80. Error Checking Block Diagram xb_error_out xb_error_parity_out Info bus from ASIC
xb error ecc out mo~ Q5 to output pads control bus voted signals ¦ \
misc. signals ~-~
xb error-lGo~ k xb_error_misc xb_error_in ~
xb_error_parity_in info bus xb_error_regs to frominput xb_error_ecc_in \modules pads control bus xb error_vote xb_error_sm voted signals 21 July 1995 136 Xbus Func~ion~ pecl..caUon ~tus Company Confidential 12.7.1 Port Llst Table 24 ~s~il,es the port list to the error checki"g and registering logic.:
T~ble 24. Error Checking and Registering Port Ust signalnamewidth l/O description clk48 1 1 48MHz clock input clk48_12_0 1 1 48MHzdockvalidat12_0 clk48_12_1 1 1 48MHz clock valid at 12_1 clk48_12_2 1 1 48MHzclockvalidat12_2 clk48 t2_3 1 1 48MHzdockvalidat12_3 dk48_24 1 1 24MHZ clock dk48_24_not 1 1 Inverted 24MHZ clock phase_12_0 1 I t2MHz phase clock 1st half of cycle 0 of phase phase_12_1 1 1 12MHz phase clock 2nd half of cyde 0 of phase phase_12_2 1 1 12MHz phase clock 1 st half of cycle 1 of phase phase_12_3 1 1 12MHz phase dock 2nd half of cycle 1 of phase phase_24 1 1 24MHZ phase clock warm_reset 1 1 Used to warm reset the gate array cold_reset 1 1 Used to cold reset the gate array alert_ 1 I Signal is as~el lèd when power to the board is not okay, used to set broken info_ai 131:0] 1 Info Bus A coming from input pads of ASIC
info_bi [31 :0] 1 Info Bus B coming from input pads of ASIC
trid_ai l6:0] I Trid Bus A coming from input pads of ASIC
trid_bi l6:0l I Trid Bus B coming from input pads of ASIC
func_op_ai_ l31:0] I Func Op A coming from input pads of ASIC
func_op_bi_ 131 :Ol I Func Op B coming from input pads of ASIC
parity_ai 1 I Parity A coming from input pads of ASIC
parity_bi 1 I Parity B coming from input pads of ASIC
info_ao l31 :0] O Registeled Info Bus A going to output pads of ASIC
info_bo l31 :~l O R89i 't 3rod Info Bus B going to output pads of ASIC

21 July 1995 137 3~

Xbus FunctionalSp ~: ~on Stratu~ ,mpanyConfidential Table 24. Error Checking and Registering Port List signal name width l/O desc,i~tio,) trid_ao 16:01 0 Re~ J~;d Trid A going to output pads of ASIC
trid bo 16:0] 0 Registered Trid B going to output pads of ASIC
func_op_ao_ 1 0 Registered Func Op A going to output pads of ASIC
func_op_bo_ 1 0 l~egistered Func Op B going to output pads of ASIC
pari,ty_ao 1 0 Reg;ste,bd parity A going to output pads of ASIC
parity_bo 1 0 Registered parity B going to output pads of ASIC
control_in_ni 17:0l I Control Bus (lle;,Jh~) coming from input pads of ASIC
control in_oi 17:0l I ControlE~us(oppo~i~e)comingfrominputpads of ASIC
controlin_pi P ~l I Control Bus (peer~ r oming from input pads of ASIC
control_out_no p:OI O Regi~ - ~d Control Bus (ne;,~l~lJor) going to output pads of ASIC
control_out_oo p:O] O Registered Control Bus (orpo~ ;~e) going to output pads of ASIC
control_out_po t7:0] 0 P(~ d Control Bus (peer) going to output pads of ASIC
control_out_ni 17 0] I Looped-back version of control_out_no, used for loop~ri ch~chi--g control_out_oi [7:01 I Looped-back version of control_out_oo, used for IOq~l~Ck CheCh;-19 control_out pi [7:0] 1 1 oopl~L~d version of control_out_po, used for loopb~ cl ~,hi.-g slot_idai 1 I Slot Id A coming from input pads of ASIC
slot_idbi 1 I Slot Id B coming from input pads of ASIC
slot_id_latched 1 0 Latched version of slot_id board_not_broken_in_n_xi 1 I Board Not Broken (n~,;gl ,~o~, x version) coming from input pads of ASIC
board not_broken_ln_n_yi 1 I Board Not Broken (neighbor, y version) coming from input pads of ASIC
board_not_broken_in_n_zi 1 1 ~oard Not Broken (-.o;gl~L,or, z version~ coming from input pads of ASIC

21 ~uly 1995 138 3~

. . ~ . ,~ _ . . .

PCT~US97109781 Xbus Functior. ,p~n ~ffon tatus Company Confidenbal Table 24. Error Chacl~ and Reg.st~ri.~ Port List signal name width llO description board_not_broken_in_o_xi 1 I Board Not Broken (opposite, x version) coming from input pads of ASIC
board_not_broken_in_o_yi 1 I Board Not Broken (opposite, y version) coming from input pads of ASIC
board_not_broken_in_o_zi 1 I Board Not Broken (opposite, z version) coming from input pads of ASIC
board_not_broken_in_p_xi 1 I Board Not Broken (partner, x version) coming from input pads of ASIC
board_not_broken_in~_yi 1 I Board Not Broken (partner, y version) coming from input pads of ASIC
board_not_broken_in_p_zi 1 I Board Not 8roken (partner, z versior ) coming from input pads of ASIC
board_not_broken_in_n 1 0 Three-way voted version of board not_borken_in_n(x,y,z) signals, goes to intemal ASIC modules board_not_broken_in_o 1 0 Thre~way voted version of board_not_borken_in_o(x,y,z) signals, goes to intemal ASIC modules board_not_broken_in_p 1 0 Three-way voted version of board_not_borken_in_p(x,y,z) signals, goes to intemal ASIC modules board_not_broken_out 1 0 Registered Board Not Broken going to output pads of ASIC
These h 'h; :. ,9 signals are specific to the Gambit gate array only reu_in_n_xi 1 I ReCC Reset neighbor (x version) coming from input pads of ASIC
reu_in_n_yi 1 I ReCC Reset ne;.JII~or (y version) coming from input pads of ASIC
recc_in_n_zi 1 I ReCC Reset nc;~hbor (z version) coming from input pads of ASIC
recc_in_o_xi 1 I ReCC Reset opposite (x version) coming from input pads of ASIC
reu_in_o_yi . 1 I ReCC Reset opposite (y version) coming from input pads of ASIC
recc_in_o_zi 1 I ReCC Reset opl~ss;t~, (z version) coming from input pads of ASIC

21 July 1995 139 ~3-., .. . .

CA 02257~11 1998-12-03 Xbus Functional Spe~ifi~ on Stratus npany Confidential Table 24. Error Checking and Regisl~..i..y Port List signal name width l/O des~ on recc_in 1 O Three-way voted version of recc_n_in (x,y,z) signals or the Three-way voted version of recc_o_in (x,y,z), goes to intemal ASIC
modules single_side 1 1 Asserted when oc.nsac.tion acbon is single sided These ~ollowing signals are specific to the Cydops gate array only recc_in_p_xi 1 I ReCC Reset peer (x version) coming from input pads of ASIC
recc_in_p_yi 1 I ReCC Reset peer (y version) coming from input pads of ASIC
recc_in_p_zi 1 I ReCC Reset peer (z version) coming from input pads of ASIC
recc_in 1 O Three-way voted version of recc_in_p (x,y,z) signals, goes to intemal ASIC modules drv_reset_out 1 0 The ReCC Reset output (u,~ ed version) recc_out_no 1 0 Reg; cteled Recc Out (ne;gl)~o.) going to output pads of ASIC
recc_out_oo 1 0 nQ~ e~¢d Recc Out (opposite) going to output pads of ASIC
recc_out_po 1 O Registered Recc Out (peer) going to output pads ot ASIC
recc_out_ni 1 1 1 oo~h~ ked version of recc_out_no, used for 1001 ~L~ck ~ he~,hil 19 recc_out_oi 1 1 ~oop~, I~Qd version ot recc_out_oo, used for loorb~ck ~;hêchill9 rscc_out_pi 1 1 l oo~t~r-~4d version of recc_out_po, used for loopl~i~ ~,hec hil 19 next_sync_out 1 I Pre-,eg;sle.ed version of sync_out, from xb_brdstate_rtLv sync_outo_ 1 0 legislèled version of next_sync_out co", ~ued with broken sync_in_xi_ 1 I Sync In (x version) coming from input pads of ASIC
sync_in_yi_ 1 I Sync In (y ~lersion) coming from input pads of ASIC
sync_in_zi_ 1 I Sync ln (z \/ersion) coming from input pads of ASIC

21 July 1995 140 Xbus run~.LonL ,~cl caffonatus CompanyConfidential Tsble 24. Error Checl~ and R~ist~ri.,~ Port List signal name width l/O description sync_in_ 1 O Three-way voted version of sync_in_zi_(x,y,z) signals, goes to intemal ASIC modules sync_outi_ 1 I Loop~~ ed version of sync_outoJ used for loopt~L~ cllechi,lg your_online_in_xi 1 1 On-line from partner (x version) coming from input pads of ASIC
your_online_in_yi 1 1 On-line from partner (y version) coming from input pads of ASIC
your_online_in_zi 1 1 On-line from partner (z version) coming from input pads of ASIC
your_online_in 1 O Three-way voted version of your_online in(x,y,z) signals, goes to intemal ASIC modules drv_my_online_out 1 1 B~e-~gi~ ed version of my_online_out coming from intemal ASIC module my_online_outo 1 O .ey;steredversionofdrv_my_online outgoing to output pads of ASIC
rny_online_out_xi 1 I Buffered, loopl~Led version of my_online_outo (x version), used for looback checking my_online_out_yi 1 I Buffered, loopt~ L~d version of my_online_outo (y version), used for looback cl)~cl~i"g my_online_out_zi 1 I Buffered, loortacL~edversion of my_online_outo (z version), used for looback cl~ i,.g These i "~ .lg signals are ~ on to both Cyclops and Gambit gate array crnp_err_in_ai 1 I These inputs come from the other side Cyclops' cmp_err_out. When ass~l tt:d, it cmp_err_ln_bl 1 1 indicates the other side Cyclops detected a IlI;_~lll~dl~: on the bus and returning data did not match the data that was driven by this ASIC.
cmp_err_out_ao 1 O These outputs are driven to the other side ASlC's cmp_err in. When asse, ted it indicates cmp_err_out_bo 1 ~ thisCyclopshas~ ~n~ da~Il;S~ll;~areonthe bus and the retuming data did not match the - data that was driven by the other side Cyclops 21 July 1995 141 ~,2 .. .. . . . . . .

CA 02257~11 1998-12-03 Xbus Functional Spe ~ h~ ~n S~atu~ mpany Confidential Table 24. Error Chccbil~ and Registering Port List signal name width IIO description drive_err_in_ai 1 I These inputs are tied to drive_err_out_a from the other side Cyclops ~"~re check. When drive_err_in_bi 1 ~ asse,t~J itindicatestheothersideCyclops has detected a ",;sco,~,~r~ on me bus in that the retuming data did not match the data that was driven by the other side Cyclops ASIC.
drive_err_out_ao 1 O These inputs are tied to the outputs from the other side Cyclops ASlC's co",~)ar~ check.
drive_en_out_bo 1 ~ Whsn asse,ted it indicates the other side Cyclops has d '-~c~d a ,n;3~""~a-e on the bus in that the retuming data did not match the data that was driven by the other side Cyclops ASIC.
cside_ctl_erri 1 I Signals that c-side gate array control signa~s dffler from dside control signals. this signal is used by the d-side cside_ctl_erro Signals that c-side gate array control signals differ from dside control signals this signal is driven by the c-side cside_ctl_err_oe_ This signal controls cside_ctl_erro if H is the c-side gate array this signal is always low (enabling cside_ctl_erro to be driven out) if it is the d-side this signal is always high info_a_reg 131:0l O RegisteredversionofInfo_ai goestointemal ASIC modules info_b_reg ~31:0] O Reg.ste,edversionoflnfo_bi goestointemal ASIC modules trid_a_reg [6:0] O Reg;sta,~d Yersion of trid_ai goes to internal ASIC modules trid_b_reg [6:0l O It2g;sI~ dversionoftrid_bi goestointemal ASIC modules func_op_a_reg_ 1 0 Reg;st~rt:d version of func_op_ai_ goes to intemal ASIC modules func_op_b_re~_ 1 0 Reg;Ot~f~dversionoffunc_op_bi_ goesto internal ASIC modu~es parity_a_reg 1 0 ne.;~.. ,t~rad version of parity_ai_ goes to intemal ASIC modules parity_b_reg 1 O Rey._~e.ed version of parity_bi_ goes to intemal ASIC modules xb_bus_reQcorr_n 1 O Reg;i~ered Fccco~ctedversionof control_in_ni[0] valid12_2to12_0 goesto intemal ASIC modules 21 July 19g5 3 142 Xbus Function~ ,pec"ication ratus Company CGIlfidel.tia' Table 24. Error Cl-ec~ and Registering Port List signal name width l/O desc,i~tion xb_bus_req_corr_o 1 O Pe~,is~f~;d, ECC cone-,t~d version of control_in_oilOI, valid 12_2 to 12_0, goes to intemal ASIC modules xb_bus_req_corr_p 1 0 ~egPt~rcd, ECC COIIe~,1ed version of control_in_pi[0], valid 72_2 to 12_0, goes to intemal ASIC modules xb_bus_req 1 O 'ored' version of xb_bus_req_corr_ (n,o,p) xb_ack 1 O 'ored' version of registered, ECC correctedcontrol_in_nilO], control_in_oi[0], control_in_pi[0] signals sampled at 12_0 and also 'ored' with registered version of dN_xb_ack own_ack 1 0 'ored' version of all internal aclu 10 IVledydS
double registered xb_rnaint_int 1 0 'ored' version of registered, ECC w"e~.~od control_in_ni [1 ], control_i n_oil 1 ], control_in_pi[1l signals ssmpled at 12_0 and ~ also 'ored' with regist~-cd version of drv_xb_maint_int xb_busy 1 0 'ored' version of registered, ECC cc",e~,t~3d control_in_nil2], control_in_oi[2], control_in_pi[2] signals sampled at 12_0 and also 'ored' with registered version of dN_xb_busy own_busy 1 O 'ored' version of all intemal BUSYs double leg; tèred xb_grant inh 1 O 'ored' version of ~a~ d, ECC corrected - control_in_ni[1], control_in_oi[11.
control_in_pi[11 signals sampled at 12_2 and also 'ored' with registered version of dN_xb_grant_inh xb_bus_err 1 0 'ored' version of leg;~.teled, ECC COIle-.t~d control_in_ni[21, control_in_oi[21.
control_in_pil21. control_in ni[31.
control_in_oil3], control in_piE3l signals sampled at 12_2 and also 'ored' with ,egist~fed version of drv_xb_bus_err_a and dN_xb_bus_err_b dN_xb_bus_re~ 1 I demultiplexed non,ey;~ter~d version of control_outlO] coming from intemal ASIC
module 21 July 1995 143 33~1 Xbus Functional Specific~. on Stratus npany Confidential Table 24. Error Checlclng and n6gis~ Port Llst signalname width l/O descri~,tiol, drv_xb_grant_inh 1 I demulbplexedno~"e~ist~~edversionof control_out[1] coming from internal ASIC
module drv_xb_bus_sn_a 1 I demulbplexed no",~ist~~cd version of control_out[2] coming from internal ASIC
module drv_xb_bus_en_b 1 I demultiplexed non-~39 ~t-~red version of control_outl33 coming from intemal ASIC
module board_specific_ack 1 I demultiplexed nol~ sist~sd term of control_out[0] coming from intemal ASIC
xb_io_reQack 1 I demufflplsxed nun.eyi ;t~,~,d term of control_out[0] coming from intemal ASIC
module drv xb_maint int 1 I demultiplexednon-~g;~tur3dversion of control_out[1] coming from intemal ASIC
module board_specific_busy 1 I demultiplexed non.~:g te~cd term of control_outl2] coming from intemal ASIC
module inpipe_busy 1 I demultiplexed nonregistered term of control_out[21 coming from intemal ASIC
module The l~ I9 signal is specific to the Cyclops gate array only ping_fifo full_busy 1 I dsmultiplexednonregisteredtemmof control_out[2l coming from intemal ASIC
module These f~ . ,9 signals are co" ,- non to both Cyclops and Gambit gate array xb_io_reg_busy 1 I demultiplexed no"-.Ji~-~red terrn of control_out[2] coming from intemal ASIC
module drv_info 131:0] I non.,2~, ~tercd version of info_o coming from intemal ASIC module drv_trid [6 0] I nor.. ~ .~c~ed version of trid_o coming from intemal ASIC module drv_func_op_ 1 I non.. ~,,k~te~ed ve,rsion of func_op_o_ coming from intemal ASIC module ping_ack_out 1 I demultiplexed no~ y i~Qred term of control_out[0] coming from intemal ASIC
module 21 July 1995 144 3 ~3 '~

CA 022~7~11 1998-12-03 Xbus Functiona ecif ,~ion ~tus Company Confidenffal Table 24. Error Checking and Registering Port List signal name width llO des.;,i~on info_a_oe_ 1 0 output enable for info_a bus info_b_oe_ 1 0 output enable for info_b bus trid_a_oe_ 1 0 output enable for trid_a bus trid_b_oe_ 1 0 out~putenablefortrid bbus func_op_a_oe_ 1 0 output enable for func_op_a_ line func_op_b_oe_ 1 0 output enable for func_op_b_ line parity_a_oe_ 1 0 output enable for parity_a line parity_b oe_ 1 0 outputenableforparity_bline control_out_n_oe_ 1 0 output enable for control_out_n line control_out_o_oe_ 1 0 output enable for control_out_o line control_out_p_oe_ 1 0 output enable for control_out_p line my_online_out_x_oe_ 1 0 output enable for my_online_out_x line my_online_out_y_oe_ 1 0 output enable for my_online_out_y line my_online out_z_oe_ 1 0 output enable for bmy_online_out_z line reset_out_n_oe_ 1 0 output enable for reset_out_n line reset_out_o_oe_ 1 0 output enable for reset_out_o line reset_out_p_oe_ 1 0 output enable for reset_out_p line sync_out_oe_ 1 0 output enable for sync_out line xb_err_drv [1 :0l 0 Indicates when and what to drive during error sequence, goes to intemal ASIC module 00- Normal o~ldtion 01 - Idle condition 10 - Drive 5s then As 11 - Drive Os drv_xbus 1 I This signal comes from the XBO, it is used to control the output buffers on the info bus early_err in_p,oyl~ss 1 0 Asserted when error state machine is currently executing the error protocol or the board is in funny_state, goes to intemal ASIC modules, it is asse, t~ d on 12_1 of Post3 and deasse, lad on 12_1 of Err2 err_in~log,ess 1 0 Asserted when error state machine is currently executing the error protocol or the board is in funny_state, goes to internal ASIC modules, it is asse-tad on 12_2 of Post3 and deasse,led on 12_2 of Err2 21 July 1995 145 33~

Xbus Functional Specificauon Stratus ~ ,pany Confidenffal Table 24. Error Checlcing and nQ~is~.in~ Port List signal name width l/O desc~ tion arb_err_in_~f~yr~ss 1 0 Asserted when enor state machine is currently executing the error protocol or the board is in funny_state, goes to intemal ASIC modules, it is asse, t~d on 12_2 of Post3 and deasse.t~d on 12_2 of Err2 pu_dside 1 I This hard-wired input indicates whether this is the C ~low) or D (high) side ASIC.
load_error_fault_bitO 1 I Enables loading of enor fauU bitO register load_error_fault_bitl 1 I Enables loading of enor fault bitl register load_error_data_match 1 I Enables loading of error data match registerload_error_control 1 I Enables loading of error control register io_reg_write_data 131;01 I The data bus that is used to write the enor ,~g;~t~.~
read_error_fault_bitO 1 I Enables reading ot error fault bitO registerread_error_fault_bit1 1 I Enablesreadingoferrorfaultbit1 register read_error_data_rnatch 1 I Enables reading of error data match registerread_enor_control 1 I Enables resding of enor control register read_bus_info_en_status 1 1 Enables reading of bus info error status register read_control_bus_err_status 1 I Enables reading of control bus error status register read_voter err_xcvr_status 1 I Enables reading of voter enor l.anscei~er status register read_bus_err_byte_status 1 I Enables reading of bus error byte status register xbus_error_regs_out 131 :0l O The data bus that is used to read the error registers control_bus_enor 1 0 Asserted when error logic sees a control busenor, used by xb_brdstate_rU.v to assert freeze_error_regs bus_error_freeze 1 0 Asserted when error logic sees a bus enor that it assei t~d, used by xb_brdstate_rtl.v to assert freeze_enor_regs voter_failure 1 O Asserted when error logic sees a voter enor,used by xb_brdstate_rtl.v to assert freeze_error_regs 21 July 1995 146 ~ 3~

-CA 022~7~ll l998-l2-03 WO 97t46941 PCT/US97/09781 Xbus Functional~ cifi~ on ' usCompanyConfiderlbal Table 24. Error Checking and Registering Port List signal name wid~ l/O desc-i~otion seen_bus_enor 1 O Asserted when error logic sees a bus error ass~l led by another board, used by xb_brdstate_rtl.v to assert freeze_enor_regs freeze_enor_regs 1 I From xb_brdstate_rtkv, used to treeze error regiate,~
break_pcib_on_en 1 I This signal is used to signify if the CPU or PCIB
should break for the arbitrary broken case broken 1 0 Asserted when board is broken, as;,e, t~d on 1 2_0 The f ~ ..9 signal is specHic to the Gambit gate anay only gam spec_broken [3:0] O Replicated versions of broken, used tor timing purposes in Gambit These i "o~ ing signals are cG"",-un to both Cyclops and Gambit gate anay set_broken 1 0 Combinatorial ~ r~ ed version of broken broken_12_3 1 O Asserted when board is broken, asse,~d on 12_3 atter broken is asse- Ied debug_conn_broken_outo 1 0 Ple,~zy,st~led version of broken qualHied with phase_1 2_0 diagnostic_broken 1 O Diagnostic version of broken, behaves the same as broken except it can be cleared with clear diay-losti~,s broken co,-,---and my_set_broken 1 O Rey,sle,~d and latched version of s~." ,eone_set_broken broken_out_ 1 O Active low version of broken three_way_loop~ch_set_br 1 0 Asserted when there is a loorb~ error on oken voted signals (reset, sync, on-line), this signal is 0 for Gambit loo~ k_enor_set_broken 1 0 Asserted when there is a loopb~ck enor on the info bus slot_parity_error_set_broken 1 0 Asserted when slot_ida and slotidb d;~g,ca other_cpu_broken 1 O Inverted version of board_not_broken from the peer board, not used in Gambit xb_arbitrary_set_broken 1 O Asserted when board breaks due to arbitrary broken xb_heuristic_set_broken 1 O Asserted when board breaks due to heuristic broken 21 ~uly 1995 147 33~

PCT/USg7/09781 Xbus Functional Spe '' ,.~on Stratus npany Confidenbal Table 24. Error ChecWng and n~gist~ring Port Llst signal name width 1/0 description xbus_cnb_en_set_broken 1 0 Asserted when there is a looph~rk error on the control bus These fs~lu~,;ng signals are specific to the Gambit gate array only xb~parity_gen_set_broken 1 O Asserted when the duplexed parity gen~,d~,s dlsag~e break Dcib 1 1 Asserted by internal Gambit ASIC module to break the PCIB
These t~ 9 signals are co,-----un to both Cyclops and Gambit gate array broken_out_oe_ 1 O Unused signal .~tl.e.~,de_set_broken 1 1 Other ASlC's my_set_broken ~U,~,~ide_set_broken_reg 1 0 Reyistered (48mhz) version of otherside_set_broken asic_specific_set_broken 1 I This signal is ass~- ted when the board needs to break for reasons that the error logic does not detect, software set broken c~------and tor example or Cougar set broken (CPU only) clr_broken_~-l-,--and 1 1 Allows so1~ to clear a broken state clr_diaa broken_co------and 1 1 Allows software to clear a diag"ostics broken state newest_cd_read_return 1 I Prevents info loopl~c'~ .;h6chil)g when it is CD
different data xb_grant_opposite 1 I From xb_arb_rU.v, ass~. ~ed when the opposite board is granted use of the bus, used to prevent looking at the middle of an info cycle when a board first comes up xb_grant_neighbor 1 I Fromxb_arb_rtl.v,asse,1edwhenthen~;!JI,bùr e board is granted use of the bus, used to prevent looking at the middle of an info cycle wh~en a board first comes up xb_grantQeer 1 I From xb_arb_rtLv, assel t~d when the peer board is granted use of the bus, used to prevent looking at the middle of an info cycle when a board first comes up xb_own_grant 1 I From xb_arb_r~.v, ass6, ted when this board is granted use of the bus, used to prevent looking at the middle of an info cycle when a board first comes up These '.,"DW.)9 signals are specific to the Cyclops gate array only 21 ~uly 1995 148 ?,~9 Xbus Functional ~dfiL~don ' ~us Company Confidenbal Table 24. Error Checbi,-g and Regisl~ri.,~ Port List signal name width l/O descripffon funny_s~tate_n 1 0 Assertedwhennei,Jlll.orboardisin funny_state, XBI uses the signal to ignore a PCIB if U is in the funny_state funny_state_o 1 0 Asserted when opposite board is in funny_state, XBI uses the signal to ignore a PCIB U it is in the funny_state 12.7.2 Functional Description The Error ChecWng and Reg;sl~,i"g logic pe, ~u-,--s two basic functions. It controls most funcffons of error ~ ,echi"g and it registers incoming and a~tg ~ ~9 Xbus signals. The error cl ,echi"g function pe,to"ns the f "Dwing tasks:
~ ECC gene-dtion and cl,e..hi-,g/conecting ~ Parity ~en~- ~ic ~ and cl ,~l~ing ~ L~opba~ k checkb,g ~ Three-way voffng ~ Controls error protocol (includes de~.;;,iûns on breaking board) ~ Provides logic for enor ins~- tion ~ Records error information The registering logic pe- hl- " ,s the f ~ . ,9 tasks:
~ registers incoming bus signals ~ ,eg; 'o~ outgoing bus signals ~ combines (logical 'or') neighbor, ûpposite~ peer versions of r~gisle~d ECC ~r,e~,t~d control signals coming into the ASIC with the delayed outgoing cu, ,esponding signal to produce a combined control signal for internal ASIC module use The Xbus can be broken into four classes of signals when ~iscussing error ~hecli"g.
Xbus Signals- these signals are protected by one parity bit, this detects an odd number of errors.
During normal A~esses the ~side gate array drives apprù,.i-- ~~tely half of the signals and the C-side drives the other haU. Please refer to Section t 1.1 on page 96 for more detail on the signals that are driven by D-side and C-side. The D-side checks the data it has driven (loophAclc drive check) and the ~side cherks the data thatthe C-side has driven with what the D-side would have driven (h:~opt;;rk co..~pa.e check). During single sided A~.ce~s9s (PCIB only) the driving ASIC
drives all of the bus signals. The driving ASIC can do a loo~ k drive check but the other ASIC
cannot do a loorl)aek co--,~ check bscAuse it has no hl ~le~e of the exact data being driven.
Control Bus Signals: these signals are protected by four bits of ECC, this detects a double bit error and can conect a single bit error. The D-side drives the signals and the Gside does the loopb~ck ~".~re check. The ~side does the ;oqJL~.~ drive check.
Voted signals: these signals are tt ipli~ated and protected by three-way voting, this allow for one out of the three lines to be in error. The ~side drives the signals and the Gside does the loop~ k 21 July 1995 149 3~ ~

,, _ Xbus Func~onal Specih~ on S~atus npany Confidenbal CO~ afe check. The D-side does the loor~(4 drive check. The one exception to ~side driving all signals is the board_not_broken_ signal. The ~side will drive the board_not_broken_ signal and the D-side will drive the enable for the board_not_broken_ signal.
Miscellaneous signals: The slot id signals are duplicated and if they are in disay,~e,.~ the board will break.
Figure 81 shows the Bus Error flow chart, please reter to sec~on 6.4 on page 30 for a detailed explanaffon of errors.

Figure 81. Bus Error Operation Flow Chart f IDLE ) j~ ~\ Error ( Err2 ) ( Err1 ) -(OPost2) (CPVTes3 -flOPost1~ fCPUPos3 ~IOTest ) 21 July 1g95 150 3l"

Xbus Functiona. "eu"~ on atusCompany Confidenffal 13. XbuslGolfbus Diffe~ences Table 25. XBus/Golfbus Differences Golfbus XBus single logical fully inter~io"nected bus, 4 distributed point-to-point buses, not duplic~ted (2 physical buses), shared duplicated, each board connects to 2 by all boards of the buses, but still ",ai"tain a sin- gle-bus view A & B buses are lockcteppe~ A & B buses are independent Special bus trdnsceivers to support No bus l~nsce.vers (except for the 3-live insertion way voted signals). To support live insertion, buses to broken boards must not be driven high.
info[63:0]: 32 & 64-bit bus widths info[31:0]: 32-bit only Post3 ACK Post2 ACK
Bus Error: Single-Phase Post4 Error Bus Error: 7-Phase Error protocol Arbitration: binary search, imple- Arbitration: distributed, in"~le",ented mented with 4-bit counter, no dead with 2-bit counter, dead cycle needed cycles between two di~teren~ boards' info's (implemented via grant_inh) Fully supported Peer-To-Peer Trans- CPU-only Peer-To-Peer T.~nsa~;Lions actions 64-bit bus func_op's unsupported transceiver loopb~ck mode unsupported unsupported New info[31:00] IOVA Address Format low_parityp:03, up_parityp:0] info_parity[0]
control signals tripl-cate~ 3-way voted control bus, w/check bits arb conlrol[0]: bus_req & ack arb_req control[11: grant_inh & maint_int ack_maint_int controll2]: bus_err_a & busy grnt_inh control[3]: bus_err_b & unused busy controlp:4]: checkbits bus_error<a,bxc,d>
gb_clk8 xb_clk8 21 July 1995 151 3~

. .

Xbus Functional Spet~ tio~ Suatu~ ~mpany Confiden~al Table 25. XBus/Golfbus Differences Golfbus XBus Other control signals: Other control signals:
recc (warm reset) ~3-way voted) slot_ida, slot_idb slot_idl3:01, parity reset (3-way voted) pair_commr7:43: ta board_not_broken (3-way voted) pair_comm[3:01: unused sync (3-way voted) btl_term even_odd_online (3-way voted) driving_bus ta_d, ta_c partners_ta_d, partners_ta_c func_opttrid[6:0]: func_op/trid~6:0]:
first cycle: first cycle:
trid[3] = slot_id~3] trid[3] = System/lOVA Address second cycle: second cycle:
func_op= obey_a func_op = unused tridl63 = obey_b trid~6] = unused trid[41 = bus size trid[4] - unused 21 July 1995 ~ . 152 .

Claims (17)

CLAIMS:
1. A fault-detecting and fault-isolating digital data processing apparatus comprising plural functional units, plurality of bus means, each connected to and providing point-to-point communication between a respective pair of functional units, the functional units each including error phase means, coupled to the respective bus means of that functional unit, for placing the functional unit in an error isolation phase substantially concurrently with the other functional units, and for transmitting test data onto at least one of those bus means during a respective portion of that phase and exclusive, with respect to that phase, of any other unit connected to that bus, bus error detecting means, coupled to the respective bus means of that functional unit, for detecting a communication error, including any of parity error, an error correction code error and a loopback error, on that bus means, and for signalling a bus error to the other functional units in the event of such communication error, error isolation means, coupled to the bus error detecting means and to the loopback error detecting means, for signaling the other functional units during the error isolation phase that the respective functional unit is faulty based on(i) whether the bus error means of that functional unit detected a loopback error, (ii) whether the bus error means of another functional unit signalled a bus error in response to test data transmitted during the error isolation phase,and (iii) whether another functional unit signaled that it was faulty.
2. A fault-detecting digital data processing apparatus according to claim 1, wherein the plurality of bus means provide point-to-point communication between each pair offunctional units.
3. A fault-detecting digital data processing apparatus according to claim 1, wherein the error phase means are responsive to a bus error signaled by any of the functional units for placing the respective functional units in an error phase.
4. A fault-detective digital data processing apparatus according to claim 1, wherein the error isolation means includes means for taking off-line a functional unit that signals that it is faulty.
5. A fault-detecting digital data processing apparatus according to claim 1, wherein the bus error detecting means of at least one functional unit comprises loopback error detection means, coupled to the respective bus means and error phase means, for comparing test data transmitted onto that bus means with data received substantially concurrently from the bus means, and for signaling a bus error to the other functional units in the event of discrepancy between the transmitted and received data.
6. A fault-detective digital data processing apparatus according to claim 5, wherein at least a selected functional unit includes a processing section for generating a datum for communication to another functional unit, a drive-side bus interface means, coupled with the processing section, for applying a first portion of that datum to at least the bus means coupling the selected functional unit to that other functional unit, a check-side bus interface means, coupled with the processing section, for applying a second, complementary portion of that datum to at least the bus means coupling the selected functional unit to that other functional unit, and each of the drive-side and check-side interface means comprise means for receiving a datum driven to at least the bus means coupling the selected functional unit to that other functional unit.
7. A fault-detecting digital data processing apparatus according to claim 6, wherein loopback error detection means comprises a loopback drive check means in each ofthe drive-side and check-side bus interface means for comparing the portion of adatum generated by the processing section and applied to the bus means by that bus interface means with a corresponding portion of the datum received from the bus means, and for signaling a bus error to the other functional units in the event of disparity between those compared portions.
8. A fault-detecting digital data processing apparatus according to claim 6, wherein loopback error detection means comprises a loopback compare check means in each of the drive-side and check-side bus interface means for comparing the portion of a datum generated by the processing section and applied to the bus means by the other bus interface means with a corresponding portion of the datum received from the bus means, and for signaling a bus error to the other functional units in the event of disparity between those compared portions.
9. A fault-detecting digital data processing apparatus comprising plural functional units, including one or more central processing units and one or more input/output interface units, plurality of bus means, each connected to and providing point-to-point communication between a respective pair of functional units, each central processing unit including dual bus interface means, each of which applies a complementary portion of a datum to the bus means coupling that central processing unit to an input/output interface unit, each input/output interface unit including dual bus interface means, each of which applies a complementary portion of a datum to the bus means Coupling that input/output interface unit to a central processing unit, the functional units each including error phase means, coupled to the respective bus means of that functional unit, for placing the functional unit in an error isolation phase substantially concurrently with the other functional units, and for transmitting test data onto the bus means during a respective portion of that phase and exclusive, with respect to that portion, of any other unit connected to that bus, bus error detecting means, coupled to the respective bus means of that functional unit, for detecting a communication error, including any of parity error, an error correction code error and a loopback error, on that bus means, and for signalling a bus error to the other functional units in the event of such communication error, error isolation means, coupled to the bus error detecting means and to the loopback error detecting means, for signaling the other functional units during the error isolation phase that the respective functional unit is faulty based on (i) whether the bus error means of that functional unit detected a loopback error, (ii) whether the bus error means of another functional unit signalled a bus error in response to test data transmitted during the error isolation phase,and (iii) whether another functional unit signaled that it was faulty.
10. A fault-detecting digital data processing apparatus according to claim 9, wherein the input/output interface units transmit information to and from PCI compatible devices.
11. A fault-detecting digital data processing apparatus according to claim 9, wherein the error phase means are responsive to a bus error signaled by any of the functional unit for placing the respective functional units in an error phase.
12. A fault-detecting digital data processing apparatus according to claim 9, wherein the error isolation means includes means for taking off-line a functional unit that signals that it is faulty.
13. a fault-detecting digital data processing apparatus according to claim 9, wherein the bus error detecting means of at least one functional unit comprises loopback error detection means, coupled to the respective bus means and error phase means, for comparing test data transmitted onto that bus means with data received substantially concurrently from the bus means, and for signaling a bus error to the other functional units in the event of discrepancy between the transmitted and received data.
14. A method of opening a digital processor of the type having plural functional units, the method comprising providing point-to-point communication between respective pairs of functional units, selectively placing each functional unit in an error isolation phase substantially concurrently with each other functional unit, during the error isolation phase, transmitting test data from each function unit onto a respective bus during a respective portion of that phase and exclusive, with respect to that portion, of any other unit connected to that bus, with each functional unit, monitoring the respective bus means for detecting a communication error, including any of parity error, an error correction code error and a loopback error, on that bus means, and signaling a bus error to the other functional units in the event of such communication error, signaling the other functional units that the respective functional unit is faulty based on (i) whether the bus error means of that functional unit detected a loopback error, (ii) whether the bus error means of another functional unit signalled a bus error in response to test data transmitted during the error isolation phase, and (iii) whether another functional unit signaled that it was faulty.
15 . A method according to claim 14, including the step of placing each functional unit in the error isolation phase in response to a bus error signaled by any of the functional unit.
16. A method according to claim 14, including the step of taking off-line a functional unit that has signalled that it is faulty.
17. A method according to claim 16, including the step of detecting a loopback error by test data transmitted onto the respective bus means of a functional unit with data received substantially concurrently from that bus means, and for signaling a bus error to the other functional units in the event of discrepancy between the transmitted and received data.
CA002257511A 1996-06-05 1997-06-05 Digital data processing methods and apparatus for fault isolation Abandoned CA2257511A1 (en)

Applications Claiming Priority (3)

Application Number Priority Date Filing Date Title
US08/658,563 US5838899A (en) 1994-09-20 1996-06-05 Digital data processing methods and apparatus for fault isolation
US08/658,563 1996-06-05
PCT/US1997/009781 WO1997046941A1 (en) 1996-06-05 1997-06-05 Digital data processing methods and apparatus for fault isolation

Publications (1)

Publication Number Publication Date
CA2257511A1 true CA2257511A1 (en) 1997-12-11

Family

ID=24641768

Family Applications (1)

Application Number Title Priority Date Filing Date
CA002257511A Abandoned CA2257511A1 (en) 1996-06-05 1997-06-05 Digital data processing methods and apparatus for fault isolation

Country Status (6)

Country Link
US (1) US5838899A (en)
EP (1) EP0979451A1 (en)
JP (1) JP2000508453A (en)
AU (1) AU725945B2 (en)
CA (1) CA2257511A1 (en)
WO (1) WO1997046941A1 (en)

Families Citing this family (58)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6032271A (en) 1996-06-05 2000-02-29 Compaq Computer Corporation Method and apparatus for identifying faulty devices in a computer system
US6161202A (en) * 1997-02-18 2000-12-12 Ee-Signals Gmbh & Co. Kg Method for the monitoring of integrated circuits
WO1998038577A1 (en) * 1997-02-26 1998-09-03 Siemens Aktiengesellschaft Redundant electronic device with certified and non-certified channels
US6275888B1 (en) 1997-11-19 2001-08-14 Micron Technology, Inc. Method for configuring peer-to-peer bus bridges in a computer system using shadow configuration registers
US6112271A (en) * 1998-05-14 2000-08-29 Motorola, Inc. Multiconfiguration backplane
US6138247A (en) * 1998-05-14 2000-10-24 Motorola, Inc. Method for switching between multiple system processors
US6161197A (en) * 1998-05-14 2000-12-12 Motorola, Inc. Method and system for controlling a bus with multiple system hosts
DE19826388B4 (en) * 1998-06-12 2007-01-11 Sgs-Thomson Microelectronics Gmbh Error processing circuit for a receiving station of a data transmission system
US6148348A (en) * 1998-06-15 2000-11-14 Sun Microsystems, Inc. Bridge interfacing two processing sets operating in a lockstep mode and having a posted write buffer storing write operations upon detection of a lockstep error
DE19835654A1 (en) * 1998-08-06 2000-02-17 Siemens Ag Device for detecting faults in electronic assemblies
EP1157324A4 (en) * 1998-12-18 2009-06-17 Triconex Corp Method and apparatus for processing control using a multiple redundant processor control system
US6574752B1 (en) 1999-07-15 2003-06-03 International Business Machines Corporation Method and system for error isolation during PCI bus configuration cycles
US6480917B1 (en) * 1999-08-19 2002-11-12 International Business Machines Corporation Device arbitration including peer-to-peer access arbitration
US6523140B1 (en) * 1999-10-07 2003-02-18 International Business Machines Corporation Computer system error recovery and fault isolation
US6701469B1 (en) * 1999-12-30 2004-03-02 Intel Corporation Detecting and handling bus errors in a computer system
TW525053B (en) * 2000-03-29 2003-03-21 Mitac Int Corp Apparatus of PCI bus cycle single step interrupt debug card and its method
US6633996B1 (en) 2000-04-13 2003-10-14 Stratus Technologies Bermuda Ltd. Fault-tolerant maintenance bus architecture
US6735715B1 (en) 2000-04-13 2004-05-11 Stratus Technologies Bermuda Ltd. System and method for operating a SCSI bus with redundant SCSI adaptors
US6691257B1 (en) 2000-04-13 2004-02-10 Stratus Technologies Bermuda Ltd. Fault-tolerant maintenance bus protocol and method for using the same
US6820213B1 (en) 2000-04-13 2004-11-16 Stratus Technologies Bermuda, Ltd. Fault-tolerant computer system with voter delay buffer
US6708283B1 (en) 2000-04-13 2004-03-16 Stratus Technologies, Bermuda Ltd. System and method for operating a system with redundant peripheral bus controllers
US6687851B1 (en) 2000-04-13 2004-02-03 Stratus Technologies Bermuda Ltd. Method and system for upgrading fault-tolerant systems
US6802022B1 (en) 2000-04-14 2004-10-05 Stratus Technologies Bermuda Ltd. Maintenance of consistent, redundant mass storage images
US6901481B2 (en) 2000-04-14 2005-05-31 Stratus Technologies Bermuda Ltd. Method and apparatus for storing transactional information in persistent memory
US6356966B1 (en) * 2000-09-12 2002-03-12 Apw Ltd. System and method for bridging bus segments on a backplane and coupling single board computers to a backplane
US6886171B2 (en) 2001-02-20 2005-04-26 Stratus Technologies Bermuda Ltd. Caching for I/O virtual address translation and validation using device drivers
US6766479B2 (en) 2001-02-28 2004-07-20 Stratus Technologies Bermuda, Ltd. Apparatus and methods for identifying bus protocol violations
GB2373607B (en) * 2001-03-23 2003-02-12 Sun Microsystems Inc A computer system
GB2373606B (en) * 2001-03-23 2003-06-04 Sun Microsystems Inc A computer system
US6928583B2 (en) * 2001-04-11 2005-08-09 Stratus Technologies Bermuda Ltd. Apparatus and method for two computing elements in a fault-tolerant server to execute instructions in lockstep
US20020188752A1 (en) * 2001-05-04 2002-12-12 Tomassetti Stephen Robert Control messaging for an entertainment and communications network
US7139629B2 (en) * 2003-04-28 2006-11-21 Palo Alto Research Center Incorporated Planning and scheduling for failure recovery system and method
US7246276B2 (en) * 2004-10-04 2007-07-17 Agilent Technologies, Inc. Error tolerant modular testing of services
JP4302078B2 (en) * 2005-04-28 2009-07-22 株式会社東芝 Electronics
US9459960B2 (en) * 2005-06-03 2016-10-04 Rambus Inc. Controller device for use with electrically erasable programmable memory chip with error detection and retry modes of operation
US7831882B2 (en) * 2005-06-03 2010-11-09 Rambus Inc. Memory system with error detection and retry modes of operation
JP2007094706A (en) * 2005-09-28 2007-04-12 Konica Minolta Business Technologies Inc Data processor, system and method for coping with cable connection abnormality
TW200712898A (en) * 2005-09-30 2007-04-01 Tyan Computer Corp Multi-processor module
US7562285B2 (en) 2006-01-11 2009-07-14 Rambus Inc. Unidirectional error code transfer for a bidirectional data link
CN100511162C (en) * 2006-09-29 2009-07-08 华为技术有限公司 Method, device and a single-board for isolating bus
US7827531B2 (en) 2007-01-05 2010-11-02 Microsoft Corporation Software testing techniques for stack-based environments
DE102007062974B4 (en) * 2007-12-21 2010-04-08 Phoenix Contact Gmbh & Co. Kg Signal processing device
FR2939923B1 (en) * 2008-12-15 2011-03-11 Converteam Tech Ltd ELECTRONIC SYSTEM WITH REDUNDANCY OF ELEMENTS, AND CHAIN OF CONTROLLING AN ENGINE IMPLEMENTING SUCH A SYSTEM
US8483071B2 (en) * 2009-09-16 2013-07-09 International Business Machines Corporation Self-healing fibre channel link
CN102628921B (en) * 2012-03-01 2014-12-03 华为技术有限公司 Integrated circuit and method for monitoring bus state in integrated circuit
DE102012010143B3 (en) 2012-05-24 2013-11-14 Phoenix Contact Gmbh & Co. Kg Analog signal input circuit with a number of analog signal acquisition channels
TWI532323B (en) 2013-08-14 2016-05-01 財團法人工業技術研究院 Digital pulse width generator and generation method thereof
EP3218826A4 (en) 2014-11-13 2018-04-11 Virtual Software Systems, Inc. System for cross-host, multi-thread session alignment
FR3036244B1 (en) * 2015-05-12 2017-06-02 Peugeot Citroen Automobiles Sa SLAVE ORGAN WITH MEANS FOR VERIFYING COMMUNICATION PROTOCOL CHARACTERISTICS, FOR A BIDIRECTIONAL VIDEO NETWORK
FR3036243B1 (en) * 2015-05-12 2017-06-02 Peugeot Citroen Automobiles Sa MASTER ORGAN WITH MEANS FOR VERIFYING COMMUNICATION PROTOCOL CHARACTERISTICS, FOR A BIDIRECTIONAL VIDEO NETWORK
US10002056B2 (en) 2015-09-15 2018-06-19 Texas Instruments Incorporated Integrated circuit chip with cores asymmetrically oriented with respect to each other
US9940235B2 (en) 2016-06-29 2018-04-10 Oracle International Corporation Method and system for valid memory module configuration and verification
JP6869660B2 (en) * 2016-08-01 2021-05-12 キヤノン株式会社 Information processing device and control method of information processing device
KR102075028B1 (en) * 2017-11-01 2020-03-11 주식회사 두시텍 Unmanned High-speed Flying Precision Position Image Acquisition Device and Accurate Position Acquisition Method Using the same
US10802932B2 (en) 2017-12-04 2020-10-13 Nxp Usa, Inc. Data processing system having lockstep operation
CN113806147B (en) * 2020-06-15 2023-07-14 英业达科技有限公司 Backboard testing system and backboard testing method
CN113055225B (en) * 2021-02-08 2023-12-05 网宿科技股份有限公司 Network fault analysis data acquisition method, terminal and server
CN113110124B (en) * 2021-03-11 2022-08-19 上海新时达电气股份有限公司 double-MCU control method and control system

Family Cites Families (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US4597084A (en) * 1981-10-01 1986-06-24 Stratus Computer, Inc. Computer memory apparatus
ATE25779T1 (en) * 1981-10-01 1987-03-15 Stratus Computer Inc DIGITAL DATA PROCESSING SYSTEM WITH RELIABILITY BUS PROTOCOL.
US4486826A (en) * 1981-10-01 1984-12-04 Stratus Computer, Inc. Computer peripheral control apparatus
US4926315A (en) * 1981-10-01 1990-05-15 Stratus Computer, Inc. Digital data processor with fault tolerant peripheral bus communications
CA1320276C (en) * 1987-09-04 1993-07-13 William F. Bruckert Dual rail processors with error checking on i/o reads
US5313627A (en) * 1992-01-02 1994-05-17 International Business Machines Corp. Parity error detection and recovery
JPH0793273A (en) * 1993-09-20 1995-04-07 Fujitsu Ltd Multi-cpu system provided with fault monitor mechanism

Also Published As

Publication number Publication date
WO1997046941A1 (en) 1997-12-11
EP0979451A1 (en) 2000-02-16
AU725945B2 (en) 2000-10-26
US5838899A (en) 1998-11-17
AU3477997A (en) 1998-01-05
JP2000508453A (en) 2000-07-04

Similar Documents

Publication Publication Date Title
CA2257511A1 (en) Digital data processing methods and apparatus for fault isolation
US5005174A (en) Dual zone, fault tolerant computer system with error checking in I/O writes
US5099485A (en) Fault tolerant computer systems with fault isolation and repair
US5499346A (en) Bus-to-bus bridge for a multiple bus information handling system that optimizes data transfers between a system bus and a peripheral bus
US4916704A (en) Interface of non-fault tolerant components to fault tolerant system
US5153881A (en) Method of handling errors in software
US5249187A (en) Dual rail processors with error checking on I/O reads
US4939643A (en) Fault tolerant digital data processor with improved bus protocol
WO1997046941A9 (en) Digital data processing methods and apparatus for fault isolation
EP0306209B1 (en) Dual rail processors with error checking at single rail interfaces
US5220668A (en) Digital data processor with maintenance and diagnostic system
US4229791A (en) Distributed arbitration circuitry for data processing system
EP0415550A2 (en) Apparatus and method for documenting faults in computing modules
US5068780A (en) Method and apparatus for controlling initiation of bootstrap loading of an operating system in a computer system having first and second discrete computing zones
EP0670551A1 (en) Remote dual copy system
US5065312A (en) Method of converting unique data to system data
JPH03184130A (en) Error processing of software
US5048022A (en) Memory device with transfer of ECC signals on time division multiplexed bidirectional lines
EP0898227A1 (en) Semaphore in system I/O space
JPH0374761A (en) Method and system for processing data
US6012127A (en) Multiprocessor computing apparatus with optional coherency directory
US5163138A (en) Protocol for read write transfers via switching logic by transmitting and retransmitting an address
US6389554B1 (en) Concurrent write duplex device
US20040255187A1 (en) Data synchronization for system controllers
EP0411805B1 (en) Bulk memory transfer during resync

Legal Events

Date Code Title Description
FZDE Discontinued