Routing traces for PCI (Altera DE2)

Factual Questions
21

[QUOTE=Alex_Dubinsky]
Hmm.. 2.5ns sounds pretty good, given that my margin is 6.
[/QUOTE]
Tht 2.5ns was just the contribution from the 47 Ohm resistors and an estimated 50pF load. The dynamic output impedance of your FPGA I/O won’t be zero, so that will be added to the 47 Ohms. Also, it might be 100pF load rather than 50pF. My point is that the series resistors might make what would have been a 66MHz-compliant design (in the absence of the series resistors) not work at that clock speed.

[QUOTE=Alex_Dubinsky]
But when you say signal traces should be as far apart as possible, do you mean I should keep them pretty thin (say, 8 mils) just to make the spacing bigger (say, 17 mils)?
[/QUOTE]
If you’ve got 25 mil spacing, I’d go for 10mil traces 15 mil apart, but it’s not that critical. Just really limit the distance where you have less than 10 mil apart. Thee are no hard and fast limits.

[QUOTE=Alex_Dubinsky]
Re: Power. I realized another thing, if I wanted to connect the +/-12v at all, it’d have to be to the same psu as the other voltages or i may get a short, wouldn’t I? But for the same reason, powering a PCI card with one psu and its host with another might burn the pci bus, no?
[/QUOTE]
No, in general you can use separate PSUs. You need to make sure that the grounds are connected together so that all the voltages are referenced to the same level, and be aware of the possibility of a ground loop, but it won’t “short” unless you make a mistake in your connection.

[QUOTE=Alex_Dubinsky]
An ATX provides a true ground from the wall, but wall-adapters (like the one powering the DE2) don’t, and I’ve measured their fake grounds to differ from real ground by dozens of volts.
[/QUOTE]
If the wall-adapter has only a 2-pin 120V plug, its DC output won’t be ground-referenced at all and will “float”. If you measure the voltage of either DC output pin (wrt ground) using a high-impedance voltmeter, you’ll see a voltage (that may be dozens of Volts) just due to picked up 60Hz mains. If you strap the 0V to a real ground (like from an ATX PSU), everything will be referenced to the same level and it’ll all be fine. Your new PCB would be connecting the “floating” ground of the DE-2 to the ATX PSU via the ground plane.

22

[QUOTE=Antonius Block]
If you’ve got 25 mil spacing, I’d go for 10mil traces 15 mil apart, but it’s not that critical. Just really limit the distance where you have less than 10 mil apart. Thee are no hard and fast limits.

If you strap the 0V to a real ground (like from an ATX PSU), everything will be referenced to the same level and it’ll all be fine. Your new PCB would be connecting the “floating” ground of the DE-2 to the ATX PSU via the ground plane.
[/QUOTE]
Ok, wow, thank you! I think all of my questions have been answered. I’ll post the schematics soon.

23

[QUOTE=Alex_Dubinsky]
I’ll post the schematics soon.
[/QUOTE]
Yes, please do! I – for one – will be happy to take a look and give feedback, as I’m sure would others who have posted in this thread.

24

Wow, this took a bloody while, but on the bright side it went pretty smooth. Sprint-Layout is a great tool. I’ve gone with a 2-layer design in the IDE-PCI-ATX configuration you suggested.

ok, lets see if my server works:
JPEG files
Gerber files
(note: the capacitors are missing, but they go to the right of the atx connector)

25

[QUOTE=Alex_Dubinsky]
ok, lets see if my server works:
[/QUOTE]
Sorry, Alex, but your site just times out for me (this was also true the other day with this link in your post #9 above).

If you could repost rar’ed or zipped files to something like MediaFire (my current favorite of the free filesending sites), and post the URL here, I’d be happy to take a look this evening and post comments either tonight or tomorrow.

ETA: Oh, and if you want to post the Sprint Layout native file as well as the Gerbers, I’ll take a look at it with the Sprint Layout “see-through” viewer.

26

Mediafire is really good.

Sprint-Layout: http://www.mediafire.com/?jnzxldemujo
Gerber: http://www.mediafire.com/?cfnz4mbme1l

27

Alex, just to give you some early feedback:
[ol]
[li]Thanks for the MediaFire upload. It works perfectly.[/li][li]If anyone else is interested in looking at the Sprint-Layout plot, one can download a free demo version of Sprint-Layout 5.0 here that will read Alex’s .LAY file, but not print or export it.[/li][li]Do you have a schematic of the circuit in addition to the PCB layers? It’s easier to see “conceptual” (as opposed to routing) errors in that format.[/li][li]On the plus side, I’m glad to see that the board is roughly what I was expecting. Early posts to the thread seemed to assume that you were laying out a board with an FPGA on it, or building a PCI peripheral board. It’s reassuring to see that the board has at least the “look and feel” of what I was expecting, i.e. a passive “adapter” board.[/li][li]It’s good to see that even with a 2-layer board, one side is very close to being a ground plane![/li][li]From a quick first impression, it looks like you’re running a few more signal traces than I would have done. 32-bit PCI will work with about 50 signal lines (cite to follow later), and as I said in post #16, you can use some of the unneeded I/Os (of the 72 available at the DE-2’s headers) as extra grounds, which not only simplifies routing, but also – more importantly – reduces “ground bounce” when several PCI signals switch simultaneously (more about this later, if you like).[/li][li]Please note that I haven’t yet taken a detailed look. My first impressions are definitely positive, but I haven’t had time to look at individual traces. A circuit schematic would really help here.[/li][/ol]Good job so far, however!

28

[QUOTE=Antonius Block]
Alex, just to give you some early feedback:
[ol]
[li]Thanks for the MediaFire upload. It works perfectly.[/li][li]If anyone else is interested in looking at the Sprint-Layout plot, one can download a free demo version of Sprint-Layout 5.0 here that will read Alex’s .LAY file, but not print or export it.[/li][li]Do you have a schematic of the circuit in addition to the PCB layers? It’s easier to see “conceptual” (as opposed to routing) errors in that format.[/li][li]On the plus side, I’m glad to see that the board is roughly what I was expecting. Early posts to the thread seemed to assume that you were laying out a board with an FPGA on it, or building a PCI peripheral board. It’s reassuring to see that the board has at least the “look and feel” of what I was expecting, i.e. a passive “adapter” board.[/li][li]It’s good to see that even with a 2-layer board, one side is very close to being a ground plane![/li][li]From a quick first impression, it looks like you’re running a few more signal traces than I would have done. 32-bit PCI will work with about 50 signal lines (cite to follow later), and as I said in post #16, you can use some of the unneeded I/Os (of the 72 available at the DE-2’s headers) as extra grounds, which not only simplifies routing, but also – more importantly – reduces “ground bounce” when several PCI signals switch simultaneously (more about this later, if you like).[/li][li]Please note that I haven’t yet taken a detailed look. My first impressions are definitely positive, but I haven’t had time to look at individual traces. A circuit schematic would really help here.[/li][/ol]Good job so far, however!
[/QUOTE]

  1. There’s really no schematic. All of the pci bus signal pins are connected to whichever header pin is closest (this gets remapped in the fpga later) except for three point-to-point signals that I have to route separate for the 2nd slot. The grounds connect to the ground plane and voltages to the atx socket on the other side. Btw, I’m sure you know the “test” tool in sprint layout.
  2. I should have clarified that, but I could tell you knew what I meant.
  3. Well actually, when the bottom layer isn’t being a ground plane, the top layer is. Is that the same thing as a solid ground plane? I twisted it around like that because I won’t have a soldermask and the soldering side alternates.
  4. Yeah, I know. I could do away with maybe a dozen signals. I’m thinking that I might want to use one of the optional signals in the future and be stuck. But really, deleting all those signals would take more work than just leaving them in. But as for ground bounce… doesn’t connecting up all of the ATX’s grounds fix the ground-pin deficiency?
  5. Thank you very much for your help! I’d link you to the long PCI pin-out, but really I don’t want you to spend your time looking/verifying at it. I’m just unsure about some of the routing techniques I used or if I made some other noob mistake (e.g. solder pads too small or the dead-end pieces of tracks I left lying around will hurt signal quality). But if I connect Reset to +V, that’s wholly on me. I went with the cheaper 2-layer design exactly so I wouldn’t feel bad about having to redo it. Coming from the background of software, it’s pretty hard for me to imagine creating something that hasn’t got at least one bug.
29

JPEG version of the PCB (for anyone who just wants to take a quick look): (click “download image” for high-res version)

Top of the board
Bottom of the board

30

I did some more work on the design, removed the little bits of dead copper, found a mis-connect, and put in the bypass capacitors and loading resistors (good thing I remembered those for the ATX psu).

I placed the orders with pcbexpress.com and digikey last night. Also bought some solder paste on ebay that came with a syringe and six different tips. (I didn’t buy a hot-air gun yet, but solder wire always pissed me off and I think I’m going to like paste).

I’ll post pics when all of this comes in the mail this week. Let me know if you’d like me to post updated design files.

31

Now, my problem is this: I need the driver writer’s manuals for the PCI boards i’ll be using, the realtek gige controller and acard sata controller. However, my requests are getting ignored by both companies. How can I better phrase myself to get those docs? Perhaps someone who works at a big firm could request them? My project is pretty screwed without them.

32

[QUOTE=Alex_Dubinsky]
I did some more work on the design, removed the little bits of dead copper, found a mis-connect, and put in the bypass capacitors and loading resistors (good thing I remembered those for the ATX psu).
[/QUOTE]
Sorry not to have replied to you before now – a non-PCB-related crisis took me away from the SDMB over the weekend. I’d started reverse-engineering a schematic from the PCB layout on Thursday, but that got interrupted…

In any case, I hadn’t seen anything fatally wrong with the design that you posted. One of the advantages of working with 2 layers rather than 4 or more is that all of your errors are (at least in theory!) correctable with a knife, a file, and point-to-point soldering, as opposed to a short in a buried layer (or 2 of 4, 6 or whatever) that you can’t get to without major surgery.

I was going to comment on all of the little “dangling traces”; they not only look inelegant and make visual inspection hard, but they also tend to take up valuable routing space and can increase inter-track capacitance and thus crosstalk. The board should look a lot cleaner, and be easier to troubleshoot, without them.

The power resistors and decoupling capacitors were things that I had mentioned upthread and was going to remind you about, but when you have large-pin connectors like the ATX, you can just solder needed components on the underside. I didn’t want to “stop the presses” over something that was easily correctable after the board was made.

You certainly could have finished the task with regular wire solder and a temperature-controlled soldering iron (there’s no fine-pitched work involved), but experience with paste may help you later on. The ATX connector may take a lot of paste, however!

If you post an updated Sprint-Layout .LAY file, I’ll take a look Tuesday night and should be able to get back to you before the board arrives.

I’m still concerned about the “ground bounce” issue. In that respect, it doesn’t matter that you have heavy connections to ground via the ATX PSU (althought that’s still a good thing!)-- it’s more an issue of ground integrity between the DE-2 board and your PCI backplane. Multiple high-speed signals can switch simultaneously at the FPGA, and there are only 2 ground pins for each of the 40-pin headers (i.e. 4 in all). Each of these pin connections has appreciable resistance and inductance. What you might want to do is run one or more (hard-wired, i.e. soldered) low-impedance wires (maybe copper braid) between the PCI adapter and good ground points on the DE-2 board (perhaps where one of the heavy-duty connectors such as the RS-232 or VGA connector has a physically-large connection to ground).

Could you please provide the exact model numbers of the boards, and preferably also the part numbers of the main PCI bridge chips on each of them? I’m not sure that i would expect the board manufacturers to provide such info, but the chipmakers should. For example, a while back I was working with some PCI video capture boards. The boardmakers were of no use for low-level work, but the Brooktree BT848 that was the “guts” of the board (carrying out all of the video work and the complete PCI interface) was fully-documented at the register level. Hopefully you can find the equivalent info for your own PCI boards…

33

[QUOTE=Antonius Block]
I’m still concerned about the “ground bounce” issue. In that respect, it doesn’t matter that you have heavy connections to ground via the ATX PSU (althought that’s still a good thing!)-- it’s more an issue of ground integrity between the DE-2 board and your PCI backplane. Multiple high-speed signals can switch simultaneously at the FPGA, and there are only 2 ground pins for each of the 40-pin headers (i.e. 4 in all). Each of these pin connections has appreciable resistance and inductance. What you might want to do is run one or more (hard-wired, i.e. soldered) low-impedance wires (maybe copper braid) between the PCI adapter and good ground points on the DE-2 board (perhaps where one of the heavy-duty connectors such as the RS-232 or VGA connector has a physically-large connection to ground).
[/QUOTE]
Ok, I think I get it now. It’s about the room for return current to the fpga. There’s something like a 10:1 ratio of signals to returns, which means each ground pin will handle 10x as much current. I’ll try running that wire. It should be easy; there’s an rs-232 half an inch from the non-soldermasked adapter.

Err, I meant the chips. Right now I’ve settled on the Marvell Yukon 88E8001 PCI/32bit/66MHz GigE controller (instead of the RealTek. This seems like the best PCI part, second only to the BCM5700 in linux benchmarks but cheap and readily available) and the Acard ATP-8620 PCI/32bit/133MHz SATAII/PATA133 controller. The ATP-8620 is pretty special not only because of its speed, but because it can operate in target mode, emulating a hard drive if connected to a PC. What I think I forgot to mention is that this whole project is about creating a SATA/AoE RAM drive.

34

Expansion.lay RTM
Gerbers, drill file, outline sent to pcbexpress

35

Ground bounce:

[QUOTE=Alex_Dubinsky]
Ok, I think I get it now. It’s about the room for return current to the fpga. There’s something like a 10:1 ratio of signals to returns, which means each ground pin will handle 10x as much current.
[/QUOTE]
Yes, that’s it. When one of your FPGA outputs switches, the initial current is going to be up to 50mA (assuming 3V and 60 Ohms – the duration of the surge will depend on the capacitance in the line). That’s per output, so if 40 lines switch at once you have an instantaneous current of 2 Amps (this is why you need good decoupling capacitors!). The outbound current is split over 40 pins, but the return path is currently only 4 pins. This puts a positive spike on the ground voltage (“ground bounce”), which can cause false triggering. If one runs everything in the FPGA synchronously and keeps the clock signal clean, then ground bounce problems can be minimized (since everything should have stabilized by the time the clock edge comes), but it’s better to kill the problem at the source by keeping ground impedances as low as possible.

i’ll repeat the question I asked above: do you have access to a good digital oscilloscope for troubleshooting?

ATX connector:
You seem to have gone with the 24-pin connector whose pinout is shown on this page. I have a couple of comments, but it might help to know the model number of the PSU that you’re using. (These comments refer to the pinout shown in the above link)
[ul]
[li]It may depend on the particular PSU, but you should probably make a link between “+3.3V sense” (pin 13) and the +3.3V line that you’re actually using to power the PCI boards (pin 12 in your .LAY file). The sense pin gives feedback to the PSU’s voltage regulator to deliver an accurate +3.3V at the board itself, accounting for voltage drop in the line. Perhaps not hugely important given your low power demands, but still something worth considering.[/li][li]I see places for capacitors in your new board (although not for -12V), but where are the power resistors to guarantee a minimum load? What values are you planning on using?[/li][li]I see that you’ve hard-wired pin 16 (“Power On”) to ground. Does your PSU have an “on/Off” switch?[/ul] [/li]Clock signal:
If there’s one signal that needs to be really clean with sharp edges, it’s the PCI clock. You might want to consider bypassing the 47Ohm series resistor on the DE-2 board for that one signal, if possible. Again, a digital oscilloscope would help check signal integrity here.

General PCB comments:
I’m glad to see that you’ve cleaned up the “orphan” traces, but you still have a lot of traces (in the area of the PCI connectors) that dead-end long beyond their useful length. In general this is bad practice because they can radiate and pick up RF noise, and they take up valuable routing space. The three traces that mar the beauty of the ground plane could easily have been routed on the same side as the others if it weren’t for the “semi-orphans”. It may sound like a nitpick, but I’ve found that an elegant-looking PCB is also best for signal integrity. It’s an art as much as a science…

Silicon datasheets:

OK, those are the primary chips of interest. What are the part #s of the PCI boards that are they on?

Do you mean that you’re using the DE-2 SDRAM as a RAM drive, connected to a PC through either SATA or AoE? Or is there another piece of hardware not yet mentioned?

36

[QUOTE=Antonius Block]
i’ll repeat the question I asked above: do you have access to a good digital oscilloscope for troubleshooting?
[/QUOTE]
No. Can I buy a good used one on ebay for ~$200? I have to buy one soon for the next stage of the project.

[QUOTE]
ATX connector:
You seem to have gone with the 24-pin connector whose pinout is shown on this page. I have a couple of comments, but it might help to know the model number of the PSU that you’re using. (These comments refer to the pinout shown in the above link)
[list]
[li]It may depend on the particular PSU, but you should probably make a link between “+3.3V sense” (pin 13) and the +3.3V line that you’re actually using to power the PCI boards (pin 12 in your .LAY file). The sense pin gives feedback to the PSU’s voltage regulator to deliver an accurate +3.3V at the board itself, accounting for voltage drop in the line. Perhaps not hugely important given your low power demands, but still something worth considering.[*]I see places for capacitors in your new board (although not for -12V), but where are the power resistors to guarantee a minimum load? What values are you planning on using?[/li][/QUOTE]
Ok, I’ll jumper pin 13 to the +3.3v line. Btw at first I thought you meant pin 13 was a dedicated v-sense signal, but now I see what you’re saying. That’s good thinking.
The capacitors are actually only on the 5v/3.3v. That thing on +12 is a loading resistor (it looks strange because I’m mounting it vertically). I’m drawing 1A on +5/+3.3 and 1/4A on +12V. There’s no capacitors on +/- 12v because those lines are rarely used. In general, I’ve realized that bypass capacitors aren’t critical because PCI cards should have a full set onboard (this also made me sleep easier regarding my voltage sources running on puny traces).

This will be difficult, since the resistor arrays are tiny and right next to the header. I’ll try to get the oscilloscope asap.

Yeah, in retrospect I could have made those criss-crossing traces vertical. I’ve read repeatedly that discontinuities in the power plane, even if it’s not a split plane, are bad. Thing is I had left off connecting them till the end, and just hooked them to whatever pin was free. However, I don’t think it’d be possible to route them entirely on top. Anyway, you’ll notice my routing scheme is very simplistic. It’s just four traces + the tiling function in Sprint. But it’s also completely symmetric and neat (except for the 3 unbussed signals), so I think it fits with what you’re saying. I guess I could have still taken the time to trim the dead ends. However, in a real, sparsely-populated PCI bus, the whole thing is one big dead end, so it can’t be that bad.

The Marvell chip is mounted on a D-Link DGE-530T. I have not found a commercial PCI card that carries the Acard controller. A sample should be available upon request from Acard, however.

Yes, that’s right. This is all just the first prototype, however. The next step is to create a board with a Cyclone III and DIMM (and PCI) connectors on it. That will be the fun part. (Although not as scary as it sounds because I have no need to run the RAM at high frequency).

37

Whew, finally got it working. Haven’t tested it thoroughly yet, but I can read DeviceIDs at 50MHz.

38

To answer my own question in the OP, edaboard.com is a good resource for info about PCB routing.

This thread, for example, gives some good rules of thumb for at what point signal quality begins to matter: http://www.edaboard.com/ftopic49082.html