Hardware Physical Address Registers

This may be a bit off topic, if you find it to be too much so, please direct
me to the correct place...

The hardware guys that are designing a pci express device for which I am
writing the driver want to conserve some space on our ASIC (surprise,
surprise). I have informed them that it would be beneficial if our ASIC's
DMA controller had a 64 bit host address register so that our device can
perform a DMA to memory above 4GB. Their solution was just to add 5 bits to
the register to allow the ASIC's DMA engine to address up to 128 GB of memory.

I am concerned that the LogicalAddress returned from AllocateCommonBuffer
will someday give me an address that does not correspond to the physical
memory address. It could give me something greater than 128GB, even though
there is not 128GB of memory installed, a logical bus address is, after all
not necessarily fixed to the physical address of the memory.

So do I need to instruct the hardware guys to give me a host address
register than can address the full 64 bit address range, or is their solution
of just addressing 128GB acceptable?

Calvin
Tue Jan 03 17:29:40 CST 2006

IMO, it's silly to build a PCIe device not being able to access full 64-bit
address space. After all, unlike pci-x, PCIe is a serial bus, all the chip
need to do is being able to understand 64bit host address and build the TLP
correctly, no extra address pin is required. That should not make the die
significantly larger.

--
Calvin Guan (Windows DDK MVP)
NetXtreme Longhorn Miniport Prime
Broadcom Corp. www.broadcom.com

"Lou Rohan" <LouRohan@discussions.microsoft.com> wrote in message
news:18A548CB-33C2-4145-B83F-6F1BFE2E4EA6@microsoft.com...
> This may be a bit off topic, if you find it to be too much so, please
> direct
> me to the correct place...
>
> The hardware guys that are designing a pci express device for which I am
> writing the driver want to conserve some space on our ASIC (surprise,
> surprise). I have informed them that it would be beneficial if our ASIC's
> DMA controller had a 64 bit host address register so that our device can
> perform a DMA to memory above 4GB. Their solution was just to add 5 bits
> to
> the register to allow the ASIC's DMA engine to address up to 128 GB of
> memory.
>
> I am concerned that the LogicalAddress returned from AllocateCommonBuffer
> will someday give me an address that does not correspond to the physical
> memory address. It could give me something greater than 128GB, even
> though
> there is not 128GB of memory installed, a logical bus address is, after
> all
> not necessarily fixed to the physical address of the memory.
>
> So do I need to instruct the hardware guys to give me a host address
> register than can address the full 64 bit address range, or is their
> solution
> of just addressing 128GB acceptable?

LouRohan
Wed Jan 04 15:07:02 CST 2006

Thank you for the hardware advice :-), but I am more curious about the
AllocateCommonBuffer and what might happen with future hw platforms and DDKs.

Thanks,
Lou Rohan

"Calvin Guan" wrote:

> IMO, it's silly to build a PCIe device not being able to access full 64-bit
> address space. After all, unlike pci-x, PCIe is a serial bus, all the chip
> need to do is being able to understand 64bit host address and build the TLP
> correctly, no extra address pin is required. That should not make the die
> significantly larger.
>
> --
> Calvin Guan (Windows DDK MVP)
> NetXtreme Longhorn Miniport Prime
> Broadcom Corp. www.broadcom.com
>
> "Lou Rohan" <LouRohan@discussions.microsoft.com> wrote in message
> news:18A548CB-33C2-4145-B83F-6F1BFE2E4EA6@microsoft.com...
> > This may be a bit off topic, if you find it to be too much so, please
> > direct
> > me to the correct place...
> >
> > The hardware guys that are designing a pci express device for which I am
> > writing the driver want to conserve some space on our ASIC (surprise,
> > surprise). I have informed them that it would be beneficial if our ASIC's
> > DMA controller had a 64 bit host address register so that our device can
> > perform a DMA to memory above 4GB. Their solution was just to add 5 bits
> > to
> > the register to allow the ASIC's DMA engine to address up to 128 GB of
> > memory.
> >
> > I am concerned that the LogicalAddress returned from AllocateCommonBuffer
> > will someday give me an address that does not correspond to the physical
> > memory address. It could give me something greater than 128GB, even
> > though
> > there is not 128GB of memory installed, a logical bus address is, after
> > all
> > not necessarily fixed to the physical address of the memory.
> >
> > So do I need to instruct the hardware guys to give me a host address
> > register than can address the full 64 bit address range, or is their
> > solution
> > of just addressing 128GB acceptable?
>
>
>

Mark
Wed Jan 04 21:34:23 CST 2006

On Tue, 3 Jan 2006 14:53:03 -0800, Lou Rohan
<LouRohan@discussions.microsoft.com> wrote:

>This may be a bit off topic, if you find it to be too much so, please direct
>me to the correct place...
>
>The hardware guys that are designing a pci express device for which I am
>writing the driver want to conserve some space on our ASIC (surprise,
>surprise). I have informed them that it would be beneficial if our ASIC's
>DMA controller had a 64 bit host address register so that our device can
>perform a DMA to memory above 4GB. Their solution was just to add 5 bits to
>the register to allow the ASIC's DMA engine to address up to 128 GB of memory.
>
>I am concerned that the LogicalAddress returned from AllocateCommonBuffer
>will someday give me an address that does not correspond to the physical
>memory address. It could give me something greater than 128GB, even though
>there is not 128GB of memory installed, a logical bus address is, after all
>not necessarily fixed to the physical address of the memory.
>

Yeah I think you have a problem. I thought that perhaps the
DeviceDesciptor allowed you to qualify addressing capabilities, but on
review, it doesn't. There is no guarantee that memory is mapped
contiguously, or that it runs from zero to sizeof ram. (Although it
certainly is very likely on most platforms.) But what prevents your
driver from running on a platform that has more than 37 bits worth of
ram?

>So do I need to instruct the hardware guys to give me a host address
>register than can address the full 64 bit address range, or is their solution
>of just addressing 128GB acceptable?

No I think it is not acceptable. I am not even convinced it is PCI
compliant. I think there are two flavors of addressing: 32bit and
64bit, and that this is not some continuum where you can support 0-64
bit addressing.


=====================
Mark Roddy DDK MVP
Windows Vista/2003/XP/2000 Consulting
Device and Filesystem Drivers
Hollis Technology Solutions 603-321-1032
www.hollistech.com