Sepherosa Ziehau [Thu, 29 Jan 2009 11:25:37 +0000 (19:25 +0800)]
busdma(9): Dup i386 busdma_machdep.c to vkernel
Sepherosa Ziehau [Thu, 29 Jan 2009 11:24:33 +0000 (19:24 +0800)]
busdma(9): Dup i386 busdma_machdep.c to amd64
Sepherosa Ziehau [Thu, 29 Jan 2009 10:51:10 +0000 (18:51 +0800)]
Unbreak LINT build
Sepherosa Ziehau [Thu, 29 Jan 2009 08:25:56 +0000 (16:25 +0800)]
sis(4): Rework busdma(9) related bits
- RX/TX descriptor rings need to be 4bytes aligned.
- RX buffers need to be 4bytes aligned.
- Create TX/RX buffer busdma map at attaching time and destroy them
at detaching time.
- Rework sis_newbuf() by utilizing bus_dmamap_load_mbuf_segment().
- Factor sis_setup_rxdesc() out of sis_newbuf() and call it on RX
error handling path.
- Rework sis_encap() by utilizing bus_dmamap_load_mbuf_defrag().
- Correct IFF_OACTIVE handling in sis_start()/sis_txeof().
Sepherosa Ziehau [Thu, 29 Jan 2009 03:39:40 +0000 (11:39 +0800)]
sis(4): Move software specific fields out of hardware descriptor structure
Sepherosa Ziehau [Wed, 28 Jan 2009 12:16:51 +0000 (20:16 +0800)]
bce(4): Remove unnecessary bus_dmamap_sync()
Sepherosa Ziehau [Tue, 27 Jan 2009 14:53:18 +0000 (22:53 +0800)]
bce(4): Adjust RX path to avoid fatal error caused by busdma failure
- Create tmp busdma map for loading mbuf on RX path, so busdma map
loading failure will not be fatal.
- Don't unload RX buffer's busdma map until we could make sure that
new RX buffer could be loaded.
- On RX path, if error happens, we only need to reconfigure RX desc,
so just call bce_setup_rxdesc_std().
- Make sure that sw_chain_cons and sw_chain_prod are same, else
log error and reconfigure the RX desc indiced by sw_chain_cons.
- Utilize bus_dmamap_load_mbuf_segment().
Sepherosa Ziehau [Tue, 27 Jan 2009 09:00:27 +0000 (17:00 +0800)]
bce(4): Factor RX desc setup out of bce_newbuf_std()
Sepherosa Ziehau [Tue, 27 Jan 2009 05:05:12 +0000 (13:05 +0800)]
bce(4): Utilize bus_dmamap_load_mbuf_defrag()
Sepherosa Ziehau [Tue, 27 Jan 2009 04:37:30 +0000 (12:37 +0800)]
bce(4): Use coherent for non-buffer data storage
- Utilize bus_dmamem_coherent_any()
- Put RX/TX pages into coherent memory
- Don't sync coherent memory
Sepherosa Ziehau [Tue, 27 Jan 2009 03:20:33 +0000 (11:20 +0800)]
bce(4): Correct parameters passed to busdma(9) functions
Sepherosa Ziehau [Mon, 26 Jan 2009 14:07:10 +0000 (22:07 +0800)]
stge(4): Remove unnecessary bus_dmamap_sync()
Sepherosa Ziehau [Mon, 26 Jan 2009 13:53:25 +0000 (21:53 +0800)]
stge(4): Adjust TX path
- Utilize bus_dmamap_load_mbuf_defrag()
- Sync TX buffer before setup the TX desc
- In stge_{txeof,start}(), use the same condition to clear/set IFF_OACTIVE
Sepherosa Ziehau [Mon, 26 Jan 2009 12:25:41 +0000 (20:25 +0800)]
stge(4): Utilize bus_dmamap_load_mbuf_segment()
Sepherosa Ziehau [Mon, 26 Jan 2009 12:08:15 +0000 (20:08 +0800)]
stge(4): Don't sync coherent memory
Sepherosa Ziehau [Mon, 26 Jan 2009 12:05:03 +0000 (20:05 +0800)]
stge(4): Adjust the error handling code on stge_dma_alloc()
Sepherosa Ziehau [Mon, 26 Jan 2009 11:45:29 +0000 (19:45 +0800)]
stge(4): Utilize bus_dmamem_coherent_any()
Sepherosa Ziehau [Mon, 26 Jan 2009 11:31:44 +0000 (19:31 +0800)]
stge(4): Regroup busdma tag creation; prepare for the upcoming changes
Sepherosa Ziehau [Mon, 26 Jan 2009 11:22:07 +0000 (19:22 +0800)]
stge(4): Correct parameters passed to busdma(9) functions
Sepherosa Ziehau [Mon, 26 Jan 2009 07:11:56 +0000 (15:11 +0800)]
et(4): Utilize bus_dmamap_load_mbuf_{segment,defrag}()
Sepherosa Ziehau [Mon, 26 Jan 2009 06:52:42 +0000 (14:52 +0800)]
et(4): Split RX/TX mbuf busdma tag
Sepherosa Ziehau [Mon, 26 Jan 2009 06:21:00 +0000 (14:21 +0800)]
et(4): Don't sync coherent memory
Sepherosa Ziehau [Mon, 26 Jan 2009 05:26:12 +0000 (13:26 +0800)]
et(4): Utilize bus_dmamem_coherent_any()
Sepherosa Ziehau [Mon, 26 Jan 2009 05:24:48 +0000 (13:24 +0800)]
busdma(9): Add bus_dmamem_coherent_any()
This function is a simplified version of bus_dmamem_coherent(),
see the comment in sys/bus_dma.h for details.
Sepherosa Ziehau [Mon, 26 Jan 2009 04:29:38 +0000 (12:29 +0800)]
et(4): Utilize bus_dmamem_coherent()
Sepherosa Ziehau [Mon, 26 Jan 2009 04:09:08 +0000 (12:09 +0800)]
et(4): Correct parameters passed to busdma(9) functions
Sepherosa Ziehau [Mon, 26 Jan 2009 03:05:06 +0000 (11:05 +0800)]
msk(4): Align TX buffer to 64bytes boundary instead of 8bytes boundary
The correct alignment boundary is found out by using busdma(9)'s
alignment constraint. However, it seems that using in-place copy
results in better performance than let busdma(9) to bounce misaligned
TX buffer, especially with tiny packets; probably due to the "in-place"
Sepherosa Ziehau [Sun, 25 Jan 2009 07:39:40 +0000 (15:39 +0800)]
msk(4): Align TX buffer to 8bytes boundary to improve TX performance
The performance improvement is ~100Mbps (from 83xMbps to 93xMbps) for
bulk data. This is accidentally found by forcing msk(4) to bounce TX
buffers to lower address during a busdma(4) test (bounce pages are
PAGE_SIZE aligned).
Sepherosa Ziehau [Sun, 25 Jan 2009 03:38:56 +0000 (11:38 +0800)]
busdma(9): Obey alignment constraint by default
Sepherosa Ziehau [Sun, 25 Jan 2009 03:17:47 +0000 (11:17 +0800)]
msk(4): Avoid busdma map leakage on error handling path
Sepherosa Ziehau [Sun, 25 Jan 2009 03:09:55 +0000 (11:09 +0800)]
msk(4): Adjust TX path
- Utilize bus_dmamap_load_mbuf_defrag().
- Fix the IFF_OACTIVE setting in msk_start(). This restores the
fragmented UDP output performance.
- Sync TX buffer before setup TX desc.
- Remove unnecessary bus_dmamap_sync().
Sepherosa Ziehau [Sun, 25 Jan 2009 02:26:31 +0000 (10:26 +0800)]
msk(4): Utilize bus_dmamap_load_mbuf_segment()
Sepherosa Ziehau [Sun, 25 Jan 2009 02:06:01 +0000 (10:06 +0800)]
msk(4): Utilize bus_dmamem_coherent() and don't sync coherent memory
Sepherosa Ziehau [Sat, 24 Jan 2009 09:04:06 +0000 (17:04 +0800)]
msk(4): Correct parameters passed to busdma(9) functions
Sepherosa Ziehau [Sat, 24 Jan 2009 06:59:14 +0000 (14:59 +0800)]
sk(4): Utilize bus_dmamap_load_mbuf_defrag()
Sepherosa Ziehau [Sat, 24 Jan 2009 06:33:42 +0000 (14:33 +0800)]
sk(4): Adjust RX path a little bit
- Utilize bus_dmamap_load_mbuf_segment()
- Don't try m_devget(); it does not help much, if the NIC is quenched
by the peer, e.g. by line rate fragmented UDP datagrams
- Move bus_dmamap_sync() from sk_rxeof() into sk_newbuf_std()
Sepherosa Ziehau [Sat, 24 Jan 2009 05:56:27 +0000 (13:56 +0800)]
sk(4): Rework busdma(9) related bits
- According to data sheet, sk(4) does support 64bits address space.
However, the RX/TX descriptors' high 32bits must be same, i.e.
descriptors' can't spread above and below 4Gbytes boundary. Since
sk(4)'s RX/TX descriptors are in contiguous memory, this means we
only need to make sure that the contiguous memory does not cross
4Gbytes boundary.
- RX/TX descriptors' is 8bytes aligned not 64bytes aligned.
- Split RX/TX descriptor rings.
- Use bus_dmamem_coherent() to allocate RX/TX descriptor rings.
- Don't sync coherent memory.
Sepherosa Ziehau [Fri, 23 Jan 2009 12:01:10 +0000 (20:01 +0800)]
sk(4): Adjust TX path a little bit.
Sepherosa Ziehau [Fri, 23 Jan 2009 10:40:18 +0000 (18:40 +0800)]
rl(4): Don't use magic number
Sepherosa Ziehau [Fri, 23 Jan 2009 10:31:25 +0000 (18:31 +0800)]
rl(4): Rework busdma(9) related bits
- Correct parameters passed to busdma(9) functions.
- Split RX/TX busdma tag.
- Use bus_dmamem_coherent() to allocate RX buffers.
- Create TX buffer busdma map at attach time and destroy them only
at detach time.
- Rework rl_encap().
- Don't sync RX buffers, which is coherent memory.
Sepherosa Ziehau [Fri, 23 Jan 2009 07:45:54 +0000 (15:45 +0800)]
bge(4): Remove unnecessary bus_dmamap_sync()
Sepherosa Ziehau [Fri, 23 Jan 2009 07:43:36 +0000 (15:43 +0800)]
bge(4): Utilize bus_dmamap_load_mbuf_{defrag,segment}()
Sepherosa Ziehau [Fri, 23 Jan 2009 07:15:00 +0000 (15:15 +0800)]
bge(4): Rework bge_newbuf_{std,jumbo}()
- Always fill the std RX ring, instead of leaving it half full.
On modern system, 1MBytes non-contiguous memory will not be
the source of trouble.
- Record RX buffer's physical address along with the RX mbuf.
- Factor out bge_setup_rxdesc_{std,jumbo}(); call them if RX errors
happen.
- Move RX std buffer busdma map sync/unload into bge_newbuf_std()
- Use temporary busdma map to load RX std buffer, so that the busdma
map loading failure will not be fatal.
Sepherosa Ziehau [Fri, 23 Jan 2009 04:35:27 +0000 (12:35 +0800)]
bge(4): Split RX/TX mbuf busdma tags
Sepherosa Ziehau [Fri, 23 Jan 2009 04:05:35 +0000 (12:05 +0800)]
bge(4): Don't sync coherent memory
Sepherosa Ziehau [Fri, 23 Jan 2009 03:55:17 +0000 (11:55 +0800)]
bge(4): Utilize bus_dmamem_coherent()
Sepherosa Ziehau [Fri, 23 Jan 2009 03:36:19 +0000 (11:36 +0800)]
bge(4): Correct busdma parameters
Sepherosa Ziehau [Fri, 23 Jan 2009 03:13:19 +0000 (11:13 +0800)]
if_bgereg.h: Remove extra white space
Sepherosa Ziehau [Fri, 23 Jan 2009 02:47:55 +0000 (10:47 +0800)]
busdma(9): Update FreeBSD RSCID.
Sepherosa Ziehau [Thu, 22 Jan 2009 13:08:48 +0000 (21:08 +0800)]
busdma(9): Don't record/use boundary in bounce zone
Bounce zone's boundary does not make any sense for PAGE_SIZE
contiguous allocation which also aligned on multiple or one
PAGE_SIZE.
Sepherosa Ziehau [Thu, 22 Jan 2009 12:45:01 +0000 (20:45 +0800)]
busdma(9): Use the passed in 'flags' instead of hard coding BUS_DMA_NOWAIT
Sepherosa Ziehau [Thu, 22 Jan 2009 12:30:57 +0000 (20:30 +0800)]
jme(4): Don't sync coherent memory
Sepherosa Ziehau [Thu, 22 Jan 2009 12:13:07 +0000 (20:13 +0800)]
re(4): Don't sync coherent memory
Sepherosa Ziehau [Thu, 22 Jan 2009 12:12:51 +0000 (20:12 +0800)]
nfe(4): Coherent memory does not need to be synced
Sepherosa Ziehau [Thu, 22 Jan 2009 11:56:17 +0000 (19:56 +0800)]
xl(4): Coherent memory does not need to be synced
Sepherosa Ziehau [Thu, 22 Jan 2009 11:35:48 +0000 (19:35 +0800)]
bfe(4): Coherent memory does not need to be synced
Sepherosa Ziehau [Thu, 22 Jan 2009 11:03:17 +0000 (19:03 +0800)]
jme(4): Move TX buffer syncing before setting up TX desc
Sepherosa Ziehau [Thu, 22 Jan 2009 10:31:53 +0000 (18:31 +0800)]
jme(4): Utilize bus_dmamap_load_mbuf_{segment,defrag}()
Sepherosa Ziehau [Thu, 22 Jan 2009 09:54:23 +0000 (17:54 +0800)]
jme(4): Depend on parent busdma tag to apply lowaddr constraint
Sepherosa Ziehau [Wed, 21 Jan 2009 14:21:59 +0000 (22:21 +0800)]
jme(4): jme_dma_free() is only called on detach path now
Sepherosa Ziehau [Wed, 21 Jan 2009 14:18:10 +0000 (22:18 +0800)]
jme(4): Utilize bus_dmamem_coherent()
Sepherosa Ziehau [Wed, 21 Jan 2009 13:49:24 +0000 (21:49 +0800)]
jme(4): Fix busdma(9) parameters
Sepherosa Ziehau [Wed, 21 Jan 2009 13:19:26 +0000 (21:19 +0800)]
jme(4): Rely on busdma(9) to do the boundary check
Sepherosa Ziehau [Wed, 21 Jan 2009 10:54:10 +0000 (18:54 +0800)]
busdma(9): Add sysctl node to show whether aligment constraint is obeyed or not
Sepherosa Ziehau [Tue, 20 Jan 2009 15:11:59 +0000 (23:11 +0800)]
busdma(9): Obey alignment constraint, disabled by default
- Obey alignment constraint
Obtained-from: FreeBSD
- Correct "cross page boundary" check in bus_dmamem_alloc()
- Make sure that after kmalloc() retry, memory will not cross page
boundary and is properly aligned. If not, print backtrace; should
panic though.
- Factor out BUS_DMA_KMALLOC(), so we don't duplicate the conditions
in bus_dmamem_alloc() and bus_dmamem_free()
- Assert the input memory is properly aligned, if caller told us so
Sepherosa Ziehau [Tue, 20 Jan 2009 12:09:57 +0000 (20:09 +0800)]
busdma(9): get rid of bounce_lowaddr, which serves no purpose
Sepherosa Ziehau [Tue, 20 Jan 2009 11:38:41 +0000 (19:38 +0800)]
xl(4): Remove unused bus_dmamap_sync()
Sepherosa Ziehau [Mon, 19 Jan 2009 13:53:08 +0000 (21:53 +0800)]
xl(4): Rework xl_encap()
Sepherosa Ziehau [Mon, 19 Jan 2009 11:52:48 +0000 (19:52 +0800)]
xl(4): Rework xl_newbuf()
- Utilize bus_dmamap_load_mbuf_segment()
- Move setting buffer address before clearing status
- Add init flag, so we could use M_WAIT during device initialization
- Move bus_dmamap_sync(POSTREAD) into xl_newbuf() and do busdma sync
and unload iff we previously loaded a mbuf.
Sepherosa Ziehau [Mon, 19 Jan 2009 11:24:09 +0000 (19:24 +0800)]
xl(4): Split RX and TX mbuf busdma tag
Sepherosa Ziehau [Mon, 19 Jan 2009 11:01:32 +0000 (19:01 +0800)]
xl(4): Utilize bus_dmamem_coherent()
Sepherosa Ziehau [Mon, 19 Jan 2009 10:46:37 +0000 (18:46 +0800)]
xl(4): Allocate parent busdma tag, which is used to constraint lowaddr
Sepherosa Ziehau [Sun, 18 Jan 2009 11:33:31 +0000 (19:33 +0800)]
nfe(4): Utilize bus_dmamap_load_mbuf_{segment,defrag}()
Sepherosa Ziehau [Sun, 18 Jan 2009 08:36:03 +0000 (16:36 +0800)]
re(4): Utilize bus_dmamap_load_mbuf_{segment,defrag}()
Sepherosa Ziehau [Sun, 18 Jan 2009 05:55:18 +0000 (13:55 +0800)]
busdma(9): Add BUS_DMA_ALIGNED hint flag
This flag is used to give hint to busdma(9) that all to-be-loaded
memory is proper aligned, thus no bounce pages should be allocated.
This will be used when busdma obeys alignment requirement.
It could save us some bounce page memory under following situation:
Some NICs require RX buffer to be 4 or 8 bytes aligned while our mbuf
cluster allocation always promises 2048 bytes alignment.
Rename BUS_DMA_ONEPAGE to BUS_DMA_ONEBPAGE, should have done that
in the first place though ...
Sepherosa Ziehau [Sun, 18 Jan 2009 03:58:03 +0000 (11:58 +0800)]
bfe(4): Utilize bus_dmamap_load_mbuf_defrag()
Sepherosa Ziehau [Sun, 18 Jan 2009 03:56:03 +0000 (11:56 +0800)]
busdma(9): Add bus_dmamap_load_mbuf_defrag()
This function is mainly used to avoid code duplication in various
(should be many) NIC driver code.
Sepherosa Ziehau [Sun, 18 Jan 2009 03:18:18 +0000 (11:18 +0800)]
bfe(4): Utilize bus_dmamap_load_mbuf_segment()
Sepherosa Ziehau [Sun, 18 Jan 2009 03:14:38 +0000 (11:14 +0800)]
busdma(9): Add bus_dmamap_load_mbuf_segment()
See the comment in sys/bus_dma.h for detailed information
about this function.
Let bus_dmamap_load_mbuf() call this function with busdma tag's
segments and nsegments.
Inspired-by: FreeBSD's bus_dmamap_load_mbuf_sg()
Sepherosa Ziehau [Sat, 17 Jan 2009 13:02:16 +0000 (21:02 +0800)]
nfe(4): busdma(9) fixes
On RX path:
- Unload busdma map iff the map has already been loaded
- Add missing bus_dmamap_sync()
On TX path:
- Reduce log verbosity
- Move buffer content's bus_dmamap_sync() to the proper place
Misc:
- Nuke unnecessary bus_dmamap_sync()
Sepherosa Ziehau [Sat, 17 Jan 2009 12:59:58 +0000 (20:59 +0800)]
busdma(9): More bounce page bug fixing
Turn on BUS_DMA_MIN_ALLOC_COMP, iff we do allocate bounce pages for
the busdma tag.
Sepherosa Ziehau [Sat, 17 Jan 2009 11:24:28 +0000 (19:24 +0800)]
busdma(9): Add tunable and read-only sysctl node for max # of bounce
pages per bounce zone.
Sepherosa Ziehau [Sat, 17 Jan 2009 11:22:57 +0000 (19:22 +0800)]
nfe(4): Always set high part of the address if the NIC supports
40bits address space accessing.
Sepherosa Ziehau [Sat, 17 Jan 2009 10:59:57 +0000 (18:59 +0800)]
nfe(4): busdma(9) correction
- Use bus_dmamem_coherent()
- Fix busdma(9) parameters
Sepherosa Ziehau [Sat, 17 Jan 2009 10:34:50 +0000 (18:34 +0800)]
re(4): add missing BUS_DMA_WAITOK
Sepherosa Ziehau [Sat, 17 Jan 2009 10:35:38 +0000 (18:35 +0800)]
bfe(4): Add missing BUS_DMA_WAITOK
Sepherosa Ziehau [Sat, 17 Jan 2009 09:00:50 +0000 (17:00 +0800)]
bfe(4): use bus_dmamem_coherent()
Sepherosa Ziehau [Sat, 17 Jan 2009 08:19:09 +0000 (16:19 +0800)]
Factor out bus_dmamem_coherent() to allocate busdma memory.
Mainly to avoid code duplication in various drivers.
Sepherosa Ziehau [Sat, 17 Jan 2009 06:26:50 +0000 (14:26 +0800)]
Change bfe(4) according to recent busdma(9) bounce page fixes
Sepherosa Ziehau [Sat, 17 Jan 2009 05:21:29 +0000 (13:21 +0800)]
Bounce page work
For NICs which support jumbo frame but need to allocate one TX
descriptor per segment, it does not make sense to allocate more
than one bounce page for a single TX mbuf busdma map. Add
BUS_DMA_ONEPAGE, so callers of busdma(9) functions could give
hint that only one bounce page is needed for a busdma map.
Sepherosa Ziehau [Sat, 17 Jan 2009 04:35:27 +0000 (12:35 +0800)]
Bounce page fix
- In bus_dma_tag_create(), make sure that:
o Alignment is power of 2
o Boundary is power of 2
o Max segment size is less than boundary
Obtained-from: FreeBSD
- When counting the pages needed by maxsize, round up the maxsize to
multiple page size first.
- For bounce zones, always round up its alignment and boundary to
page size.
- Pass 'flags' to alloc_bounce_pages(), so we could assign 'mflags'
to contigmalloc().
Sepherosa Ziehau [Sat, 17 Jan 2009 02:14:38 +0000 (10:14 +0800)]
Bounce page fixes
- Free accumulated bounce pages in the bus dma map if we could not
reserved enough bounce pages.
- In bus_dmamap_load_mbuf(), if _bus_dmamap_load_buffer() returns
ENOMEM on non-first mbuf fragment, we adjust the error to EFBIG:
o There is at least one bounce page available, so we could expect
that after mbuf defragmentation, enough bounce pages (normally
one bounce page for non-jumbo frame) may be available.
o Most callers defragment the mbuf chain only if error is EFBIG.
Sepherosa Ziehau [Sat, 17 Jan 2009 01:58:26 +0000 (09:58 +0800)]
Increase RE_TXDESC_SPARE to cooperate 9K jumbo frame.
Sepherosa Ziehau [Fri, 16 Jan 2009 14:52:20 +0000 (22:52 +0800)]
Allocate at least one bounce page if the dma tag indicates bouncing
is needed. Bug was found by forcing re(4) to bounce mbufs to lower
physical memory (re(4)'s TX and RX buffers use different bounce zones).
Sepherosa Ziehau [Fri, 16 Jan 2009 13:00:57 +0000 (21:00 +0800)]
Split RX/TX bus dma tag. RX buffers should to be aligned on 8 bytes.
Sepherosa Ziehau [Fri, 16 Jan 2009 12:35:06 +0000 (20:35 +0800)]
Correct busdma(9) parameters
Sepherosa Ziehau [Fri, 16 Jan 2009 11:42:30 +0000 (19:42 +0800)]
There should be no 2 bytes alignment requirement
Sepherosa Ziehau [Fri, 16 Jan 2009 11:41:23 +0000 (19:41 +0800)]
Add assertion to make sure that the segment index is in valid range.
Sepherosa Ziehau [Fri, 16 Jan 2009 11:16:39 +0000 (19:16 +0800)]
Add assertion to make sure that bounce zone is in sane state.
Sepherosa Ziehau [Thu, 15 Jan 2009 13:28:38 +0000 (21:28 +0800)]
Free the bounce pages allocated and reserved by the busdma map,
if _bus_dmamap_load_buffer() fails because of too many segments.