So Phoronix pointed out we have a KMS/DRI2 3D regression vs UMS/DRI1, two things sprang to mind wrt to places we lost some of this so I did a quick benchmark on my laptop.

Thinkpad T60P, rv530 FireGL V5200, Intel Core Duo T2500 CPU 2Ghz, 1600x1200 LCD

F-12 mostly.

So I ran openarena with Eric Anholt demo at 1400x1050 (not fullscreen)

5 tests, non-kms, kms out of the box, kms with color tiling, kms with vsync remove, kms with vsync removed + colortiling

non-kms: 840 frames 25.3 seconds 33.2 fps 9.0/30.2/55.0/6.7 ms
kms-nontiled: 840 frames 38.2 seconds 22.0 fps 9.0/45.5/81.0/9.4 ms
kms-tiling: 840 frames 31.6 seconds 26.6 fps 19.0/37.6/72.0/7.2 ms
kms-nontiled-novsync: 840 frames 35.7 seconds 23.5 fps 15.0/42.5/77.0/8.9 ms
kms-tiled-novsync: 840 frames 28.7 seconds 29.2 fps 11.0/34.2/66.0/7.6 ms

So there is some explaination of where some of the performance went, I'll definitely enable KMS tiling on r300->r500 where
we should have enough stuff in mesa to deal with all the sw fallbacks (gallium may need some work). The vsync vs non-vsync thing isn't something I really want to change, basically when DRI2 does the back->front copy it doesn't do it when the scanout is in the area being copied so with DRI1 you'd get tearing, with dri2 you get none. We'll also get DRI2 page flipping support at some point, which for a fullscreen app should really improve things. Besides that I'm sure there is lot of optimisation that can be done in the KMS codebase to improve this situation. I (or someone else please feel free) need to break out the profiling tools (perf, sysprof etc) and track down where we can do better.

At the moment the priority is still to make things work first, then make them fast.

Finally got some time to take another look at radeon displayport support.

I wrote initial userspace modesetting patches in June to get EDID and by mid July I had it to the stage where I could
get a display if I started X, then used xrandr to switch to a different mode. I ran out of time to continue investigations then.

Last week Alex Deucher rebased my code into the latest DDX and cleaned up a lot of the insanity in it. At the same time I started porting the code to the kernel for KMS support. Alex also picked that up and made the userspace and kernel drivers pretty much act the same. However at the end of this neither of us actually had reliable display port support.

So this morning I discovered that actually every second modeset was working, with ones in between turning off power to the monitor, so I moved a few things around in the displayport link training initialisation code and voila reliable userspace displayport. I then ported the same fix to the kernel and it didn't make it work. Another 2-3 hours of guessing found a missing
return statement which meant we ended up with a version 0 DP identification which broke stuff. With that fixed, KMS displayport started working.

Where can I get this?
UMS/DDX: git://git.freedesktop.org/git/xorg/driver/xf86-video-ati displayport branch
KMS: git://git.kernel.org/pub/scm/linux/kernel/git/airlied/drm-2.6.git drm-radeon-dp branch

TODO:
a) When r600 grows IRQ support we need to use the hotplug irq to detect the monitor appearing/disappearing
so we can retrain the link with it, otherwise unplugging the monitor means it won't be there on replug,
unless you change modes, for userspace we probably need to use a timer to detect if the monitor goes away or we lose
training.
b) cleanup code, maybe see what we can share with Intel.
c) merge, I'd say we can nearly merge the UMS code to master, and KMS can definitely go in the next merge window.

[updated for pedants].

So I (and other radeon developers) debug a lot of radeon problems, both locally and with people over irc/bugzilla, and I often am quite slow to deal with bugs that I can reproduce locally, its usually a last resort to do remote debugging and its unfortunate for people who have hw bugs that we can't reproduce locally.

So what prompted this post?
https://bugzilla.redhat.com/show_bug.cgi?id=527874
KMS:RV515:X1400 Thinkpad T60 resume fails

So first up, my local Thinkpad T60p with an rv530 always resumed fine, my local T42 with 7500 mostly works okay as well, so
there goes my local reproduction.

So Peng works for Red Hat in Beijing and the week before kernel summit I sat down on irc for most of two days with him running various tests. We tracked it down in that 2 days to the fact that his video RAM wasn't getting setup properly on resume. The NMI on resume let me track down that when the gpu accessed the ring, it generated an error on the PCI bus, this led to checking the contents of the PCIE gart table (with a detour through kernel vmap page handling). The PCIE gart table is in VRAM, and upon checking it on resume noticed when we copied it back into VRAM it was getting mangled. So we could deduce VRAM was broken.

I handed off to Jerome and he got traces of the BIOS posting using VBE and using ATOM (which we use in the kernel), Jerome ruled out different parts of the engines and we got reports of it working for some people when powered down for a long time or other randomness, and we were going back and forth with ideas on what might be going wrong, and had started thinking we should power off various parts of the hw before suspend, and the problem was due to inconsistent hw state on resume.

Peng happened to visit Brisbane for a Red Hat meeting this week and brought the T60, and this morning I swapped my laptop for his. First of all I tried to play by plugging in a VGA monitor, this produced another bug where the LVDS would die when starting KMS, so I fixed that quickly first. So the VGA screen was also corrupted, and VRAM wasn't enabled at all. Next I tested vbetool posting worked, also suspend/resume to corrupt, unload radeon, vbetool post and load radeon worked. Then I started testing with Jerome's userspace atom init tool, doing a s/r, unload radeon, atom post, load radeon also worked fine. This is where it started to make no sense, since Jeromes tool was doing the exact same thing as the kernel parser. I started by blaming the atom delay code in the kernel but that proved a dead end after an hour or two. Next thing I enabled the kernel atom debugging and all of a sudden it resumed fine. So it was a timing issue somewhere in atom parser running the init code.

So enabling debugging put enough of a delay between operations that something that wasn't working before now succeeds. I started bisecting the debug messages, I removed half the debugging at a time until after about 3 hrs I got it down to one printk happening between two atombios commands. The surrounding code was reading and writing one of the memory controller setup registers on the GPU, so it pointed to some register write not getting fully into the hw before we read it back and write it again later. I changed the atom code to do a read back before writing regs for certain operations and viola all resumed fine.

So this took the best part of 8 hours, I reckon if I'd been doing the same over irc with Peng it would have taken at least a month of back and forth on irc to figure it out. Having the hardware locally even for a day made it possible to track it down and figure it out so much quicker and efficently. So the bad news for anyone with bugs we can't reproduce locally is that we generally will fix any bugs we can locally first just from a efficiency point of view, since we can fix them so much quicker and faster.

I've pushed out the drm-next branch to

http://git.kernel.org/?p=linux/kernel/git/airlied/drm-2.6.git;a=shortlog;h=drm-next

A few highlights include:
firmware loading for r128/mga/radeon + radeon kms drivers. (for all the Debian fans).
Intel -next tree pulled in.
radeon r100/r200 kms command stream checking
radeon kms tv-out support
merged fb handling for all kms drivers
DMT timings + better HDMI EDID decoding
R600 3D support
R600 KMS support (under staging drivers along with radeon KMS).

along with numerous other patch.

The r600 3D support enables r600 3D without KMS.

The r600/r700 KMS support in that tree enables full 2D/3D/Xv accel + DRI2,
you need a master libdrm, master ati DDX and master mesa. Jerome and myself
at Red Hat have been working on this tag team along with Alex at AMD. It may
not be 100% stable yet and I'm sure we can make things a lot faster, but
the basics all work for me here. So get testing and reporting please.

Fedora rawhide users, just wait for a day and it should all be available.

I've done some boot testing on my rs780, rv635, rv610 and rv730 so far
and all seem to at least start and run my X session.

So X.org has long contain a layer known as RAC or Resource Access Control. This layers job was to arbitrate between multiple X.org drivers running inside a single server for access to shared resources. A resource is defined as an I/O region, MMIO region, FB region, VGA I/O, VGA FB apertures. Now back before we had operating systems that actually set hardware up properly, X used to do a lot of work. For example it pretty much tried to fully control the PCI bus, and was able to allocate resources to GPUs when the BIOS or OS didn't do so. It was able to dynamically switch on/off any resources a GPU could care to want.

This system was engineered with some really scary scenarios in mind, it could deal with sharing PCI decode space between two devices (something PCI can't really do). It could deal with setting up GPUs the OS hadn't assigned PCI resources to. It could share the limited x86 I/O ports between multiple cards. It could also do all the simple scenarios well.

Now back then maybe some of these things were necessary however nowadays really only two shared resources are left that we want
X drivers to care about. We hope the OS can assign all the PCI I/O space and PCI memory space so that X doesn't need to try and multiplex it.

Also this system has some major disadvantages, two X servers couldn't be used as the locking was all done inside one single server, so a second server would just race with the first. Graphics cards could never reliably use interrupts, since if you disable a cards MMIO region and it receives an interrupt, generally bad things will happen.

So fast forward to 2009, where we have pretty much OSes that do things correctly and we have only two shared resources we need to care about VGA i/o and VGA memory (0xa0000). Now the way VGA works on modern PCI systems, is there is a bit in each bridge which denotes whether that bridge routes the VGA decodes to its bus. On that bus there is allowed to be one device decoding VGA resources at any one time. The bridge forwarding is controlled by a bit in the bridge PCI config, the VGA decoding is controlled by the PCI COMMAND_IO and COMMAND_MEM bits. These bits however disable *all* IO and MEM accesses to the PCI device. Now most modern GPUs can disable the VGA decoding on its own, however this is GPU specific and there is no generic PCI config space bit to do this (FAILLL!!!!).

So the OS kernel needs to provide some sort of arbitration logic between processes wanting to access the VGA decodes. Reasons for wanting the VGA bits enabled for a GPU are generally doing any int10 call (like card posting or VBE mode setting), or doing console save/restore when X is starting/stopping. Some GPUs use the VGA i/os still for modesetting but these are generally very old cards. Some nasty ACPI/BIOS implementation also seem to like having VGA routed correctly for suspend/resume.

So Ben Herrenschmidt, Tiago Vignatti and myself brought the VGA arbiter into the world, the code has taken a couple of years to get the attention it needs to go upstream. The vga arbiter exposes a /dev/vga_arbiter node, that X can open and use to route the VGA io/mem decodes between any GPUs in the system. So how does this deal with the issues RAC had:

1. Multiple processes:
Since this is in the kernel, the lock is now available across processes so multiple X servers can in fact fight over the VGA.

2. Interrupt handling:
Since most of the IRQ handling for GPU devices is done in the DRM layer, the arbiter allows a callback to be registered for devices which cannot disable VGA decodes on-card, so that they can disable their irqs when their memory/io decodes are turned off.

3. Cards with local ability to disable VGA mem/io
Cards can register that they can disable VGA mem/io locally. This means these cards are removed from VGA arbitration completely. Generally for this to occur a KMS or other proper kernel driver is preferred. Since if X decides a card doesn't need arbitration another process might decide it does, so having a master kernel driver that owns the hw makes this a lot easier.

4. Crazy ACPI cards.
The arbiter does nothing if only a single card is registered with it. So it will leave the VGA decodes enabled on all single GPU systems. If a GPU was hotplugged, the driver will get a callback saying it needs to disable VGA decoding on itself. Generally we won't see the crazy ACPI issue on multi-gpu systems.

However there is one issue we are seeing with the VGA arbiter that RAC never really addressed either, interaction with kernel DRI. So I suspect we will disable DRI access for any GPU that requires VGA routing. The issues I've seen so far are all contentions issues with the arbiter lock. They can probably be worked around on a case by case basis, but really just get KMS already.

So we are hoping to upstream the kernel code for 2.6.32 and push the libpciaccess and X.org server patches to their master repos
around the time the patch is in the pci queue.

I'm sure I've missed something as I've been working on this for 4 days now, though I lost a day going in a redesign loop :-)

So we've had an -ati DDX with KMS support in a branch for quite a while, but it was starting to grow into a very big mess, and some of the hacks in it were quite unmaintainable.

So started cleaning it up and pushing the bits to master.

Step 1 was adding macros in all the places in the accel code to abstract away the different command submission methods, and
add some ifs to do KMS specific things. Once this was done in theory the accel code wouldn't functionally regress and wouldn't require any more changes.

Step 2 was bringing over the kms and DRI2 support files from the branch.

Step 3 was making the decision between if (kms) if (!kms) blocks all over radeon_driver.c in all the various functions or having a nearly completely separate KMS/DRI2 driver file. The original code has the if approach and it was an unmaintainable nightmare, so I opted for approach 2 and it definitely is the best. At driver probe time in radeon_probe.c, I now do the KMS check if the pciaccess probe is called (kms without pciaccess is probably not going to matter). If I get KMS supported the driver picks a nearly completely different set of functions for PreInit/ScreenInit etc. I now have a separate radeon_kms.c file which has all the DDX interface in it. Of course if we have any changes they may need to be done in two places, but its a lot cleaner than it was in the other codebase.

I also ported the KMS code to use the libdrm_radeon buffer management code which is shared with mesa, instead of the DDX having its own buffer manager code base. This code is well tested via mesa, however it hasn't got all the features/optimisations that I've added to the DDX bufmgr over the last while.

So what's left:
Missing optimisation from old buffer manager:
1. buffer in VRAM? has the buffer ever explicitly been validated to VRAM, this allow for an optimisation on download from screen.
2. Download from screen, with driver pixmaps we don't know if the buffer is in VRAM or GTT, with (1) we can either blit if in VRAM, or just memcpy if in GART.
3. force a buffer to validate in GTT or force a buffer to stay in fast CPU access space - this was really useful some sw fallbacks where a buffer would end up in VRAM and then get used by the CPU from there. It probably only really takes the place of fixing EXA properly so the pixmap scoring is separate from the offscreen memory, and having a XA that works with driver pixmaps a lot better.
4. bugs and crashes I appear to be hitting a realloc crash at some point where glibc reenters itself and fails.

So there is a kernel conference on in Brisbane next month, being run by Sun.

Now when this was announced initially it was proposed as an all kernel hackery type get together for folks in the region, not matter which kernel they cared about.

(I did propose to talk about kernel graphics at this and got refused - so maybe I'm just being bitchy, however this is my blog).

20 speakers break down as follow:
12 Sun
1 Intel - Sun manager
1 RH
1 OpenBSD
1 FreeBSD
4 misc.

2 of the misc talks are in some way OSOL related,

the OpenBSD talk is about networking and pf, the RH talk is about security, FreeBSD about storage.

Now really if you aren't into OSOL or ZFS (3 slots OSOL FS related), why would you go. This conference is
local to me and I still couldn't justify paying the signup fee/taking the time to my manager at all. Now if
one of the main kernel and X.org hackers who lives in Brisbane can't be bothered to go, I do wonder
why anyone who isn't into OSOL kernels might be tempted.

There was talk of a .au unconference at one point, which maybe when the whole swine flu escapade is over might
actually be a useful meetup for the aussie open source community.

Okay radeon TTM/KMS has landed in Linus tree under staging.

To enable it you need to enable CONFIG_DRM_RADEON_KMS, which relies on CONFIG_STAGING being set.

please read the CONFIG_STAGING warnings, esp the
"Please note that these drivers are under heavy
development, may or may not work, and may contain userspace
interfaces that most likely will be changed in the near
future."

Now to get a userspace that can use this code you need to get

git://git.freedesktop.org/git/mesa/libdrm master branch
and build it with --enable-radeon-experimental-api and install that.

git://git.freedesktop.org/git/xorg/driver/xf86-video-ati kms-support branch
build that second

git://git.freedesktop.org/git/mesa/mesa.git master branch
build this with libdrm_radeon somewhere that pkgconfig can find it.

You should either have KMS + DRI2, or a pile of smoking trash.

Please report any mode type issues on #radeon or dri-devel mailing list.

If you can't compile or configure your system to use this please wait until you have a distro do it for you.

If you are using Fedora 11, grab the latest xf86-video-ati from koji and all you need is the new kernel bits.

Known issues:
My DDX reports something about DRM 2.0.0 and wanting 1.2.x or something like that, you messed up setting up the DDX or are
still using the system DDX.
Xv might be broken on resize (or normally)
r600/r700 doesn't work (no surprise its not ready yet)

So I merged the rewrite of the radeon/r200/r300 to the mesa master today.

On a non-KMS DRI1 system we are hoping these patches are mostly regression free in terms of lockups and possibly fix a fair few lockups thanks to Maciej and Jerome's work.

Of course on a KMS/DRI2 system, this work enables GL on r100-r500 families of radeon cards and we hope to add more features to the ones we gain. We should now at least have FBOs where we never did before (in DRI2/KMS mode).

R600 work is ongoing in a branch still so this announcement has no effect on r600 or above.

I've also sent Linus a drm pull request with all of Intels drm-intel-next branch + all the patches I had sitting in my queue.

Early next week I'll send him the KMS patches for radeon, which will be in the radeon driver but the enable switch will be hidden in staging, while we stabilise it, since its such a huge amount of code. So don't expect miracles out of it anytime soon.

On a PCI ID table comparison Intel KMS have 28 IDs to support, radeon KMS has ~350, granted this code has no memory manager for r600 and up yet (kms code sohuld all be there), so its probably close to 200 GPU variants (+/- 50 for ids not seen in the wild maybe).

So Gia, Isabel and I are off to Ireland for 3 weeks from the 7th -> 31st of May.

Radeon KMS is all I've been doing lately and my feeling is the F11 driver is in a fairly good state, a lot better than F10. 3D seems to be working okay, however we have some speed regressions that I will be attempting to track down in earnest on my return. AGP cards got hit a bit lately as a fix for a real bug slowed an optimisation I previously made down.

For mesa I'd really like to merge radeon-rewrite to master already, but I expect I should wait until I get back as I need to be more reactive to fixing regressions in it.

So I've been slowly getting radeon-rewrite into a useful state, I diverted myself on Friday evening to take a quick look at FBOs on KMS/DRI2. I'd already layed down most of the basis for FBO code so I thought it might not take too long to get working.

So today I merged the FBO code into the radeon-rewrite branch, its not perfect but its a good place to start, its only enabled for KMS/DRI2 setups of course.

So there are a few things left to get radeon-rewrite into master:

1. Fix hangs on r100/r200 that some people have reported. I'm not having a huge amount of success in reproducing this, but I'll try and give it a larger effort asap. Its hanging in the buffer manager aging code somewhere. It smells like a GPU hang but its hasn't actually hung, killing the app, and starting it again, keeps things going.

2. r200 depth buffer is broken.

3. DRI2 single buffered rendering isn't drawing to the window where it should.

4. Have to run piglit regression tests on r100/r200/r300/r400/r500, under current mesa master, radeon-rewrite, radeon-rewrite DRI2 with swtcl and hwtcl paths. So I think it works out around 30 tests cases to try, and then fixup the regressions!!

5. lots of other things I've forgotten.

Post merging, I need to add support for two things back:
1. Color buffer tiling under KMS/DRI2. Works fine under DRI1.
2. Texture tiling under *.

previous quote by radeonhd devloper: "And this is real cutting edge stuff, we're doing things no-one else is doing"

mailing list today:
"A question the for the radeonhd developers: Why are you not using the register access routines provided in compiler.h for the purpose? By not using those you break portability."

These two things may not be equal.

In order to further Red Hat's commitment to KMS on radeons, I've been adding support for a bufmgr to all 3 radeon 3D drivers.

The bufmgr essentially abstracts the lowlevel buffer management from the state machine of the driver, so we can implement a simple userspace buffer manager which operates on the current system, and a memory managed buffer manager which operates on the GEM kernel API we defined for radeon.

Jerome Glisse and Nicolai Haehnle wrote the initial r300 bufmgr, texturing and mipmap tree code. I decided that instead of trying to port it to r100/r200, I took the opportunity to merge as much of the common code in radeon/r200/r300 drivers.

So armed with piglit to fight regressions and a lack of sleep, I started merging.

So I've pushed the results to the radeon-rewrite branch of mesa, it works for me on all the radeons I've played with in legacy mode.

My future plans for the codebase are:
1. get libdrm_radeon on modesetting-gem autodetected
2. Implement radeon/r200 userspace clear code. This is in-kernel at the moment but kms needs it out.
3. Play with DRI2 - should already be supported on KMS/GEM stack. I suspect I need to fix some state handling for this.
4. make it go fast on kms - work with compiz and a few games for F11.

Oh I forgot to mention the best bit:
100 files changed, 11289 insertions(+), 15487 deletions(-)

4000-5000 less lines of code ftw. I have a few more license headers in new files :)

(meant to post this pre-baby)

When I originally started to work for Red Hat I relocated to Brisbane so I could be based in an office and for other personal reasons. I was the first kernel/X.org/OS engineer based in the Brisbane office and the rest of the X.org team was based in Westford, MA mostly. I didn't think at the time this would change much, I mainly wanted to have the office as an option to work in as I can get demotivated working from home, changing scenery helps.

When RH moved offices in Brisbane last year, a new lab was built in the new office, and I got assigned a nice desk + rack + remote power + remote KVM over IP. This has proven really useful for doing development on all my different machines.

So mid-last year RH desktop team got the opportunity to hire Peter Hutterer who was based in Adelaide at the time, and we persuaded him to relocate to Brisbane once he finished a few things in Adelaide. He finally relocated a couple of months ago and is hacking lots on getting X.org input to a better place.

Late last year we also had a replacement position open up in the X group, and after some searching, Ben Skeggs, one of the top two nouveau developers (nvidia reverse engineering project) was interviewed and accepted the position. Ben was based in Tasmania, and agreed to relocate to Brisbane and started this week in the office. Hopefully this will mean good things for Fedora + nouveau development.

Its nice to have a team in one place for a few reasons,
a) people to talk technical to in the same timezone!!
b) easier to keep a hw library in one place, since we mainly do hardware enablement on Intel/AMD/nvidia hw having access to as many cards/systems as possible in one place very winning. With having more people in the office we can work on getting more hw directly shipped here and build up a better development environment.
c) talking technical can involve drinking.
d) easier to attract more people to the sunniest place to work :)

update from between the baby naps :)

So back in 2002 while trekking in Nepal I got a dog bite from a possibly rabid dog, hilarity ensued (well actually scaryness), recently the story got picked up again by an Irish medical communications company, and has now appeared in the Irish Evening Herald and Longford Leader with at least one other paper possibly running an article.

http://www.herald.ie/national-news/city-news/warning-after---irish-trekker-in-rabies-ordeal-1587917.html
http://www.longfordleader.ie/news/Longford-man-relives-rabies-horror.4848488.jp

update: rapid->rabid :-)

Isabel was delivered this morning, at 11:54am, at 8lb13oz (4.07kg).

I'll be slow on anything non-baby related for a while :)

AMD today pushed the initial code to support acceleration on the r600/r700 range of GPUs.

This consists of r6xx-r7xx-support branches in the drm, radeonhd and a new r600_demo repo.

This code is really only for developers at this point but its great to see AMD finally get things lined up to allow this code to be released.

I've only been barely involved in the r600 code so far, I wrote the original drm over a few days and handed it over to AMD to continue on with, as I wanted to concentrate on the kms work. Hopefully I can get some time to look at it over the next while (yeah right, new baby still not here).

So I sent a drm pull request that includes the kernel modesetting core + intel i915 driver supporting it.

This is a major milestone for a project I started working on in a previous job, and I barely remember burning through the initial code for the initial prototype in a week of little sleep.

To enable the code you need to set the CONFIG_DRM_I915_KMS, this isn't enabled by default, as we don't have a userspace that supports it available yet for general consumption. If you enable kms now, you will more than likely get a broken X for your trouble as the kernel drivers aren't compatible with having userspace drivers trample the hardware.

So where is ATI at?

although we are shipping radeon code in F10, the code is based on the TTM memory manager, which isn't really in an upstreamable
state in its current form. Hopefully a newer TTM codebase might become available that can be used upstream. If that doesn't happen, we might rearchitect the core memory manager code of the radeon system now that we have the API mostly proven. So I'm not sure when we will upstream it, it all depends on how much time I can work on it.

Baby status: due today, no sign yet, will severely impact amount of time I spend on this stuff :)

So we merged r500/r600 via atombios TV-out support, but we haven't enabled it by default.

You need to add

Option "ATOMTvOut" "true" to xorg.conf to enable it.

We'll hopefully get to fix up some of the remaining corner cases and issues or enable it at
least on a card by card basis.

Okay I've just spent a few days doing output enablement on -ati, since I got an rv730. I decided to try and get tv-out up and going again.

I've pushed a branch to the main xf86-video-ati repo called "atom-tvout"

Please test this on any r500 -> rv7xx cards if you are interested in TV-out support on this range in open source drivers.

It mostly uses atombios to set the cards up, with one exception for the scaler on the R500 cards which I'm discussing with ATI.

I've also added and tested DCE3.2 output support to master in the past day or two, I just need to add the official PCI IDs for all the rv7xx variants.