forums.ps2dev.org

[ps2devman]

Since a parameter of hypervisor's function used to request a bitblt from GPU looks like the one we would use ourself in a Nvidia miniport push buffer (alias FIFO) if we had direct access to RSX... I think there is really a push buffer/FIFO used by hypervisor. A circular main memory area, where a few dwords are written (1 command + few parameters).
There is certainly a variable (tail) that points at the end of these list.
Area may be very short if hypervisor knows linux runs and only bitblt commands will be issued.

When this variable (tail) changes, that means new commands just got written. A direct write to GPU mapped registers is probably made at the same time to warn GPU it should start reading them and where it should stop.

If we manage to survey that variable (tail) and if we manage to put our own commands and parameters in the FIFO/push buffer, immediately after this change, it's our commands that will be executed (if it's too fast, we can leave alone first commands and target next ones).

I'm not naive I doubt software tricks can bypass memory addresses range verification when CPU attempts a write/read somewhere in memory.
But who knows, with marcus' other os demo...

I'm rather thinking about a ram attack. That is hardware mods allowing to spy/change main ram.

I also heard a rumour about a usb chipset in PS3 hardware, for which we have all direct access, and that may act as a dma controller, thus allowing to read/write anywhere.

What's your opinion about this idea?

[ldesnogu]

EDIT : Sorry I read your post too fast, I thought you wanted to do it in a pure software way :)

[StrontiumDog]

[ps2devman]

That's good and bad news.

Bad news:
That means the variable (tail) is probably in a non dma-able area.

Good news:
Still hope for the FIFO/push buffer itself!
Instead of surveying the variable (tail), we can survey the first command and anticipate what command it will be and the number of parameters involved.

First step would be to explore all dma-able areas to check for remaining traces of one of the parameters of the bitblt : "(xres << 16) | yres"
Usually, what is written in FIFO/push buffer is not erased after being
read by GPU. It will just be overwritten when all buffer is used (a jump
to head will occur) since it's a circular buffer. It shouldn't be a too common
value in main ram... Also it should be repeated in memory at exact
same "distance" (a few dwords).

Remember, ralferoo, pointed at this kernel call (let's say HV API) :

[ps2devman]

Maybe search for
L1GPU_FB_BLIT_WAIT_FOR_COMPLETION | (xres << 16) | yres
as well... or only look for relevant lower bits.

[mc]

[rz950]

The ps3 activates the hypervisor when it boots into the other os? This is the only thing I don't understand, what actually tells the hypervisor to activate, isn't it not "on" when running a game? I understand the other os or "boot loader" doesnt actually call the hypervisor, if it did we could have had a modded boot loader but I am guessing the ps3 firmware tells the hypersior to go into block rsx mode(yeah I come up with some good names xD) when the other os is running. If I am wrong please feel free to correct me, I would like to know.

Friend just got a ps3 yesturday (version 1.51) and I have been reading up on it a bit more then before.

[mc]

When booting, the system checks a bit in the flash which says if it should boot GameOS or "Other OS". If it's set to "Other OS", it starts the hypervisor which will in turn start otheros.bld. So when otheros.bld starts, the hypervisor is already active.

Whether GameOS runs without a hypervisor, or just with a different hypervisor, is unknown at this point.
_________________
Flying at a high speed
Having the courage
Getting over crisis
I rescue the people

[rz950]

ah, I see now thanks for explaining. Other OS is just responsible of launching linux basically then even through modded ones can do so many other things. I know it wont do much but a modded other os that would call the hypervisor, it should report it as active, just a thought?

Well, thanks again for explaining so quick and nice work on the modded other os mc :)

Edited: This is a bit off-topic but I am wondering if I should update or not, I told my friend not too and he knows why and understands that he shouldn't but just want to know what anyone here thinks.

[mc]

The "Other OS" mechanism is intended to support launching any OS under
virtualization, but since this virtualization is not fully transparent, the
OS needs to be adapted to run in this environment, and currently Linux
is the only OS for which such an adaption has been done (since that's
the OS which Sony themselves chose to do it for). But yes, "otheros.bld"
is just intended as a launcher ("bld" = Bootloader) which loads the
actual OS from the harddrive. But any handover between bld and OS
is up to them, the hypervisor will not care but present them with the
exact same virtualized environment.

[IronPeter]

The new sources for ps3fb.c http://www.everfall.com/paste/id.php?kjjqgvpbncu0 contain GPU_CMD_BUF_SIZE macro.

It's clear that the memory region [ ps3fb_videomemory.address + ps3fb_videomemory.size - GPU_CMD_BUF_SIZE, ps3fb_videomemory.address + ps3fb_videomemory.size ) is FIFO buffer.

This region dump: http://www.everfall.com/paste/id.php?ieto25cyoy0g

PS. excuse my English.

[IronPeter]

Yes, it is realy push buffer.

The good idea is to test all undocumented entries from http://wiki.ps2dev.org/ps3:hypervisor:lv1_gpu_context_attribute , get push buffer dump for the each one and compare with nouveau NV40 push buffer database.

[IronPeter]

fresh news about fifo control regs.

There is lpar_dma_control area. It can be iomapped. This area is filled with zeroes, only 3 dwords are non-zeroes. dwords with indices 0x10, 0x11, 0x15.

The value of dword 0x11 just after hypervisor blit is 0xe1f0000, few ms later is 0xe1f0048 and 0xe1f00b8 finally.

compare with
http://nouveau.cvs.sourceforge.net/nouveau/nv40_demo/main.c?revision=1.5&view=markup

[IronPeter]

I've parsed blit push buffer into dma packets ( size of packet, subchannel id, tag ):

http://www.everfall.com/paste/id.php?z51ttk37j71s

My screen resolution is 1280 x 1024 ( 0x500 x 0x400 ).
Funny, hypervisor made 2 blits ( one 1024 x 1024 and one 256 x 1024 ), so parameters of hypervisor call are not "preformated for th GPU".

You can refer this document relating push buffer tags:

http://gitweb.freedesktop.org/?p=nouveau/xf86-video-nouveau.git;a=blob;h=ed7fe9a628e09227531ac701868264cafded4638;hb=62294d9be8ec6f1159e3c2ad273649f937b7aa7e;f=src/nouveau_reg.h

[jimparis]

Nice work, this is good stuff.

[Warren]

[ps2devman]

Congratulations IronPeter! Smells good, very good!

[IronPeter]

I was able to run blit push buffer from the user land using fifo control regs.

There was some kind of protection. Very weak protection.

It works unstable for now, but it does work. Probably, it's possible to write some kind of 2D support ( stretched blits, color fills, etc ).

The main question is about 3D support. We need so-called "context objects" to be properly initalized. Probably, hypervisor does this work for us. All we need are handles ( and lpar_dma_reports contains something that looks like this handles ). To initialize these objects "by hands" we need to access to very special RSX registers, so called RAMIN area.

PS. I investigate RSX with only open-source information. I have no signed NDA with Sony or NVidia.

[ps2devman]

Ok, things are getting serious now... Hehe.
What is your firmware version?

We need to be careful and detect when this new trick will become unusable in future firmware versions. Someone with infectus and the ability to swap firmware would be the best person for detecting such infamous change...

Thanks for your great work, IronPeter!

PS: If you could publish a minimal source, even unstable, that can be used to test this new trick for each firmware version that would be great! Thanks!

[IronPeter]

The firmware version is 1.8

The trick is very simple, I can describe it without posting the full sources.

Look at the push buffer dump:

http://www.everfall.com/paste/id.php?uxdlpwlbfpo9

It is the image of push buffer after hypervisor blit. The end of buffer is at 0xb8.

The last packet sends zero to the subchannel zero with tag 0x110. Replace the tag with NOP ( 0x100 ) while buffer is kicked by the hypervisor and is executing with the RSX.

Fill push buffer with N + 1 copies of the first 0xb8 bytes.

In cycle for( int i = 0; i < N; ++i ) modify client screen buffer in some way ( fill with random numbers ), kick push buffer via writing ( (uint32_t *)ioremap( lpar_dma_control, 1024) )[ 0x10 ] = 0xe1f0000 + 0xb8 * ( i + 1 ); sleep ( 1 ); The client screen in the xdr memory will blits in the videomemory.

If nobody is unable to repeat these steps I post the full sources.

Edited: bugfix

[mc]

Very nice work, IronPeter!

I find it intriguing that not only are you allowed to define the FIFO region
in user memory, but that you are also allowed to map the control area.
This suggests to me that Sony actually intended to support HW 3D under
Linux, as they could just as easily have made the control area accessible
only to the hypervisor.
_________________
Flying at a high speed
Having the courage
Getting over crisis
I rescue the people

[IronPeter]

[mc]

Well, yeah, but that only makes it more likely that it is intentional
that this particular part can be mapped. It would make sense
to only expose the parts needed for performance (= FIFO interface)
and handle access to other parts through the HV.

Thanks for the link, BTW.
_________________
Flying at a high speed
Having the courage
Getting over crisis
I rescue the people

[IronPeter]

Yes, mc, you are right.

The only thing that needs direct FIFO access is real time 3D acceleration. 2D part does not need that. So Sony probably wants to expose 3D driver.

[IronPeter]

It's push buffer dump after hypervisor FB_SETUP. Programmers from Sony decided not to clean up push buffer after context objects set up.

http://www.everfall.com/paste/id.php?ew29498z816w

Enjoi it.

[Glaurung]

Hi all,

I could reproduce the FIFO hack described by IronPeter, using firmware v1.93. I had to setup a large blit (which is decomposed into many 1024x1024 blits by the HV) in order to have time to tweak the FIFO area. Instead of patching with a NOP (which works), I chose to set the FIFO end pointer two operations back. I also had to remove the L1GPU_FB_BLIT_WAIT_FOR_COMPLETION flag from the call to lv1_gpu_context_attribute(), so to sum it up:

[Seather]

Would this hypervisor be anything close to the Xen from Sun?

[jimparis]

1) Xen isn't from Sun, it's from XenSource.
2) This thread has nothing to do with that, please keep it on-topic.
3) It is the same in theory but there are no similarities that are useful to us.

[IronPeter]

Glaurung, nice.

I want to say few words about testing methodology. Segmentation fault in the kernel mode is not very good. I made few things to make life easier:

1.) I extended ps3fb's memory mapping for the last 65536 bytes. open( "/dev/fb0" ), mmap it, use push buffer in the client mode.

2.) I extended ps3fb's ioctl for the fifo control registers read/write.

3.) lv1_gpu_context_intr ( interruption ) is very useful for you. Extend ioctl with this function. And it is better to disable (also by ioctl) regular kernel's lv1_gpu_context_intr in the driver.

[IronPeter]

Glaurung, would you like to test blit from videomemory to the system area? I have no access for ps3 for a short time.

http://wiki.ps2dev.org/ps3:hypervisor:lv1_gpu_memory_allocate returns only 252 megs, the top of the videomemory seems to contatin RAMIN area ( the global GPU control area ).

This area must be protected from read/write, but probably...