[porting suggestion] Quake 3 for PSP-Slim (with 64Mb RAM)?

Discuss the development of new homebrew software, tools and libraries.

Moderators: cheriff, TyRaNiD

Be3f
Posts: 59
Joined: Thu Mar 15, 2007 9:28 pm

[porting suggestion] Quake 3 for PSP-Slim (with 64Mb RAM)?

Post by Be3f »

Finally, PSP-Slim (PSP-200X) is avaluable and it runs homebrew - my congratulations to Team C+D (for the Magic Battery) and to Teem m33 (for the Custom Firmware v3.60)! As you know, slim has 2X more main RAM, than "fat" (PSP-100X) - 64 Megs!

Low RAM (32Megs of system memory with only 24Megs avaluable for programms) was the bottleneck of the PSP-100X, that left no chance to port such huge games as Quake 3 to PSP, but now i can't find any reason, that makes Quake 3 port for the PSP-Slim (only) impossible!
McZonk's Quake-2 PSP port seems to run full singleplayer game on Slim too... :rolleyes:

So, does Slim has 56 free megs of the user memory (64 system - 8 Mb kernel memory)?

I've posted this once at the qj forums, just copy-paste here:

I already have a portable Quake 3 on my PocketPC (Dell Axim x51v) - Q3 was ported to WinCE in 2005 by NoctenWare (port is called Q3CE), it features both software and hw rendering (uses OpenGL-ES) - runs at ~15 average FPS on my x51v with max graphics (hardware rendering)... and it's package contains very interisting thing for us - pakconvert tool whitch reduces content in Quake3 *.PAK files: downsizes textures & gfx(2x), level & model geometry, converts sounds and videos to save RAM & increase perfomance - after running pakconvert scripts, main PAK0.PAK downsizes from 450 to 117 Megs and Q3CE runs on PPCs with only 36Mb free RAM! Also, Q3CE partitions memory automaticly, instead of original Q3 (and com_hunkMegs & com_zoneMegs commands do not work in it...).

This is how Q3CE accelerated runs on my PPC:

Image
(Q3DM3 map, 2 bots deathmatch, max graphics (but downsized data), 19 FPS at the moment)

Unfortunetly, NoctemWare website (noctemware.com) is currently dead, but you can find some info about Q3CE on it's offical forum:

http://www.pdaground.com/forum/viewforum.php?f=14

=Update=

Yesterday I've spoken to the Q3CE developer and asked him for the source code. Here are the latest q3ce v1.1b sources:
http://www.filecrunch.com/file/~euhoj4
Last edited by Be3f on Sun Sep 16, 2007 3:55 am, edited 2 times in total.
00000110 00000110 00000110
User avatar
Wally
Posts: 663
Joined: Mon Sep 26, 2005 11:25 am

Post by Wally »

ooohhhh this could be interesting :)

Slim only homebrew :D
Murdock
Posts: 110
Joined: Sun May 21, 2006 2:14 am

Post by Murdock »

I am no coder, but from what you wrote above, isn't it possible to downsize the game (i.e. gfx etc.) any further to make Q3A run on the old PSP, too? I mean of course, it'd look not that nice, but it "should" run.

Or is this 36 MB RAM the least?
W00fer
Posts: 40
Joined: Fri Apr 13, 2007 8:02 am

Post by W00fer »

Ehm, what about the Dell Axim processor ?!

The psp still has only 333mhz not 624 mhz like the Dell !

Thats almost twice as much as the psp. It would be fun though if it could be done !
Be3f
Posts: 59
Joined: Thu Mar 15, 2007 9:28 pm

Post by Be3f »

Murdock wrote:I am no coder, but from what you wrote above, isn't it possible to downsize the game (i.e. gfx etc.) any further to make Q3A run on the old PSP, too? I mean of course, it'd look not that nice, but it "should" run.

Or is this 36 MB RAM the least?
I've posted this onсe on qj:

About a year ago, PeterM said:
PeterM(dcemu) wrote: id's way of coding is to have numerous large global variables. Last time I looked, Q3 had over 20MBs of them, which is unfortunately the total amount of free RAM available for use on the PSP...
Unfortunately, Quake 3's global variables use up the rest...
Many coders tried to port Q3 engine to the PSP-100X, but they were unlucky and gave up...
W00fer wrote:Ehm, what about the Dell Axim processor ?!

The psp still has only 333mhz not 624 mhz like the Dell !

Thats almost twice as much as the psp. It would be fun though if it could be done !
But PSP's GU is much faster, then i2700g in Dell ;)

Also, I've tested some original Quake 3 Arena maps with downsized (2x) textures on the LTE v2.2 engine (it uses PSP-GL for rendering - this is slower then PSP-GU code!): small maps run ~perfect at ~30-60 FPS (PSP was overclocked to 333/166MHz, Vsync turned on...) with mipmapping & lightmaps (but with some, not too heavy rendering artifacts).
Here's a photo of the Q3DM3 map:

Image

...but large maps refuse to start - PSP freezes, while loading... out of RAM or VRAM? Maybe LTE loads all textures & mips into VRAM and freezes, when VRAM gets full? I don't know exactly, but I think, that framebuffer is set to 16 bit - if it's so, we have about 1.3MB of free VRAM for textures, right?

PSMonkeys Iris engine v0.15 also supports Q3BSP and it loads all original Q3A maps, but without textures. So, VRAM must be the main limit, but it is possible to make the engine put into main RAM all textures, that do not fit in VRAM.
Also, LTE precashes everithing while loading - level (and other...) data streaming may save much memory...
Last edited by Be3f on Thu Sep 13, 2007 3:44 am, edited 1 time in total.
00000110 00000110 00000110
User avatar
Raphael
Posts: 646
Joined: Tue Jan 17, 2006 4:54 pm
Location: Germany
Contact:

Post by Raphael »

W00fer wrote:Ehm, what about the Dell Axim processor ?!

The psp still has only 333mhz not 624 mhz like the Dell !

Thats almost twice as much as the psp. It would be fun though if it could be done !
Ugh... I don't understand why people don't get that you can't just compare clock speeds. You are comparing apples and eggs here. The Dell has a ARM cpu (XScale), while the PSP has a MIPS. Though both are RISC, they are different in design and instructions.
Apart from that, the PSP has much more additional hardware, like the GU which easily outplays the Intel 2700g, a second CPU (ME) plus a VFPU for doing all the vector and matrix maths.
...but large maps refuse to start - PSP freezes, while loading... out of RAM or VRAM? Maybe LTE loads all textures & mips into VRAM and freezes, when VRAM gets full?
I doubt the VRAM is the problem, as I'm pretty sure the LTE guys were at least clever enough to swap textures to sysram if they don't fit into VRAM anymore. The real problem is the sysram that runs out. Especially mipmaps take quite some additional space (though they improve rendering speed), plus all the data from the BSP tree and the map mesh. Those easily count up to more than you imagine.
Also, LTE precashes everithing while loading - level (and other...) data streaming may save much memory...
Except that Q3 maps weren't designed for streaming, so that would involve a lot of coding to get done.
<Don't push the river, it flows.>
http://wordpress.fx-world.org - my devblog
http://wiki.fx-world.org - VFPU documentation wiki

Alexander Berl
Be3f
Posts: 59
Joined: Thu Mar 15, 2007 9:28 pm

Post by Be3f »

So, it's near impossible to dev Q3 port for the "fat" PSP...

WTH is going on with memory partitions in PSP-Slim RAM?
kururin wrote:partition 2: (user)

topaddr = 0x08800000, size = 0x1800000 (24 MB), attr = 0x0F
-this is old one, same with PSP-100X, used for homebrew

and 2 new kernel partitions implemented for UMD-cache in Slim:
kururin wrote:Partition 8: (new in the psp slim!)

topaddr = 0x8A000000, size = 0x1C00000 (28 MB), attr = 0x0C

Partition 10: (new in psp slim!)

topaddr = 0x8BC00000, size = 0x400000 (4 MB), attr = 0x0C

So it seems at first the new partitions are for kernel mode use, but i guess they can be unlocked for user mode use with the SetDdrMemoryProtection function :)
If only these 3 partitions could be merged to have 56 free megs of RAM for homebrews...
00000110 00000110 00000110
J.F.
Posts: 2906
Joined: Sun Feb 22, 2004 11:41 am

Post by J.F. »

I'm watching this myself... I also tried porting Q3 (ioquake3 to be exact), and while it would compile and run, it would just start initializing the subsystems, and then error out as it ran out of memory. While it's possible you could make something play Q3 levels using another heavily modified engine, you won't get Q3A itself going on the old PSP. Too little memory. The new PSP should have enough, and is plenty powerful enough for Q3. This issue of memory partitions needs to be looked into first.
Vincent_M
Posts: 73
Joined: Tue Apr 03, 2007 4:16 am

Post by Vincent_M »

Wow, I hadn't realized that this already happened. I've been a stranger to the homebrew scene for like two weeks now. That's cool that the PSP Slim has double the system ram. At first, I thought it was only going to have more flash memory, but system ram is better by far... I've seen that their's compatibility issues with PSP homebrew running on the PSP Slim. What is that exactly? Does it have anything to do with the firmware or the hardware?
Be3f
Posts: 59
Joined: Thu Mar 15, 2007 9:28 pm

Post by Be3f »

J.F. wrote:it would just start initializing the subsystems, and then error out as it ran out of memory. While it's possible you could make something play Q3 levels using another heavily modified engine, you won't get Q3A itself going on the old PSP. Too little memory.
Oh, this words are the absolutely truth! There was such thread: http://forums.ps2dev.org/viewtopic.php?t=8465 :)

Do you have any plans to port Q3 to the Slim, when patritions problem is over? ;)
Vincent_M wrote:I've seen that their's compatibility issues with PSP homebrew running on the PSP Slim. What is that exactly? Does it have anything to do with the firmware or the hardware?
Unfortunetly, 3.60-m33 custom firmware hasn't got the 1.50 FW part (like all previous Custom FWs for PSP-100X had to run homebrews in the 1.50 kernel mode) - only 3.60 part, so most of the homebrews, compiled for 1.50 currently don't work on Slim :( Team m33 says, they tried to implement 1.50 part into 3.60 m33, but got lots of errors (no display and other...). Let us hope, that one day 1.50 part will appear in custom FWs for Slim... And also someone will find a way to use 56 Megs of free RAM for 3.XX homebrews ;)
00000110 00000110 00000110
Be3f
Posts: 59
Joined: Thu Mar 15, 2007 9:28 pm

Q3CE source code

Post by Be3f »

Yesterday I've spoken to the Q3CE developer and asked him for the source code. Here are the latest q3ce v1.1b sources:
http://files.filefront.com/q3ce+11b+src ... einfo.html
00000110 00000110 00000110
Be3f
Posts: 59
Joined: Thu Mar 15, 2007 9:28 pm

Re: Q3CE source code

Post by Be3f »

sorry, the q3ce-src archive in the previous post is corrupt :( here is the fixed link:
http://www.filecrunch.com/file/~euhoj4
00000110 00000110 00000110
PeterM
Posts: 125
Joined: Sat Dec 31, 2005 7:25 pm
Location: Edinburgh, UK
Contact:

Post by PeterM »

Thanks - good to have this file around now that Noctemware vanished. :-)
http://aaiiee.wordpress.com/

I can no longer do any homebrew PSP development nor discuss PSP specific topics.
J.F.
Posts: 2906
Joined: Sun Feb 22, 2004 11:41 am

Post by J.F. »

PeterM wrote:Thanks - good to have this file around now that Noctemware vanished. :-)
Actually, I already had it. You can find it on the net with about 10 to 15 minutes of googling. It wasn't "lost", just misplaced. :)
Be3f
Posts: 59
Joined: Thu Mar 15, 2007 9:28 pm

Post by Be3f »

Finally, custom FW 3.71 is out and RAM partitions problem is over:
Dark_AleX (3.71-m33 readme) wrote:- Psp Slim: umdcache was allocating memory even when homebrew was launched, wasting
memory that homebrew programs may want to use. Now umdcache module is stopped before
it can allocate any memory, only in the case homebrew is launched.

Also, memory is unprotected for user memory usage by M33 core (only when homebrew is launched).
Developer, for a sample of how to use the extra memory, see the extra ram sample of the M33 sdk.

Included is a sample that uses the 32 MB of extra ram of the psp slim.
00000110 00000110 00000110
SamuraiX
Posts: 76
Joined: Tue Jan 31, 2006 6:28 am
Location: USA
Contact:

Post by SamuraiX »

It is great news to have the additional 32 MBytes of ram available. But now it requires some sort of memory manager to fully utilize the extra ram. Would it be possible to integrate the additional memory when using malloc() and not require us to generate our own memory management schemes?
Be3f
Posts: 59
Joined: Thu Mar 15, 2007 9:28 pm

Post by Be3f »

SamuraiX wrote:It is great news to have the additional 32 MBytes of ram available. But now it requires some sort of memory manager to fully utilize the extra ram. Would it be possible to integrate the additional memory when using malloc() and not require us to generate our own memory management schemes?
well...
moonlight wrote:Techincal stuff, the memory protection of te extra slim memory is in these hardware registers: 0xbc000040-0xbc00007F.

Set all to 0xFF, and you have access to it in user mode (note: sceKernelSetDdrMemoryProtection wouldn't do it)., but anyways is done automatically by m33 when launching homebrew, so no need to care about that.
Visit this thread for details and view include dir in (a sample) the 3.71-m33 SDK ;)
00000110 00000110 00000110
SamuraiX
Posts: 76
Joined: Tue Jan 31, 2006 6:28 am
Location: USA
Contact:

Post by SamuraiX »

Using:

Code: Select all

u8 *buf = &#40;u8 *&#41;0x0a000000;
is not the same as doing buf = malloc((32*1024*1024) + (20*1024*1024)); and having it return a buf with a size of 52 Mbytes.



Otherwise we are going to have to do some serious memory management not that I'm against challenges. ;p
J.F.
Posts: 2906
Joined: Sun Feb 22, 2004 11:41 am

Post by J.F. »

SamuraiX wrote:Using:

Code: Select all

u8 *buf = &#40;u8 *&#41;0x0a000000;
is not the same as doing buf = malloc((32*1024*1024) + (20*1024*1024)); and having it return a buf with a size of 52 Mbytes.



Otherwise we are going to have to do some serious memory management not that I'm against challenges. ;p
You want an adaptation of the valloc code Raphael did.
TyRaNiD
Posts: 907
Joined: Sun Jan 18, 2004 12:23 am

Post by TyRaNiD »

The trouble is the malloc code assumes a contiguous block of memory to work on as its heap. As the PSP still keeps its stacks at around 09XXXXXX there is no contiguous run up to the new memory for malloc to work.

What would have beem nice would be if partition 2 had actually been extended in sysmem to encompass the entire memory region then the kernel would have allocated stacks at top end of that region giving you a nice 50+ meg contiguous block which you would have been able to allocate quite easily :P

Of course it still might be possible to do with a new operating mode (i.e. have a compatibility mode where things are the same as before but with the extra ram maybe, and a super mode with one large memory space). Still probably too late as now people will write using absolute pointers :)
J.F.
Posts: 2906
Joined: Sun Feb 22, 2004 11:41 am

Post by J.F. »

I think the best thing to do would be to write your own malloc that used regular malloc until it returned 0, then pass on to new routine based on valloc but with the new memory as the base. It wouldn't be that hard. You wouldn't get a block larger than ~32MB, but you'd get all 56MB in chunks... which is how most apps use memory anyway.
PeterM
Posts: 125
Joined: Sat Dec 31, 2005 7:25 pm
Location: Edinburgh, UK
Contact:

Post by PeterM »

Indeed, and using something like Doug Lea's malloc to handle the hard stuff would help you out considerably.
http://aaiiee.wordpress.com/

I can no longer do any homebrew PSP development nor discuss PSP specific topics.
J.F.
Posts: 2906
Joined: Sun Feb 22, 2004 11:41 am

Post by J.F. »

PeterM wrote:Indeed, and using something like Doug Lea's malloc to handle the hard stuff would help you out considerably.
Or maybe ptmalloc. http://www.malloc.de/en/
TyRaNiD
Posts: 907
Joined: Sun Jan 18, 2004 12:23 am

Post by TyRaNiD »

Perhaps just dont use malloc in the first place ? :)
J.F.
Posts: 2906
Joined: Sun Feb 22, 2004 11:41 am

Post by J.F. »

TyRaNiD wrote:Perhaps just dont use malloc in the first place ? :)
Some programs have their own memory allocation handling - you just give it a pointer to a block of X amount of memory, and it does the rest. Most of the id programs are that way. In that case, just give it the address of the 32MB block, and it'll do all the memory handling on its own. I wonder if that's enough for Q3A... I'll have to try that.
PeterM
Posts: 125
Joined: Sat Dec 31, 2005 7:25 pm
Location: Edinburgh, UK
Contact:

Post by PeterM »

Sounds doable.
http://aaiiee.wordpress.com/

I can no longer do any homebrew PSP development nor discuss PSP specific topics.
Cpasjuste
Posts: 214
Joined: Sun May 29, 2005 8:28 am

Post by Cpasjuste »

I noticed that when i reboot psplink in vsh mode, i get this :

Code: Select all

host0&#58;/> meminfo
Memory Partitions&#58;
N |    BASE    |   SIZE   | TOTALFREE |  MAXFREE  | ATTR |
--|------------|----------|-----------|-----------|------|
1 | 0x88000000 |  6291456 |   2270464 |   1866496 | 000C |
2 | 0x08800000 | 50331648 |  24760576 |  23950592 | 000F |
3 | 0x88000000 |  6291456 |   2270464 |   1866496 | 000C |
4 | 0x88600000 |  2097152 |   2097152 |   2097152 | 000C |
5 | 0x0B800000 |  8388608 |   8388608 |   8388608 | 000F |
What does that mean ? Is there a way to have a continus memory parition by this way ?
TyRaNiD
Posts: 907
Joined: Sun Jan 18, 2004 12:23 am

Post by TyRaNiD »

Well as pointed out you could do something similar in normal game mode to allocate all of ram in one massive chunk of memory in partition 2 but that requires firmware support as early as possible which probably isn't going to happen any time soon.
moonlight
Posts: 567
Joined: Wed Oct 26, 2005 7:46 pm

Post by moonlight »

TyRaNiD wrote:What would have beem nice would be if partition 2 had actually been extended in sysmem to encompass the entire memory region then the kernel would have allocated stacks at top end of that region giving you a nice 50+ meg contiguous block which you would have been able to allocate quite easily :P
Well, if i ever continue with this shit, I may try that.
SamuraiX
Posts: 76
Joined: Tue Jan 31, 2006 6:28 am
Location: USA
Contact:

Post by SamuraiX »

Hate to bring up an old topic but it seems Moonlight has incorporated the addition 32 MBytes to Malloc() when using his SDK for M33-3 (Looking at sample memory test).

This is great news, meaning if you include the SDK with your current project you will now have direct malloc access to the additional memory.

I can't wait to incorporate support for additional memory on my project (OpenBOR) now!


Thank You Moonlight!!!
Post Reply