Building PRXes in PSPSDK

Discuss the development of new homebrew software, tools and libraries.

Moderators: cheriff, TyRaNiD

Post Reply
TyRaNiD
Posts: 907
Joined: Sun Jan 18, 2004 12:23 am

Building PRXes in PSPSDK

Post by TyRaNiD »

Okay, this is probably a long post but it is something I should have done before ;P

As some people know pspsdk has had limited prx building for a while now, now with a major overhaul and integration into the normal build system I felt it was time to explain a bit about how to build prxes in pspsdk, some things to bear in mind and some information on using the tools.

For people who are too lazy to read a long post, this is a summary of the _improvements_ in the new build system.
  • - Now links with the pspsdk crt0. You no longer need to manually setup your initial thread or parse the arguments. Just place a main function in your source code and it will just work. If you don't want an initial thread (e.g. in a pure export library) then you can specify the PSP_NO_CREATE_MAIN_THREAD() in your source, main will then be called from the module loader thread, just don't sleep in it :P.
    - No longer need to specify a different template makefile to build a prx, just set BUILD_PRX to 1 in your makefile and it will do everything necessary.
    - psp-prxgen now removes undefined re-locations, this was a bug which should have been fixed long ago ;P
    - No longer need to write an export file if you are not interested exporting anything other than the default module_start and module_info.
    - Sort of related, added a PSP_MAIN_THREAD_NAME(name) definition so you can change the default thread name. Not really required but makes it easier to see what modules are what in the list.
    - Oh and malloc should now work, you will need to set PSP_HEAP_SIZE_KB(size_kb) though as prxes have a default heap of 64kb if you need to use more than that. It should be noted on this though that if you are running the prx from a _normal_ app which also uses malloc you dont actually have any memory left for the prx to allocate.
A BIG NOTE: Building prxes does not mean you can make encrypted ones which will work on anything, it will only permit you make plain text modules, and you might need to apply some kernel patches to get them to load (e.g. pspSdkInstallNoPlainModuleCheckPatch).

-= And so we begin =-

First off to use some of this you need to update both pspsdk and newlib to the very latest versions.

For a simple example how do you build a prx in pspsdk? If you have got a pretty basic application (ideally a user mode one) you can build a prx by simply specifying BUILD_PRX=1 in your makefile.
Rebuild and if all went well you will get a .elf and a .prx file. The .elf should only be used for extracting debugging information, it almost certainly wont run on the psp. Copy your prx to your psp and run using your favourite method (this is possibly more complicated than it sounds of course :P).

To make a kernel mode prx just specify 0x1000 in your PSP_MODULE_INFO attribute. The sdk will worry about the rest for you.

Before going further some other useful new makefile definitions.

USE_KERNEL_LIBS=[1,0] - Only link in kernel mode libraries. If you are building a kernel mode prx this is highly recommended. You can also use it for normal apps but it is inadvisable. The reason for this is the default build mode will link in user libraries before kernel libraries where it can, this allows you to run 100% native kernel mode, eliminating the syscalls for abit extra speed. It should be noted however that you can use user mode libs as well, but you will need to specify them manually.

USE_KERNEL_LIBC=[1,0] - This is a counterpart to USE_PSPSDK_LIBC, it will use the kernel's very limited libc instead of using newlib or pspsdk libc. Use this only if you are making very limited usage of libc and want to go for size.

PRX_EXPORTS=file.exp - This specifies the exports file to use for your prx. It must have an exp extension for the default build rules to work. NOTE: This has superseded the old method where you specified it in the OBJS. For the file format see below on the section about psp-build-exports.

-= The tools =-

There are two tools in pspsdk in support of prxes. One you should never need to worry about but might as well explain it as I go.

psp-prxgen - This is the guts of the prx generation, without it you are screwed. It takes a specially linked elf file (you cannot run it on an elf built using the normal method) and converts it to a valid prx file. That's it, the make system will handle it for you, it would only be called outside of this in order to diagnose problems (run with the -v flag to dump alot of debug output).

psp-build-exports - This tool takes a text file and builds a .c file containing code to establish the exports for the prx. Even if you are not exporting any functions it is still important because standard executables need at least two exports in order to run.

The text file consists of a list of commands which are read sequentially in order to make your exports, the testprx sample has a simple one to demonstrate usage, but I will reproduce it here.
PSP_BEGIN_EXPORTS

PSP_EXPORT_START(syslib, 0, 0x8000)
PSP_EXPORT_FUNC(module_start)
PSP_EXPORT_VAR(module_info)
PSP_EXPORT_END

PSP_EXPORT_START(MyLib, 0, 0x0001)
PSP_EXPORT_FUNC(getModuleInfo)
PSP_EXPORT_END

PSP_END_EXPORTS
This should be placed in a file named something like exports.exp, and referenced in your makefile with the PRX_EXPORTS directive. If you only want to export module_start and module_info you can use the default exports table which will get linked in if you don't specify PRX_EXPORTS.

A description of the commands.
  • - PSP_BEGIN_EXPORTS - This just starts the export tables, it is really just a place marker.
    - PSP_EXPORT_START(name, ver, attributes) - Begin an export library with the specified name, version and attributes. When defining the default export the prx does not use a name (it sets the pointer to 0) so the made up name of syslib is used for this as a place marker. The version is major/minor arrangement, one in each byte of this 16bit value. It is important as you can use this to ensure you can only link to a certain library version. For our purposes it isn't really important. The attributes are not quite clear, 0x8000 is a special case for syslib. For exporting a set of functions from a library to be used by another prx (i.e. user to user) of the same type specify 0x0001. For exporting functions from a kernel prx to be used by user and kernel mode applications through a syscall gateway specify 0x4001.
    - PSP_EXPORT_FUNC(name) - Export a function. The SHA1 nid will be autogenerated from the name you give it.
    - PSP_EXPORT_VAR(name) - Same as above but for a variable.
    - PSP_EXPORT_FUNC_NID(name, nid) - Export a function with a specific name and SHA1 hash value. This is only needed if you either want to export a function with a randomish hash value or you don't know the
    real name of the function to generate the _proper_ nid.
    - PSP_EXPORT_VAR_NID(name, nid) - Same as above but for variables.
    - PSP_EXPORT_FUNC_HASH(name)/PSP_EXPORT_VAR_HASH(name) - Synonyms for PSP_EXPORT_FUNC/VAR, there due to legacy :P
    - PSP_EXPORT_END - End the current export library.
    - PSP_END_EXPORTS - Place holder for the end of all the exports.
Specifying syslib is absolutely mandatory, else your prx wont work (however you can add additional exported functions such as module_stop which is called when your module is being shut down). The rest can be added as needed, there is obviously a limit to the number of exports but not a practical one.

While in normal operation you shouldn't need to call psp-build-exports manually there is one important time when you do. There would be no point in exporting functions from your module if nothing can import it. This is where the --build-stubs and --build-stubs-new options come into play. By running the tool with one or other of these options and providing the name of the export text file each of the libraries in the file (barring syslib) will be written to a file of the form libname.S. These can then be included in your project which needs the functions imported. The difference between them is --build-stubs generates the old style assembly files which only have to be included in your project, --build-stubs-new generates the new style files which need to be specially built to get them to work, the benefit to using this is along with psp-fixup-imports it eliminates unused functions from the modules imports (saving space). Unless you are a glutton for punishment or you are adding these to pspsdk (where the usage of this form is close the mandatory) it probably isn't worth the effort :)

Okay that's enough for one day. Have fun coding.
Oobles
Site Admin
Posts: 347
Joined: Sat Jan 17, 2004 9:49 am
Location: Melbourne, Australia
Contact:

Post by Oobles »

Over the last week I've been attempting to create, load and use my own prx with little success. This is some notes which after discussions with qubitz on IRC seem to be required.

The situation is that I have a user memory prx which is to be called from a user memory thread in an application which has been compiled to allow kernel access on startup. After lots of trial and error this is what we discovered:

1. You must call pspSdkInstallNoPlainModuleCheckPatch from the kernel thread before attempting to laod your module from the memory stick.

2. The load module function requires the full path to the PRX.

3. You can load and start your module in either the kernel thread or user thread.

4. To access your PRX functions you must manually call pspSdkFixupImports. This function can only be called in a kernel thread. This makes point 3 invalid, as you must then load and start your prx in a kernel thread. I'm very curious as to why this needs a kernel thread, and it needs more investigation.

4. As noted above the PSP_HEAP_SIZE_KB is required to get things using malloc working.

After all this there is still problems with bus errors, likely from malloc returning 0. I'm yet to try this specific recipe myself, but it seems getting the memory allocation right is still tricky and will also need further investigation.

A few things I'd like to understand better.

1. In what situations is pspSdkFixImports required? Why doesn't the load or start module fix these imports?

2. Why does pspSdkFixImports need to be in a kernel thread? My intention was to eventually allow LuaPlayer to load PRXs, however this might be more difficult if I can't search and call the PRX exports.

3. Why is it that even when you set PSP_HEAP_SIZE_KB in both the PRX and the application that you can continue to get 0 from malloc? Is there any restrictions on which libc is used, etc?

Hope that helps anyone having troubles with PRXs, and hopefully we can sort out the rest of the issues.

David. aka Oobles.
TyRaNiD
Posts: 907
Joined: Sun Jan 18, 2004 12:23 am

Post by TyRaNiD »

Okay let me try and answer some questions.
1. In what situations is pspSdkFixImports required? Why doesn't the load or start module fix these imports?
The automatic fixup in the kernel has a couple of restrictions, the main one though being that it will not link up module exports for two different mode prxes unless you set the 0x4000 flag in the libraries attributes (and then this only allows a user mode app to link to a kernel mode export). This I guess is to prevent accidently passing kernel addresses to a user mode app or to stop kernel mode functions being directly linked to (with a j address instruction) from a user mode app (as it could not jump to the address). The typical example of this is the net libs, they are user mode modules so the functions will not get auto linked into a kernel mode program without manually fixing them up.
2. Why does pspSdkFixImports need to be in a kernel thread? My intention was to eventually allow LuaPlayer to load PRXs, however this might be more difficult if I can't search and call the PRX exports.
In order that SdkFixImports can work it calls sceKernelFindModuleByName which is a kernel mode only function. You could rewrite it so it scans memory and trys to find heuristically the module export table but that is considerably more annoying to implement correctly. If I recall there was at some point code which did this in the sdk which was removed as it was unreliable.
3. Why is it that even when you set PSP_HEAP_SIZE_KB in both the PRX and the application that you can continue to get 0 from malloc? Is there any restrictions on which libc is used, etc?
Well I don't really know, if you are going to use a malloc implementation I would probably recommend using the one in newlib (along with its sbrk). Think the one in pspsdk libc is still broken (or at least not been updated since the overhaul of the prx builder). It all depends on how you are testing your stuff. One thing to bear in mind is the libc init call (in crt0) uses malloc to strdup some strings (which is kindof annoying if you ask me :P) so if you were testing by trying to malloc all your memory in one chunk it is no doubt going to fail.

Based on what I guess is what you are trying to do with lua let me make a few suggestions ;)

1. Make luaplayer a user mode prx only and bootstrap it into memory. This will ensure you have maximum compatibility for linking libraries in (such as net etc.) The bootstrap could operate in a similar fashion to psplink by loading a kernel mode module into memory which could also contain a number of key kernel exports (with 0x4001 as its attribute) which would allow access to sio, adddrv, kernel loadmodule etc. It is worth noting that if you provide a function which is called through the syscall gateway then set the k1 reg to 0 before calling the _real_ kernel function it will operate as if it was a kernel mode function (not a limited usermode one).

2. Don't bother trying to auto import loadable lua prx modules, on the main user mode luaplayer export a library with function calls to register and deregister a library set. Then get the loaded module to call that during its initialisation, you might have race condition issues I guess but it would be more generic. It is worth nothing this is how irx exports worked.
Oobles
Site Admin
Posts: 347
Joined: Sat Jan 17, 2004 9:49 am
Location: Melbourne, Australia
Contact:

Post by Oobles »

Just to clarify a couple of points for the slow people like myself. :)

To use the newlib libc and malloc you must remove any USE_KERNEL_LIBC or USE_PSPSDK_LIBC from your makefile. The default is that newlib libc will be linked.

The concept of bootstrapping is to make your application a usermode PRX. Make the main elf/eboot a kernel mode application. You keep the bootstrap short and perform only those functions which require kernel model. Your application will then be correctly in user mode and you have less or the kernel/user mode pains.

David. aka Oobles.
qubitz
Posts: 32
Joined: Sun Apr 03, 2005 10:30 am

Post by qubitz »

Using the bootstrap method I have been able to get user PRX exports and memory to work properly. However, the results are not so successful when I don't bootstrap. I haven't been able to track it down yet, but there seems to be something broken.
raf
Posts: 57
Joined: Thu Oct 13, 2005 7:38 am

Post by raf »

Well; I've been playing with PRXs some more, and these are my findings...

1- Most of the problems I had the last time are gone! (malloc / crt0 linkage / etc)!

2- Loading a usermode PRX from a usermode thread in a kernel mode application is NOT the same as loading a usermode PRX from a 100% usermode application. This explains why using a bootstrap works.
I have a kernel mode bootstrap that loads my app -- which is now a usermode prx. Then the app loads other usermode prx's no problem.
There is no need to call pspSdkFixImports or anything. BUT, pspSdkInstallNoPlainModuleCheckPath NEEDS to be called (in the bootstrap).

3- Both my main app(prx) and the other prx are linked against newlib. I found that malloc works fine now from both; but other stdio functions fail from the child prx. Functions like fopen() fail, and generate an exception that points to newlib's findfp.c funcion, or refill.c.
I ended up exporting some functions from the main prx to the child, and this seems to have solved some issues. But I wonder if there is something else I need to do..
*Do I need to initialize newlib or newlib's stdio in the PRX?

4- I tried exporting variables, but when I add an entry to my .exp file like:
PSP_EXPORT_VAR(var_name)
Then when I run psp-build-exports, the .S file has no reference whatsoever to my variable...
*Am I missing something?


Thanks!,

Raf.
TyRaNiD
Posts: 907
Joined: Sun Jan 18, 2004 12:23 am

Post by TyRaNiD »

*Do I need to initialize newlib or newlib's stdio in the PRX?
You shouldn't need to afaik, the newlib in the child prx _should_ be totally separated from the parent, and so the theory goes if your parent is working fine then so should your child. I assume you are linking in with the default crt0 for prxes? If not then there is a libc init function which needs to be called at some point but last time I looked it didn't really do an awful lot. Perhaps it is a malloc issue or something, maybe.
Then when I run psp-build-exports, the .S file has no reference whatsoever to my variable...
*Am I missing something?
Nope you are not, I haven't got around to working out the variable import structures so while you can export variables you cannot import them, hence why they are not in the .S. When I get some time I'll try and fix that. In many ways it is easier and probably cleaner to treat a module as an object and provide accessor methods to manipulate variables.
raf
Posts: 57
Joined: Thu Oct 13, 2005 7:38 am

Post by raf »

TyRaNiD wrote:
*Do I need to initialize newlib or newlib's stdio in the PRX?
You shouldn't need to afaik, the newlib in the child prx _should_ be totally separated from the parent, and so the theory goes if your parent is working fine then so should your child. I assume you are linking in with the default crt0 for prxes? If not then there is a libc init function which needs to be called at some point but last time I looked it didn't really do an awful lot. Perhaps it is a malloc issue or something, maybe.
That's what I thought.. Is there a problem with using PSP_HEAP_SIZE_KB in both PRXs?
I use the standard crt0, I'm not doing anything too weird.
I'm using the PSP_NO_CREATE_MAIN_THREAD() in the child PRX though; but I tried without it with same results.
Then when I run psp-build-exports, the .S file has no reference whatsoever to my variable...
*Am I missing something?
Nope you are not, I haven't got around to working out the variable import structures so while you can export variables you cannot import them, hence why they are not in the .S. When I get some time I'll try and fix that. In many ways it is easier and probably cleaner to treat a module as an object and provide accessor methods to manipulate variables.
[/quote]

That explains it. OK. Yes, I am using objects and interfaces for communications. I was just trying to avoid linking against the same libraries on both prxs, and trying to export some globals to get the child to link without the whole lib.

Thanks,

Raf.
raf
Posts: 57
Joined: Thu Oct 13, 2005 7:38 am

Post by raf »

TyRaNiD wrote:
*Do I need to initialize newlib or newlib's stdio in the PRX?
You shouldn't need to afaik, the newlib in the child prx _should_ be totally separated from the parent, and so the theory goes if your parent is working fine then so should your child. I assume you are linking in with the default crt0 for prxes? If not then there is a libc init function which needs to be called at some point but last time I looked it didn't really do an awful lot. Perhaps it is a malloc issue or something, maybe.
OK Now I have a better grasp on what's going on, and why the theory of "if it works on the parent should work on the child" fails in my case.
These are just facts I have discovered, and are purely symptomatic (don't really understand the causes completely, hopefully you can help).

-When a PRX is loaded/started, main() runs in its own thread. In this thread everything I've tried works fine. newlib included.
The "parent prx" runs everything from this main, and so everything works fine.
-Now, when the parent PRX calls an exported function from the child PRX, this function is executed in the parent's thread, but the data (data/bss segments?) seems to not be shared as they would be if the function resided in the same module. And so in this environment, newlib doesn't work.

Digging some deeper, it seems that the main problem with newlib's stdio in particular, is a global variable "_global_impure_ptr" which is dereferenced from most stdio's functions (for some reentry purpose).
This variable is declared in newlib's reent/impure.c (defined in sys/reent.h) as:

Code: Select all

static struct _reent __ATTRIBUTE_IMPURE_DATA__ impure_data = _REENT_INIT (impure_data);
struct _reent *__ATTRIBUTE_IMPURE_PTR__ _impure_ptr = &impure_data;
struct _reent *_CONST __ATTRIBUTE_IMPURE_PTR__ _global_impure_ptr = &impure_data; 
If the address of impure_data(or what's pointed to by _global_impure_ptr) is logged from the prx main(), it appears correct. But once logged from an exported function, it is garbage:

This is from the parent PRX:

Code: Select all

09&#58;07&#58;43.253&#58;ScreenHandler/ScreenHandler.cpp@160<20>&#58; In PSPRadioPRX&#58; _global_impure_ptr=0x088CD340, _impure_ptr=0x088CD340
This is logged from the child's PRX's exported function:

Code: Select all

09&#58;07&#58;43.757&#58;main.cpp@47<20>&#58; In TextUI- Before&#58; _global_impure_ptr=0x00000040, _impure_ptr=0x00000020
I have a work-around that seems to work for me right now, but it is kind of a hack, and hope to be able to remove it once this is all figured out.
My work around involves declaring a local impure_data structure in the prx, and then making _impure_ptr and _global_impure_ptr point to it instead of the default one. Once this is done, all calls to the prx that need stdio work fine.

Code: Select all

#ifndef __ATTRIBUTE_IMPURE_DATA__
#define __ATTRIBUTE_IMPURE_DATA__
#endif
static struct _reent __ATTRIBUTE_IMPURE_DATA__ my_impure_data = _REENT_INIT &#40;my_impure_data&#41;;
extern struct _reent *_global_impure_ptr __ATTRIBUTE_IMPURE_PTR__;
void FixNewLibStdio&#40;&#41;
&#123;
	ModuleLog&#40;LOG_LOWLEVEL, "In TextUI- Before&#58; _global_impure_ptr=%p, _impure_ptr=%p", _global_impure_ptr, _impure_ptr&#41;;
	_impure_ptr = &my_impure_data;
	_global_impure_ptr = &my_impure_data;
	ModuleLog&#40;LOG_LOWLEVEL, "In TextUI- After&#58; _global_impure_ptr=%p, _impure_ptr=%p", _global_impure_ptr, _impure_ptr&#41;;
&#125;

IPSPRadio_UI *ModuleStartUI&#40;&#41;
&#123;
	ModuleLog&#40;LOG_LOWLEVEL, "ModuleStartUI&#40;&#41;"&#41;;

	FixNewLibStdio&#40;&#41;;
		
	return new CTextUI&#40;&#41;;
&#125;
ModuleStartUI() is my exported function. It is called once, it 'fixes' stdio, and then returns the object I use for successive calls.

This code generates this log entries:

Code: Select all

09&#58;07&#58;43.618&#58;main.cpp@55<20>&#58; ModuleStartUI&#40;&#41;
09&#58;07&#58;43.757&#58;main.cpp@47<20>&#58; In TextUI- Before&#58; _global_impure_ptr=0x00000040, _impure_ptr=0x00000020
09&#58;07&#58;43.895&#58;main.cpp@50<20>&#58; In TextUI- After&#58; _global_impure_ptr=0x08D2E9B8, _impure_ptr=0x08D2E9B8
09&#58;07&#58;43.905&#58;TextUI.cpp@55<10>&#58; CtextUI&#58; Constructor start
09&#58;07&#58;44.055&#58;TextUI.cpp@71<10>&#58; CtextUI&#58; Constructor end.
The only problem with this approach is that I'm redifining _global_impure_ptr, as it's originally defined as _CONST (read only) in reent.h.. To get around that, I use a local reent.h without this restriction (this makes this an even bigger hack).

I also found that sometimes, if PSP_HEAP_SIZE_KB is not used in the PRX, the _global_impure_ptr pointer points to, apparently, valid data for a while, and the app may run a little further before crashing. (and you are of course limited to the default of 64KB or heap)
I'm using PSP_HEAP_SIZE_KB(2048); right now and get garbage assigned initially.

So all in all, I'm happy that I finally got my child prx running correctly. But there's gotta be a better way of doing this... Or maybe I'm just doing something completely wrong? Don't know. Hopefully you can enlighten me.

Thanks!,

Raf.
TyRaNiD
Posts: 907
Joined: Sun Jan 18, 2004 12:23 am

Post by TyRaNiD »

Sounds very odd, based on your description it would indicate to me that either newlib or you have a buffer overflow bug somewhere.

One possible way of fixing it would be to actually implement the reentrant stuff (to do with storing the impure ptr's in the data block referenced by the k0 register) if that was done then if you called a newlib function inside your child it would actually use the reent data from the calling thread and not the statically linked stuff in newlib, unfortunately this isn't the smallest amount of work as I have other stuff to consume my attention atm. Will do it sooner or later.

However that said I don't really understand why the impure pointers in newlib would not be defined correctly in your child, sounds very strange. I'll try and look at it this weekend for you :)
raf
Posts: 57
Joined: Thu Oct 13, 2005 7:38 am

Post by raf »

TyRaNiD wrote:Sounds very odd, based on your description it would indicate to me that either newlib or you have a buffer overflow bug somewhere.
That may very well be. I'll try to re-create with the sdk sample code, and let you know.
However that said I don't really understand why the impure pointers in newlib would not be defined correctly in your child, sounds very strange. I'll try and look at it this weekend for you :)
Thanks, man. Whenever you have time. Like I said, at least I got my stuff working atm. Still, your help is greatly appreciated!.

Thanks again,

Raf.
raf
Posts: 57
Joined: Thu Oct 13, 2005 7:38 am

Post by raf »

OK, I was able to recrate the behavior using the latest prx sample from the sdk source.

ftp://ftp.berlios.de/pub/pspradio/pre_r ... x-test.zip

All I did was commenting out the USE_PSPSDK_LIBC from the Makefiles (so it uses newlib). I also added some printf's to show the pointer addresses in different places.
I found that printf (not sceDebugScreenPrintf) works from the main() of the PRX but not from the exported function, it causes an exception if called there.
So I print the pointer using the debug function from the exported function..

Thanks for looking into it.

Raf.
TyRaNiD
Posts: 907
Joined: Sun Jan 18, 2004 12:23 am

Post by TyRaNiD »

Well I have found the bug, and it isn't good :)

As far as I can tell gcc is totally ignoring the G0 switch which is supposed to disable use of the gp register, however if you disassemble the resulting code you will see it is still using it. So the reason it is breaking is fairly simple, when it gets impure_ptr it references it via the gp register which is different for a different module and thus returns garbage.

As for a fix well, I have looked around in gcc, it certainly seems that it is setting the G flag correctly to 0, and there seems to be a check for the size, however there is also a second place where the symbol could be marked as gp relative so it might actually be that which is causing it, bypassing the G flag entirely :/

As a quick and dirty fix you could probably wrap any exported functions with code to set the gp register to the value in the module itself, e.g.

Code: Select all

void* setgp&#40;void* gp&#41;
&#123;  
     unsigned int oldgp;
     asm&#40;
           "move %0, $gp\n"
           "move $gp, %1\n"
           &#58; "=r"&#40;oldgp&#41;
           &#58; "r"&#40;gp&#41;
          &#41;;
     return oldgp;
&#125;

extern void _gp;

void modexport1&#40;void&#41;
&#123; 
     void* oldgp;

     oldgp = setgp&#40;&_gp&#41;;

     /* Do code */

     setgp&#40;oldgp&#41;;
&#125;
raf
Posts: 57
Joined: Thu Oct 13, 2005 7:38 am

Post by raf »

TyRaNiD wrote:Well I have found the bug, and it isn't good :)

As far as I can tell gcc is totally ignoring the G0 switch which is supposed to disable use of the gp register, however if you disassemble the resulting code you will see it is still using it. So the reason it is breaking is fairly simple, when it gets impure_ptr it references it via the gp register which is different for a different module and thus returns garbage.

As for a fix well, I have looked around in gcc, it certainly seems that it is setting the G flag correctly to 0, and there seems to be a check for the size, however there is also a second place where the symbol could be marked as gp relative so it might actually be that which is causing it, bypassing the G flag entirely :/

As a quick and dirty fix you could probably wrap any exported functions with code to set the gp register to the value in the module itself, e.g.

Code: Select all

void* setgp&#40;void* gp&#41;
&#123;  
     unsigned int oldgp;
     asm&#40;
           "move %0, $gp\n"
           "move $gp, %1\n"
           &#58; "=r"&#40;oldgp&#41;
           &#58; "r"&#40;gp&#41;
          &#41;;
     return oldgp;
&#125;

extern void _gp;

void modexport1&#40;void&#41;
&#123; 
     void* oldgp;

     oldgp = setgp&#40;&_gp&#41;;

     /* Do code */

     setgp&#40;oldgp&#41;;
&#125;
Thanks for taking the time to look into this!. So gcc is not using the -G0 flag; interesting. The psp-gcc man page also says that for -G to work, all modules must be compiled with the flag; I wonder if something in the sdk/newlib doesn't have it?
I'll try your work around right now, and let you know! (It would be painful to use it, though, as I would need to do it for every method in my interface).

Thanks again!,

Raf.
raf
Posts: 57
Joined: Thu Oct 13, 2005 7:38 am

Post by raf »

OK, I think I know how to fix this.

BTW, I tried your workaround, but I was still getting invalid imputr_ptr. So I started trying all the gcc flags that had to do with relocations, and GOT, etc.

I found that when I used -mno-explicit-relocs in my code, I was able to access the module's imputr_ptr correctly from the exported function. But the app would still crash when a stdio (newlib) call tried to access it.

So I recompiled newlib modified to use this flag as well. Once the process is linked against the newlib with the flag, calls can access the pointer correctly, and everything works!.

I'm not sure what other implications can this flag have, but maybe we should patch newlib in svn to use it? It's up to you; but it'd be appreciated so I don't need to have my own newlib.

Code: Select all

~/Projects/newlib/newlib $ svn diff configure.host
Index&#58; configure.host
===================================================================
--- configure.host      &#40;revision 1808&#41;
+++ configure.host      &#40;working copy&#41;
@@ -574,7 +574,7 @@
          mipsallegrex*-psp-*&#41;
                sys_dir=psp
                syscall_dir=syscalls
-               newlib_cflags="$&#123;newlib_cflags&#125; -G0 -DCOMPACT_CTYPE -DCLOCK_PROVIDED -DMALLOC_ALIGNMENT=16 -I$&#123;prefix&#125;/psp/sdk/include" ;;
+               newlib_cflags="$&#123;newlib_cflags&#125; -G0 -mno-explicit-relocs -DCOMPACT_CTYPE -DCLOCK_PROVIDED -DMALLOC_ALIGNMENT=16 -I$&#123;prefix&#125;/psp/sdk/include" ;;
          *&#41;
                newlib_cflags="$&#123;newlib_cflags&#125; -DMISSING_SYSCALL_NAMES" ;;
        esac
Thanks!,

Raf.
raf
Posts: 57
Joined: Thu Oct 13, 2005 7:38 am

Re: Building PRXes in PSPSDK

Post by raf »

TyRaNiD wrote: - Now links with the pspsdk crt0. You no longer need to manually setup your initial thread or parse the arguments. Just place a main function in your source code and it will just work. If you don't want an initial thread (e.g. in a pure export library) then you can specify the PSP_NO_CREATE_MAIN_THREAD() in your source, main will then be called from the module loader thread, just don't sleep in it :P.
In my application, I'm using the PRXs in this mode, as a pure export library.
I'd like to just have an empty main(), as all the work is performed by the exports; but I noticed that when the prx main() returns, exit() is called, which in turns causes all memory (allocated by the sbrk) to be released. Is exit() really need to be called here? Isn't memory released by the system when the PRX is unloaded? Or leave it up to the PRX to call exit?
Maybe if PSP_NO_CREATE_MAIN_THREAD() is used, then don't exit? doesn't exiting make the no main thread scenario basically useless?

Thanks for your help,

Raf.
*eyelash
Posts: 3
Joined: Wed Jan 10, 2007 8:07 am

Post by *eyelash »

raf or tyranid, please, to make a app using net lib (3.03 OE with WPA support) i need to make a kernel prx or 100% user mode works?

thanks!
*eyelash
PSP TA-082 with FW 3.03 OE-A2
worster
Posts: 1
Joined: Tue Feb 06, 2007 7:04 pm

Post by worster »

TyRaNiD wrote:
*Do I need to initialize newlib or newlib's stdio in the PRX?
You shouldn't need to afaik, the newlib in the child prx _should_ be totally separated from the parent, and so the theory goes if your parent is working fine then so should your child. I assume you are linking in with the default crt0 for prxes? If not then there is a libc init function which needs to be called at some point but last time I looked it didn't really do an awful lot. Perhaps it is a malloc issue or something, maybe.
Then when I run psp-build-exports, the .S file has no reference whatsoever to my variable...
*Am I missing something?
Nope you are not, I haven't got around to working out the variable import structures so while you can export variables you cannot import them, hence why they are not in the .S. When I get some time I'll try and fix that. In many ways it is easier and probably cleaner to treat a module as an object and provide accessor methods to manipulate variables.

Yes, I sure agree with your opinion, it's a good method to use accessor methods to access variable that in prxes. But i'm sorry about that i really need to import the variables exported by some prxes. And then how could i do this? Beg your help...
jockyw2001
Posts: 339
Joined: Thu Sep 29, 2005 4:19 pm

Post by jockyw2001 »

My ME kernel mode prx needs access to global vars which are defined in the parent user mode prx and vice versa. Is there a solution for this? I tried the PSP_EXPORT_VAR() macro, but it doesn't do anything.

And can functions be exported bidirectionally, i.e. parent to child and child to parent?
TyRaNiD
Posts: 907
Joined: Sun Jan 18, 2004 12:23 am

Post by TyRaNiD »

The stubs do not currently support variables and I dont think they ever will, it would require too much hackery or modification to ld to be really worth it. If you design needs global variables then imo you haven't designed it correctly ;)
jockyw2001
Posts: 339
Joined: Thu Sep 29, 2005 4:19 pm

Post by jockyw2001 »

TyRaNiD wrote:The stubs do not currently support variables and I dont think they ever will, it would require too much hackery or modification to ld to be really worth it. If you design needs global variables then imo you haven't designed it correctly ;)
True :)

I still have this question: can functions be exported bidirectionally, i.e. parent to child and child to parent?
TyRaNiD
Posts: 907
Joined: Sun Jan 18, 2004 12:23 am

Post by TyRaNiD »

I still have this question: can functions be exported bidirectionally, i.e. parent to child and child to parent?
In theory as long as you export with the delayed link option.
Anissian
Posts: 16
Joined: Fri Jan 26, 2007 8:40 pm

Post by Anissian »

TyRaNiD wrote:The stubs do not currently support variables and I dont think they ever will, it would require too much hackery or modification to ld to be really worth it. If you design needs global variables then imo you haven't designed it correctly ;)
Heh I agree :o). This may not be the right thread, but checking the code for psplink, I thought that this could be worth mentioning (although admittedly offtopic, is there a better place for bug reports?). svn seems down so I cannot check if it is fixed in last version

Code: Select all

bash-3.2# diff -urN libs.c.old libs.c
--- libs.c.old  2007-02-12 19&#58;21&#58;22.000000000 +0100
+++ libs.c      2007-02-12 19&#58;22&#58;15.000000000 +0100
@@ -209,7 +209,7 @@
                                &#125;
                        &#125;

-                       if&#40;pImp->funcCount > 0&#41;
+                       if&#40;pImp->varCount > 0&#41;
                        &#123;
                                printf&#40;"Variable Imports&#58;\n"&#41;;
                                for&#40;count = 0; count < pImp->varCount; count++&#41;
seems like a typo to me, (apologies if it is not) but it's really minor (won't go into the for loop anyway)

thanks
Post Reply